ABSTRACT

This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.

The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.

Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.

The purpose of this mixed methods research study was to assess the impact and influence of a pre-international experience course on Arizona State University (ASU) students before they study or intern abroad. Currently, the study abroad pre-departure orientation for ASU participants consists of online modules and a two-hour face-to-face orientation. In this action research study, the practitioner-researcher re-designed an ASU School of Politics and Global Studies (SPGS), one-credit course that focused exclusively on cross-cultural awareness and sensitivity. A needs assessment was distributed to a sample of 800 returning study abroad participants and was used to influence the study, along with an extensive literature review and two cycles of action research. The dissertation research and study was conducted during the ASU fall 2013 semester. Quantitative data and qualitative data were collected using eight different measures. To better understand the impact of a pre-international experience curriculum for ASU study abroad and international internship participants before they go abroad, this research study investigated the following research questions: (1) What cultural impact does a pre-international experience course have on students who complete the course before studying or interning abroad? (2) What specific cultural competencies are gained by the participants after participating in the pre-international experience course? (3) How has developing the curriculum, teaching the curriculum and implementing the innovation influenced and informed my practice as an international educator and the Assistant Director of the Arizona State Study Abroad Office? The following five assertions were identified within the quantitative and qualitative analysis of the collected data to answer the three research questions: (1) Students are more confident in their abilities to cross cultures after successfully completing taking the new course; (2) Students are more aware of other cultures and their own culture after successfully completing taking the new course; (3) Students gained important knowledge about understanding others' worldviews after successfully completing taking the new course; (4) Students gained general openness toward intercultural learning and to people from cultures different from their own after successfully completing the new course; (5) Developing and implementing a pre-international experience course changed me as a leader, instructor and researcher. Implications for future implementation and research are discussed.

Virtual Patient Simulations (VPS) are web-based exercises involving simulated patients in virtual environments. This study investigates the utility of VPS for increasing medical student clinical reasoning skills, collaboration, and engagement. Many studies indicate that VPS provide medical students with essential practice in clinical decision making before they encounter real life patients. The utility of a recursive, inductive VPS for increasing clinical decision-making skills, collaboration, or engagement is unknown. Following a design-based methodology, VPS were implemented in two phases with two different cohorts of first year medical students: spring and fall of 2013. Participants were 108 medical students and six of their clinical faculty tutors. Students collaborated in teams of three to complete a series of virtual patient cases, submitting a ballpark diagnosis at the conclusion of each session. Student participants subsequently completed an electronic, 28-item Exit Survey. Finally, students participated in a randomized controlled trial comparing traditional (tutor-led) and VPS case instruction methods. This sequence of activities rendered quantitative and qualitative data that were triangulated during data analysis to increase the validity of findings. After practicing through four VPS cases, student triad teams selected accurate ballpark diagnosis 92 percent of the time. Pre-post test results revealed that PPT was significantly more effective than VPS after 20 minutes of instruction. PPT instruction resulted in significantly higher learning gains, but both modalities supported significant learning gains in clinical reasoning. Students collaborated well and held rich clinical discussions; the central phenomenon that emerged was "synthesizing evidence inductively to make clinical decisions." Using an inductive process, student teams collaborated to analyze patient data, and in nearly all instances successfully solved the case, while remaining cognitively engaged. This is the first design-based study regarding virtual patient simulation, reporting iterative phases of implementation and design improvement, culminating in local theories (petite generalizations) about VPS design. A thick, rich description of environment, process, and findings may benefit other researchers and institutions in designing and implementing effective VPS.

ABSTRACT Education policymakers at the national level have initiated reforms in K-12 education for that past several years that have focused on teacher quality and teacher evaluation. More recently, reforms have included legislation that focuses on administrator quality as well. Included in far-reaching recent legislation in Arizona is a requirement that administrators be evaluated on a standards-based evaluation system that is linked to student outcomes. The end result is an annual summative measure of administrator effectiveness that impacts job retention. Because of this, Arizona administrators have become concerned about rapidly becoming proficient in the new evaluation systems. Administrators rarely have the explicit professional development opportunities they need to collaborate on a shared understanding of these new evaluation systems. This action research study focused on a group of eight administrators in a small urban district grappling with a new, complex, and high-stakes administrator evaluation that is a component of an all-encompassing Teacher Incentive Fund Grant. An existing professional learning time was engaged to assist administrators in lessening their concerns and increasing their understanding and use of the evaluation instrument. Activities were designed to engage the administrators in dynamic, contextualized learning. Participants interacted in a group to interpret the meaning of the evaluation instrument share practical knowledge and support each other's acquisition understanding. Data were gathered with mixed methods. Administrators were given pre-and post-surveys prior to and immediately after this six-week innovation. Formal and informal interviews were conduct throughout the innovation. Additionally, detailed records in the form of meeting records and a researcher journal were kept. Qualitative and quantitative data were triangulated to validate findings. Results identified concerns and understanding of administrators as they attempted to come to a shared understanding of the new evaluation instrument. As a result of learning together, their concerns about the use of the instrument lessened. Other concerns however, remained or increased. Administrators found the process of the Administrator Learning Community valuable and felt their understanding and use of the instrument had increased. Intense concerns about the competing priorities and initiatives led to the administrators to consider a reevaluation of the competing initiatives. Implications from this study can be used to help other administrators and professional development facilitators grappling with common concerns.

This mixed methods participatory action research study explored how an instructional coach influenced a state mandated curriculum adoption at a Title 1 urban middle school. The purpose of this study was to identify ways in which an instructional coach supported a veteran staff during the adoption of new curriculum standards. The instructional coach/action researcher employed a three pronged coaching approach that incorporated individual and team coaching sessions and increased networking to encourage and support the development of social capital. This study was informed using Vygotsky's Social Learning Theory, Wenger's Communities of Practice, Coleman's Social Capital Theory, and Hall and Horde's Concerns-Based Adoption Model. The study is heavily weighted in favor of qualitative data which includes participant reflections, coach individual session and team session reflections, field-notes, team meeting videos, and exit interviews. Several themes emerged supporting the use of a differentiated coaching approach, the promotion of social capital, and the identification of initiative overload as a barrier to curriculum adoption. The quantitative data analysis, pre and post study Stages of Concern Questionnaires, produced evidence that participants experienced minor shifts in their concerns relating to the adoption of Common Core State Standards. Results were used to inform coaching decisions based on individual participant needs as well as to augment the qualitative findings. Ideas for further research are discussed.

As students gain access to personally-owned Mobile Communication Devices (MCDs), schools have begun to embrace MCDs as mobile-learning (m-learning) teaching and learning tools. A research gap currently exists for the innovation of m-learning with student-owned devices, which this study attempts to fill by answering the following Research Question: What are the Bring Your Own Device (BYOD) Levels of Use of teachers at a high-performing, high SES suburban high school? To answer this question, I answered 5 sub questions: (1) What instructional decisions did BYOD user-level teachers make with regards to m-learning? (2) How did teachers collaborate on BYOD with colleagues during implementation? (3) How did teachers participate in voluntary professional development for BYOD and m-learning? (4) Was there a difference in Levels of Use between early career and veteran teachers? (5) What barriers to successful implementation did teachers at this school report? To answer these questions, I conducted a Levels of Use interview with 2-3 teachers from each academic department (n=28), at a school that was in its third year of BYOD implementation, as well as observed 18 of the teachers during instruction. I triangulated data from a first and second interview with observation data, and analyzed these data sets to profile the different Levels of Use among the teachers, and present recommendations for research and practice. I rated all participants between Level 0: non-use and Level IVB: refinement; no teachers in this study were above Level IVB. The findings indicate that teachers made instructional decisions based on their Level of Use, and although they did not participate in ongoing professional development specific to BYOD, they did work with others based on their Level of Use. Few teachers participated in voluntary professional development, and cited time as a factor. This study also finds that personal experience with technology and lesson planning for student-centered learning is a greater indicator of successful BYOD implementation than age or teaching experience. Finally, the most commonly reported barriers to successful implementation of BYOD were time, equity/access, and student behavior.

The pathway for entering the K-12 classroom as a teacher varies compared to what was once the traditional model of teacher preparation. In this mixed-methods action research study, I explore supporting first year alternatively certified urban and rural intern teachers through a multicomponent distance induction program. The induction model in this study was based on the theoretical framework of Bandura's social learning theory and Wenger's communities of practice. The purpose of this study was to identify the extent in which a multicomponent distance learning induction program impacts first year intern teachers' sense of self-efficacy, understand their successes, their challenges, and to identify how intern teacher evaluations change. Quantitative data included results from a self-efficacy survey and the Student Teaching Assessment Instrument (STAI). Qualitative data was collected through intern teachers' blogs, cadre leader video narratives, and cadre leader STAI narrative responses. Six themes emerged including topics such as building relationships with other education professionals, receiving feedback from the cadre leader, identifying struggles and application of college coursework into the K-12 classroom. Key findings reveal interns are least efficacious in student engagement, relationships with other educators support an intern teacher emotionally and pedagogically, intern teachers struggle with work-life balance, and cadre leaders observed intern teachers as having improved their skills in student engagement, instructional practices, and classroom management. Implications to practice include a structured approach to introducing student engagement, creating a best practices library of video examples, and a pre-orientation (Super Saturday) of topics prior to stepping into the classroom with students.

This mixed methods action research study explores the impact of a multilevel intervention on retention indicators of special education induction teachers and the leadership capacities of the special education induction coaches and coordinator. The purpose of this investigation was to understand the impact of developing and implementing an action research study on three different levels of participants the special education induction coaches, teachers and me. A theoretical framework based upon Bandura's (1977, 1982) work in Social Learning Theory, and in self and collective efficacy informs this study. The conceptual framework developed based upon the tenets of Authentic Leadership Theory and special education mentor programs inform the development of the intervention and data collection tools. Quantitative data included results collected from the Psychological Capital Questionnaire (PCQ), Authentic Leadership Questionnaire (ALQ), and the Special Education Induction Teacher Questionnaire (SEITQ). The qualitative data included results collected from the SEITQ open-ended questions, Email Reflective Response (ERR), organic and structured focus groups, fieldnotes, and the Teachers' Final Letter. Findings include: a) I changed as a leader and a researcher, b) the special education induction coaches began to think and act as leaders, c) the special education induction teachers' retention indicators increased, d) by actively participating in the co-construction of the special education induction program, both the coaches and the teacher provided valuable insights as pertains to developing a program that supports special education induction teachers. Implications and next steps are discussed.

The use of educational technologies as a tool to improve academic achievement continues to increase as more technologies becomes available to students. However, teachers are entering the classroom not fully prepared to integrate technology into their daily classroom teaching because they have not been adequately prepared to do so. Teacher preparation programs are falling short in this area because educational technology and the role of technology in the classroom is seen as an extra component to daily teaching rather than a central one. Many teacher preparation programs consist of one stand-alone educational technology course that is expected to prepare teachers to integrate technology in their future classrooms. Throughout the remainder of the program, the teachers are not seeing educational technologies modeled in their other core courses, nor are they getting the hands-on interaction necessary to become more confident in using these technologies with their future students. The purpose of this study was to examine teachers' views of educational technology in the classroom from those enrolled in a graduate program. The study consisted 74 first- and second-year teachers who were enrolled an alternative teacher preparation program. Thirty-four of the teachers received the Integrating Curriculum and Technology (iCAT) intervention and the remaining 40 teachers were part of the control group. Each teacher completed a pre- and post-intervention questionnaire and 23 of the 74 teachers participated in one of three focus group interviews. Additional data from the teachers' course instructors were gathered and analyzed to compliment the focus group and quantitative data. Results showed that iCAT participants' scores for confidence in using technology and efficacy for using educational technology increased at a faster rate than the control group participants' scores. Similarly, confidence in using technology, perceptions about integrating technology in the classroom, and efficacy for using educational technology could be predicted by the amount of hands-on interaction with technology that the teachers received during their graduate course. The discussion focuses on recommendations for infusing technology throughout teacher preparation programs so that teachers have the tools to prepare their students to use a variety of technologies so that their students can be better prepared to complete in today's workforce.

ABSTRACT Professional Development (PD) is an important tool in the field of education. Successful PD programs are those that include adult learning methods and opportunities for experiential learning and discussion. The university where this action research was conducted does not offer formal training to adjunct instructors. The adjunct instructors are hired based primarily on their content knowledge. This research was conducted to understand, if the application of a blended training model for adjuncts influences the adjunct's perception of meeting their student's educational needs and the student's perception that their personal education needs are met. The blended learning included the delivery of a framework that incorporated Andragogy, Content Knowledge and Technology (ACKT). The purpose of the ACKT framework is to supplement adjunct's content knowledge expertise with adult learning methods and technology. The effectiveness of the framework was measured by using a quasi-experimental, pre to post intervention assessment. The treatment group and control group each contained twenty-two adjunct instructors from the university. The treatment group received training on the framework prior to commencing the class and participated in two focus groups during the semester. In addition, the treatment group was observed teaching in their classroom. The control group did not receive training, or participate in focus groups; however they were observed teaching in their classroom. The results of the action research showed significant improvement for the adjunct instructors in the treatment group. Specifically, knowledge of and application of andragogy showed a large improvement. In addition, the social influence of the adjuncts in the treatment group showed a large improvement. Less significant was the improvement in the efficacy of the students in the treatment group classes compared to those in the control group classes. However, the data suggests that the students in the treatment group better applied the content learned and they were more aware of other's educational needs than their peers in the control group. The study supports the need for adjunct instructor PD. Through a PD program adjunct instructors increase their own efficacy and this improvement translates into increased content transfer for the students in the classroom. Based on the strong evidence for adjunct instructor improvement this research will continue by expanding the blended learning model to more of the adjunct instructors at the university, and continuing to evaluate the effectiveness of the model in meeting student's educational needs.