Performance Outcomes

Performance Outcomes

2021

Impact on P-12 Learning and Development

For the 2019-2020 cycle, the EPP developed new instruments, Completer Visit to the Classroom Assessment Instrument and Completers’ Interview on Teaching Effectiveness and P-12 Students Impact, to assess completers’ impact on P-12 students learning and development. In addition, the EPP did and action research to explore completer’s self-efficacy perception regarding teaching effectiveness and P-12 students’ impact.  All these measures evidenced that EPP completers have a positive impact on P-12 students learning and development.

The Completer visit to the Classroom Assessment Instrument is a rubric based on the InTASC Standards for effective teachers. It was developed to observe completers performance teaching P-12 students in a school scenario. The instrument is distributed in four stages of proficiencies and three in-between stages. The stages are: Distinguished (4); Proficient (3); Emerging (2); and Underdevelop (1). The in between stages describe that completers are close to achieving the prior stage but is still in progress. The rubric has a validity index of .89 and a Cronbach’s alpha index of .974 (See Exhibit 1.0 EPP Created Instruments). The Completers’ Interview on Teaching Effectiveness and P-12 Students Impact is an eleven open-ended questions interview. It was developed to assess completers’ perceptions on teaching effectiveness and P-12 students’ impact. The interview was validated for content by an expert panel (See Exhibit 1.0 EPP Created Instruments). The EPP did not have access to the State value-added measures on P-12 students’ growth and developed the action research to evidence completers teaching effectiveness and P-12 impact using multiple measures (qualitative and quantitative).

Completer Visit to the Classroom Assessment Instrument

Six (37%) of the 16 EPP completers for 2019-2020 were observed teaching a group of P-12 students in virtual classroom scenarios. The EPP used the Completer Visit Classroom Assessment Instrument to record completer’s compliance with InTASC categories: The Learner and Learning, Content Knowledge, Instructional Practices, and Professional Responsibilities.  Although proficient in all items scored, completers scores of (3.08, 3.16, 3.25 out of 4.0) were related to InTASC standards #3, The teacher understands how students differ in their approach to learning and creates instructional opportunities that are adapted to diverse learners; standard #9, The teacher is a reflective practitioner who continually evaluates the effects of his/her choices and actions on others and who seeks out opportunities to grow professionally; and standard #5, The teachers uses understanding of individual and group motivation and behavior to create a learning environment that encourages positive interaction, active engagement in learning and self-motivation. All completers are teaching in virtual platforms and are adapting teaching and learning resources to satisfy student needs. In addition, completers are inexperienced in the field of distance education and that could have been a critical factor creating engaging learning environments. Nevertheless, results show that EPP completers are performing at a level of proficiency of 3.08 and above in all the InTASC categories and standards (See Table 1.0 2019-2020 Completers Visit to the Classroom Assessment Instrument Scores Disaggregated Data by InTASC Standard and Specialization.

 

Table 1.0 2019-2020 Completer Visit to the Classroom Assessment Instrument Scores Disaggregated Data by InTASC Standard and Specialization
Completer Major Scored by InTASC Standard and Item Number
  InTASC St 1 InTASC St 2 InTASC St 3 InTASC St 4 InTASC St 5
N=6 #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11
ESL Elementary 4 4 4 4 4 3.5 4 4 4 4 4
Preschool 4 4 4 4 3.5 3.5 3.5 4 3.5 3.5 3.5
K-3 4 4 4 4 3.5 3.5 3 3 3.5 4 3.5
Special Education 3 3 3.5 3 3 3 2.5 3 3 2.5 3
4-6th 4 4 3.5 3 3.5 3 3 3 3 2.5 3
Preschool 3 3 4 3 3 3.5 2.5 3 3.5 3 2.5
Mean 3.66 3.66 3.83 3.50 3.50 3.33 3.08 3.33 3.41 3.25 3.25

 

Completer Major Scored by InTASC Standard and Item Number
  InTASC St 6 InTASC St 7 InTASC St 8 InTASC St 9 InTASc St 10
N=6 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21
ESL Elementary 4 4 4 4 4 4 4 4 4 4
Preschool 3.5 4 4 4 4 3.5 4 4 4 4
K-3 4 4 4 3.5 4 4 3 3.5 3.5. 3.5
Special Education 3 4 3 2.5 3 3 3 3 3 2.5
4-6th 3 4 3 2.5 3 3 2.5 3 2 2.5
Preschool 4 4 3.5 3.5 2.5 2.5 2.5 2.5 3.5 3
Mean 3.58 4.00 3.58 3.33 3.58 3.33 3.16 3.33 3.33 3.25

 

Completers’ Interview on Teaching Effectiveness and P-12 Students Impact

The EPP interviewed seven (44%) 2019-2020 completers using Completers’ Interview on Teaching Effectiveness and P-12 Students Impact. Data reflected that completers are using the assessment cycle to revise planning and to use a variety of teaching strategies to approach each learner needs. The interviews were coded and categorized for analysis. The themes that emerged from the categories were: Practical Courses, Real-Life Scenarios, Parent’s Disengagement, Individualized Education, Curriculum Adaptations, and Authentic Assessment. The findings show that completers are aware that P-12 student needs are diverse and that they have the responsibility and commitment to address them. In addition, they are using the assessment cycle to revise planning, reteach information, and to use a variety of learning strategies to ensure P-12 students learning success. All the interviewees reported P-12 students growth academically and demonstrating better dispositions to learn.

Table 1.1 Completers’ Interview on Teaching Effectiveness and P-12 Impact Themes

Themes

Practical Courses

Real-Life Educational Scenarios

Parent’s Disengagement

Individualized Education

Curriculum adaptations

Authentic Assessment

 

Action Research

The EPP’s 2019-2020 action research explored three (20%) completer’s self-efficacy perception regarding teaching effectiveness and P-12 students’ impact. The research questions were:

  • What EPP preparatory experiences help develop teacher efficacy?
  • How do completers perceive their self-efficacy?
  • How do completers perceive their teaching effectiveness?

The EPP used various instrument to triangulate data: Completers Interviews on Teaching Effectiveness and P-12 Impact, Teachers’ Sense of Self-Efficacy Scale (a proprietary assessment), Principal’s Evaluations, Completer Visit to the Classroom Assessment Instrument, and Completers’ Artifacts (students’ assessments) (See Exhibit 1.1 Action Research). Completers provided evidence of classroom artifacts that have been used to evidence student’s growth. In general, all P-12 students were positively impacted by EPP completers by improving their academic performance. Completer 001 student’s performed at the expectancy level for Teaching Strategies Creative Curriculum. Completer 002 students’ had academic learning growths of 80% in English Oral Communication Skills, and Completer 003 students demonstrated gains in performance from 16% in the pretest to 42% in the posttest in the Kindergarten Performance Test. The EPP completers demonstrated a high sense of teacher efficacy that have served as a motivator to look for innovate ways of teaching and to help P-12 students attain the desired levels of performance. One of the recommendations of the study was to add P-12 student evaluations as another measure of impact for future studies and for EPP continuous improvement. In addition, the Puerto Rico Department of Education is working on resuming standardized testing for next year. This past two years (2018-2020) the tests have been cancelled due to the earthquakes and the COVID-19 pandemic. The standardized test will provide the EPP with a State value-added measure.

Indicators of Teaching Effectiveness

For 2019-2020, the EPP developed various instruments with the purpose of measuring completers teaching effectiveness. As stated in Measure 1, the Completer Visit to the Classroom Assessment Instrument is a rubric based in the InTASC Standards for effective teachers. The rubric was developed to observe completers performance as teachers with a group of P-12 students. The instrument is distributed in four stages of proficiencies and three in-between stages. The stages are: Distinguished (4); Proficient (3); Emerging (2); and Underdeveloped (1). The in-between levels are: Partial success at rating “4” (3.5); Partial Success at rating “3” (2.5); Partial Success at rating “2” (1.5).  The rubric has a validity index of .89 and a reliability index (Cronbach’s alpha) of .974 (See Exhibit 1.0 EPP Created Assessments). Six completers out of sixteen (37%) were observed in virtual classrooms while teaching a group of students. The same faculty member visited all completers minimizing differences on criteria and assuring reliability. Although proficient, completers scores of 3.08, 3.16, 3.25 out of 4.0 were related to InTASC standards #3, The teacher understands how students differ in their approach to learning and creates instructional opportunities that are adapted to diverse learners; standard #9, The teacher is a reflective practitioner who continually evaluates the effects of his/her choices and actions on others and who seeks out opportunities to grow professionally; and standard #5, The teachers uses understanding of individual and group motivation and behavior to create a learning environment that encourages positive interaction, active engagement in learning and self-motivation. As with other measures, the EPP have noted consistency in completer’s areas that are related with the transitioning from face-to-face teaching to virtual scenarios. Completers are teaching in digital platforms that are relatively new to everyone. They had no mentorship in this transitioning stage because it was an abrupt but necessary adoption for the school system due to the pandemic. Other than that, completers did perform at the distinguished and proficient stages. Three completers performed at partial success of a proficient rating (2.5) in some items and one performed, in one item, at the emerging stage (2) (See Table 1.0 Completers’ Visit to the Classroom Assessment Mean Scores Disaggregated by InTASC Standards and Specialization). Those completers that performed in some items below 3.0 belong to the Special Education, 4th-6th, and Preschool Education programs. Special Education and Preschool programs rely on hands-on instructional strategies that empower student’s construction of learning. The EPP analysis is that changes in teaching scenarios could account for these programs differences in scores comparing to other EPP programs. The EPP will present the information in the June Stakeholders data retreat for further analysis and to find other ways to integrate innovative ways to address learning for functionally diverse and preschool students.

In addition, the EPP conducted an action research to explore completer’s self-efficacy perception regarding teaching effectiveness and P-12 student impact (See Exhibit 1.1 Action Research). The EPP collected data from three completers (20%) using various instruments: Completers’ Interview on Teaching Effectiveness and P-12 Impact, Teachers’ Sense of Self-Efficacy Scale, Completers’ Visit to the Classroom, Principals Evaluations and Completers’ Artifacts. The research questions were:

  • What EPP preparatory experiences help developed teacher self-efficacy?
  • How do completers perceive their self-efficacy?
  • How do completers perceive their teaching effectiveness?

The findings for research question 1 (RQ1) suggested that completers were satisfied with the preparation experiences received at the EPP. The findings suggested that having four practical experiences during preparation: Field Experiences I, Field Experiences II, Clinical Experiences I, and Clinical Experiences II were critical in their development of teacher self-efficacy. The EPP provide candidates with performance experiences in their core and specialization courses. In these courses they acquire content and pedagogical knowledge, teaching skills, and dispositions that give them self- confident in their capabilities. In addition, EPP school stakeholders provide opportunities for candidates to observe highly effective teachers as they interact with P-12 students. These observations are valuable examples that become part of the completers teaching repertoire. The findings are in accordance with what Maddux and Gosselin (2013) mentioned about self-efficacy development and performance and vicarious experiences.

The findings of RQ2 suggested that completers perceived themselves as having high self-efficacy.  Completers demonstrated that they have that confidence and self-control necessary to produce gains in students learning. In addition, individuals high in self-efficacy are known to be self-drive and to motivate others to engage in productive behaviors. Self-efficacy represents an individual’s belief in his/her ability to execute behaviors necessary to produce specific and productive outcomes (Bandura, 1977, 1997).  Tschannen-Moran and Woolfolk (1998) argued that competence, self-efficacy, and commitment are intertwined. Competence and commitment are also characteristics of effective teachers. Moreover, Lukácová et al. (2018) found that the higher the level of teachers’ self-efficacy, the better the teacher is in their tendency to adopt an active attitude to the teaching process. Teacher efficacy is also related to the positive use of teaching strategies and to be more students oriented.

The findings of RQ3 suggested that completers perceived themselves as effective teachers. Completer’s concept of effective teaching goes beyond grades as they seek for the social and emotional wellness of their students and their development of a sense of competence. In addition, effective teachers contribute to positive academic, attitudinal, and social outcomes for students. Completers also included parents and other professionals to individually planned and intervene for students success. These perceptions are in accordance with Goe et al. (2008) findings that effective teachers have high expectations for all students and help students learn. Goe et al. (2008) pointed that effective teachers collaborate with other teachers, parents, and professionals to ensure students success.

EPP completers demonstrated, through multiple sources of evidence, that they are effective teachers.

 

Table 1.0 Completers Visit to the Classroom assessment Instrument Disaggregated Mean Scores by InTASC Standard and Specialization.
Completer Major Scored by InTASC Standard and Item Number
  InTASC St 1 InTASC St 2 InTASC St 3 InTASC St 4 InTASC St 5
N=6 #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11
ESL Elementary 4 4 4 4 4 3.5 4 4 4 4 4
Preschool 4 4 4 4 3.5 3.5 3.5 4 3.5 3.5 3.5
K-3 4 4 4 4 3.5 3.5 3 3 3.5 4 3.5
Special Education 3 3 3.5 3 3 3 2.5 3 3 2.5 3
4-6th 4 4 3.5 3 3.5 3 3 3 3 2.5 3
Preschool 3 3 4 3 3 3.5 2.5 3 3.5 3 2.5
Mean 3.66 3.66 3.83 3.50 3.50 3.33 3.08 3.33 3.41 3.25 3.25

 

Completer Major Scored by InTASC Standard and Item Number
  InTASC St 6 InTASC St 7 InTASC St 8 InTASC St 9 InTASc St 10
N=6 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21
ESL Elementary 4 4 4 4 4 4 4 4 4 4
Preschool 3.5 4 4 4 4 3.5 4 4 4 4
K-3 4 4 4 3.5 4 4 3 3.5 3.5. 3.5
Special Education 3 4 3 2.5 3 3 3 3 3 2.5
4-6th 3 4 3 2.5 3 3 2.5 3 2 2.5
Preschool 4 4 3.5 3.5 2.5 2.5 2.5 2.5 3.5 3
Mean 3.58 4.00 3.58 3.33 3.58 3.33 3.16 3.33 3.33 3.25

 

Satisfaction of Employers and Employment Milestones

For 2019-2020, the EPP used a new instrument Employers’ Satisfaction with Completer Survey to assess completer’s mastery of teaching and professional competencies as perceived by employers. The instrument has a content validity of .90 and a Cronbach’s alpha index of .802 (See Exhibit 1.0 EPP Created Assessments). The EPP sent the Employers’ Satisfaction with Completer Survey and had a 69% (9 out of 13 employers) return. Some employers used the Google Forms instrument, while others filled out a copy of the instrument and returned it by email. Three completers, out of sixteen, are not currently employed in teaching positions.

Five of those nine employers (55%) recruited candidates who did their Student Teaching at their schools. Employers’ answers fell between Strongly Agree and Somewhat Agree which demonstrated that employers perceived that completers are well prepared to perform as effective teachers. Completers scores (3.44 and 3.55 out of 4) corresponded to items #5 and #8. Item #5 is about creating learning environments, and item #8 is about the uses of formative and summative assessments. The EPP analysis reflected that completers received their education in traditional classrooms but had to transition to virtual classroom for their first experiences as teachers. These completers have been teaching virtually without prior experience in virtual teaching platforms. Changes in teaching modalities could account for the 3.4 scores particularly in students’ motivation, the creation of learning environments, and assessing learning. Nonetheless, all scores were excellent and demonstrated completer’s strengths and competencies as required by InTASC standards for teaching effectiveness. In addition, 33% (3) of employers gave perfect scores to completers. The EPP have started modifications in all courses to resolve areas for improvement related with the transition to virtual education scenarios. At this moment, two full-time faculties are being trained in the development of digital resources and educational materials appropriate for functionally diverse students. In addition, the university has been diligent to provide professional development related to virtual learning environments, rubric development, assessment strategies, Blackboard Collaborate, and reasonable accommodations, among others. It has been demonstrated that the EPP is using employer’s information to work toward continuous improvement. In addition, some employers are part of the EPP Steering Committee and will participate in the 2021 June data retreat for further analysis and discussion of these findings.

As far as employment milestones, five of the thirteen (38%) completers of 2019-2020 that are currently working as teachers, were employed in the same school that they did the student teaching. This datum reflected employer’s satisfaction with EPP completers’ performance. Retaining student teachers after graduation demonstrates that EPP completers are meeting employers’ expectations on teaching effectiveness and professional standards. Table 3.0 shows employers’ satisfaction with completer survey mean scores disaggregated by item number and employer response. Table 3.1 shows employers satisfaction with completer aggregated data.

In general, EPP completers are exceeding employers satisfaction with their performance as it is evidenced by the data provided and by retaining student teachers once they graduate and become completers.

Table 3.0 2019-2020 Employers’ Satisfaction with Completer Survey Mean Scores Disaggregated by items and Employers Response
Employers Mean Scores by Item Number
N=9 #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 Mean
1 3 4 4 4 3 4 4 4 4 4 3.40
2 4 4 4 4 3 3 3 3 4 3 3.20
3 4 4 4 4 4 4 4 4 4 4 4.00
4 4 4 4 4 4 4 4 3 4 4 3.90
5 4 4 3 3 4 4 4 3 4 4 3.70
6 4 4 4 3 3 4 4 3 4 4 3.70
7 4 4 4 4 4 4 4 4 4 4 4.00
8 3 3 3 3 3 4 3 3 4 4 3.30
9 4 4 4 4 4 4 4 4 4 4 4.00
Mean 3.77 3.88 3.77 3.77 3.55 3.88 3.77 3.44 4.00 3.88  

 

Table 3.1 2019-2020 Employers’ Satisfaction with Completer Survey Mean Scores Aggregated Data (new instrument)
Employers N= 9 Mean Scores by Item Number
#1 #2 #3 #4 #5 #6 #7 #8 #9 #10
Mean 3.77 3.88 3.77 3.77 3.55 3.88 3.77 3.44 4.00 3.88
Percent values 94% 97% 94% 94% 88.75% 97% 94% 86% 100% 97%
Satisfaction of Completers

In 2019-2020, the EPP administered a satisfaction survey to 16 (100%) completers through telephone calls. The purpose of the survey is to learn completers’ satisfaction level with their preparation and to use that information for the continuous improvement of the Educator Preparation Program.  The instrument has been developed as part of EPP quality assurance process Satisfaction of Completers’ Survey (See Exhibit 1.0 EPP Created Assessments), which explains why the EPP reported one cycle of data for it. The instrument was validated for content .90 and its reliability was established by Cronbach’s alpha index of .910. The Completers’ Satisfaction Survey consists of 17-questions grouped by: Content Knowledge, Instructional /Pedagogical Knowledge and Skills, Diversity, Integration of Technology, and Research Skills that are aligned to InTASC, CAEP, and Puerto Rico Department of Education Standards. These areas represent the knowledge, skills, and dispositions that effective teachers exhibit in their classroom.  The survey has a Likert Scale of four where 4 means Well Prepared (very confident and very capable of doing the tasks), 3 means Sufficiently Prepared (Confident and capable of doing the tasks), 2 means Not Sufficiently Prepared (capable of doing the tasks with some effort or support), and 1 means Not At All Prepared (insecure of my ability for doing the task). The 2019-2020 completers obtained more than 3, Sufficiently Prepared, in all areas except item #16 where the mean score was 2.75 and is related to research skills. In general, the Special Education program obtained scores (means between 2.17 and 3.29) which represented that they are less confident in their knowledge and skills to do the tasks required to teachers. The Special Education Specialization is a K-12 program and is going through a systemic revision. The EPP is recommending the development of courses emphasizing elementary education, teaching methodologies, and instructional strategies for this specialization. It seems that courses are broad because there is too much content (k-12) and completers do not perceive that they have the mastery required to feel confident in the school scenario. Completers ’perceptions on their confidence and abilities provided critical information for the improvements that are needed regarding their preparation, specifically on Special Education. Table 4.0 shows completers’ satisfaction survey mean scores disaggregated by specialization.

In general, EPP completers are satisfied with their preparation as teachers and feel confident and capable of doing the tasks expected of an effective teacher.

Table 4.0 2019-2020 Completers’ Satisfaction Survey (new instrument) Mean Scores Disaggregated by Specialization
Completer

 Major (N=16)

  Completers ’Satisfaction Survey Scores by  Items
#1 #2 #3 #4 #5 #6 #7 #8 #9 #10
ESL Elementary 4 4 4 3 3 4 4 3 3 2
ESL Elementary 3 3 3 3 3 3 3 3 2 3
ESL Elementary 4 4 4 4 3 4 4 4 4 4
K-3 4 4 4 4 2 4 4 2 4 4
K-3 3 3 3 4 3 3 3 3 3 3
K-3 4 4 4 4 4 4 4 4 4 4
Special Education 3 3 3 3 3 3 4 4 4 4
Special Education 2 2 2 3 4 1 2 2 2 3
Special Education 3 3 2 2 3 2 2 2 2 2
Preschool 4 2 4 4 4 4 2 2 4 4
Preschool 3 4 4 4 4 4 4 4 4 3
4-6 2 2 2 4 4 3 3 3 4 4
History 4 1 2 3 2 2 2 2 2 4
Mathematics 4 4 4 3 3 2 4 3 4 4
ESL Secondary 4 4 4 3 4 4 4 4 4 4
ESL Secondary 4 4 3 3 3 3 4 4 4 3
Mean 3.43 3.18 3.25 3.75 3.25 3.12 3.31 3.06 3.37 3.43

 

Completer

 Major (N=16)

Completers ’Satisfaction Survey Scores by  Items
#11 #12 #13 #14 #15 #16 #17 Mean
ESL Elementary 2 4 4 4 3 3 3 3.11
ESL Elementary 4 4 4 4 3 2 3 3.11
ESL Elementary 4 3 3 4 4 4 4 3.82
K-3 2 2 2 4 4 2 4 3.29
K-3 3 3 3 3 3 3 3 3.05
K-3 3 4 4 4 3 3 3 3.76
Special Education 4 4 4 4 2 2 2 3.29
Special Education 2 2 2 2 2 2 2 2.17
Special Education 2 2 2 2 2 2 2 2.17
Preschool 2 4 4 4 4 2 2 3.29
Preschool 4 4 4 4 3 3 4 3.76
4-6 4 4 4 4 2 3 2 3.17
History 4 4 2 2 4 2 2 2.58
Mathematics 3 3 4 3 3 3 4 3.41
ESL Secondary 4 4 4 4 4 4 4 3.94
ESL Secondary 3 3 3 3 4 4 4 3.47
Mean 3.12 3.37 3.31 3.43 3.12 2.75 3.00  
Graduation Rates

The EPP selected  2014 cohort to establish graduation rates at six years or less. The cohort for undergraduate education students was defined by:

  • First year students at education programs
  • Regular type of admission
  • Full time enrollment for their first term as undergraduate students (Fall).

After selecting the cohorts for each year of admission, students are followed during six years to determine how many could complete requisites and graduate. The quantity of graduates is divided by the base and multiply by a hundred. The accumulative graduation rate at six years reflect the percent of students that graduted in six or less years from the Teacher Preparation Programs.

The EPP, like other Educator Preparation Programs across Puerto Rico and the United States, has experimented a decrease in admissions, especially in Secondary Education programs. Nevertheless, the graduation rates of  EPP students have been consistant demonstrating the quality of the program.

Accumulative Graduation Rates at 6 years or less of studies – Undergraduate Education Programs
Cohort Base Graduate % Graduate
2014 17 2 12%
2013 33 3 9%
2012 60 8 13%
2011 59 17 29%

 

Accummulative Graduation Rate at 6 years or less of studies- Disagreggated by Specialization for Undergradute Education Programs
 

Academic program

 

 

Cohort Base 2014

 

Graduates From Education Programs

 

% Graduation

 

144- SEC EDUC TEACHING OF HISTORY

 

2

 

0

 

0%

147- SEC ED TEACH ENGLISH 2ND LANGUAGE 4 1 25%
206- ELEM ED TEACH ENGLISH SEC LANG  

1

 

0

 

0%

236D- EARLY CHD ELEM LVL K3 INTERNET  

1

 

0

 

0%

236- EARLY CHILDHOOD ELEM LVL K3  

3

 

0

 

0%

243- EARLY CHILDHOOD PRESCHOOL LVL  

3

 

1

 

33%

TOTAL  

17

 

2

 

12%

 

Ability of Completers to Meet Licensing

In 2019-2020, the EPP integrated pedagogical situations in core courses to prepare candidates to solve educational problems using evidence-based research and content and pedagogical knowledge. These revisions sought to strengthen candidates’ skills and to improve the Puerto Rico Teacher Certification Test scores (PCMAS). Nevertheless, the 2020 PCMAS administration, scheduled for March 17, was cancelled due to the COVID-19 pandemic island-wide lockdown. The College Board Administration of Puerto Rico administered the test this past March2021 and results would not be available until June 2021. In the academic year 2018-2019, 15 completers took the PCMAS. Table 6.0 shows the single-assessment level pass-rate data for regular teacher preparation program for 2018-2019. Table 6.1 shows Aggregate Assessment Level Pass-Rate Data for EPP. There is no data for the 2010-202 Licensure Exams and the submission of it has been waived due to the pandemic.

 

Table 6.0 EPP Completers Single-Assessment Level Pass-Rate Data (PCMAS) for 2018-2019 Disaggregated by Tests
Type of assessment Assessment Code Number No. of Students taking Assessment No. of Students Passing Assessment Institution Pass Rate Statewide Pass Rate *Test Takers rate
PCMAS General  

PR10

 

 

10

 

 

9

 

9/10 = 90%

 

94%

 

67%

PCMAS General Elementary  

PR21

 

9

 

8

 

8/9 = 89%

 

93%

 

60%

PCMAS General Secondary  

PR25

 

1

 

1

 

1/1 = 100%

 

98%

 

7%

Specialization Spanish  

PR30

 

 

 

 

89%

Specialization English  

PR40

 

 

3

 

2

 

2/3 = 67%

 

95%

 

20%

Specialization Mathematics  

PR50

 

1

 

1

 

1/1 = 100%

 

88%

 

7%

Specialization Social Studies  

PR60

 

 

 

 

92%

Specialization Science

 

 

PR70

 

88%

*Test takers Rate: The ratio of aggregate number of students taking the assessment to the number of program completers for the institution and a specific academic year.

Table 6.1  2018-2019 Aggregate Assessment Level Pass-Rate Data for EPP.
Type of Assessment Assessment Code Number N. of Students Taking Assessment No. of Students Passing Assessment Institution Pass Rate Statewide Pass Rate
PCMAS General PR10 10 9 9/10 = 90% 94%
PCMAS General (Elementary/Secondary)  

PR21, PR25

 

10

 

9

 

9/10 = 90%

 

94%

Specialization PR30, PR40, PR50, PR60, PR70  

4

 

3

 

3/4 = 75%

 

93%

Summary Pass Rate**   10 9 9/10 = 90% 588/636 = 92%

**Summary Pass Rate: The proportion of program completers who passed all tests they took for their areas of specialization among those who took one or more tests in their specialization areas.

 

Ability of Completers to be hired in Education Positions

One of the EPP goals is that all its completers get hire in education positions. For the 2019-2020 cycles, 12 out of 16 (75%) completers have been hired as teachers. Of those four (33%) are working in the public system of education and eight (66%) are working in the private educational system.  Two (12.5%) 2019-2020 completers are self-hired as teachers for homeschool students.  Table 7.0 shows completers hired in education positions disaggregated by specialization and school system.

The EPP recognize that completers are being successfully hired in positions related to education and that the educator preparation program have been strengthened through the application of data driven decisions.

Table 7.0 2019-2020 Completers Hired in Education Positions Disaggregated by Specialization and School System
Completers Specializations Number of Completers

 

Private School System Public School System
Preschool 2 1 1
K-3 3 3  
4-6 1 1  
History 1   1
Math 1   1
ESL Elementary 1 1  
ESL Secondary 1 1  
Special Education 2 1 1
Total 12 8 4

 

 

Student Loan Default Rate

The EPP received in September 2020 the FY 2017 Cohort Default Rate form the United States Department of Education. The FY 2017 Cohort Default Rate was established at 3.7.  As established by the Higher Education Act of 1965 (HEA), as amended, the Higher Education Reconciliation Act of 2005 (HERA), Pub. L. 109-71 and the Department of Education’s regulations, the Inter American University, Metro Campus, is not subject to any sanctions based on the FY 2017 CDR. Schools with a cohort default rate of less than 15.0 percent for each of the most recent fiscal years may disburse loans in a single disburse. Table 8.0 shows Aggregated EPP Cohort Default Rate for years 2014 through 2017. Table 8.1 shows Disaggregated EPP Cohort Default Rate Information.

The EPP has maintained an excellent record through the years as shown in the information provided by the United States Department of Education.

Table 8.0 Aggregated EPP Cohort Default Rate for years 2014 through 2017
Year Cohort Default Rate
2014 10.5 %
2015 7.9 %
2016 2.7 %
2017 3.7 %
Table 8.1 Disaggregated EPP Cohort Default Rate Information.
OPE ID School Type Control PRGMS   FY2017 FY2016 FY2015
 

 

 

003940

 American University of Puerto Rico- Metropolitan Campus

Highway 1KM 16.3 Corner Francisc

San Juan, PR 00919-1293

 

 

Master’s Degree or Doctor’s Degree

 

 

 

Private

 

 

 

Both

(FFEL/FDL)

 

Default Rate

 

3.7

 

 

2.7

 

7.9

No. in Default  

305

 

230

 

1888

No. in Repay  

8,223

 

8, 340

 

23,686

Enrollment Figures

 

 

Percentage Calculation

10,514

 

 

 

 

78.2

11,060

 

 

 

 

75.4

0

 

 

 

 

0

2020

Impact on P-12 Learning and Development

EPP’s Response:

Currently, the EPP is working on the construction of its own link within the Education Department’s website. Next May, in time for CAEP’s site-visit, the website will be ready, including a particular link for CAEP’s related information and data.

In recent years, the EPP had very limited accurate and valid data to assess the impact of its completers on Pk-12 students. For future annual reports, the EPP will be using the Puerto Rico Department of Education Measurement and Evaluation Standardized Tests (META) to evaluate the impact of its completers on P-12 students. META tests were developed to comply with Every Student Succeeds Act (ESSA) of 2015. The tests are administered, once a year in April, to the third, fourth, fifth, sixth, seven, eight, and eleven grades. The tests measure students’ competencies in Spanish, Mathematics, English, and Science. META results are also used by the Puerto Rico Department of Education to evaluate teachers' effectiveness. To evaluate the impact of its completers on P-12 students at private schools, the EPP will be using the Learn Aide standardized test. The Learn Aide test is administered to students at private schools in Puerto Rico. To find out the impact on PK-12 students, the EPP will compare the results of the completers’ pupils, in the META and Learn Aide exams, with those of the rest of the systems, public and private, respectively.

Besides the test’s results, the EPP is working on a multiple case study design to measure completer’s impact on PK-12 students.

 

Proposal for Future Assessment

Direct Measures of P-12 Learning Specific Assessment Validity and Reliability of Instruments
Puerto Rico Department of Education is going to provide student learning data.

 

Puerto Rico Private School System is going to provide student learning data

META (Standardized Test)

 

 

Learn Aide

(Standardized Tests)

http://de.pr.gov/meta-pr/index.html

 

 

 

https://learnaidpr.com/medicion.html

Completer Case Study EPP proposed a design for the multiple case study The design addresses components 4.1 and 4.2 including completers observations, students’ pre- and post-assessment, student work samples, and completers’ interviews.

 

Completer Teaching Effectiveness

EPP’s Response:

The EPP used three measures to evaluate candidates’ teaching effectiveness:

  1. High quality test of teachers’ knowledge and skills (PCMAS).
  2. Feedback from employers.
  3. Clinical experiences Formative and Summative Evaluations.

 

According to Puerto Rico Department of Education Public Policy for Teacher Certification (Circular Letter No. 13-2019-2020), the Teacher Certification Exam is competency-based. It evaluates the competencies that completers need to become an effective teacher as established by state and federal regulations. Table 1.1 shows EPP completers Teacher Certification Exam pass rates compared to statewide pass rate. EPP completers had been maintained in the upper rank for the past years.

The EPP used the Employers Satisfaction Survey as a measure of completers teaching effectiveness. Unfortunately, the employers’ sample was small, but it demonstrated that EPP completers performed above average in the competencies needed for effective teaching. Table 4.3 shows the means scores of the Employers Satisfaction Survey by item.

The EPP also used Cooperative Teacher and University Superior Formative and Summative Clinical Experiences Evaluation as a measure of completers teaching effectiveness. Table 1.3 shows paired T-test results for formative and summative scores for Fall 2018 candidates. The scores suggested that the clinical experiences contributed positively to candidates’ performance. Table 1.4 shows paired T-test results for formative and summative scores for winter 2019 candidates. The result suggested that candidates’ scores increased as they did their clinical experiences. These measures evidenced that EPP completers are well prepared to teach PK-12 students and that their teaching practices are effective.

The EPP acknowledge that completers performance assessments are more reliable measures for teacher effectiveness. As part of the EPP re conceptualized QAS, a performance assessment has been created to measure 2019-20 completers. The EPP will be using:

  • Completers Observation Visits to the classroom.
  • PK-12 students survey on completers teaching effectiveness.
  • Completers Survey for Teaching Effectiveness.
  • Completers Focus Groups.

 

Indicators of Teaching Effectiveness:

 

 EPP’s 2018-2019 Completers PCMAS Pass Rates

Type of Assessment No. of students passing assessment No. of students passing assessment Institution

Pass Rate

Statewide

Pass Rate

PCMAS 10 9 9/10=90% 94%
PCMAS General Elementary 9 8 8/9=89% 93%
PCMAS Secondary 1 1 1/1=100% 98%
Specialization English 3 2 2/3=67% 95%
Specialization Math 1 1 1/1=100% 88%

 

 

Paired T-Test Fall 2018 Clinical Experiences Formative and Summative Scores

Cooperative Teacher Evaluation Formative Summative
Mean 2.7017 2.8600
Standard Deviation 0.2247 0.1152
University Supervisor Formative Summative
Mean 2.6400 2.7750
Standard Deviation 0.2713 0.2835

 

T-Test Spring 2019 Clinical Experiences Formative and Summative

Cooperative Teacher Evaluation Formative Summative
Mean 2.1117 2.6400
Standard Deviation 0.5004 0.2689
University Supervisor Formative Summative
Mean 2.3533 2.5700
Standard Deviation 0.2730 0.1658

 

Employers Satisfaction with EPP’s Completers

Employer Evaluation of Completer IP-12 Mean Score
Mastery of content Knowledge 87.3%
Content and Pedagogical Knowledge 83.3%
Use of Technology 94.4%
Classroom Management Skills 88.5%
Dispositions and Engagement with the profession 95.8%
Diversity and Inclusion 100%
Research and reflexive thinking competencies 94.4%
Number of employers that are TEP completers 4
How do you evaluate completers’ professional performance? Excellent
How do you perceive completers preparation? Excellent
Will you be willing to employ other TEP completers? Yes
Will your school be willing to participate as a center for candidates’ clinical experiences? Yes
Recommendations Positive discipline workshops and English Integration
Employers Satisfaction with EPP's Completers

EPP’s Response:

The EPP administrated a survey to the employers of 2018-2019 completers. The employer response rate was 25%. This percentage is based on the eight completers (53%) that are currently working as teachers. Employers were surveyed using IP-12 instrument that have 25 items and a Likert Scale of: Most-Acceptable (3); Acceptable (2); and Unacceptable (1). In addition, the instrument provides 6 open-ended questions regarding employers’ perceptions of completers professional performance. Overall, all employers rated EPP completers as most acceptable and acceptable.

 

Employers Satisfaction with EPP’s Completers

Employer Evaluation of Completer IP-12 Mean Score
Mastery of content Knowledge 87.3%
Content and Pedagogical Knowledge 83.3%
Use of Technology 94.4%
Classroom Management Skills 88.5%
Dispositions and Engagement with the profession 95.8%
Diversity and Inclusion 100%
Research and reflexive thinking competencies 94.4%
Number of employers that are TEP completers 4
How do you evaluate completers’ professional performance? Excellent
How do you perceive completers preparation? Excellent
Will you be willing to employ other TEP completers? Yes
Will your school be willing to participate as a center for candidates’ clinical experiences? Yes
Recommendations Positive discipline workshops and English Integration

 

Completer Satisfaction

EPP’s Response:

The EPP administrated a Satisfaction Survey to 2018-2019 completers. The survey assessed the degree of satisfaction with their teaching preparation in different areas including the faculty. Completers rated their overall satisfaction with their preparation as “Quite Satisfied” and/or “Satisfied”. The areas assessed and percentages of “Quite satisfied” completers’ responses were:

 

Completers Satisfaction Survey

Criteria Percentage
Content Knowledge 90%
Methodology and Courses Organization 86%
Fundamental Teaching Skills 83%
Diversity an Inclusion 85%
Classroom Management 79%
Assessment 92%
Faculty 87%

 

Graduation Rates

EPP’s Response:

Graduation rates at the Interamerican University of Puerto Rico are calculated annually through the Institutional Postsecondary Education Data System (IPEDS). For CAEP’s 2020 Annual Report, the graduation rates are based on an admission cohort selected six years earlier. The EPP's graduation rate for the 2018-2019 academic year, based on the 2013 admission cohort, is 48.4%. In 2013, the EPP admitted 33 new students of which 16 graduated in May 2019, for a 48.4% graduation rate, based on six years study program.  It is important to note that in 2013 the EPP offered 15 different concentrations. In the 2018-2019 academic year, the EPP revised its academic offering, reducing the number of concentrations by 5, from 15 to 10.

Completers Licensing

EPP’s Response:

PCMAS Teacher Report Card Data for Institutional pass rates and single assessment pass rates for initial programs are provided for academic year 2018-2019. EPP completers pass rates remained in the upper rank compared with statewide pass rates. The College Board Administration administered the Puerto Rico Teacher Certification Exam (PCMAS) once a year. PCMAS results include the 2018-2019 completers and students from other graduation cohorts. Out of 24 students that took the PCMAS, 21 (87%) approved it satisfactorily.

EPP’s  2018-2019 Completers PCMAS Pass Rates

Type of Assessment No. of students passing assessment No. of students passing assessment Institution

Pass Rate

Statewide

Pass Rate

PCMAS 10 9 9/10=90% 94%
PCMAS General Elementary 9 8 8/9=89% 93%
PCMAS Secondary 1 1 1/1=100% 98%
Specialization English 3 2 2/3=67% 95%
Specialization Math 1 1 1/1=100% 88%

 

 

Completers Hires in Education

EPP’s Response:

            Almost half of EPP completers, year 2018-2019, were successfully hired as teachers. Of the total 16 completers, 8 were hired in private and public-school systems. It is important to note that, in recent years, many of the students who graduate from Puerto Rico universities prefer to emigrate to the United States in search of better employment opportunities. It is difficult for the EPP to follow up on students who emigrate.

 

Loan Default Rates

EPP’s Response:

When the 2020 Annual Report was written, the last yearly Federal report on Students Default Rates was the 2016 one. According to the Higher Education Act of 1965 (HEA), the Higher Education Reconciliation Act of 2005 (HERA), Pub. L. 109-71 and the Department of Education regulations, the Interamerican University of Puerto Rico is not subject to any sanctions based on the FY 2016 cohort Default Rate. Interamerican University of Puerto Rico 2016 Cohort Loan Default Rate was 2.7.