Understanding the impact of waiving proctoring polices for PAEA End of Rotation™ exams
PAEA End of Rotation exam national comparative data remain stable year-round. National comparative data are based on the performance of a reference population of student test-takers, not current student test-takers. That reference population took End of Rotation exams in past years with either live in-person or live remote proctors.
Under the current, temporarily waived proctoring policies for the End of Rotation exams, national data will not be influenced. This is because the national comparative data, found on our website and on End of Rotation exam score reports, are from a reference population. Therefore, it will remain unchanged.
As changes in proctoring have occurred, the way programs interpret student test-taker scores should be reviewed. The national comparative data on the PAEA End of Rotation exam website and on exam score reports consist of student test-takers who took the exam under live-proctored controlled environments. Given the challenges created for PA programs by the COVID-19 pandemic, PAEA has waived proctoring practices, which has resulted in a spectrum of local exam administration practices. Some of these local practices continue to involve high security, while some involve less security. If your program has elected exam administration practices that involve little-to-no security, you may not want to compare your students to the national comparative data, which was gathered in a reference population who took the exams under live-proctored controlled environments. You also may want to use caution when comparing this year’s local cohort to last year’s local cohort, as they also took the exam in a live-proctored environment.
If you are currently administering exams in an unproctored and/or non-secured environment, you may want to consider factors that can influence test scores such as a student using a textbook during the exam. If this is the environment that is best suited to your program’s current needs, considerations should be made to compare exam scores at the local level to previous local cohorts or to the national comparative data. Programs can still compare scores, but they must take into consideration that the test environment is different than previous End of Rotation exam administrations, which may have a positive or negative impact on scores.
Aside from proctoring, there are other factors that can influence scores. Everyone right now, whether they are aware of the feeling or not, is in a state of stress. High-stake exams, in typical environments, are stressful for students. Combined with the daily stressors students may be facing related to COVID-19, their performance could be impacted. Consideration should be given to students and the impact current events may have on their scores during this time.
Another consideration is a sudden and potential change in length of rotations. Programs have been innovative in bridging the gap of lost time and experience, but this circumstance is still new for many students and these changes can impact student exam performance. Almost all programs have moved to an online format, but as with any new teaching strategy, there is a learning curve for both the instructor and the student. Many students have been following a sequence of a clinical rotation then an exam — and that balance has been suddenly disrupted. Thought should be given to the impact this may have on scores as well.
All student test-takers’ scores starting from March 13, 2020, will not be included in the forthcoming national comparative data.
Update: Version 7 of End of Rotation and will remain active and in circulation for current and new cohorts. When we release version 8, programs will receive ample notice. You can find additional information here.