Exams Security Enhancement: FAQ

This update isn’t triggered by anything in particular, but as the PAEA Assessment program grows, so does our need to address exam security. Historically, less than one percent of exams have had a concerning number of alerts, but as more people rely on the program, the risk increases.

These changes will increase the security of our exams while we work on ways to integrate lockdown technology into our platform to prevent examinees from looking at anything other than their exam or using any controls that are not strictly necessary. Over the next year, we will be developing and testing (both from a technological and psychometric perspective) lockdown browser options, with the goal of fully launching this enhancement before the End of Curriculum exam is released, estimated for early 2020.

Proctors should investigate every notification and alert — always. The level of intervention that proctors have with an examinee is up to each program, but proctors should always investigate any potentially inappropriate behavior to check for honor code and academic integrity violations.

Notifications that proctors might receive:

1. Viewing Other Content

Three of the following triggers an auto-pause alert, and two more (total of five) triggers an auto-end alert.
This could refer to any of the following being open:

• Other browser windows or tabs (It’s a good idea to check browser history for recent websites visited in case the student has closed what they were looking at.)
• Other applications, even calculators (Calculators aren’t needed for PAEA exams.)
• Chat and Instant Messaging applications
• Search boxes – a Control+F/Command+F search box is technically outside the exam window and will trigger an alert.
• System updates or alerts

2. Inappropriate Key Stroke (Mac-specific alert – A key combination that includes Command+Shift)

One of these triggers an auto-pause alert, and two more (total of three) triggers an auto-end alert.
This could refer to:

• Screenshots – usually digital
• Toggling between browser windows
• Toggling between applications
• Zoomed text (This is the most common example of a benign trigger for this alert, but functionality has been added to the ExamDriver student window to avoid this trigger.)

3. Screen Capture Detected (PC-specific alert – Print Screen button)

One of these triggers an auto-pause alert, and two more (total of three) triggers an auto-end alert.
This refers to:

• Screenshots – digital or to the networked printers

Follow these step-by-step instructions to help the process go smoothly and ensure an appropriate resolution.

1. Don’t panic. That will panic the students as well.
2. Claim the alert by clicking the red Investigate button. This will signal to other proctors that you are the one communicating with the student.
3. Investigate and intervene. Once an investigation has begun, the investigating proctor should be absolutely sure that the student is not engaging in inappropriate exam behavior before allowing them to continue.
4. Convey the situation to the student — ideally using scripted language developed by the program. One program shared this script:
   Your exam has been auto-paused due to multiple system alerts, which I will need to investigate. Please remain calm and cooperate with my investigation. These alerts can be triggered by both appropriate and inappropriate actions, so I need to be sure before we continue. If my investigation does not uncover inappropriate behavior, I will be able to resume your exam with all exam answers and time intact.
5. Look for any of the indicators listed in this resource. If the alert is Viewing Other Content, the proctor should definitely check the browser history.
6. Enter the results of the investigation into ExamDriver, then either Resume or End the student’s exam.
   • If the exam is resumed, there is no need to reauthorize. The student will continue where they left off.
   • If the exam is ended, have the student pack up their things and leave quietly. PAEA will be in touch with the program via email to discuss next steps related to the student’s score.

Follow these step-by-step instructions to help the process go smoothly and ensure an appropriate resolution.

1. Don’t panic. That will panic the students as well.
2. Claim the alert by clicking the red Investigate button. This will signal to other proctors that you are the one communicating with the student.
3. Investigate and intervene. Auto-ending means that the student has continued to engage in potentially inappropriate behavior, so the investigating proctor will need to look carefully for anything they might have been doing. PAEA has the ability to resume the exam after it has been auto-ended, but the proctor should be absolutely sure that the continued behavior is not a problem.
4. Convey the situation to the student — ideally using scripted language developed by the program. One program shared this, a slightly edited version of the auto-pause script:
   Your exam has been auto-ended due to additional system alerts, which I will need to investigate. Please remain calm and stay connected as I complete my investigation. As with the auto-pause, these alerts can be triggered by both appropriate and inappropriate actions. If my investigation does not uncover inappropriate behavior, you do not need to be concerned. Your exam will be resumed or rescheduled as soon as possible with all exam answers and time intact.
5. Look for any of the indicators listed in this resource. If the alert is Viewing Other Content, the proctor should definitely check the browser history.
6. Enter the results of the investigation into ExamDriver, and hit Save and Close. This will trigger an email to you and to PAEA.
7. Wait for PAEA to respond with next steps based on the findings of the investigation. If you do not hear back in five minutes, initiate a chat with Exam Support and they will move things along.
8. Respond to PAEA’s email with confirmation of the decision to either Resume or End the exam. A discussion with the program about scores can happen after the rest of the students have completed the exam.
9. If the exam is to be resumed and the student has closed their browser window, you will need to reauthorize them to continue.
10. If the exam is to be ended, there are two options for handling the scores, depending on the program’s academic integrity policy:
    • Complete the exam now to score the material they have already answered. This will score the portion of the exam that the student has completed. We recommend this in most cases. If the program wants to give the student a lowered score as a penalty, or retest them in the near future and ensure they get a different exam form, this is the appropriate course of action.
    • Invalidate the exam. This will remove all results and activity from the system. As a general rule, we do not recommend that programs do this unless they intend to give the student a score of 0. If the program chooses to retest the student, we can invalidate the ended exam afterwards so that they are definitely assigned a form they have not yet seen.

These processes apply to all proctored exams. If the program chooses to deliver PACKRAT in a proctored environment, like a classroom, the alert triggers apply. If the program chooses to deliver the PACKRAT exam without a proctor, selecting the “non-proctored” option when scheduling, the exams are not at risk of auto-pause or auto-end, as there is no one there to investigate. Some programs administer PACKRAT as a non-proctored and open book exam, since it is a self-assessment (which is why programs cannot administer PACKRAT for a grade).

Some students will always be anxious when taking exams, no matter how many exams they’ve taken by the time they are in a master’s program. The vast majority of students will not notice anything related to these changes. While we cannot control all variables (e.g., internet connectivity or facilities issues), we can help ease stress that may arise through explanation and assurance that as long as they’re focused on the exam, their experience should be free from interruption or penalty. Here are some things you can do to help reduce their anxiety:

• Give Clear Pre-exam Instructions: Telling students what behaviors will trigger alerts should deter inappropriate behavior and ensure that they do not inadvertently trigger alerts and an investigation. Click here for a model script, which you can adapt for your class.
• Set up Privacy Barriers: Some students may benefit from privacy during their exam in the form of physical barriers placed around students’ desks. While you may need to proctor from the back of the room instead of the front in this case, it could ease the pressure from the glances of other students.
• Establish Investigation Protocols: Ensure that you do not start your investigation with an accusation, as it could distract other students. (Model language is in step #4 above.) There are benign causes of alerts, so approach the student with compassion to foster cooperation with your investigation. Remain calm yourself, but be firm that you need to investigate any possible case of inappropriate behavior. Let the student know that their exam will be resumed if you do not find anything.
• Share Online Resources: Almost every testing service and education publication has an available guide to help examinees with test anxiety. The suggestions provided here are specific to our exams, but students may find these resources helpful as well: ETS, Princeton Review, U.S. News and World Report