Subjective assessments, such as writing exams or project evaluations, often require multiple perspectives to ensure a fair and comprehensive evaluation. Recognizing this need, YouTestMe provides a robust feature that allows multiple graders to collaboratively assess a single attempt, enhancing accuracy and reducing bias in grading.
In this article, we will explore the Multiple Graders feature, its functionality, and how to maximize its benefits.
The Grader Role #
If a user is designated to grade only the finished attempts of a specific test, assigning the Grader role to their account ensures efficient and secure access management. Users with the Grader role can exclusively access and evaluate the finished test attempts for which they have been explicitly assigned as Graders. This targeted access enhances security and streamlines the grading process.
When assigned to grade a test attempt, the grader will receive an invitation that they can accept or reject.
For a detailed overview of user permissions and roles, please refer to this article.
How to Enable the Multiple Graders Feature #
To allow multiple graders to assess the finished attempts of a test, the Multiple Graders feature should be enabled. This can be done either during test creation or from the settings of an already-created test.
Note: The multiple graders feature is only available for manually created tests and tests generated based on question categories.
Enabling During Test Creation #
At the Grading Step of the test creation process:
Click on Show Advanced Settings.
- Enable the Multiple Graders option.
- Define the minimum number of graders required to release the results.
- Enable the Auto Assign Graders option if you want the system to automatically assign new graders from the graders list when one or more graders cannot grade the attempt.
For more information on test creation, please read this article.
Enabling from Test Settings #
Navigate to Tests, select the Manage Tests tab, and choose the desired test.
Go to the Grading Configuration in the Settings tab.
Click on Show Advanced Settings.
- Enable the Multiple Graders option.
- Define the minimum number of graders required to release the results.
- Enable the Auto Assign Graders option to allow automatic reassignment of graders.
Assigning Graders #
Graders must be assigned to the test to receive grading requests and evaluate the finished attempts.
To assign Graders:
- Navigate to the Tests tab, select Manage Tests, and choose the test.
- Navigate to Test Administration
- Select the Graders tab.
- Select the test version (if there are multiple versions).
- Click on the Assign Graders button.
- Find the desired users in the dialog and check the boxes beside their usernames.
- Use the Confirm button to add the graders to the list.
The list of assigned graders will be displayed. You can set the order of graders, which the system will respect when auto-assigning graders. For example, if the minimum number of graders is set to three, the system will invite graders 1, 2, and 3. Grader four will be automatically invited if one of these graders rejects the invitation.
Note: Once invited to grade a finished attempt, graders cannot be unassigned from the test. If a grader cannot or does not wish to participate, they must reject the grading invitation.
Grading Attempts – Grader’s Perspective #
Once a candidate submits their test attempt, the required number of graders will receive a notification inviting them to grade the attempt.
The Grader can access the grading invitation and grade the attempt by clicking the Preview button or accessing their profile and navigating to the My Grading Reports tab.
To access the candidate’s answers, the grader should click on the Report button from the Actions column.
- Click the Reject Request if you are unable to grade the attempt
- Click the Accept Request to proceed and grade the attempt
If the invitation is accepted and you want to proceed and grade the candidate’s answers to the questions, click on the Grade button in the Actions column next to each question’s text to preview the candidate’s answers and grade them.
Enter the number of points to award for each question and use the Submit button to save.
After grading all questions, click Submit to submit the grading report.
Personal Report – Candidate’s Perspective #
The candidate’s results will only be finalized once the defined minimum number of graders submit their grading reports. Until then, the candidate’s personal report will not display a score but will indicate whether they have passed or failed the exam.
