M09a4 - Anonymous Marking in Moodle Assignment

On this page:

Introduction

The guidance below takes you through several different anonymous marking scenarios with Moodle Assignments afforded by UCL's Marking Policy.

See Section 7.4 of the Academic Manual Academic Regulations Anonymity for full details of UCL regulations.

| Top of page |

Terms

• Anonymous marking: see below for a definition and explanation.
• Bulk-download: downloading the contents of a submission inbox for marking outside Moodle.
• Candidate number: a unique identifier which students receive annually.
• Combinations of blind/open, full/sampled second marking: refer to the UCL Marking Policy.
• External examining: refer to the UCL Marking Policy.
• Grader: Moodle's own marking environment.
• Identifier: the unique number that Moodle gives to each anonymised student (n.b. this doesn't correspond to the Student Number or Candidate Number).
• Intermediary: somebody without marking responsibilities, perhaps in an administrative or technical role.
• Markings: all notes, specific and general comments, and numeric marks made by markers.
• Moderating: refer to the UCL Marking Policy.
• Proxy identifier: The recommended proxy identifier is the 'Candidate Number'.   See below for a definition and explanation.
• Student Record Number (SRN): this appears on the front of every student's ID card.
• Submission filename: the name of the file(s) submitted; this becomes important in cases where students' work is downloaded for marking outside Turnitin.
• Submission title: the title students type into Turnitin when submitting their work. This displays in the Turnitin inbox and can be used to allocate submissions to subject specialist markers.

| Top of page |

Definition of anonymous marking for online assessment

After considering a range of possible definitions, the Board of Examiners has settled on the following definition of anonymous marking on Moodle, Turnitin or other online environment: student names are absent from their submissions at the point of marking.

Instead of their name, students are instructed to use a unique proxy identifier.

There is no expectation that student work remains permanently anonymous to a marker after they have finished marking.

| Top of page |

What proxy identifier should be used instead of the student name?

Why is a proxy identifier needed at all? Because staff, sometimes working offline with downloaded files, need a robust, convenient way to refer to any given student, reconcile their markings with those of other markers, and update the student's record in the absence of a name.

Until a technical solution becomes available, students should be instructed to include this proxy identifier in both the title and file name of their submission.

The recommended proxy identifier is the Candidate Number. The Student Record Number (SRN) may be used instead, if preferred.

The proxy identifier isn't generated automatically, so students need be instructed to title their submission and file name so as to include it.

| Top of page |

Open second marking

Full open second marking

Where? First and second marking can take place either online or offline. First marking online may make things easier for two reasons - feedback will be ready for release to students, and the Second Marker will be able to log in directly rather than having to cross-reference between emailed or shared files and Moodle.

Feedback. If the submitted work is in PDF format markers can directly annotate online; if more than one marker is doing this then each needs to distinguish between different markers' notes by typing initials (e.g. M1 or M2) into their own comments i.e. to fulfill record-keeping requirements. However, if First Marker feedback is the only feedback students are expected to see, then the Second Marker needs to make their comments outside Moodle, bulk downloading the inbox along with a Grading Worksheet is a good way. If the submitted work is word processed or other file format, it can be bulk downloaded for annotation using e.g. MS Word Comments, and bulk uploaded as feedback files (caveat - only one marker should carry out a bulk upload. The reason is that a second upload would overwrite the first because the file names are necessarily identical).

Grades. It may best fulfill record-keeping requirements to use the Moodle Assignment's Grading Worksheet which can be downloaded and completed by the first marker and passed to the second marker, who may complete their own separate Grading Worksheet. These can be compared when the time comes to reconcile the grades, and a single finalised version uploaded to Moodle. Alternatively, this process can be carried out online directly in the Moodle Grader but be aware that when the feedback is released, students will see all marks or comments there, so it may need to be tidied, which poses obstacles to record keeping for the examinations board, external examiners &tc.

Records may take the form of the Grading Worksheet, feedback comments on Moodle, and files of annotated student work (separate for First and Second Marker).

| Top of page |

Sampled open second marking

As above, but with a sample defined by local policy.

To obtain the sample, the Moodle Assignment inbox can be sorted by grade to easily find Firsts, Fails and borderlines and access the submissions. This may be the most convenient approach.

If an offline Grading Worksheet has been used, this can also be sorted using e.g. MS Excel, but finding the corresponding students' work may be less convenient since the proxy identifier on the worksheet and in the filename of the submission is a longish number.

| Top of page |

Open second marking after releasing first marker feedback

This allows a quick return of feedback to students.

Before the First Marker feedback is released (which necessarily lifts student anonymity) the Second Marker downloads the still-anonymous inbox and any feedback. The Second Marker can refer to the First Marker's markings and carry out Second Marking outside Moodle. If students are to receive Second Marker comments (as well as First Marker comments), these need to be uploaded individually as separate feedback files (i.e. they cannot be bulk-uploaded since the original filename acts as a unique identifier, required to match files with students, and so risks overwriting First Marker feedback which may have been bulk uploaded.

Students should be made aware that the First Marker's marks are provisional and therefore subject to change.

| Top of page |

Blind second marking or independent marking

Full blind second marking

Second marking takes place outside Moodle. Before first marking marking begins, the Second Marker bulk downloads the inbox and Grading Worksheet.

Feedback can be returned to students by bulk uploading the marked submissions, and the completed Grading Worksheet.

| Top of page |

Sampled blind second marking

First marking can take place in Moodle, while second marking takes place outside Moodle. Before First Marker marking begins, the Second Marker bulk downloads the inbox and identifies the sample (either by eye or by carefully filtering the data in the Grading Worksheet) according to the agreed policy.

First and Second Markers reconcile the marks and keep records of both sets of marks and feedback.

| Top of page |

Blind second marking after releasing first marker feedback

First marking can take place in Moodle, while second marking takes place outside Moodle. Before First Marker marking begins, the Second Marker bulk downloads the inbox of submissions along with a Grading Worksheet.

After first marking - which may take place on Moodle - is complete, this feedback is released to students. Students should be made aware that the First Marker's marks are provisional and therefore subject to change.

Meanwhile second marking is carried out. First and Second Markers reconcile the marks. If students are to receive Second Marker comments, these need to be uploaded individually as separate feedback files (i.e. they cannot be bulk uploaded since the original filename acts as a unique identifier, required to match files with students, and so risks overwriting First Marker feedback which may have been bulk uploaded.

| Top of page |

Moderation or Check Marking

If first marking is completed in Moodle, then before the marks are released to students (i.e. while student anonymity is still in place), moderation or checking is carried out.

| Top of page |

Sampled moderation of Check Marking

As above, but filter the inbox by grade to find the sample you need.

| Top of page |

Moderation or Check Marking after releasing first marker feedback

First marking is completed on a bulk-downloaded inbox, which allows a still-anonymous marked work to be passed to checkers or moderators outside Moodle. Meanwhile it can be bulk-uploaded for students to access their marks and feedback on Moodle.

| Top of page |

External examining

See below.

| Top of page |

Sampled external examining

See below.

| Top of page |

External examining after marks have been released

This is the usual situation. Students need to remain anonymous, and marking and how marks are arrived at must be documented and transparent for external examiners (as well as examination boards, students and, if necessary, grievance panels).

So External Examiners can be given the marked submissions and completed Grading Worksheet. Note that this isn't - can't be - downloaded from Moodle, but already exists with the marker(s). It's probably a good idea to work out a departmental system - perhaps a shared network drive since this will be both secure and backed up - for keeping these.

| Top of page |

Questions and Answers

See our Questions and Answers for anonymous marking on Turnitin since there is considerable overlap between Turnitin and Moodle with respect to the questions.

Do word processed documents include the author names in the properties?

Microsoft documents may include personal or hidden data. Of necessity, the responsibility to provide an anonymised document must fall to each student - see Microsoft's guidance on removing personal or hidden data in documents. But the short answer is, anonymity means students names are removed from the script at the point of marking. The personal data isn't blatantly obvious, so if you don't look for it you won't be confronted with it.

| Top of page |

Is it possible to hide the grader's name from the student?

In the settings of a Moodle assignment, under Grade you can set Hide grader identity from students to yes to hide the name of the grader to the student. This can avoid student confusion, especially when multiple markers and administrators handle assignment submissions.

| Top of page |

Guidance on anonymous marking in Moodle - including second marking, moderating and external examining by the Digital Education team  is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at https://wiki.ucl.ac.uk/display/MoodleResourceCentre/.

Save