Workshop is one way Moodle staff can set up student peer marking in Moodle. The activity can accommodate different kinds of submitted work and can happen before, alongside, or independently of the standard tutor assessment. Workshop lets staff set up forms, rubrics or marking guides to support students in making judgements. Student assessments can receive feedback only, or feedback and numeric grades. Tutors can optionally assess the student submissions and assessments, and weigh their own judgements in peer relations. Peer submissions and assessments can be kept anonymous if needed. Workshop also allows self-assessment.
The basic stages of a Workshop activity, set out in more detail below, are:
Peer marking requires students to make judgements about the work of their peers. Peer markers may be expected to give feedback only (which we'll refer to here as 'peer feedback'), or feedback along with a numeric mark (which we'll refer to as 'peer assessment').
A well-conceived, well-enacted peer assessment activity can advance:
Keith Topping (2009) suggests explaining to students,
'...that peer assessment involves students directly in learning, and should promote a sense of ownership, personal responsibility, and motivation. Teachers can also point out that peer assessment can increase variety and interest, activity and interactivity, identification and bonding, self-confidence, and empathy with others.'
Won't students take each others' ideas? A widely held concern - but Richard Milne (UCL Centre for Virology) comments on his own experience of setting up peer review activities, ' I wasn't worried about students stealing each others' ideas. ...you discuss a subject with somebody else and then formulate your own way of thinking about it based on the conversation you’ve had'. Students can be encouraged to credit each others' ideas (and a convention can be agreed upon for circumstances of anonymity).
Can students at any level of knowledge carry out good peer reviews? Potentially, yes - a meta-analysis by Falchikov and Goldfinch (2009) found that peer assessment at introductory levels was as valid as at higher levels. They attribute this to good preparation. McConlogue (2014) suggests beginning early with low stakes assessment, perhaps the introduction of a draft essay, and building up student expertise over the duration of a programme.
Can peer assessment work in every subject area? Yes, it seems so - Falchikov and Goldfinch (2009) found that subject area has little effect on validity or reliability of peer assessment. They also report that peer assessment of academic products (e.g. essays, posters) or processes (e.g. oral presentation skills, groupwork participation) tend to have more validity than those in the context of professional practice (e.g. internships). This may relate to students' greater familiarity with academic products and processes. Their research also suggests students may struggle with peer assessment in multiple unfamiliar disciplines - so stick to one new discipline.
Will students be willing to do it? That depends. Read on.
For general peer assessment design principles and case studies, see the University of Strathclyde's PEER Toolkit and contributions from Eva Sorensen (UCL Chemical Engineering), Richard Milne (UCL Virology) on UCL's Teaching & Learning Portal.
This displays for students on the front page of the activity just under the Planner (dashboard). Give a brief motivating introduction to the activity. (Instructions for each phase come later).
Below you will be asked to submit your case study, peer mark two other students' case studies, and self-mark your own. Your overall mark will be based on the marks you receive for your submission from student peers, and the marks you receive from a tutor for the feedback you give as a peer marker.
You will find the precise deadlines on the Planner here.
Participation - the way we have set up this activity, only students who make a submission will be able to peer mark. Because, at the deadline, Moodle is set to automatically allocate submissions to peer markers, we cannot give extensions. If you cannot submit work then you need to go through the extenuating circumstances procedure.
Anonymity - both the work you submit and the peer assessments you give will be anonymous to fellow students - you will not know whose work you are marking nor who has marked yours. However, tutors will be able to see this information.
How you will be marked - the mark you receive for your submission will be based on the average mark you received from your peer markers, and will be 50% of your overall mark. The feedback you give as a peer marker will be marked in turn by a tutor, and an average of those marks will comprise the other 50% of your overall mark.
Grading strategy determines the assessment form used and the method of grading submissions.
There are 4 alternative grading strategies:
Grade for submission and Grade for assessment allow you to set a maximum grade which can be obtained on a piece of work.
The Submission grade to pass and Assessment grade to pass boxes allow you to set a minimum grade required for students to pass the assessment.
Instructions for submission display to students above the place where they submit their work. It is a good place to explain the settings and any conditions you have set up.
Example submission instructions:
Submit your work below [link to guidance] Please note that there are no late submissions for this activity - you need to submit work before the deadline in order to be allocated others' work to mark, and in order to receive marks and feedback yourself.
Maximum number of submission attachments - if set to none then students will need to paste their submission directly into a Moodle text field. Since Workshop does not have autosave, you might suggest they draft elsewhere and paste into Moodle. Alternatively you can allow files, in which case let students know how many you expect and in which format.
Submission attachment allowed file types - leaving this field blank allows students to upload all file types. Clicking 'Choose' allows you to create a list of allowed file types for student submission.
Late submissions - if late submissions are allowed there is no way to automatically allocate peer markers to them. You'll need to to manually allocate markers to these late submissions, or mark them yourself.
Instructions for assessment - this displays to students during the assessment phase and appears just under the Planner and above their allocation. Like the instructions for submission they are important for orienting and motivating students.
Example assessment instructions:
Immediately after the submission deadline you will be asked to complete a practice assessment. Once you've done this you'll be able to start giving feedback on fellow students' work you have been assigned, until the assessment deadline.
The submissions have been assigned to you randomly and automatically.
Use self-assessment - this allows students to be allocated their own work to assess.
Overall feedback mode - provides a field for students to type summary comments which they have not made elsewhere.
Maximum number of overall feedback attachments - this may be none. Attachments come in handy if you are, asking peer markers to comment directly onto the work. There's no need to ask peer markers to upload a completed marksheet here, since Moodle Workshop provides you with ways to make these (see below).
Conclusion - this displays to students after the activity has been closed i.e. when they can come and collect their feedback and/or marks. It appears just under the Planner and above their feedback.
Example submissions are a practice assessment opportunity for students.
Use examples - this will allow you to give students a piece of work so they can practice peer marking without their feedback reaching a genuine peer. You will also provide an exemplary assessment for them as a reference.
Modes of examples assessment - choose when students encounter the example, and whether they have to mark it.
Here you set dates for the Submission and Assessment phases.
Switch to the next phase after the submissions deadline setting - this begins the Assessment phase automatically at the Submission deadline (the alternative is to manually switch phases when you're ready). If you are going to switch automatically you will almost certainly need to set up Scheduled Allocation - see below.
These are the same as other Moodle activities.
You will be able to edit many of these settings retrospectively, but be aware that many of them display in the students' Planner view, who may plan around them - so once the Submission Phase is underway it's best to leave the common module settings alone unless you have a chance to negotiate changes with students.
When you are ready click 'Save and Display'; the Setup Phase page displays.
During this phase, only Moodle editors are active - students aren't involved yet.
The example Planner shown below is telling us that before we switch to the Submission Phase, we first need to edit the Assessment Form and prepare Example Submissions. Under that, the Description that was added in the Workshop Settings will be displayed.
To set up the assessment, in the Setup phase panel on the left, click the link to Edit Assessment Form; what displays here depends on the grading strategy you chose in the Grade settings component of the Workshop settings. (You can also change this during the Setup Phase).
If you chose Accumulative Grading within the grade settings of the Workshop, you'll be asked to define one or more aspects. For each criterion aspect you'll specify the maximum points available or which scale to use. You can have different points or scales for each aspect. You'll also be asked to give each aspect a weighting in relation to the others. Peer markers will see the definition of the aspect, a place to enter the grade or scale, and a field for their comments.
If you chose Comments within the grade settings of the Workshop, you'll be asked to define one or more aspects. These are the questions or feedback points which will display to peer markers, with a blank text field under each. This is a feedback-only marking strategy so there are no numeric grades or weightings to set.
If you chose Number of Errors within the Grade settings of the Workshop, you'll be asked to define one or more assertions along with words for peer markers to use to indicate whether these have been met or not met. You'll also be asked to give each assertion a weighting in relation to the others. Save And Continue Editing to activate the Grade Mapping Table, which turns the weightings you have chosen into a number of possible errors and relates these to the overall number of points available for the Workshop activity - so what you then need to do is make sure that the values in each menu is the same as their adjacent values.
If you chose Rubric within the grade settings of the Workshop, you'll be asked to define your criteria and levels of achievement. You need to assign a unique grade to each level. Extra levels will appear 'on demand' - first complete those that display by default, then click Save And Continue Editing. When the page reloads it will display extra blank levels. You will be able to choose whether the rubric should display to students as a (vertical) list or (horizontal) grid (list is best where there are many levels).
When you are ready you will need to manually enable the Submission Phase - during this phase students submit their work. Only after the Submission Phase deadline do students begin to peer mark.
Still in the Setup Phase - if, in the initial Workshop Settings, you enabled the example submission, the Planner will now prompt you to add one by clicking the button.
You are then prompted to paste or type an example submission directly into Moodle, which you then 'mark' in an exemplary way according to your chosen grading strategy. This becomes a reference for students. If you want to show students a range of work, you can add more than one example submission (If you do add more than one example submission, it is probably not a good idea to force the student to assess them all before being able to proceed).
When a student assesses the example, their assessment does not accrue marks. They are shown the example submission and the Assessment Form we set up earlier, and after completing the form and saving the assessment, they are presented with the tutor's reference assessment for comparison.
Students have the opportunity to re-assess the example.
Students tend to ask for anonymity, both for their submissions and assessments. Our evaluations of Workshop have indicated that without anonymity (and depending on the experience and trust among students) 'everyone is holding back' on passing judgement on their peers' work. This is to be expected - at least while they are inexperienced at peer marking. Anonymity is something to negotiate with students and be very clear about.
In the event that a workshop doesn't require it, here is how to remove anonymity for both Author and Marker (which Moodle Workshop refers to as 'Reviewer').
Even if you enabled a date in the Workshop Settings, you will still need to go to the Planner and manually click the link in the Submission Phase column, to enter Submission Phase. The column will then appear shaded green.
It often helps to send a News Forum message reminding students that the submission phase is underway, its duration, and where to find it.
To monitor submissions, you can refer to the Planner to monitor how many students have submitted work so far, and below that you can filter by Group (if you've set them up).
Allocation of submissions to peer markers could be done as student submissions arrive, but it is probably easiest to wait until after the deadline - and easier still to set Moodle Workshop up to do this automatically.
Scheduled allocation (automatic after the Submission Phase ends)
You may not want to do this if you are expecting students to submit late. However, it does offer the benefits of moving things smartly along and allowing students to peer mark while their own work is still fresh in their minds.
The most automated this gets is as follows - in the Workshop settings, in the Availability section:
There's one more thing to do for these settings to be enacted.
Decide whether students should self-assess.
You will see a list of the participants (students on the course), and to the left a column to assign them a reviewer, and a right column to assign them a reviewee. A participant must submit something before they can be assigned a reviewer, however you can assign students to review someone else's work even if they have not submitted themselves.
The other way you can allocate reviewers and reviewees is by random allocation. If you have groups set up on the Workshop activity then you can set the allocations to distribute among group members. You set how many students you want to review each piece of work. You can choose to override any previous allocations, for example if you have some extra submissions you may wish to re-do the random allocations. There is also the option to allow those who have not submitted to be reviewers, if you do not tick this then anyone who has not submitted will not be allocated a reviewee. Once you have selected the relevant settings click 'Save changes'.
When the random allocation has completed you will get a confirmation screen, which will also note any problems, such as not being able to allocate to individuals who have not submitted.
Once submissions are completed and reviewers have been allocated it is time to move into the Assessment phase. This can be done manually, but if you set it up in the initial settings then this will happen automatically at the set date. From a tutor or course administrator perspective there is not much more to do at the Assessment phase, once you have activated it. This is the students' time to review their peers work and whilst doing so also reflect on their own pieces.
As the grades or comments are given by peers to one another you will start to see these appearing in the table. This will allow you to chase up anyone who has not peer marked before the assessment phase closes. The names highlighted in red are of those still to give feedback, the ones in black have completed, and in the case where grades are given you can see what they have given.
This phase is all about the tutors or course administrators, as it is your chance to review the comments and grades given by students and make any amendments where they have been either too lenient or too harsh. It basically enables you to mitigate any negativity and ensure all grading has been done fairly. You can also allocate marks for the reviewers based on the quality of their reviews if you wish to.
Once you turn on the Closed phase then the students can see any grades or comments they have been left by the other students. This is also the end of the process.
If you find any inaccurate or missing information you can even update this yourself (it's a communal wiki).
If you have a specific question about the tool please contact the Digital Education team.
See this brief separate guide for students using the Moodle Workshop activity.
Sadler, D. (2010) Beyond feedback: developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535-550.