These recommendations are intended for colleagues involved in planning, delivering or marking departmental assessments on AssessmentUCL for 2022-23. These recommendations outline best practices and are designed to help you get the most out of the platform and avoid some known pitfalls.  Like any educational technology, AssessmentUCL has limits in terms of the range of requirements it can accommodate. The product owners (UNIwise) are working in close partnership with UCL to enable enhancements in response to feedback and use cases from departments. The roadmap for these developments can be found here.

The recommendations cover five key areas, organised in stages of the assessment journey and are relevant to different users (depending on your role in assessments):

1. Planning (including are your assessments suitable?)
2. Setting up
3. During the assessment
4. Marking and feedback
5. Post-assessment

 Useful resources:

• Steps for departments: Delivering assessments on AssessmentUCL 22-23 (visual).pptx. 
• AssessmentUCL Resource Centre for staff
• AssessmentUCL guidance for students

RECOMMENDATIONS

1. PLANNING 

Planning is arguably the most important stage when running assessments on AssessmentUCL and will help avoid issues later in the process. Planning should be led by the Course or Module lead in collaboration with the Administrators, Markers and other team members as outlined under 1.iii below.

i. Contact Digital Assessment Advisory team to discuss suitability

If you are new to Assessment UCL, it is imperative to discuss your requirements in depth with a member of the Digital Assessment Advisory team (assessment-advisory@ucl.ac.uk) to confirm that AssessmentUCL is the right fit for you.  This discussion should include key members of your team (including Administrators) and the Faculty Learning Technology Lead.  If you are already familiar with Assessment UCL, you may decide you don’t need a formal meeting, but the opportunity is there in case you wish to review and clarify arrangements for the coming year.  

To get you started, please consider the following suitability criteria: 

Are your assessments eligible?

Your assessments must:

• be listed on Portico 
• be summative in nature (support for formative is expected by Jan 2023)
• correspond with assessment types available on the platform:
  https://wiki.ucl.ac.uk/pages/viewpage.action?pageId=161883404#GettingstartedwithAssessmentUCL(AUCL)-WhatassessmentscanberunonAssessmentUCL?

Are your assessments suitable for AssessmentUCL?

Suitable:

• Text-based assessments where the entire paper/submission is single or double marked
• Multimedia (i.e. video) upload assessments that are single or double marked
• MCQ type exams that are automatically marked
• Dissertations that are double blind marked

Not suitable (currently):

• assessments requiring question-based marking (tutors marking different questions / sections) and where the student chooses which questions to answer (e.g. Answer 3 questions out of 9)
• MCQ type exams that require double marking
• Mixed MCQ & Essay based question exams - possible but there are challenges and workarounds involved depending on the assessment and marking set up required.
• assessments where feedback is released prior to second marking and/or moderation.

ii. Gather key assessment data

Following the preparatory meeting/communication, you will be asked to complete a spreadsheet with details such as; the dates for your assessment, marking period, who your assessors will be, and so on. These are required to set up the assessments on the platform.

iii. Establish capacity and availability of key staff.

 To run successful assessments on the platform, your department needs to:

• Have a team in place, including administrators, markers, the course/module/programme lead and any other academics and/or externals. Administrative support is crucial as their role plays an essential part in ensuring assessments run smoothly from beginning to end. They are usually the central point of contact for your team and liaison for the Digital Assessment Team who provide advice and guidance.
• Ensure capacity of your team members to commit to training and carry out AssessmentUCL related tasks -these commitments should be accounted for in your team’s workload model;
• Ensure your team understands their roles and responsibilities on AssessmentUCL. These are determined by the designated roles on the platform (see What role might I have on AssessmentUCL?)
• Ensure that the right people have the right roles on SITS so that they can be brought through using integration (see How are staff assigned in AssessmentUCL?)
• Communicate with your team about expectations, timelines and so on and ensure that they have access to key assessment information and any relevant training guidance.
• Identify how and when to get support (see Support during an AssessmentUCL assessment)
• Plan marking and feedback processes well in advance (see section directly below);

iv.  Plan marking and feedback

To make marking and feedback run smoothly, we recommend that you plan by:

• Reviewing your marking processes. Are they overly complex? Is there any way to streamline workflows? Are your marking workflows suitable for AssessmentUCL?  Please note: Currently there are no re-useable comments (e.g., Turnitin Quickmarks) on AssessmentUCL but there are comment and annotation tools, as well as rubrics. Rubric marking on AssessmentUCL can vary greatly from Turnitin, so it is important to check these features are compatible with your marking
• Agreeing on a shared marking approach which all markers should follow. Unlike Moodle or Turnitin, each marker in AssessmentUCL has their own copy of a student submission which allows for first and second marking to be recorded separately. A nominated marker must then reconcile marks and enter a final agreed mark. Whilst this can seem like more work compared to Moodle or Turnitin, it ensures clearer records of double marking within the one platform. In future, we hope to automate aspects of the sampling and the reconciliation of marks.
• Ensuring markers have sufficient time to undertake training and ask them to schedule this in advance.

See also 4. Marking and Feedback

v. Provide support and information to students

Following the setup of assessments on the platform, students are added to practice assessments on AssessmentUCL and advised to do a practice run in advance of their live assessment. It is recommended that students complete their practice assessments to mitigate the likelihood of experiencing fewer issues during their live exams, such as leaving it too late to upload their submissions or browser or device issues that prevent accessing information and saving of answers. Therefore, we recommend:

• Familiarising yourself with student support guidance
• Communicating with students early about requirements and support available. A template for departments outlining essential information and guidance to students will be provided - departments should amend this template for their own use and circulate to their students.
• Include a practice and Q&A session in scheduled teaching that give students the opportunity to complete the standard practice assessments and ask questions about their assessment. This is especially relevant if your assessments require complex scanning and/or upload requirements.

2. SETTING UP 

i. Creating Assessments (Flows)

Once the key assessment data has been established, the assessments (referred to as “flows” in the platform) should be set up by the person with the Manager role. This is the Administrator and/or Exam Liaison Officer associated with the assessment(s). Please note academics are not allocated the Manager role.

Set up includes:

• creating the flow assessment instance and ensuring that all the settings are correct (see setting up an assessment);
• attaching the assignment & supporting materials (see add assignment);
• assigning missing Assessors and Reviewers and allocating them to students (see Add and allocate staff). See 4iii below for recommendations on FLOWmulti (MCQ type exam) marking;
• adding additional time for any students with SORAs or ECs (see Add extra time)
  
  

ii. Marking and review settings

These settings determine how and when feedback and grades are finalised and released to students, amongst other things. Agreement on marking and feedback processes should happen at the Planning stage.

A new function which will be introduced in October 2022 is the ability to set the default sharing options of annotations and comments (i.e., sharing feedback to co-assessors, students etc). It is important to set this up correctly for your assessment needs and to communicate with markers and reviewers how this will work for them.

iii. Creating content on the platform (MCQs, Rubrics etc.)

If running an online exam (FLOWmulti) or using a rubric (marking scheme), these need to be created directly on the platform (see Create assignment and Create rubric). The person who creates this content on the platform is called an Author and this role can be assigned to either academics or administrators.

To complete this task, we recommend:

• Agreeing on a timeline for creating and approving assessment content and attaching to the flow;
• When creating your rubric, use the template rubrics provided to save time. In the Author role, on the Rubrics page, under Published Rubrics, search ‘Template.’ You can click on the menu option to copy a rubric to edit.
• Ensuring that there is contingency planning for ownership of author groups. You may want to collaborate on building questions or give access to externals or other colleagues to check and approve questions. In this case you will need to create author groups and add these colleagues to the group. If you create an author group, the owner is the only person who can add new members to that group;
• Keeping a master copy of question text (in a word document). You may have questions already on Moodle that can be imported into Assessment UCL (see Import questions).
• Considering how you will organize your online exam questions. On Assessment UCL, online assignments are structured in layers (from top to bottom); the Assignment ‘shell’ of settings > Items that act as pages or sections of your exam > and finally Questions. Items help structure your paper by theme and for the student they represent a page of the exam. If your exam is a MCQ exam, it is best to have one question per Item. If you have a series of questions under a theme then you could put them all into a single Item (group of questions). However, be aware that:
• you cannot randomise questions within Items
• student submissions will not save until they have completed all questions in that Item
• you cannot convert questions back into separate Items.
• ‘Promoting’ questions to the Content Bank, where you can create/share new items and add questions from there onto your Flow.

iv. Activating assessments

Once the flow has been set up correctly, with the relevant content created and attached, the flow must be activated (see Activate assessment). Activating an assessment means that students can view the assessment instance when they log into the platform but will not be able to access the assignment content until the participation start date/time. Upon activation, students will receive an email notification with the date/time of their assessment and a platform link to the assessment.
Departmental flows are activated by the person with the Manager role, e.g., Exam Liaison Officer or teaching administrator.

v. Messaging via the platform

As mentioned, students will be notified by email when the Manager has activated the assessment on AssessmentUCL. Students will also receive a reminder message 24 hours before the start date/time of the assessment that they are due to take the assessment on the platform.

Students are not notified when marks and feedback are released on the platform – for this, and other notifications, you may wish to schedule a platform message in advance (see Send messages via AssessmentUCL).

3. DURING THE ASSESSMENT 

As with any departmental assessment, it is recommended that Administrators are on hand to monitor student progress and provide support where applicable (particularly for timed assessments).

i. Monitoring student progress

Administrators with the Manager role can track student engagement with an assessment in real time using the Participation monitor.

ii. Student support

When taking an assessment on AssessmentUCL, students may contact you for assistance. Please direct them to the Support during your assessment page which outlines the various support that students can access at different stages of their assessment.

For example, if students are experiencing technical issues (i.e. logging in or submitting), and there is still time left before the deadline, they can contact the ISD helpdesk for support.

For other matters, there is an Assessment Query Form attached to every flow that students can access on the overview page of their assessment. Once submitted, these queries will be responded to by the central assessment team.

4. MARKING AND FEEDBACK 

Marking and feedback is one of the main concerns for staff in the assessment process.  Completing the planning phase of marking and feedback (see 1. PLANNING) will make life easier for markers and moderators and help the whole process run smoothly.  Planning how your department will mark is essential to consider as certain marking workflows work better than others, and others are not yet suitable for the platform (see examples here).  Step-by-step guidance on marking can be found here.

i. Before marking period begins – we recommend that the Course or Module lead:

• Informs markers about assessment processes, timelines, training requirements and how to get support. Ideally hold a Teams or Zoom prep meeting.
• Agrees a standardised marking framework for all markers (where possible); this will make the management of marking much easier and guarantee that students are provided with the same depth of feedback.
• Understands marking options available (most of this would be discovered during the planning stage). Markers can mark using annotations, comments, a digital rubric and/or grades. They can choose to share any of their marking with students and/or keep some or all private for other markers - understanding this will avoid manual work re-sharing marking.
• Informs markers who the Administrator is (Manager role on Assessment UCL). If they cannot access assessments, it is most likely that they haven’t been allocated to submissions and the Manager can do this.
• If you have a lot of different markers, create a Teams site so that they can communicate, be supported as a group and all have access to key information and guidance in a timely way. Add your DAA to support.
  
  

ii. Double-blind marking

• If the assessment is double blind marked, you may not know who your co-assessor is. The system does not allow for first markers to alert second markers that they have completed marking. Therefore, you might want to agree on whether to mark in parallel or on a timeline for completing first marking.

• Specific recommendations for marking FLOWmulti exams (e.g. MCQ type exams)

Online exams (FLOWmulti) work best when set up for automatic grading.  Manual marking should be avoided where possible, especially if you have several markers.  Please note also that online exams cannot be automatically re-graded if an error or additional answers is found after the exam is run. It is possible to view student performance statistics and to export student MCQ answers to make manual corrections in Excel. For more complex automatically marked questions (e.g. short answer, drag and drop) this approach is not recommended and will require thorough manual checking of answers if they need to be regraded.

5. POST-ASSESSMENT 

Once marking is completed, there are several tasks the administrator in the Manager role must complete:

• Checking any late or word count penalties. Penalties are best applied and recorded in Portico.
• Notifying students that feedback has been released. They won’t get an automatic email but you can schedule a platform message in advance (see section 2v).
• Exporting grades to Portico. This can only be done once and requires the marking to be complete and the marking period to be closed.
• Providing External Examiners with information and guidance. External examiners will have the Reviewer role and can be allocated to a sample or all papers.
• Archiving/un-archiving. Flows get archived and become read-only 30 days after marking closes. They can still be viewed by students, markers and external examiners in their respective roles, but they appear filed under ‘Archive’. The Manager can un-archive a flow to make changes e.g. add an external examiner.