Crowdmark: Online Collaborative Grading for Written Exams

What is Crowdmark? #

Crowdmark is an online collaborative grading and analytics platform that helps educators, working alone or in teams, to more effectively evaluate student work. The platform allows for easy distribution and collection of student work, offers tools for the assembly and management of grading teams, and provides an online workflow that streamlines grading and the delivery of rich feedback to students. The primary feature for Crowdmark is its Administered Assessments feature, in which scanned written exams can be graded on a digital platform.

  • Crowdmark, Administered Assessments (for in-person written exams)
    • The instructional team (professor, TAs, etc.) is responsible for specialized printing, handing each student the physical test papers in the exam hall, collecting these papers when time is up, removing staples, scanning, and uploading

Crowdmark Workflow (How it works) #

1. Set Up Assessment: Generate and print exam booklets with features like multiple choice or create assignment questions and distribute them to students.

2. Upload Completed Work: Instructors or the grading team will have to scan and upload student work. In a paper based exam, this means removing the stapled end with a paper cutter and scanning each page. Each Crowdmark exam booklet page has a QR code to keep track of each student’s exam. Once each page is scanned, you will be able to upload the PDFs to Crowdmark.

3. Grade and Return: Leave feedback with Crowdmark’s online grading tools and share frequently-used comments with your grading team. Grades can be returned to students online.

4. Analyse Learning: See detailed information about student performance on each question.

Roles and Permissions #

When setting up your course, there are several permission levels you can assign your team:

  • Instructor: Has access to all features.
  • Facilitator: Has access to all features in assessments, but limited access to the course.
  • Grader: Can grade assessments and see grading progress. Does not have access to any identifiable student information (including cover pages), evaluations left by others (unless enabled), or assessment administration.
  • Printer-Uploader: Only has access to the Uploads page of assessments, which includes an upload area as well as a button to download the printable PDF of blank assessments.

A user with Instructor permissions can change team roles at any time. Read more details about permissions here.

Why use Crowdmark over traditional grading? #

There are many distinct advantages of Crowdmark that add up to a big advantage over the traditional 100% paper-based approach for in-person assessments.

  • Improved management of TA team during marking process
    • No passing of papers or need to mark in a secure physical location and lower possibility of lost papers
    • Real time knowledge of everyone’s marking progress and exact marking time by TA and by question
    • Easier to monitor comments being used and marks being given: can fix any issues promptly
    • Improved communication: TA can ask about a paper via e-mail and instructor can see it
    • Marking is entirely anonymous: graders cannot access students’ identifying information
    • Reduces clerical errors in marking (no need for human addition, marks entry, copying to front, etc.)
  • Allow comments on students’ work to be efficiently reused and/or revised
    • This leads to students getting more and better comments on their work, even in large classes
  • Improved marking experience and marking quality even when instructor marks everything themselves
  • Instant return of students’ work when marking is finished
    • Do not need to delay until next class or tutorial
    • Do not need to waste class or tutorial time with administrative tasks
  • All students have ready, continuing, and secure access to their marked work organized by course
    • No uncollected papers and no misplaced work
    • No temptation to change answer and submit it for remark and no suspicion around remark requests
    • Students can write in pencil and use an eraser during invigilated work (no need to require using a pen
    • Improved confidence in accuracy of recorded marks and appreciation of increased transparency
    • Remark requests can be done electronically: no need for passing papers back-and-forth

Other potential benefits of Crowdmark #

  • Opens door for promising pedagogical approaches: we keep thinking of new things
    • Two-staged tests
    • More flexible class participation and without cost to students (unlike iClicker or TopHat)
  • Managing a TA budget
    • Can divide marking duties far more than would make sense if TAs had to share physical papers and flip to a particular page
      • More specialization leads to better and more efficient marking
      • Can productively use TAs even if they have a very small amount of marking time
    • Can keep track of actual marking hours to ensure equity across TAs by end of course