Collaborative assessments in on-line classrooms

Abstract

With massive open on-line courses (MOOCs) gaining momentum, it is now common for thousands of students to enrol in a course, making manual assessments by teachers simply unfeasible. Peer assessments is one way to go when auto-scoring approaches are not possible. Current on-line courses usually use a simple aggregation of peer assessments, but these suffer from two main pitfalls. First, simple aggregation does not take into consideration the reliability of a peer assessment. Second, simple aggregation calculates what the students think of an assignment as opposed to what the teacher thinks of it (the far more important opinion). This work proposes two different models to address these two different pitfalls. These models lay the foundation for future work, where we intend to combine both models into a single one that addresses both pitfalls at once. The aim is to build an automated assessment system that results from the collaboration of both students and teachers.