Algorithms for assessing the quality and difficulty of multiple choice exam questions
View/ Open
Date
27/06/2016Author
Luger, Sarah Kaitlin Kelly
Metadata
Abstract
Multiple Choice Questions (MCQs) have long been the backbone of standardized
testing in academia and industry. Correspondingly, there is a constant need for the
authors of MCQs to write and refine new questions for new versions of standardized
tests as well as to support measuring performance in the emerging massive open online
courses, (MOOCs). Research that explores what makes a question difficult, or what
questions distinguish higher-performing students from lower-performing students can
aid in the creation of the next generation of teaching and evaluation tools.
In the automated MCQ answering component of this thesis, algorithms query for
definitions of scientific terms, process the returned web results, and compare the returned
definitions to the original definition in the MCQ. This automated method for
answering questions is then augmented with a model, based on human performance
data from crowdsourced question sets, for analysis of question difficulty as well as
the discrimination power of the non-answer alternatives. The crowdsourced question
sets come from PeerWise, an open source online college-level question authoring and
answering environment.
The goal of this research is to create an automated method to both answer and
assesses the difficulty of multiple choice inverse definition questions in the domain of
introductory biology. The results of this work suggest that human-authored question
banks provide useful data for building gold standard human performance models. The
methodology for building these performance models has value in other domains that
test the difficulty of questions and the quality of the exam takers.
Collections
Related items
Showing items related by title, author, creator and subject.
-
Automated question answering for clinical comparison questions
Leonhard, Annette Christa (The University of Edinburgh, 2012-06-25)This thesis describes the development and evaluation of new automated Question Answering (QA) methods tailored to clinical comparison questions that give clinicians a rank-ordered list of MEDLINE® abstracts targeted to ... -
Intra-cranial circulation in the closed and in the unclosed skull; with special reference to the question of the movements of the cerebro-spinal fluid under ordinary physiological conditions.
Elder, George (The University of Edinburgh, 1896)The remarkable functions performed, by the brain and the great readiness with which these functions are, under many conditions, interfered with, have naturally led to a large amount of discussion and investigation as ... -
Puerperal infection: with special reference to the value of blood cultures in the various stages of this condition, and to the question of 'autoinfection'
Eeles, Jessie (The University of Edinburgh, 1924)Since the introduction of aseptic and antiseptic methods into Midwifery the incidence of Puerperal Fever has diminished enormously. But when we compare the results obtained in Midwifery, with those obtained in Surgery ...