Very short answer questions (VSAQs) have gained attention for several advantages as compared to multiple-choice questions (MCQs). While VSAQs require knowledge recall, MCQs often only involve knowledge recognition. This difference may lead to varying cognitive workloads, defined as the amount of mental processing in working memory. Eye tracking provides objective, process-level indicators of cognitive workload. This study investigated whether answering VSAQs requires a higher cognitive workload than answering MCQs.
In a within-subject randomized crossover experiment, sixth-year medical students answered both VSAQs and MCQs. Cognitive workload was measured using screen-based eye tracking, focusing on the number of fixations and revisitations as objective indicators of mental effort. Data were analyzed using mixed-effects models. Thirty-four medical students participated, yielding 1,326 observations, which is the multiplication of the number of students by the number of questions (39 questions).
Analyses indicated a significant effect of question type on both workload indicators: VSAQs elicited more fixations and revisitations than MCQs. This effect remained after controlling for accuracy. Incorrect answers were associated with higher workload. Heatmaps confirmed these findings, showing denser fixations on key diagnostic features for VSAQs and on answer options for MCQs.
We find that answering VSAQs imposed a higher cognitive workload than MCQs. The presence of answer options in MCQs may reduce workload by providing unintentional cues, while VSAQs require active retrieval. Eye tracking proved valuable for distinguishing cognitive workload across assessment formats.
Queiroz, Maria Gabriela Carneiro; Junior, Francisco Carlos Specian; Filho, Pedro Tadao Hamamoto; Santos, Thiago M.; Schauber, Stefan; Woltman, Andrea M. & Cecilio-Fernandes, Dario (2026). Comparison of cognitive workload between very short answer questions and multiple-choice questions: an eye-tracking experiment. Medical Education Online. 31(1). doi: 10.1080/10872981.2026.2621434.