Written by Tim Sommers, fourth-year graduate student in the Department of Philosophy

What kinds of questions, short-answer or multiple-choice, are best to use on a test? In my own unscientific surveys over the years, I have found that many people in my field, philosophy, think of multiple-choice questions as a way to save time on grading, but believe that, all other things being equal, essay questions are better. I have often seen fellow teachers treat questions like “Which kind of tests questions are better?” as if we couldn’t really know, beyond a priori speculation, what the answer to such a question might be. But here’s the answer. Research suggests that multiple-choice questions are no worse and no better than short-answer questions. 

Wait. What does better mean? In this context, by better I mean better in terms of: 

  • Completion rate. (Students presented with the test actually finish it.)  
  • Reliability. (Students will tend to score the same each time they are presented with it.) 
  • Validity. (Students’ scores reflect their knowledge level.) 
  • Retention. (Students score the same if presented with test again later.)

How do I know that multiple-choice questions are no better or worse than essay questions by these criteria? Because it’s been studied empirically. A lot. See some sample sources below.  

Tradition determines much of how we grade. But we could also rely on the growing body of scholarship on teaching and learning. 

Here’s another example. In grading, should we prefer a more modest or a more expansive scale? For example, in grading a short essay question is it better to prefer a 1-10 scale or a 1-100? There’s no certain and final answer, but in general, it’s probably better to prefer a more compressed scale. Every measurement is vulnerable to chance error. The difference between a 10 and a nine, or even an eight, is as likely to be chance as knowledge. But expanding the scale creates more “cut-off boundaries” and is even more susceptible to chance errors than a more modest scale. Further, a larger scale may introduce more subjectivity. Think of it like this. You probably have a pretty good sense of the difference between an essay answer that’s a nine and one that’s an eight. But do you really have a solid, reliable grasp on the difference between an essay that’s a 92 and one that is an 82? 

We can, and should, take an evidence-based approach to grading and pedagogy. 

For a nice survey of some of the education research surrounding grading, see "What We Know About Grading: What Works, What Doesn't, and What's Next."

Some References on Short-Answer Versus Multiple-Choice Questions 

Becker, William E., Johnston, Carol. (1999) The Relationship between Multiple Choice and Essay Response Questions in Assessing Economics Understanding. The Economic Record, Vol. 75, No. 231, pp. 348-357. 

Blum, Abraham & Azencot, Moshe (1986) Multiple‐choice versus equivalent essay questions in a national examination, European Journal of Science Education, 8:2, 225-228. 

Gray, Lorraine, (1984) The Comparative Effects of Multiple-Choice versus Short-Answer Tests on Retention. Journal of Educational Measurement, Spring, 1980, Vol. 17, No. 1 (Spring, 1980), pp. 45-50. 

Kuechler, W. L. & Simkin, M. G. (2003). How Well Do Multiple Choice Tests Evaluate Student Understanding in Computer Programming Classes? Journal of Information Systems Education, 14(4), 389-400. 

Megan A. Smith & Jeffrey D. Karpick. (2014) Retrieval practice with short-answer, multiple-choice, and hybrid tests, Memory, 22:7, 784-802. 

Pepple, Dagogo J., Young, Lauriann E. and Carroll, Robert G. (2010) A Comparison of student performance in multiple-choice and long essay questions in the MBBS stage I physiology examination at the University of the West Indies (Mona Campus). Advanced Physiological Education, 34: 86-89. 

Pettijohn, Terry F., Sacco, Matthew F. (2007) Multiple-Choice Exam Question Order Influences of on Student Performance, Completion Time, and Perceptions. Journal of Instructional Psychology, v. 34, #3, 142-149. 

Tozoglu, Dogan, Tozoglu, Musavver, Gurses, Ahmet, Dogar, Cetin The Students’ Perceptions: Essay versus Multiple-Choice Type Exams. Journal of Baltic Science Education, No. 2 (6), 2004-10-30; pp. 52-5 

Walke YSC, Kamat AS, Bhounsule SA. (2014) A retrospective comparative study of multiple-choice questions versus short answer questions as assessment tool in evaluating the performance of the students in medical pharmacology. International Journal of Basic Clinical Pharmacology 3:1020-3.