Designing tests is an important part of assessing students understanding of course content and their level of competency in applying what they are learning. Whether you use low-stakes and frequent evaluations–quizzes–or high-stakes and infrequent evaluations–midterm and final–careful design will help provide more calibrated results.
Here are a few general guidelines to help you get started:
- Consider your reasons for testing.
- Will this quiz monitor the students’ progress so that you can adjust the pace of the course?
- Will ongoing quizzes serve to motivate students?
- Will this final provide data for a grade at the end of the quarter?
- Will this mid-term challenge students to apply concepts learned so far?
The reason(s) for giving a test will help you determine features such as length, format, level of detail required in answers, and the time frame for returning results to the students.
- Maintain consistency between goals for the course, methods of teaching, and the tests used to measure achievement of goals. If, for example, class time emphasizes review and recall of information, then so can the test; if class time emphasizes analysis and synthesis, then the test can also be designed to demonstrate how well students have learned these things.
- Use testing methods that are appropriate to learning goals. For example, a multiple choice test might be useful for demonstrating memory and recall, for example, but it may require an essay or open-ended problem-solving for students to demonstrate more independent analysis or synthesis.
- Help Students prepare. Most students will assume that the test is designed to measure what is most important for them to learn in the course. You can help students prepare for the test by clarifying course goals as well as reviewing material. This will allow the test to reinforce what you most want students to learn and retain.
- Use consistent language (in stating goals, in talking in class, and in writing test questions) to describe expected outcomes. If you want to use words like explain or discuss, be sure that you use them consistently and that students know what you mean when you use them.
- Design test items that allow students to show a range of learning. That is, students who have not fully mastered everything in the course should still be able to demonstrate how much they have learned.
Multiple choice exams
Multiple choice questions can be difficult to write, especially if you want students to go beyond recall of information, but the exams are easier to grade than essay or short-answer exams. On the other hand, multiple choice exams provide less opportunity than essay or short-answer exams for you to determine how well the students can think about the course content or use the language of the discipline in responding to questions.
If you decide you want to test mostly recall of information or facts and you need to do so in the most efficient way, then you should consider using multiple choice tests.
The following ideas may be helpful as you begin to plan for a multiple choice exam:
- Since questions can result in misleading wording and misinterpretation, try to have a colleague answer your test questions before the students do.
- Be sure that the question is clear within the stem so that students do not have to read the various options to know what the question is asking.
- Avoid writing items that lead students to choose the right answer for the wrong reasons. For instance, avoid making the correct alternative the longest or most qualified one, or the only one that is grammatically appropriate to the stem.
- Try to design items that tap students’ overall understanding of the subject. Although you may want to include some items that only require recognition, avoid the temptation to write items that are difficult because they are taken from obscure passages (footnotes, for instance).
- Consider a formal assessment of your multiple-choice questions with what is known as an “item analysis” of the test.
- Which questions proved to be the most difficult?
- Were there questions which most of the students with high grades missed?
This information can help you identify areas in which students need further work, and can also help you assess the test itself: Were the questions worded clearly? Was the level of difficulty appropriate? If scores are uniformly high, for example, you may be doing everything right, or have an unusually good class. On the other hand, your test may not have measured what you intended it to.
“Essay tests let students display their overall understanding of a topic and demonstrate their ability to think critically, organize their thoughts, and be creative and original. While essay and short-answer questions are easier to design than multiple-choice tests, they are more difficult and time-consuming to score. Moreover, essay tests can suffer from unreliable grading; that is, grades on the same response may vary from reader to reader or from time to time by the same reader. For this reason, some faculty prefer short-answer items to essay tests. On the other hand, essay tests are the best measure of students’ skills in higher-order thinking and written expression.”
(Barbara Gross Davis, Tools for Teaching, 1993, 272)
When are essay exams appropriate?
- When you are measuring students’ ability to analyze, synthesize, or evaluate
- When you have been teaching at these levels (i.e. writing intensive courses, upper-division undergraduate seminars, graduate courses) or the content lends it self to more critical analysis as opposed to recalling information
How do you design essay exams?
- Be specific
- Use words and phrases that alert students to the kind of thinking you expect; for example, identify, compare, or critique
- Indicate with points (or time limits) the approximate amount of time students should spend on each question and the level of detail expected in their responses
- Be aware of time; practice taking the exam yourself or ask a colleague to look at the questions
How do you grade essay exams?
- Develop criteria for appropriate responses to each essay question
- Develop a scoring guide that tell what you are looking for in each response and how much credit you intend to give for each part of the response
- Read all of the responses to question 1, then all of the responses to question 2, and on through the exam. This will provide a more holistic view of how the class answered the individual questions
How do you help students succeed on essay exams?
- Use study questions that ask for the same kind of thinking you expect on exams
- During lecture or discussion emphasize examples of thinking that would be appropriate on essay exams
- Provide practice exams or sample test questions
- Show examples of successful exam answers
Assessing your test
Regardless of the kind of exams you use, you can assess their effectiveness by asking yourself some basic questions:
- Did I test for what I thought I was testing for?
If you wanted to know whether students could apply a concept to a new situation, but mostly asked questions determining whether they could label parts or define terms, then you tested for recall rather than application.
- Did I test what I taught?
For example, your questions may have tested the students’ understanding of surface features or procedures, while you had been lecturing on causation or relation–not so much what the names of the bones of the foot are, but how they work together when we walk.
- Did I test for what I emphasized in class?
Make sure that you have asked most of the questions about the material you feel is the most important, especially if you have emphasized it in class. Avoid questions on obscure material that are weighted the same as questions on crucial material.
- Is the material I tested for really what I wanted students to learn?
For example, if you wanted students to use analytical skills such as the ability to recognize patterns or draw inferences, but only used true-false questions requiring non-inferential recall, you might try writing more complex true-false or multiple-choice questions.
Examinations are a very common assessment and evaluation tool in universities and there are many types of examination questions. This tips sheet contains a brief description of seven types of examination questions, as well as tips for using each of them: 1) multiple choice, 2) true/false, 3) matching, 4) short answer, 5) essay, 6) oral, and 7) computational. Remember that some exams can be conducted effectively in a secure online environment in a proctored computer lab or assigned as paper based or online “take home” exams.
Multiple choice questions are composed of one question (stem) with multiple possible answers (choices), including the correct answer and several incorrect answers (distractors). Typically, students select the correct answer by circling the associated number or letter, or filling in the associated circle on the machine-readable response sheet.
Example: Distractors are:
A) Elements of the exam layout that distract attention from the questions
B) Incorrect but plausible choices used in multiple choice questions
C) Unnecessary clauses included in the stem of multiple choice questions
Students can generally respond to these type of questions quite quickly. As a result, they are often used to test student’s knowledge of a broad range of content. Creating these questions can be time consuming because it is often difficult to generate several plausible distractors. However, they can be marked very quickly.
Tips for writing good multiple choice items:
In the stem:
In the choices:
In the stem:
In the choices:
Suggestion: After each lecture during the term, jot down two or three multiple choice questions based on the material for that lecture. Regularly taking a few minutes to compose questions, while the material is fresh in your mind, will allow you to develop a question bank that you can use to construct tests and exams quickly and easily.
True/false questions are only composed of a statement. Students respond to the questions by indicating whether the statement is true or false. For example: True/false questions have only two possible answers (Answer: True).
Like multiple choice questions, true/false questions:
- Are most often used to assess familiarity with course content and to check for popular misconceptions
- Allow students to respond quickly so exams can use a large number of them to test knowledge of a broad range of content
- Are easy and quick to grade but time consuming to create
True/false questions provide students with a 50% chance of guessing the right answer. For this reason, multiple choice questions are often used instead of true/false questions.
Tips for writing good true/false items:
Suggestion: You can increase the usefulness of true/false questions by asking students to correct false statements.
Students respond to matching questions by pairing each of a set of stems (e.g., definitions) with one of the choices provided on the exam. These questions are often used to assess recognition and recall and so are most often used in courses where acquisition of detailed knowledge is an important goal. They are generally quick and easy to create and mark, but students require more time to respond to these questions than a similar number of multiple choice or true/false items.
Example: Match each question type with one attribute:
- Multiple Choice a) Only two possible answers
- True/False b) Equal number of stems and choices
- Matching c) Only one correct answer but at least three choices
Tips for writing good matching items:
Suggestion: You can use some choices more than once in the same matching exercise. It reduces the effects of guessing.
Short answer questions are typically composed of a brief prompt that demands a written answer that varies in length from one or two words to a few sentences. They are most often used to test basic knowledge of key facts and terms. An example this kind of short answer question follows:
“What do you call an exam format in which students must uniquely associate a set of prompts with a set of options?” Answer: Matching questions
Alternatively, this could be written as a fill-in-the-blank short answer question:
“An exam question in which students must uniquely associate prompts and options is called a
___________ question.” Answer: Matching.
Short answer questions can also be used to test higher thinking skills, including analysis or
evaluation. For example:
“Will you include short answer questions on your next exam? Please justify your decision with
two to three sentences explaining the factors that have influenced your decision.”
Short answer questions have many advantages. Many instructors report that they are relatively easy to construct and can be constructed faster than multiple choice questions. Unlike matching, true/false, and multiple choice questions, short answer questions make it difficult for students to
guess the answer. Short answer questions provide students with more flexibility to explain their understanding and demonstrate creativity than they would have with multiple choice questions; this also means that scoring is relatively laborious and can be quite subjective. Short answer
questions provide more structure than essay questions and thus are often easy and faster to mark and often test a broader range of the course content than full essay questions.
Tips for writing good short answer items:
|Type of question||Avoid||Do use|
Suggestion: When using short answer questions to test student knowledge of definitions consider having a mix of questions, some that supply the term and require the students to provide the definition, and other questions that supply the definition and require that students provide the term. The latter sort of questions can be structured as fill-in-the-blank questions. This mix of formats will better test student knowledge because it doesn’t rely solely on recognition or recall of the term.
Essay questions provide a complex prompt that requires written responses, which can vary in length from a couple of paragraphs to many pages. Like short answer questions, they provide students with an opportunity to explain their understanding and demonstrate creativity, but make it hard for students to arrive at an acceptable answer by bluffing. They can be constructed reasonably quickly and easily but marking these questions can be time-consuming and grader agreement can be difficult.
Essay questions differ from short answer questions in that the essay questions are less structured. This openness allows students to demonstrate that they can integrate the course material in creative ways. As a result, essays are a favoured approach to test higher levels of cognition including analysis, synthesis and evaluation. However, the requirement that the students provide most of the structure increases the amount of work required to respond effectively. Students often take longer to compose a five paragraph essay than they would take to compose five one paragraph answers to short answer questions. This increased workload limits the number of essay questions that can be posed on a single exam and thus can restrict the overall scope of an exam to a few topics or areas. To ensure that this doesn’t cause students to panic or blank out, consider giving the option of answering one of two or more questions.
Tips for writing good essay items:
Suggestions: Distribute possible essay questions before the exam and make your marking criteria slightly stricter. This gives all students an equal chance to prepare and should improve the quality of the answers – and the quality of learning – without making the exam any easier.
Oral examinations allow students to respond directly to the instructor’s questions and/or to present prepared statements. These exams are especially popular in language courses that demand ‘speaking’ but they can be used to assess understanding in almost any course by following the guidelines for the composition of short answer questions. Some of the principle advantages to oral exams are that they provide nearly immediate feedback and so allow the student to learn as they are tested. There are two main drawbacks to oral exams: the amount of time required and the problem of record-keeping. Oral exams typically take at least ten to fifteen minutes per student, even for a midterm exam. As a result, they are rarely used for large classes. Furthermore, unlike written exams, oral exams don’t automatically generate a written record. To ensure that students have access to written feedback, it is recommended that instructors take notes during oral exams using a rubric and/or checklist and provide a photocopy of the notes to the students.
In many departments, oral exams are rare. Students may have difficulty adapting to this new style of assessment. In this situation, consider making the oral exam optional. While it can take more time to prepare two tests, having both options allows students to choose the one which suits them and their learning style best.
Computational questions require that students perform calculations in order to solve for an answer. Computational questions can be used to assess student’s memory of solution techniques and their ability to apply those techniques to solve both questions they have attempted before and questions that stretch their abilities by requiring that they combine and use solution techniques in novel ways.
Effective computational questions should:
- Be solvable using knowledge of the key concepts and techniques from the course. Before the exam solve them yourself or get a teaching assistant to attempt the questions.
- Indicate the mark breakdown to reinforce the expectations developed in in-class examples for the amount of detail, etc. required for the solution.
To prepare students to do computational questions on exams, make sure to describe and model in class the correct format for the calculations and answer including:
- How students should report their assumptions and justify their choices
- The units and degree of precision expected in the answer
Suggestion: Have students divide their answer sheets into two columns: calculations in one, and a list of assumptions, description of process and justification of choices in the other. This ensures that the marker can distinguish between a simple mathematical mistake and a profound conceptual error and give feedback accordingly.
Cunningham, G.K. (1998). Assessment in the Classroom. Bristol, PA: Falmer Press.
Ward, A.W., & Murray-Ward, M. (1999). Assessment in the Classroom. Belmont, CA: Wadsworth Publishing Co.
This Creative Commons license lets others remix, tweak, and build upon our work non-commercially, as long as they credit us and indicate if changes were made. Use this citation format: Exam questions: types, characteristics and suggestions. Centre for Teaching Excellence, University of Waterloo.