Assessment support

toolbox

Constructing questions

Starting point: test specification table

When you have decided to use a written test as an assessment method, your first step will be to construct a blueprint for your test. This can be done by constructing a so called "test specification table". With this you can ensure that your learning objectives are covered and that the questions are on the right level (of Bloom). It helps to construct a test that focuses on the key areas and weights different areas based on their importance. The table helps to make a resit test or a test a next year similar to your original test. Under Design you can find more explanation and a format. 

Decide about question type

A decision that is also involved in constructing your specification table, is the decision about question type. As an examiner, you have to choose the appropriate question type for what you want to assess. Validity is the main issue here. Do you really test what you want to test (your learning objectives)? Will the score and grade accurately indicate the extent to which the student has the intended level of knowledge and skills?
Efficiency may play a role in the choice of question format. Consideration can be given to what is the best possible choice given the circumstances.  Combination of question types in one test are certainly possible. So you can offset (dis)advantages of one question form with another question form.

A distinction in types of questions that is often made: 

Closed questions, where the answer options are provided, such as

• Multiple choice questions. A question (or partial statement) and multiple answer options are given. One answer is the correct answer or best answer. 
• True/False questions. A question is followed by true/false answer options.
• Completion question. A statement of paragraph  is given with one or more blank parts, which should be filled in (chosen) by the students. 
• Matching questions. For instance two rows of items are porvided and the student has to indicate how the stems or problems on one side are related to answers on the other side. They have to find relationships. 

Open questions, for which students have to come up with their own answers, such as: 

• Completion of fill-in question. A statement of paragraph  is given with one or more blank parts, which should be filled in by the students. 
• Short answer questions. A prompt is given and it demands a written answer that varies in length but will be brief (just one word, a number, a short list, a few sentences). 
• Essay questions. A prompt is given, that requires an elaborate answer. Sometimes the question can be complex and might contain several sub-questions. The response asked for can be: extended (free writing) or restricted (requirements are given, such as: compare in your answer X and Y).

Within these main types, there are other sub-types and variations. For instance one can use a case as basis for the questions or provide data to which operations should be applied. 
The different types of exam questions have their own advantages, and disadvantages.Open questions will require more attention and time for reviewing compared to closed questions. But creating good quality mc questions is quite hard to do and takes a lot of time beforehand. 

The following sites give a nice overview about the mentioned question types:  Exam Questions: Types, Characteristics, and Suggestions | Centre for Teaching Excellence | University of Waterloo (uwaterloo.ca) and Advantages and Disadvantages of Different Types of Test Questions | UAMS Educational Development 

Nb. People often think that closed questions can only be used for lower-order cognitive skills. To check if students can recall some factual knowledge for instance. But closed questions can be used for higher-order thinking (application, analysis, evaluation) too. Although it is not easy to come up with good questions. This handout show some examples. Under 'Tips & resources" further below, more resources for formulating higher-order closed questions are provided.   

In "Assemble the test" we will provide more infromation about considerations when putting the test together. For instance: How many questions are needed to make a test reliable? How many questions can be put in a test, when there is limited time available?  

Tips & Resources