Oral Exams

What

An oral exam assesses what the student has learned based on the spoken word. Students are asked to use the spoken word, guided by questions or small tasks, to show they have achieved the learning objectives. It can be conducted for individuals or organized for groups of students.
For this table oral exams are considered to be a summative assessment method, but they can be used in a formative way as well. 

How

The format and content of an oral exam are determined by the learning outcomes of the course and the purpose of the oral exam.
Some considerations, tips, and tricks can be found <here>. 

Time

An oral exam might take usually about 30 minutes, including direct preparation and grading. But it depends on the purpose.

Student preparation

Inform the students explicitly and elaborate on the procedure. If possible, let them practice or show a simulated setting in a video to give an impression. Tell them that the oral is meant to provide an opportunity to show what they have learned. Make beforehand preferable clear: 
> the setting (who will be there; who will ask questions etc.) 
> the purpose of the oral exam and the percentage contribution to the overall grade
> what will be addressed (and what not) (e.g. will it just be about the project they did or will there also be questions about theories
   addressed in the lectures or materials)  
> the criteria for the award of grades and how the grade will be determined.

Advantages

You can test understanding of the topic individually in a reliable way.  Can be efficient up to 100 students. 

Disadvantages

It takes time and it's quite strenuous for the assessor. When more assessors are involved but they work parallel or separately, consider how to guard the inter-rater reliability. Do the first few together. Use a clear checklist or rubric.

Concerns

> A rule in the bachelor EER:  the oral examination is public unless the Examination Board has determined otherwise in a special case. 
> For this Teaching Guide, oral exams are considered to be a results-oriented (summative) exam type. When choosing an exam type
   and designing a specific exam, consult the exam regulations of the respective institution. They will provide information on how the
   exam should be designed and what is and is not allowed.
> Most programmes require a second pair of eyes at oral exams. This means either having a second person present or making a
   recording. There are privacy regulations around recordings. They should not be available to anyone other than the parties in the
   recording and must be removed after a prescribed period.
> Assessors must be able to be objective in their assessment and be aware of bias effects (see Grading pitfalls)

Ideas, specifics

An oral exam can be done in small groups. Interaction can be stimulated. 
An oral exam can also be combined with an assignment, for instance, a (group) report or presentation.
A tip by Kristina Edström ¹: reverse the burden of proof.  Ask the students in the first 7 minutes of an oral exam to show you that he/she has reached the learning outcomes. Follow-up questions will then pop up.  Ask students kindly how they think it went after the exam.  Make sure they know they have to show ”real” understanding, in real time.

Grading, reliability and quality assurance

Use a grading scale or rubric to stay consistent and fair. A filled in standard form will help to substantiate your judgment in retrospect and upon request. A recording can be considered. The use of two assessors will increase reliability, but this might not always be possible. Assessment might be biased because of assessing pitfalls if the student is not anonymous.  Student’s articulateness, shyness, speed of thought, gender, language skills, etc. can influence the judgment. Keep in mind during the oral exam that a student can be very nervous. Be aware of influencing factors that should not be allowed to contribute to the determination of the grade.
See one of the external resources for examples of scoring forms. 

Point of attention: the first student can pass on the questions to other students. Consider whether this is objectionable and may disadvantage or benefit the earlier, later students. Similar but different question scenarios can be used to overcome this point.

Tools for online

Microsoft Teams, Canvas Conference. There can be situations in which you might want to make use of remote oral exams. During the Covid-19 pandemic this was done more often. On the Remote assessment | Oral exams (utwente.nl) page you can learn more about how to create a virtual room and time slots in Canvas and using MS Teams to set up and record individual or group meetings.
In the Covid period, a how to guide (2 pages) was set up:  how-to-design-and-execute-an-oral-exam-2.pdf (utwente.nl).  

Useful extra  information & tips

CELT guideline: oral-exam-make-it-valid-reliable-and-transparent.pdf (utwente.nl) 
Guideline for online oral exams: How to design and execute an oral exam.  

External sources:
>  A short guide to oral assessment. Gordon Joughin, 2010. Leeds Metropolitan University/University of Wollongong.  
> Oral Exams Teaching Guide for Higher & Professional Education ZHAW Center for Innovative Teaching and Learning (Zurich
   University of Applied Sciences) - Contains nice checklists and templates for scoring and reporting.