Digital versus paper-based exams
The advantages and disadvantages of digital assessment for efficiency purposes
At the UT it is possible to administer exams digitally for summative and formative purposes. This offers efficiency benefits, compared to paper-based assessments. For closed questions, the benefit is in having the results automatically reviewed. Afterwards, you get the scores, a test analysis and, if desired, a grade overview.
For written exams with open (essay) questions, efficiency is enhanced because it is easier to check per question (often a faster procedure ) and the questions can easily be distributed among several people who can check them in parallel. Not having to decipher handwritten text or having to find out exactly where the answers are and what applies when something has been partially crossed out, also makes marking easier.
Students nowadays will often prefer typing rather than having to write everything down by hand. For them, it is also a benefit when during test taking they can easily make changes in their answers.
But there are also disadvantages and points to consider. Especially for using Remindo, you will have to invest time to get familiar with the system and process. A logistic issue might be that if you want to use Remindo (suitable for closed and open-ended questions) on Campus, in combination with Chromebooks (laptops), this will require timely reservation of the Chromebooks and everything should be well prepared and set up in the scheduled room. For technical issues, it is helpful if there is someone available to assist. Scheduling the Chromebooks will become more difficult if the demand for it will increase significantly.
So, efficiency can play a role when you considering digital testing, however, there may be other considerations at play. Check out a more detailed list of the advantages and disadvantages of using a digital exam system (for summative purposes) to make an informed choice.
Advantages of administering a written exam digitally - Automatic scoring. The scoring for closed (MC) exams is done automatically. The results can be available quickly and it requires limited actions from the teacher. There is less chance of errors during marking (unless systematic errors, e.g. because the wrong key is indicated).
- Easier review of typed work in fixed order. For open questions, you no longer have to bother deciphering handwritten text or figuring out where the answer to which question is (if no format has been used for the questions and students do not follow the given order).
- Review on a question-by-question basis. Thereby staying more focused and consistent. You can assign the marking of certain questions to certain assessors. The marking can be done in parallel.
- Getting psychometric analysis results automatically. You don't have to use extra programs, like Excel, for this. Having the data makes it easier to check the quality of questions and detect if there are questions with which there is something problematic (e.g. the wrong key was indicated or the question was so unclear that no one knew the right answer). The information is also valuable for evaluating your teaching process.
- Students may prefer typing over handwriting.
- Direct feedback for students. If wished for and for instance the system is used in a formative way, immediate feedback can be provided. for the students.
- Some anti-cheating measures are easier to implement. For instance, questions can be given randomized.
- Diverse question types can be used and multimedia. E.g. if the system has this feature, video's or animations can be used.
- Special features can be used. Such as time limits for questions, disallowing going back to questions or - for advanced systems and use - tailoring the questions on student performance.
- Remote assessment can be done more easily. NB. lleaving aside attention points such as application of proctoring systems, which makes it a bit more complicated. In principle it can facilitates students taking exams from various locations at the same moment.
- Centralized data storage in systems, without the need for physical storage space. Results can be easily stored, accessed, and analyzed. Also for longer times.
- Reduced Paper Use. Digital assessments can decrease the use of paper, although it might increase the use of electricity.
- Customizability for students with specific needs. For isntance adapting the e.g. font size adjustment.
- The scoring for closed (MC) exams is done automatically. The results can be available quickly and it requires limited actions from the teacher. There is less chance of errors during marking (unless systematic errors, e.g. because the wrong key is indicated).
- For open questions, you no longer have to bother deciphering handwritten text or figuring out where the answer to which question is (if no format has been used for the questions and students do not follow the given order).
- You can review more easily on a question-by-question basis (thereby staying more focused and consistent) and assign the marking of certain questions to certain assessors. The marking can be done in parallel.
- You will get psychometric analysis results automatically, which makes it easier to check the quality of questions and detect before the grading questions with which there is something problematic (e.g. the wrong key was indicated or the question was so unclear that no one knew the right answer). The infromation is also valuable for evaluating your teaching process.
- If wished for: immediate Feedback for the students.
- Some anti-cheating measures are easier to implement. For instance, questions can be given randomized.
- Diverse question types can be used and multimedia, e.g. (if the system has this feature) video's or animation.
- Special features can be used, such as time limits for questions, disallowing going back to questions or - for advanced systems and use - tailoring the questions on student performance.
- Remote assessment can be done more easily (leaving aside attention points such as application of proctoring systems, which makes it a bit more complicated). It facilitates students taking exams from various locations at the same moment.
- Centralized data storage in systems, without the need for physical storage space. Results can be easily stored, accessed, and analyzed. Also for longer times.
- Reduced Paper Use: Digital assessments can significantly decrease the use of paper, contributing to more sustainable practices in education.
- Environmental benefit due to reduced paper use: Digital assessments can significantly decrease the use of paper, although it will increase the use of elektricity.
- Customizability for students with specific needs. For isntance adapting the e.g. font size adjustment.
- :
- Adaptive Testing: Exams can be tailored in real-time based on student performance, offering a more personalized assessment experience.
- questions and secure browsers, reducing the risk of cheating.
- Quick Results: Automated grading for objective questions allows for instant feedback, which can enhance the learning process. Analytics: Digital platforms can provide detailed analytics on student performance, helping examiners identify trends and areas for improvement.
- Enhanced Security:
- Anti-Cheating Measures: Digital exams can include various security features like randomized questions and secure browsers, reducing the risk of cheating. Easier Monitoring: Digital environments can facilitate proctoring and monitoring to ensure integrity.
- Customizability:
- Adaptive Testing: Exams can be tailored in real-time based on student performance, offering a more personalized assessment experience. Variety of Question Types: Digital formats can support multimedia questions (e.g., video, audio), which can enhance the assessment.
For Assessing:
- Accessibility:
- Anti-Cheating Measures: Digital exams can include various security features like randomized questions and secure browsers, reducing the risk of cheating. Remote Access: Students can take exams from various locations, increasing accessibility for those who may have transportation or mobility issues.
- Data Management:
- Centralized Data Storage: Results and performance data can be easily stored, accessed, and analyzed, facilitating long-term tracking of student progress. Better Reporting: Digital assessments can generate comprehensive reports that help in understanding group performance and informing curriculum changes.
- Flexibility in Format:
- Diverse Question Types: Digital exams can incorporate a variety of formats, including interactive simulations and multimedia elements, which can assess a wider range of skills. Time Management: Students may benefit from digital timers and flexible time limits, enhancing the exam experience.
- Environmental Impact:
- Reduced Paper Use: Digital assessments can significantly decrease the use of paper, contributing to more sustainable practices in education.
Overall Impact:
Digitally administered exams can lead to a more efficient, flexible, and inclusive assessment environment. They enable examiners to focus on enhancing educational outcomes through immediate feedback and data-driven insights, while providing students with a more varied and accessible assessment experience.
Disadvantages of administering a written exam digitally Most of the following disadvantages do not apply to the Contest system or Grasple.
Contest uses pre-printed forms on which students fill in their answers. The questions will be on a seperate paper. In that sense, it resembles a paper exam more. However, the forms can still be processed automatically afterward.
Grasple is mostly used for mathematic courses and more for homework and formative tests.
- Learning to use the system. It will take some time to learn especially how to use Remindo.
- Technical issues: Digital examination relies on technology. The hardware and software may fail and disrupt the exam process.
For individuals or larger groups of students. A technical assistant during test taking is needed when the present teacher cannot provide this assistance. Even with support, major technical problems will disrupt or necessitate the cancellation of the examination process. - Getting familiair for students: Students need to become familiar with digital examination. It is especially important for new students (first-year Bachelor's or Master's students) to receive explanations about the systems to be used and preferably to let them practice with them.
- Reliability risks. When using Chromebooks, which only will give access to programmes the examiner permits and what is permitted is limited to the test itself (no Internet), this won't be less of a problem, but when students can use own devices, ensuring academic integrity can be challenging. Students may easily look at unauthorized resources or commuicate with other students.
- Screen fatigue: Prolonged exposure to screens, especially when concentration is needed, can lead to fatigue and decreased concentration, which might negatively impact student performance.
- Limitation for question formats or use of (mathematical) symbols: Although a lot is possible in the digital systems, some question types with complex layouts or which uses specific symbols (in formula) might not be possible. NB. Grasple offers special possibilities for symbls and formula.
- Data privacy concerns: Digital exams may raise concerns about data security and privacy. For that reason it is important that only checked and authorized systems are used at the UT.
What to do when you want to use a digital system
Weighing the pros and cons, you might choose for digital testing. Luckily a lot of organizational, and logistic aspects will be organized for you by the Examinations Office | CES-SA&L. The office is responsible for managing and coordinating institution-wide all aspects of the test-taking process for written tests. On the website, you can inform yourself about the test-taking process in general (1. Scheduling exams 2. Preparing exams 3. Preparing exams location 4. Administer exams 5. Assess exams 6. Register exams results 7. Extra Facilities Form).
If you administer the test digitally, you can inform yourself about the practical issues via this site: Digital exams, with a nice overview of the Step-by-step instructions. If you want some personal support or have questions, you can also contact the e-learning specialist at your faculty.
Information about the digital systems in use (central licensed) at the UT, can be found in the table below. NB. Your faculty or some programmes might have another system in place and their own license. Ask around at your faculty.
Tools for assessing summatively and/or formatively
Contest | Uses forms that can be automatically processed. Can only be used for closed questions and only for questions with one correct answer per question. Processing takes three business days. You can request different kinds of reports afterward, see the link on the left for examples and more information (also via video) about the process.
|
Remindo | Mostly used in combination with Chromebooks on Campus. Can be used for open and closed questions. A lot of question types are possible (see the practice test). Remindo can be used formatively in Canvas too: Remindo in Canvas. |
Grasple | Mainly used for the mathematics courses in the Math Line, providing homework and diagnostic (formative) tests to the students. |
Canvas - quiz Instructor Guide - Quizzes | Mainly used for formative (mock) exams or quizzes, for practicing and self-assessment purposes. NB. There is an online Canvas lecturer course that can be gone through at your own convenience and which addresses also how to make use of quizzes: look on this page and under "Learn Canvas online" to enroll. |
Wooclap | Wooclap can be used to ask questions during a lecture. This way you can engage your students and it can be used for self-assessment purposes. At the same time provides you as a teacher insight into possible problem areas. WooClap is a stand-alone application. It can be used in lecture rooms and online sessions. Wooclap can be integrated into PowerPoint, so you can prepare and ask questions without the need to open a separate tool. It can be accessed via app.wooclap.com/home
|