Maritime Jobs
Wednesday, August 5, 2020

Training Tips for Ships: Never Give the Same Exam Twice

October 8, 2019

© smolaw11/AdobeStock

© smolaw11/AdobeStock

We’ve all heard the stories, and many of us have seen it first-hand: the completely ineffective assessment practice of giving the same multiple-choice exam to people over and over. The only time this is ever acceptable is when all of the people writing the exam are doing so at the same time and are supervised to ensure they are not sharing answers. However, giving the same exam to different people over a period of months or years ensures that exam grades are going to go up over time – sometimes quite markedly. Sadly, this is not because people are learning more. It is because the people who wrote the exam earlier are sharing the answers with those writing it later. This is no surprise, and it creates real issues.
The most obvious problem with the practice of exam reuse is that the assessment results become meaningless. They are not an accurate reflection of the trainee’s knowledge, and they cannot be used as a measure of training quality. And, of course, with exam answers in hand, trainees will bypass the learning materials and go straight to the exam – rendering even well-designed training useless.

Possibly the most damaging effect, however, is that the training program loses credibility and respect in the eyes of trainees. To a trainee, reuse of exams signals laziness and a lack professionalism in the training organization. They will respond in kind.

Thus, the lesson here can hardly be overstated: never give the same exam twice. If you are doing so, now is a great time to stop.

Does This Mean Constantly Creating New Exams?
No – though it may mean some changes. Most people who are reusing exams are doing so because they are delivering paper-based exams or using poorly built computer-based training.  In the case of paper exams, it is difficult and expensive to create a new exam each time it is given. In the case of poorly constructed computer-based training, the program is “static”, so the same exam is delivered to every person who completes the training.

The alternative is to randomize your exams. With exam question randomization, no two trainees will get the same exam. Instead, they will each get a different mix of exam questions, and in a different order. This kind of exam randomization requires a learning management system (LMS). But fortunately, nearly all modern professional LMSs support exam question randomization. So if you have access to an LMS, there is a good chance that you already have the capability.

What Does This Mean for Assessment Consistency?

The idea of giving each trainee a different exam may raise the concern that we lose consistency in our assessment practices. After all, it is critical that we measure everyone by the same standard. However, making it easy to share answers by reusing exams completely distorts the playing field – so we already have a problem. Additionally, exam randomization does not have to be quite as “random” as it sounds. Instead, done correctly, controls can be applied to ensure that the same competencies are covered, to the same degree, with questions of roughly equal difficulty. How do we do this?

Check back here next month and we will see in Training Tips for Ships! Until then, safe sailing!


(As published in the September 2019 edition of Maritime Reporter & Enginerering News)



computer-based training

Featured Jobs

Deputy Pilot Openings in Florida

Board of Pilot Commissioners

Able Seaman

Military Sealift CommandNorfolk, VA, USA

Chief Radio Electronics Technician (IAT)

Military Sealift CommandNorfolk, VA, USA

Electronics Technician

Military Sealift Command

First Radio Electronics Technician (IAT)

Military Sealift CommandNorfolk, VA, USA

Electrician

Military Sealift CommandNorfolk, VA, USA

Steward Cook

Military Sealift Command

Assistant Cook

Military Sealift CommandNorfolk, VA, USA

Assistant Damage Control Officer

Military Sealift Command

Third Officer

Military Sealift CommandNorfolk, VA, USA

Featured Employers

Military Sealift Command

The Military Sealift Command (MSC) is a United States Navy organization that controls most of the re

Fraser River Pile and Dredge

Fraser River Pile & Dredge (GP) Inc. (FRPD) is Canada’s largest Marine & Infrastructure, Land Founda

Board of Pilot Commissioners

The Board of Commissioners of Pilots (the “Board”) is a public agency, created by the New York State