When I design a training module, the first thing I do is constructing the performance test. Later in the design process, I may not use this test, but the time I invest in constructing it makes the subsequent instructional design better.
I do not use paper-and-pencil tests as final tests — unless my training deals with some paper- and-pencil activity such as business writing. In most situations, my final tests present authentic scenarios and require actual performance.
What We Planned for the Final Test
Here’s an example:
I recently designed a training package for Customer Service Representatives (CSRs) on how to empathize with upset customers during telephone conversations.
This was the plan for our final test:
It will involve scenario-based roleplays over the telephone. This test will present unpredictable telephone calls from upset customers. The participant’s performance will be evaluated through an objective checklist.
To create the scenarios, we will work with experienced CSRs, subject-matter experts, and real-world customers. The final version of the roleplay performance test will involve 10 different situations that represented different types of internal and external customers complaining about different products and services. We will create five different personality profiles for the customers (ranging from the meek and compliant to the abusive and aggressive). By using these five personalities with each of the 10 situations, we will end up with 50 variations.
To assess the competency of each test-taker,we will ask them to roleplay two randomly selected scenarios. To ensure approximately equal difficulty level among the random scenarios we will hire improv actors to make the telephone calls. These actors will be provided with the scenarios but not with scripts. They will call the test-taker on the telephone and improvise the conversation in response to this persons’ questions, statements, and reactions. They will push the participant to a medium level of difficulty (irrespective of the personality profile assigned to them). The telephone conversation will be recorded and evaluated on an objective rating scale.
What We Actually Did
Because of time and budget constraints, we did not implement these plans for our final performance test. However, we did adopt and adapt aspects of this test-construction plan to make our training design faster and better.
We used the planned test as the operational definition of our training goal. We created several scenarios according to the original plan and incorporated them in different parts of the training session. We created the checklist for rating participants’ performances and used it to define the content and objectives of our training.
The Training Sequence
The facilitator walks participants through the rating checklist items and explains appropriate behaviors for demonstrating empathy during a conversation with an upset customer.
The facilitator conducts a roleplay (inviting a participant to play the role of an upset customer) to demonstrate appropriate empathic behaviors.
The facilitator assigns different scenarios to teams of participants. These teams analyzethe scenarios and prepare an outline for a simulated telephone conversation. They enact this conversation as an audio drama. After each enactment, participants from the other teams (and the facilitator) provide feedback on which empathic behaviors were demonstrated effectively and which behaviors could be improved in future conversations.
Individual participants use other scenarios to conduct rapid roleplays to improve their fluency with empathic behaviors.
As a final activity, each participant is given a different scenario. A randomly selected participant roleplays a CSR while the facilitator roleplays an upset customer (as in the case of the improv actor in our original plan.)
Suggestion for Instructional Designers
Early in the training design process, construct an authentic performance test. Use components of this test to operationalize the training goal, objectives, and content. Design your trainingto enable participants to perform effectively in this test – even if you don’t actually use it in the original form.