Posttest Paradox

Michael Allen in his book Designing Successful e-Learning: Forget What You Know About Instructional Design and Do Something Interesting refers to a phenomenon he calls the “posttest paradox.” It appears based on a Google search of the term that he must have coined it, as no one else has used it (until now!). The phenomenon, however, has certainly been described. It’s the idea that while high stakes testing is generally intended to raise accountability and therefore motivation and therefore learning, it is possible it may actually decrease useful learning.

There are a number of aspects to this. One is that test preparation changes how participants engage with a course. Attending a course in order to get better at some real world task encourages learners to think about the concepts and principles and strategies taught in the class through the lens of how they will apply in the real world, which helps them encode information in a usable, retrievable way. The presence of a high stakes test on the other hand refocuses learners so they think about the content in terms of how it might be tested, which is a less useful way to encode the information in the long term, making it less accessible after the test is complete.

Tests also create anxiety, which can mask performance and inhibit learning.

In terms of adult learners, high stakes testing does not honor their ability or responsibility to engage with the material in the course that is most meaningful to them most heavily and with irrelevant material less. Anything could appear on the test.

It would also be fair to say that as tests become more high stakes, the administrative burden climbs. High stakes incentivises cheating, for instance, plus it becomes critical that records be flawless. The resources spent policing the test and keeping records are resources not devoted toward creating the best possible learning experience.

In short, post-testing as a means of creating motivation carries costs, many of them hidden.

Advertisements

2 thoughts on “Posttest Paradox

  1. Rob Foshay

    Well said, Bob.
    That’s why I’ve always been interested in performance-based assessment, and always considered paper and pencil tests to be limited in their value. This is especially true when we are interested in learning outcomes that can’t be measured well in conventional tests, such as complex ill-structured problem solving.
    What’s interesting is that the first and second generations of psychometricians seemed to be aware of the limitations of conventional tests in a way that has somehow gotten lost in the current generation of psychometricians. I took my first applied psychometrics course in 1971, from E.L. Thorndike, Jr. – son of one of the founding fathers of psychometrics. He constantly reminded us that conventional tests were at best only imperfect correlates of what we really wanted to measure. Somehow, as the theory of domain-referenced testing developed, that basic principle seems to have been ignored – or honored more in the breach than the practice.

    None of this is original to me: there is a counter-revolution going on in testing, led mostly by learning psychologists and some of the “old school” psychometricians such as Robert Mislevy. For more on this, see the reports of the Gordon Commission on assessment, a project of Educational Testing Service. In particular, look into Mislevy’s work on Evidence Centered Design. I’m a huge fan, and I’m applying the principles of ECD to performance-based assessment for professional certification/credentialing at The Institute for Performance Improvement (TIfPI.org).

    Reply
  2. Pingback: Do Exams in Corporate Training Increase Engagement, Learning, or Satisfaction? | Engaged

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s