Desirable vs Undesirable Difficulties

I like how the authors of Make It Stick use the terms desirable and undesirable difficulty to describe positive and negative traits of learning situations. Undesirable difficulty, I think, is more inclusive than extraneous cognitive load because it includes internally-generated emotions like anxiety.

The authors indicate that there is some experimental evidence that anxiety can be ameliorated by acknowledging that the material is difficult and talking with learners about how struggle is healthy and desirable because it is a sign that learning is happening.

Addendum: The authors give credit later in the chapter to Elizabeth and Robert Bjork for coining the term desirable difficulties. They quote the Bjorks: “[Desirable difficulties] trigger encoding and retrieval processes that support learning, comprehension, and remembering. If, however, the learner does not have the background knowledge or skills to respond to them successfully, they become undesirable difficulties.”


Varied Practice

Make It Stick starts the chapter on massed practice with an interesting research anecdote about a group of eight-year-olds who practiced throwing a beanbag into a bucket three feet away. Half the group practiced with the bucket three feet away and half practiced by throwing the beanbag into buckets two and four feet away.

They did this for 12 weeks and then were tested throwing a beanbag into a bucket three feet away. The subjects who practiced on the two and four foot away buckets did much better than the ones who practiced the actual task. Interesting!

And counterintuitive. I would have anticipated the opposite, on the strength of the principle that the closer the practice is to the real world task (or the exam), the better the preparation.

If I wanted to learn a song in a piano book, clearly I’d be better off practicing that song than the song before and after it in the book. But, thinking broadly, maybe I would learn the song better if I practiced it in varied keys. And in the long run perhaps learning a variety of styles of music would help me become stronger in my preferred genre.

In terms of cognitive skills, it’s hard to know how far to take this, and the authors acknowledge that more research is needed. However, it seems reasonable to me that if you were an aspiring CPA and were learning how to audit cash at financial institutions (banks), you’d benefit from practice auditing cash at other types of businesses. The risk is that you waste time practicing learning and applying principles and facts that don’t apply to any of your actual clients, but the upside, maybe, is that the varied practice makes you a stronger auditor in financial institutions.

Actually, the bigger risk I see is that if you specialize in auditing financial institutions, taking time to practice auditing other kinds of institutions may force you to deal with concepts that are foreign to you, raising the cognitive load higher than it needs to be. Cognitive load is not an issue when you are throwing beanbags into buckets, but it is with complex cognitive problem solving.

Thus, a model for using varied practice to learn cognitive skills would have to include guidance on what cognitive variations are useful and which are harmful.

Memorization vs Complex Problem Solving

In my last post I commented that the emphasis in Make It Stick on recall as opposed to application made the book feel a little academic. The authors do address this point, though they are a little snarky.

Hmmm. If memorization is irrelevant to complex problem solving, don’t tell your neurosurgeon….Pitting the learning of basic knowledge against the development of creative thinking is a false choice. Both need to be cultivated. The stronger one’s knowledge about the subject at hand, the more nuanced one’s creativity can be in addressing a new problem. (p. 30)

It’s certainly true that knowledge within a problem domain is essential, and that greater depth of understanding will likely produce better solutions. But that understanding has to be built on coordinating facts, concepts, principles, and well- to ill-structured procedures–distinctions I wish the authors of Make It Stick had more carefully addressed. The type of knowledge needed to solve problems matters, and I could see how critics can interpret Make It Stick as focused primarily on fact-level learning.

The Testing Effect

I’ve written previously about the Posttest Paradox. Makes it Stick in contrast speaks of a “testing effect,” which is the idea that retrieving information from memory–say, for an exam–increases your ability to retrieve that knowledge later.

I don’t like the term “testing effect,” because it implies formal exams. I’ve spent the last few years cautioning my firm that exams have hidden costs and may not be the best way to achieve their objectives, so calling this the “testing effect” undermines that when in reality, it sounds like the effect had more to do with practice and application than exams per se.

Their alternative name, the “retrieval-practice effect,” is a little better, but not exactly memorable.

There’s also a lot of emphasis in the book on retrieval. While the ability to access important knowledge is important for problem solving, in the real world, people can also look stuff up. I’d have liked to see more focus on conceptual understanding and the ability to generalize to related problems.

Reflection Is a Form of Practice

The authors of Make it Stick highlight the principle that reflection is itself a form of practice, that thinking about a problem is a form of useful rehearsal.

The recently revised standards for learning for CPAs put out by the National Association of State Boards of Accountancy (NASBA) stipulate for the first time that classroom learning must be active. For now, the rules for minimum interactivity are, indeed, minimal. To give formal credit, classes have to have one interaction per hour, of which that interaction can be nearly anything, including asking participants to reflect silently for a few seconds on a given question.

Lecturing for 45 minutes and then asking participants to reflect silently on a question asked by the lecturer is not necessarily effective design. But I applaud NASBA both for requiring interactivity and for keeping the requirement open. The more prescriptive the targets, the more proforma the execution. Keeping it open will result in many developers trying to do the minimum, for sure, but it may also help developers take ownership of active learning and try to understand NASBA’s intent.

Quiz Before Teaching

One of the core principles in Make it Stick is “trying to solve a problem before being taught the solution leads to better learning, even when errors are made in the attempt.” (p. 4)

I get it; finding out you don’t know something as well as you thought can make for a powerful learning moment.

On the other hand, I’ve tended to advise course developers to avoid asking learners right/wrong multiple choice questions before actually teaching the content, arguing that they are unfairly setting learners up for failure. (Intentional and actionable diagnostic pre-testing is an exception.) Philosophically and temperamentally, I much prefer trying to set learners up for as much success as possible. But I do understand that there are proponents of failure-based learning and am open to the possibility that there could be elements here that I should add to my playbook.