Ergodic Switch, part 2: Chubby Thighs

Part 1

If people aren’t ergodic, that means in theory exams and other cognitive assessments have no predictive value for individuals, only for populations. Intuitively, of course, that’s not true. If I needed a Spanish translator, and all I knew from among the candidates was their grades in their most recent Spanish class, I’d pick someone with high scores. That wouldn’t guarantee success, but I’d expect it to correlate with success.

The example Rose uses is instructive. If you hold up a newborn so that his or her legs are dangling, he or she will move his or her legs in a walking reflex. This walking reflex disappears after a time, only to reappear later. Researchers hypothesized that the suppression of this walking instinct is related to cognitive development.

They measured the point at which babies on average lose the walking instinct and compared that average to cognitive development known to happen in that same timeframe and linked the two.

Some doctors, then, naturally, would express concern to parents if their baby didn’t lose their walking reflex in that timeframe. It became a symptom of delayed cognitive development.

That is, until a researcher came along and instead of worrying about how babies compared against the average baby, measured each baby’s loss of walking reflex against a variety of other physical measurements specific to that baby. The culprit: chubby thighs. As babies put on weight, their leg muscles simply can’t power their legs anymore. The reflex returns as their legs strengthen.

According to Rose, the difference between the two research approaches is “aggregate, then analyze” versus “analyze, then aggregate,” with the former being more typical given our drive to compare ourselves and others against averages.

I’m not positive Rose’s is a great example of the problems of bias toward comparisons with normal. The original researchers” focus on an incorrect hypothesis (cognitive development versus physical development) left them trying to find a correlation to something unobservable. Still, Rose’s point is that had researchers focused on measuring, in each subject rather than across subjects, the relationship between the walking instinct and cognitive development, they may have discovered decades earlier the lack of a correlation.

More to come…

Ergodic Switch

Every measurement of cognitive skills or attributes, such as an academic exam or a personality assessment, is just a sample that, because of luck or fatigue or other factors, may or may not represent your true self. The only way to find out your true score would be for you to take the test a bunch of times and average your score.

But that doesn’t work in the real world because taking the test changes you. So instead a bunch of people take the test and we use the distribution of scores to statistically determine your likely range of scores.

This type of reasoning is ergodic. Ergodic theory, according to Todd Rose, allows you to “use a group average to make predictions about individuals if two conditions are true: (1) every member of the group​ is identical, and (2) every member of the group will remain the same in the future” (p. 63). Ergodic theory was developed by physicists to study the question of whether one could use the average behavior of a group of gas molecules to predict the average behavior of a single gas molecule.

Turns out, according to Rose, most gas molecules are not ergodic. Likewise, Rose cites the work of Peter Molennar to argue that people aren’t ergodic, either, throwing into doubt the validity of instruments like exams and diagnostic tests.

That’s interesting. More soon.

Design Flourishes

I designed a business simulation a few years ago where teams compete to be the most profitable. It is a simulation of a CPA firm, so they are competing to win the audit, tax, and consulting work of various local, fictional businesses. I named the businesses after jazz musicians I was listening to at the time, which was fun for me, but not something I expected anyone else to catch. I was delighted to receive an email recently from one of the instructors sharing the news that one of her students was a jazz piano minor in college and was very enthusiastic about the jazz references in the course. It enriched the course for him.

That made my day.

Way back at the beginning of my career, I was writing worskills exams for PLATO Learning (then TRO Learning) and I remember creating a whole little fictional universe with recurring businesses. It helped make development come alive for me, and I have to believe the more engaged the designer, the better the design, and in the end the better the learning.

At some level, elegant design is about small, delightful details, whether the details themselves get noticed or not (thought it’s great when they are noticed).

Learning Athlete, Part 2: Temperature

I hate freezing cold classrooms.

One of the areas of improvement we are looking at is whether we can gain better control over temperature in classrooms in the hotels and other venues we hold classes. First, we had to ask, what is the optimal temperature range for learning?

The team assigned to this couldn’t find research specific to learning and classrooms, but they did find research on workplace productivity. This metastudy talks about how productivity drops two percent for every degree Celsius outside of 21 to 24 degrees Celsius (70 to 75 degrees Fahrenheit). That’s a little warmer than I expected.

Of course, the age and gender of participants matters.

Then I got schooled by the folks at RSM who work with the hotels. Apparently, it’s not that hotels aren’t aware that their rooms are too cold or too hot; they just have less control than we realize. The way the HVAC systems are set up makes it often impossible to make all of their classrooms comfortable at the same time.

We decided to focus our efforts on making sure site staff are equipped to address concerns as honestly and realistically as possible. It’s no good to tell conference participants that the hotel staff has been notified and is working on it if we know darn well that the situation won’t change. Making someone believe that change is coming if it isn’t is worse than just leveling with them.

Acknowledge their concerns. Be honest about what’s been done and the prospects (or lack of) for change. Offer to get them a hot cup of coffee to hold. But don’t imply that it will get better because that will distract them from learning when it doesn’t.

Don’t Remind Me Later

I’ve had a Yahoo! mail account for a long, long time. I’ve always used it as the address I can give to commercial websites, forum logins, etc.; anything that could generate spam. Works great. I check it daily either through the web interface or through the Gmail app on my phone.

The past few months, every time I visit through the web interface, several seconds into the visit I get the following pop up.


I believe this is because I also have Yahoo! mail hooked up to the Gmail app. I tried using the official Yahoo! app a few years ago, but it was a battery draining mess. It may be better now, but I’m not inclined to try again and it works fine through Gmail. So every single time I click “I understand the risks.” Of course, it often appears in the middle of doing something else like opening an email, which then falls, so I have to reopen that email. First world problems.

What bothers me is that there is no “Don’t show this again” checkbox. This means Yahoo!’s strategy is to annoy me into doing what it wants. That’s no way to treat users. If they just want to disallow third party logins, fine, go ahead. If on the other hand this is a courtesy warning to help me, great, then leave me alone once I’ve confirmed I understand the risks.

I see this a lot in UI design, usually in the form of the interface asking me to do something and giving two choices: OK and Remind Me Later. There are certainly instances where this is the appropriate set of choices, such as if you are using a trial version of a piece of software. But usually this sort of thing is used to try and force me into doing things that they want me to do, like rate their app, rather than being reserved for things I logically need to do.

It takes a special disrespect for users to try and annoy then into doing what you want them to do. Take a stand, UI designers; the choices should be OK, Remind Me Later, and No.

Learning Athlete, Part 1: Breaks

We put together some internal working teams at my firm to address some learning-related questions that are sometimes posed to us. One of the teams is focused on energy management. Intuitively, what we eat and drink and how we sleep and feel affects our ability to learn, but framing this in terms of “energy management” was new to me–an interesting frame*.

We are looking at several energy management topics. One of them is breaks. Are more shorter breaks better than fewer longer ones?

The individual assigned to this question came back unable to find empirical guidance. It doesn’t seem there is a definitive answer (though my intuition says more, shorter breaks is better for learning, though if asked I suspect more learners would choose fewer longer breaks [or fewer shorter breaks to end the class earlier] so they can use the time more effectively to check in with clients or respond to emails).

To her credit, she quickly focused on what can be done from an instructional design perspective to get participants to move around and/or mentally shift gears, which should offer a mental refocus, not unlike a break. Makes sense to me.

I wonder, though, if there isn’t guidance or inspiration we can take from best practices around breaks as it relates to workplace productivity.

*I recognize that the practical impact of energy management is probably swamped by quality of the instructors, relevance of the instruction, instructional design, etc. This is strictly all-else-being-equal territory.

1912 Vision for Schools

I’m reading The End of Average by Todd Rose. Really interesting so far.

Early in the book he talks a little about the history of education in this country and the struggle between humanist education and Taylorist.

He includes a quote I enjoyed, from the General Education Board, funded by John D. Rockefeller, in 1912. I love the clarity of the message (if not the message!).

We shall not try to make these people or any of their children into philosophers or men of learning or of science. We are not to raise up from among them authors, orators, poets, or men of letters. We shall not search for embryo great artists, painters, musicians…nor lawyers, doctors, preachers, politicians, statesmen, of whom we have ample supply….The task that we have set before ourselves is very simple as well as very beautiful…we will organize our children into a little community and teach them to do in a perfect way the things their fathers and mothers are doing in an imperfect way.