If people aren’t ergodic, that means in theory exams and other cognitive assessments have no predictive value for individuals, only for populations. Intuitively, of course, that’s not true. If I needed a Spanish translator, and all I knew from among the candidates was their grades in their most recent Spanish class, I’d pick someone with high scores. That wouldn’t guarantee success, but I’d expect it to correlate with success.
The example Rose uses is instructive. If you hold up a newborn so that his or her legs are dangling, he or she will move his or her legs in a walking reflex. This walking reflex disappears after a time, only to reappear later. Researchers hypothesized that the suppression of this walking instinct is related to cognitive development.
They measured the point at which babies on average lose the walking instinct and compared that average to cognitive development known to happen in that same timeframe and linked the two.
Some doctors, then, naturally, would express concern to parents if their baby didn’t lose their walking reflex in that timeframe. It became a symptom of delayed cognitive development.
That is, until a researcher came along and instead of worrying about how babies compared against the average baby, measured each baby’s loss of walking reflex against a variety of other physical measurements specific to that baby. The culprit: chubby thighs. As babies put on weight, their leg muscles simply can’t power their legs anymore. The reflex returns as their legs strengthen.
According to Rose, the difference between the two research approaches is “aggregate, then analyze” versus “analyze, then aggregate,” with the former being more typical given our drive to compare ourselves and others against averages.
I’m not positive Rose’s is a great example of the problems of bias toward comparisons with normal. The original researchers” focus on an incorrect hypothesis (cognitive development versus physical development) left them trying to find a correlation to something unobservable. Still, Rose’s point is that had researchers focused on measuring, in each subject rather than across subjects, the relationship between the walking instinct and cognitive development, they may have discovered decades earlier the lack of a correlation.
More to come…