Every measurement of cognitive skills or attributes, such as an academic exam or a personality assessment, is just a sample that, because of luck or fatigue or other factors, may or may not represent your true self. The only way to find out your true score would be for you to take the test a bunch of times and average your score.
But that doesn’t work in the real world because taking the test changes you. So instead a bunch of people take the test and we use the distribution of scores to statistically determine your likely range of scores.
This type of reasoning is ergodic. Ergodic theory, according to Todd Rose, allows you to “use a group average to make predictions about individuals if two conditions are true: (1) every member of the group is identical, and (2) every member of the group will remain the same in the future” (p. 63). Ergodic theory was developed by physicists to study the question of whether one could use the average behavior of a group of gas molecules to predict the average behavior of a single gas molecule.
Turns out, according to Rose, most gas molecules are not ergodic. Likewise, Rose cites the work of Peter Molennar to argue that people aren’t ergodic, either, throwing into doubt the validity of instruments like exams and diagnostic tests.
That’s interesting. More soon.