I’ve always wanted to use experience sampling method as a way to study the real time impact of instruction, particularly synchronous webcasts. Experience sampling method is closely associated with Mihaly Csikszentmihalyi, who used it study how and when people experienced flow, which is the condition of being absorbed by whatever they are doing. In the old days, subjects would carry pagers and whenever they went off, the subject was to record their state of flow and the activity they were engaged in.
I’ve always been intrigued by the possibilities of using something like this during a webcast to provide information to the instructor about how his or her learners are responding, since they can’t see them like classroom instructors can. When are learners engaged? When are they tuning out? Two things have kept me from ever trying experience sampling methods during webcasts.
One is just the logistics and technology. Sure, experience sampling method would be possible with pagers or, more recently, apps, but it would be much more convenient to have it built into webcast tools like WebEx. Some tools have experience reports that you can pull after the webcast to show at any given moment how many participants had clicked away from the program window so that it no longer had focus. This approach is imperfect–just because the window doesn’t have focus doesn’t mean I’m not paying attention–but interesting, though unfortunately unavailable on any of the tools I’ve used. In any case, a more active form of experience sampling would be even better.
The other barrier is that I’m uncomfortable interrupting learners during a learning experience. Flow is fragile, and it’s possible that distracting the wrong learner for the wrong moment could cause the firm harm. So experience sampling method as Csikszentmihalyi practiced it is probably too invasive.
I was intrigued to read in Thinking, Fast and Slow by Daniel Kahneman about the day reconstruction method, which asks subjects to recreate after the fact when they were in flow and when they weren’t, and why.
I tried a primitive form of this in graduate school and found that participants had trouble stating with any precision what they felt and when, but perhaps Kahneman has elevated this method of sampling to increase its sensitivity. Definitely something to keep in mind.