I’ve been reading Ruth Colvin Clark’s Evidence-Based Training Methods. Good book.
Effective use of end-of-course evaluations has been much on my mind lately, so I was very interested in Clark’s reference to a Sitzmann et al (2008) assertion that measures of self-efficacy are the best indicators of achievement of the learning objectives (short of actual testing).
Self-efficacy is our own perception of our competence at a given skill. High self-efficacy is synonymous with self-confidence that you can achieve a particular objective.
I went out to the Sitzmann article to see if it had more specific guidance. The article turned out to be a meta-analysis of level one evaluation. And, indeed, they conclude that the only elements on end-of-course satisfaction surveys that actually correlates to learning are self-efficacy questions. That’s useful to know, and my next question was: What makes a good self-efficacy question?
I turned to Bandura (2006). He presents this form:
The attached form lists different activities. In the column Confidence, rate how confident you are that you can do them as of now. Rate your degree of confidence by recording a number from 0 to 100 using the scale given below:
Cannot do at all (0)
Moderately certain can do (50)
Highly certain can (100)
Bandura recommends a 100 point scale because it is more sensitive and reliable than a smaller scale. He recommends at least not going below a 10 point scale.
The way it is laid out by Bandura would require customization for each course as you’d have to list out all the objectives and have learners rank their self-efficacy in each one. In a higher volume environment, that may not be practical. Clark suggests as a generic alternative: “Rate your level of confidence to apply the workshop skills to your job.”
I’ll definitely be thinking more about this in April, which is when we traditionally take a look at our evaluation templates for the coming year. It would be really interesting to correlate self-efficacy ratings with quiz scores to attempt to replicate Sitzmann et al’s findings.
Bandura, A. (2006). Guide for constructing self-efficacy scales. In F. Pajares & T. Urdan (Eds.). Self-efficacy beliefs of adolescents, (Vol. 5., pp. 307-337). Greenwich, CT: Information Age Publishing.
Sitzmann, T., Brown, K.G., Casper, W.J., Ely, K. & Zimmerman, R.D. (2008). A Review and Meta-Analysis of the Nomological Network of Trainee Reactions. Journal of Applied Psychology, 93 (2), 280-295.