Some dark thoughts from Geoff Norman in one of his iconic editorials:
In some of my darker moments, I can persuade myself that all assertions in education (a) derive from no evidence whatsoever (adult learning theory), (b) proceed despite contrary evidence (learning styles, self-assessment skills), or (c) go far beyond what evidence exists. I suspect most readers of AHSE are aware of the first two kinds of assertion, but in this editorial I want to elaborate on the third, the challenge of arriving at general conclusions about the way the world works based on the empirical evidence derived from limited studies.
The link between a particular study, and generalisability is of course what science is all about: this is why much science is about theory, rather than A/B testing. It is why RCTs are not usually about science, but about product comparisons.
Some Much medical education research is hurtling into the abyss of studies that have little interest beyond the fact that they got published, and that somebody has a copy of SPSS.
At this stage I always muse over out Clark Glamour’s statements about departments of education:
‘We leave training to schools of education, the bottom feeders of academe’,
‘Almost all advanced degrees held by teachers are in education, which means that they are academically rock bottom’
I could feel guilty for sharing such thoughts, if it were not for the fact that Glymour is a leading figure in statistics, learning and philosophy; and one who has made major contributions to how we can envision learning (see for instance his wok on Bayes Nets and Causal thinking), or his work on statistics teaching and learning, described in Galileo in Pittsburgh).
A long time ago I enjoyed a book called ‘What is this thing called science‘. The one on medical education has yet to be written. Norman’s voice is the closest we have to it. Somehow we have to carve out a space between the sorts of biology that explain human psychology and culture that the Premacks describe (see for instance, Original Intelligence: The Architecture of the Human Mind); George Steiner’s, Lessons of the Masters; and the knowledge of the bazaar that reminds us that we have understood certain aspects of apprenticeship and craft learning for at least a thousand years. And finally, the sorts of deep insights based on experiments that often appear deceptively simple, that some areas of experimental psychology are well known for.