Asked at a conference what I thought was the best book on education research I’d read recently, I was quick to answer, “Moneyball.” Moneyball? But that’s a baseball book! Well, yes and no. Michael Lewis’s story tells how Oakland A’s General Manager Billie Bean got the lowest payroll baseball team in America to challenge the American League record for consecutive wins; the A’s went on to repeated success by dispensing with preconceived notions of what makes for a good baseball player and letting comprehensive data analysis inform their decision making throughout the organization.
.” Moneyball? But that’s a baseball book! Well, yes and no. Michael Lewis’s story tells how Oakland A’s General Manager Billie Bean got the lowest payroll baseball team in America to challenge the American League record for consecutive wins; the A’s went on to repeated success by dispensing with preconceived notions of what makes for a good baseball player and letting comprehensive data analysis inform their decision making throughout the organization.Many of the insights offered in the book are good re-tellings of the classic writings of baseball statistician William James. Here is just a sampling of insights from James that can be applied to education research:
“One absolutely cannot tell, by watching, the difference between a .300 hitter and a .275 hitter.” The difference between a good hitter and an average hitter is simply not visible — it is a matter of record.”
Educators, like baseball fans, are convinced that they know good teaching when they see it, that they can assess student ability and performance without systematic measurement, and that careful observation and understanding of individual student and teacher differences must be taken into account. Of course there is some truth to the latter, but there is so much to be learned over time that even semi-frequent snapshots of the classroom experience (alone) are likely to be inadequate.
“When the numbers acquire the significance of language, they acquire the power to do all of the things which language can do: to become fiction and drama and poetry” [quoting Bill James]. Or, as Lewis notes, “The statistics were not merely inadequate; they lied.”
As in baseball, so in education, we must be watchful to assure that the statistics we gather and the analyses we pursue tell true stories of what is significant in the processes and the outcomes they produce. This comes from not only assuring that the measures we use are gathered without bias, but that they are in fact the appropriate measures in the first place!
“The problem is that baseball statistics are not pure accomplishments of men against other men, which is what we’re in the habit of seeing them as. They are accomplishments of men in combination with their circumstances.”
To the extent that we fail to account for the overall circumstances of our students, teachers, and schools, we will miss important lessons, and may ultimately draw incorrect conclusions about effective instructional strategies. For example, I recently toured the most well-regarded school in a local school district, where I’d considered placing my son. Standardized test scores were higher than the district average; the school was neat and orderly; the teachers followed the curriculum. And the students were practically falling asleep at their desks out of sheer boredom! It was painful to watch. The Assistant Principal went on about how their test scores and successful secondary school placements outperformed the rest of the district. But it was a magnet school, with less then half the district average for students receiving Free or Reduced Price Lunch, and it more closely resembled local private schools in the composition of the student body than any other school in the district. With all that taken into account, it should have been performing better than it was. (And it could, with more engaging curriculum and instruction.)
The lessons to be drawn are too numerous to detail. What do good teachers (fielders?) do that we don’t or can’t measure? How do we tell the difference between a few well executed lessons (at-bats?) and a teacher’s long term effectiveness with students (on-base plus slugging?). What is the optimal mix of talents among a team of teachers in a school, family, or grade level? What are the most effective coaching strategies for school principals and leaders? What is the best mix of student talents and experiences for the overall learning of a group of students? Insight into these and other questions can be found here. It’s a good read…if you like baseball.