When asked publicly or privately about high stakes assessments for teachers and schools, we always say the same thing: don’t go there. Using value-added models based of student test scores to reward or punish teachers misdiagnoses educator motivation, guides educators away from good assessment practices, and unnecessarily exposes them to technical and human testing uncertainties. Now, to be clear, we do use and value standardized tests in our work. But here’s a 10,000-foot view of why we advise against the high stakes use of value-added models in educator assessments:
- Using value-added scores to determine teacher pay misdiagnoses teacher motivation.
When Wayne Craig, then Regional Director of the Department of Education and Early Childhood Development for Northern Melbourne, Australia, sought to drive school improvement in his historically underperforming district, he focused on building teachers’ intrinsic motivation rather than the use of external carrots and sticks. His framework for Curiosity and Powerful Learning presented a matrix of theories of action that connect teacher actions to learning outcomes. Data informs the research that frames core practices, which then drive teacher inquiry and adoption. The entire enterprise is built on unlocking teacher motivation and teachers’ desire to meet the needs of their students. (more…)
Continue readingOur clients are by and large genuine change-makers, motivated to measure and achieve the positive outcomes they seek. And one of our most important jobs is helping them develop and use appropriate data to enhance discovery, analysis, insight and direction. But client commitment and our professional responsibility don’t always avoid some common data collection pitfalls.
Through countless evaluations of school and district level educational programs, as well as multi-site, statewide initiatives, we have identified the pitfalls that follow. They may seem like no-brainers. But that’s what makes them so easy to fall into, even for seasoned evaluators and educational leaders. We highlight them here as a reminder to anyone looking to accurately measure their impact:
1. Asking leading a question versus a truly open-ended one. If you aim for honesty, you must allow respondents to give negative responses as well as positive ones. For instance, asking:
“How would putting an iPad into the hands of every student in this district improve teaching and learning outcomes?”
…assumes teaching and learning outcomes will be improved, at least to some degree. (more…)
Continue reading
Asked at a conference what I thought was the best book on education research I’d read recently, I was quick to answer, “Moneyball.” Moneyball? But that’s a baseball book! Well, yes and no. Michael Lewis’s story tells how Oakland A’s General Manager Billie Bean got the lowest payroll baseball team in America to challenge the American League record for consecutive wins; the A’s went on to repeated success by dispensing with preconceived notions of what makes for a good baseball player and letting comprehensive data analysis inform their decision making throughout the organization.
.” Moneyball? But that’s a baseball book! Well, yes and no. Michael Lewis’s story tells how Oakland A’s General Manager Billie Bean got the lowest payroll baseball team in America to challenge the American League record for consecutive wins; the A’s went on to repeated success by dispensing with preconceived notions of what makes for a good baseball player and letting comprehensive data analysis inform their decision making throughout the organization.Many of the insights offered in the book are good re-tellings of the classic writings of baseball statistician William James. Here is just a sampling of insights from James that can be applied to education research: (more…)
Continue reading