As educators, we talk about data, collect data, wade through data, analyze data, and draw conclusions from data that hopefully demonstrate how and why our interventions led to the achievement of our goals. But sometimes there seems to be so much data, so many things we could measure, that it’s difficult to know where to start.
Burying one’s head in the sand – i.e., not planning for the appropriate collection and use of data to drive decision-making – is clearly not the answer. But where to begin? In a guest blog post for ASCD, 30-year educator, administrator and author Craig Mertler shared his top five ways to achieve strategic data use in planning and decision-making. We’ve adapted them here:
1. Find your focus. Planning starts with identification. Mertler suggests zeroing in on a specific “problem of practice” that you want to improve or otherwise address and using that to brainstorm about the types of data you may wish to collect.
2. Keep data collection manageable. We’ve shared elsewhere that we typically encourage clients to “keep it simple,” with the caveat that this does not always mean keeping things “light.” Engaging in a reflective process while constantly focusing on your problem of practice will help you narrow the possibilities for data collection. We also encourage you to think ahead to what the reporting requirements will look like for each measure you’re considering. Can you structure data collection efforts so that it will be easy for participants to report at a level that helps you inform future decision-making?
3. Be strategically forward-thinking, and value data sources appropriately. It’s important to stay focused, but it’s also okay to track more data than you can immediately process. In fact, there will likely be future instructional cycles and decision-making opportunities associated with similar problems of practice toward which this additional data can be applied, and you may be glad you collected it across earlier cycles. Even large-scale data like state accountability measures, suggests Susan Brookhart, a former teacher-turned-author, can be instructive if used appropriately (e.g., in the case of state assessments, to raise questions “at the 30,000-foot level”).
4. Seek patterns, trends, and outliers. We touched above on just how much data is out there, and school systems are by no means exempt from sometimes frenzied-seeming collection efforts. Mertler suggests starting your analysis of any data set by identifying patterns or trends (e.g., are there problems, items, or tasks that many students answer or perform incorrectly?) and outliers (e.g., are there instances where only a couple of students make mistakes, but the mistakes are the same or are similar in nature?). This assumes that you have access to detailed results and not just summary proficiency levels, and it can help you suss out specific aspects of your intervention that should be targeted for replication, change and/or improvement.
5. Remember that data-driven decision-making is a cyclical process. The good news, Mertler reminds us, is that you rarely need to solve all of your problems of practice in one round of decision-making. What you learn in one cycle can be used to inform the progress and instructional improvements you make in future cycles, and the continual refinement will enable you to hone your data collection and reporting efforts across time as well.