In a classroom at Madison East High School, Tom Kanies meets with half a dozen teachers. With laptops out and color-coordinated spreadsheets open, the group is ready to start its weekly discussion on how their ninth grade students are performing.
But in this discussion, the numbers do the talking.
At East, teams of teachers meet weekly to discuss datasets prepared by on-track coordinators like Kanies and a team of administrators from the Madison Metropolitan School District to help identify students who are struggling, especially on the ninth-grade-on-track benchmark. Freshmen who have at least a 90 percent attendance rate and 3.0 grade point average, and no more than two failing grades, are considered on track.
MMSD under Superintendent Jennifer Cheatham has implemented more ways to help track data in real-time. By identifying trends sooner, teachers can make adjustments more quickly, instead of waiting for grade reports and standardized test scores to come out. Using data, the district wants to quickly identify how best to help students succeed and buck the trend of a persistent achievement gap.
“Data use for us is one method of fueling our inquiry. We are constantly trying to figure out how to serve our students better,” said Cheatham, who has led the district since 2013. “It’s never about data use in isolation.”
Tracking data in schools has an expansive history dating back to the “A Nation At Risk” report in 1983, which found that American public school students significantly trailed their counterparts in other industrialized nations across most subjects, according to Wisconsin Department of Instruction spokesman Tom McCarthy. Further developments in the use of data in schools came with the No Child Left Behind Act in the early 2000s, renewal of the Americans with Disabilities Act in 2004 and the Every Student Succeeds Act in 2015.
The federal stimulus package in 2009 gave states the authority to create data systems that track student information across multiple years in order to make data-informed decisions, according to DPI’s data division team leader Melissa Straw. DPI launched several databases for districts to have access to data that gets collected for state and federal reporting requirements.
In Madison, the big push for using data has come in the last decade.
“I’ve seen a big difference in the last 10 years for sure, but in the last four years in particular, it’s really ramped up,” Kanies said. “In the past, we had some individual teachers tracking data individually, but it wasn’t as accessible as it is now.”
The teacher teams at East use Data Dashboard, an online portal developed by the private software company Hoonuit, to track grades, attendance records and behavioral data.
In 2011, the high school started a Freshmen Academy that focused on helping students make the transition from middle school more successful. It’s evolved and matured since MMSD focused on the ninth-grade-on-track benchmark, including the devoted time teacher teams have to analyze the data and adjust their instruction.
Recent research shows that success in ninth grade is extremely predictive of future success. Students who are on track after their freshman year are 3.5 times more likely to graduate on time, according to MMSD.
And at East, absenteeism has been a problem. About 18 percent of students over the course of three years were “chronically absent,” according to the school’s state report card, meaning they were absent at least 16 percent of the time.
East uses incoming students’ eighth grade attendance and grades to sort the students into four categories: high opportunity, opportunity, vulnerable and high risk. Kyle Mayne, a special education teacher at East, said that some weeks he’s focused on students who might just be under the 3.0 grade point average to help them realize the importance of being on track for graduating.
“Having these elaborate teacher teams built into our schedules allows us to zero-in on what we want to look at during a specific week,” Mayne said. “The key to using data for us has been having the time to discuss and reflect on what we’re doing in the classroom.”
Other weeks, the focus of teachers has been on students who had high grades and attendance records in eighth grade but have seen a significant drop off in high school.
“We can then work with students to figure out what’s going on. Sometimes it’s helping students find out ways to open up communications with their teachers or getting them to learn how to organize better,” Mayne said. “Sometimes a student misses a day of school and they don’t know how to get their work for the day they missed and how to catch up.”
Teachers have had to adjust to learning more about how to implement data in their instruction, according to East principal Mike Hernandez.
“Getting a bunch of numbers can be scary if you’re not confident in looking at it,” Hernandez said.
Hernandez said that some of the real-time data school leadership looks at has helped teachers figure out what courses the school should and shouldn’t offer.
Part of getting teachers on board has been creating a culture around valuing data, according to Kanies.
“We’ve all sort of become data geeks through this process,” Kanies said. “At ninth grade at least, there’s been a real coordinated effort. But in order for these efforts to succeed there needs to be a school culture around data, and I love East culture’s around data.”
MMSD uses a slew of different data systems beyond Data Dashboard. The district started using the private software system eduCLIMBER this school year to track behavioral data, for example. Teachers also use the private software programs OASYS and Infinite Campus to track special education data and student grades, respectively.
Michael Jones, a special education teacher at Black Hawk Middle School, noted that having data spread out across different databases has been a challenge.
“It’s something that’s not necessarily the district’s fault since there are different places for this data because of proprietary reasons, but you end up having to go from one data warehouse to another often,” Jones said.
Andrew Statz, the district’s data coordinator, said data from the different programs are fed into a data warehouse each night, which then goes into Data Dashboard for educators to use.
“A lot of the programs are really good at one specific thing, which is why we have to use different programs,” Statz said. “We recognize that we have a responsibility to train users and keep them updated on changes to the programs we use.”
District officials have emphasized that ninth-grade-on-track data offer a broad view for principals like Hernandez to see what is happening in the classrooms and adjust to the trends that are developing.
“How teams of teachers use formative data like student grades and work to inform these faster cycles of inquiry is really at the heart of the work that teachers do,” Cheatham said. “So if you’re going to talk about data use in MMSD, you can’t decouple that from the work we’re doing to strengthen teacher teams.”
Much of the data efforts used in MMSD revolve around predictive analytics, according to UW-Madison School of Education professor Rich Halverson.
“Predictive analytics is where you try to use records of student performance to predict where they’re going to be so you can reach out to students and intervene,” said Halverson, who serves as the associate dean for innovation, outreach and partnerships.
Some districts in the state have tried to use personalized learning as an approach to using data, according to Halverson.
In a personalized learning environment, teachers work with a student to gauge a student’s interest in a certain topic, what the order of the lessons should be and how a student will show they mastered the material.
“It’s much more about instruction than diagnosis, and it changes the way that teaching happens,” Halverson said. “It’s quite controversial in some ways because if it’s not done right, it’s not effective. It’s not necessarily better or worse, but a different way that some districts have used data to do instruction.”
Statz said use of the data tools available to teachers has increased and that questions about how to use the data have become more advanced.
“They have gone from, ‘How do I export results or set a filter?’ to ‘What else can we add to this report to make it a more complete picture?’” Statz said.
But that complete picture also includes data that can’t be controlled by educators.
“We do know that ninth grade is really important for success, but my guess is that still the most predictable data point on academic success is socioeconomic status,” said Tim Slekar, the dean of Edgewood College’s School of Education. “You can’t ignore the socioeconomic situations that families and children live in if you’re going to be worried about educational attainment.”
Cheatham noted that making data-supported decisions doesn’t end with just the data.
“We’re going far beyond using the typical datasets to make decisions,” Cheatham said. “We’re talking about the fine-grained information … I actually think that’s how you start to change some of the predictability around outcomes — getting beyond the big datasets and getting into real conversations about real students by name in a very caring and holistic way.”
MMSD and districts across the state are facing the challenge of building datasets that measure qualitative aspects of a student’s life: data beyond quantitative data such as test scores and GPAs.
Social and emotional learning, which refers to how children manage emotions and build positive relationships, is an example of qualitative data that has been hard to measure.
“The practices of measuring social and emotional learning are not well articulated in research yet,” Halverson said. “There’s not a widely shared metric people can use.”
At East, English teacher Gloria Kok has students write journal entries regularly and work on other assignments that get them reflecting about their personal lives, so she has a better idea of who they are. That informs how she might address a particular srudent’s situation.
“Part of that is the beauty of being an English teacher, but I’m constantly trying to get qualitative data where I can,” Kok said.
Another piece of qualitative data being collected is when students are asked if they have a trusted adult at the school who they can talk to, and who that person is, according to Kanies.
Cheatham stressed that though the field is evolving around social and emotional learning data, the district wants to find ways to incorporate more perspectives on students’ social and emotional needs.
“Across the country, I think people are asking themselves, how can we do a better job of balancing the kinds of information we compile to have a more well-rounded picture of how students are doing in school,” Cheatham said.
Del Underbakke, a retired East teacher who worked in the district from 1975 until 2016, notes the district needs to focus on the social and emotional learning components to make a dent in the achievement gap and meet the needs of students who are identified as at-risk.
“Data Dashboard accomplishes compliance and provides numbers for the district report card,” Underbakke said. “That data is too generalized and too averaged to meet the needs of individualized students who have individual needs who need customization in order to move forward.”
Underbakke said data such as attendance and behavior incidents, while useful, don’t always give teachers and administrators the full picture.
“Just because someone has a chronic illness and they don’t show up to their doctor’s appointment — that data wouldn’t touch on why they didn’t show up,” Underbakke said. “The numbers on how many days someone doesn’t attend school really doesn’t get to why that person isn’t attending school, and other data needs to be collected to get that information.”
Underbakke has spent much of his time in retirement advocating for more resources and focus on at-risk youth. MMSD identified 1,348 students as at-risk during the 2017-18 school year, according to district records. East had 213 students identified as at-risk, the most of the city’s four comprehensive high schools.
“A big part of the discussion surrounding data has to be responsibility of not only gathering data, but gathering data that results in kids making progress,“ Underbakke said. “That’s not a common discussion that goes on.”
Mayne, the East teacher, said the school has emphasized building meaningful relationships and learning partnerships with students to help adjust their instruction.
“One big intervention we’ve had in our learning partnerships is bringing a student who might have shown a consistent trend of being disengaged in to attempt to make them feel more invested in the classroom environment,” Mayne said. “We see them engage more when we have them feel like they were part of building a lesson or part of looking at why a lesson didn’t succeed.”
The district formed the Office of Youth Re-engagement this school year to help find ways to get students identified as “opportunity youth” more engaged in school, which is headed by former La Follette assistant principal Paris Echoles.
Cheatham noted that the work of Echoles and Ricardo Jara, the superintendent’s chief of staff, has pushed the district’s thinking on what data use can look like.
Jara last year conducted countless interviews with students who qualify as opportunity youth to learn about their experiences. And that, Cheatham said, is the kind of important data the district can focus on.
“What we’re learning is how to use both quantitative and qualitative data to find out what is happening holistically with a student — and in this case, forming that picture with the student,” Cheatham said. “It’s a really powerful picture of what high quality data use can do. It’s not just about numbers on a spreadsheet.”
Humans, not numbers
Nearly everyone interviewed for this story noted the importance of remembering that data points are not simply numbers, but the lives of kids. By more thoroughly combining both student-level and system-level approaches, data could continue to play a larger role in how MMSD serves its students.
“There’s a tendency in predictive analytics to sort people into groups that they don’t necessary belong to,” Halverson said.
Halverson noted that data in law enforcement and marketing sometimes profiles people, which can happen in predictive analytics if data isn’t collected properly.
Additionally, educators can, like the teams at East, tap into teachers’ valuable insight from being in the classroom to analyze the data, as it sometimes confirms what they’re seeing already.
“(Teachers) can often already see if students are misbehaving or are not being engaged,” Halverson said. “It’s best to combine the real-world experiences and the data system signals so leadership in schools can really see what students are struggling with and how they’re succeeding.”
This combined approach is what the district hopes to accomplish in the future, which could include newer ways to visualize the data.
“There are a lot of emerging practices that could help convey individual students’ stories within the big picture,” Statz said. “We are interested in the humanization of data where a student’s experience generates data that tells a story about that student as well as tells a story about the whole.”
Using a more holistic approach will likely be what’s key to using data effectively, instead of just letting the system-wide numbers do the talking.
“If teacher teams are looking at data just for the sake of looking at it ... then it’s a waste of time and we’re not doing it right,” Cheatham said. “(Using data) has to be part of the cycle of inquiry, and that’s what we’re trying to accomplish.”