In the News

Schools turn to data to improve academics

By ROB ANTHES, Hamilton Post

Note: This is the second of four stories on the Hamilton Township School District’s improvement plan. A piece on the expanded school day for the township’s middle and high schools appeared in the November 2011 edition of the Post. Stories on two other measures will be published in the coming months.

Ever since their employer landed on the state’s Districts in Need of Improvement list last year, officials at the Hamilton Township School District have been working hard to turn the bad news into positive reforms.

School district administrators identified three major needs—increased parent involvement, more time on-task and a shift to data-driven instruction—the district needed to address, and coupled them with a reform of the district’s oft-criticized hiring practices.

The good news: Students in Hamilton hit important standardized test benchmarks in 2010–11 that promoted the district from District In Need of Improvement status—a stigma placed on just 10 percent of districts in New Jersey last school year—to “on hold” status. Another year of adequate test scores will clear the district of any DINI designation. And the programs meant to address the district’s needs are just starting to come online this year.

But there’s a catch. Education experts are at odds about whether one of the initiatives—data-driven instruction—will actually improve performance in the classroom.

The term “data-driven instruction” can mean any instruction method that uses data specific to the district or a school, class or student. In Hamilton, teachers will use students’ performances on district-developed standardized tests to gather information about the students, and use that information, in theory, to not only predict how each student will perform on state standardized tests but also develop lessons to help the students perform better in school.

However, critics like Columbia University professor Aaron Pallas debate whether data-driven instruction actually helps children.

“There’s a lot of interest and excitement in using data to drive classroom instruction,” said Pallas, a professor of sociology and education at Columbia’s Teachers College. “There’s very little evidence to point to an increase in academic achievement.”

Various members of the district administration emphasized they decided to pursue data-driven instruction only after doing some research of their own. The district studied the work of education expert Douglas Reeves and other well-known educators in the field of data-driven instruction. Interim superintendent James Sheerin said the district made an effort to ensure it uses only methods backed by research.

It also developed a protocol for how each school should use data, as well as objectives for each school and benchmarks that are common to all schools. The tests and the standards are aligned with the state’s standards for language arts and mathematics, said Lois Braender, the district’s director of elementary education.

After taking an initial benchmark test, each student’s answers are recorded and put into a system called a data warehouse, which allows teachers to see all of their students’ responses to each question. The district hopes these reports will help teachers identify students’ weak areas and generate ways to work with individual students on improving performance. In other cases, it can identify students performing above grade level and generate activities that will challenge those students.

“This is the type of data that every teacher has to be accountable for,” said Michael Gilbert, the district’s director of secondary education. “It’s not just what did we teach today, how did we teach it, how are we grading it. It is driving right down to the proficiencies and skill level for every student.”

The district began testing the method last school year, using a state-purchased system called Learnia. When the state announced last year they would no longer pay for the system, district administration looked at 11 new data warehouses. After analyzing each, they selected a system called LinkIt because it would give the district the most information on its students while also giving teachers tools to use the data, Braender said.

The school board approved the LinkIt purchase July 27, and schools began using it this school year. Elementary students took district-developed benchmark tests in math and literacy in the fall. Braender said the high schools will start using the system this month.

LinkIt can also suggest lessons and worksheets, as well as assist teachers in making tests and tracking results and student progress. Essentially, LinkIt can do everything, and the district administration wants teachers to use those tools to drive instruction.

It is a piloted program, so some schools are ahead of others, Braender said. But the district administration believes, in the end, every student and staff member will benefit from data-driven instruction.

“Now that we have a data warehouse and a data management system, we have a centralized electronic capability that enables our staff, in an automated fashion, to retrieve information in a simple manner,” district testing evaluation and data systems specialist William Paul said. “That really is helping us be more efficient and provide the information we need to address what the student needs. All the initiatives ... they’re all based on solid research.”

However, Pallas said there aren’t many good studies on data-driven instruction, and the studies out there are lukewarm on the approach. The issue of improving flagging performance may be better dealt with by changing curriculum, he said.

Drew Gitomer, a former researcher at Educational Testing Service and chairman at Rutgers University’s Graduate School of Education, took a less hard-line stance, pointing to the importance of data to decision making in the business world.

“Most reasonable people would say it’s a sensible thing to use data,” he said. “But it really comes down to how these things are implemented.”

(Both professors were asked about data-driven instruction in general, and did not comment directly about Hamilton’s plan.)

Gitomer seconded Pallas’ opinion that altering curriculum may prove more effective, saying standardized tests are good for finding trends or patterns in a large population but they don’t do a good job finding deficits in particular skills because they are given in a high-stress environment where students must answer a lot of questions in a short period of time. There are also the cases of students who excel in the classroom but, for one reason or another, do not test well.

Pallas worries that data-driven instruction could lead to the narrowing of American students, especially if it winds up as the primary tool for school improvement. More simply put, it could lead to teachers focusing overwhelmingly on the subjects addressed on the standardized tests.

He did concede that teachers could find “blind spots” in their methods by finding trends in their class’ test scores, but maintained that shaping instruction based on test scores is an “ambiguous and ambitious task.”

This is especially true, he said, because the people who are in charge of data often are faced with a mess of information and aren’t trained properly to make sense of it or select which information is important and relevant.

“Test scores are hard things to make sense of,” Pallas said.

Hamilton Township School District has hired seven teacher data coaches to try to combat this issue. There is one special education coach, three elementary school coaches and three middle/high school coaches.

Braender said the district certainly could use more teacher data coaches, but the current ones have been working hard to ensure folks at the schools know how to work with the data collected from assessments. The coaches hold meetings at each school and work with teachers so they can use the data to improve their instruction in the classroom.

“We’re teaching them how to use the system,” district language arts supervisor Kelli Kish said. “We’re teaching them how to look at data, how to plan instruction from the data, and we’re also following through with them. So if the teachers find they need to work on something in a specific area, the coaches will come back in and give their ideas for working with the students in a classroom.”

Kish said teachers often go to a one- or two-day workshop to learn new methods, and may never fully grasp what the workshop taught. The teacher data coaches, on the other hand, are always there to meet with a teacher about a question or to reinforce a point.

“The coaches are really there to support them,” Kish said.

Gitomer said districts turning to data-driven instruction need to offer that kind of support, but also need to be careful about relying too heavily on test scores as a mode and indicator of improvement.

“What has happened in education is there has been a very large amount of attention placed on test scores,” Gitomer said. “But they have limited ability to answer all the questions. The tests give you a general sense of a student’s achievement level. There’s other data that has to be collected to determine what the scores really mean.

“If I was a [school] board member, I would ask, ‘What other data are we collecting to understand the data we have?’ That’s going to tell us what actions we can take. [Using test scores is] a good step. It tells us something important. But it doesn’t make much sense if that’s all we’ve got.”