Think Recent Test Scores Show Schools are Failing? Think Again!

Over the past month, media outlets across the country have been abuzz with stories about student test scores. In February, the U.S. Department of Education released the 2005 National Assessment of Educational Progress (NAEP) scores for twelfth graders, and the country was astounded to find out that the number of students achieving “proficiency” in reading has dropped significantly since 1992, even while high school GPAs have been on the rise. The next week, the U.S. Chamber of Commerce released a state-by-state “report card” that graded every state on, among other things, fourth and eighth grade NAEP scores (this “report card” has also received strong criticism for the methodologies it used).

Reports such as these two are given wide coverage because they are shocking and straight-forward, but in reality they have very little substance. People interested in “grading” public education should be looking elsewhere for measures of success. New Jersey, which has been in the news recently over Governor Corzine’s school funding proposals, provides an instructive example of what people should actually be talking about when judging the success of schools.

Test Scores Are Simplistic and Misleading

Average test scores, despite the play they get in the media, have very little meaning as measures of student achievement, because they can hide gains made by increasingly large minority populations. This is known as Simpson’s paradox, which is explained here. For example, the twelfth grade NAEP data for English language learners showed a slight increase in the last 15 years (though not by a statistically significant amount). ELL students, who tend to score lower, make up a much greater percentage of students today than in 1992, and thus have the effect of “bringing down” the average despite stable or improved performance.

As Michael Martin, a research analyst at the Arizona School Boards Association, has noted, today’s population of students are much harder to teach. Minority populations, English language learner populations, and populations of students from single-parent or foster homes have all increased. In addition, more students today are in college preparatory programs; many of the students who are today achieving lower scores would have, in past decades, been tracked into non-college preparatory classes such as vocational education.

Finally, Martin also notes that graduation rates for students of all races and backgrounds are significant higher today than in previous decades and are increasing. Lower twelfth grade test scores, he argues, are the result of our successes in keeping students in school who would otherwise have dropped out. We are looking at the wrong measures of success, he argues, and we are therefore drawing the wrong conclusions.

Better Measures of Success

In late February, New Jersey Governor Jon Corzine released his proposed budget for next fiscal year – a budget that included $328 million in additional school aid. This sum included $125 million more for the state’s 31 “Abbott districts” – New Jersey’s poorest districts that, since the 1998 state Supreme Court decision in Abbott v. Burke, have received the bulk of New Jersey’s school funding increases. However, for the first time since 2000, Corzine’s proposed budget contains significant increases in the funding for other low-income and middle-class districts. This new aid, Governor Corzine has said, will be targeted to expand to many new districts the programs that have been implemented in the Abbott districts over the last decade, such as preschool, full-day kindergarten, and intensive literacy reforms.

Corzine’s eagerness to expand these programs comes from the under-reported but significant successes in the Abbott districts. In 2005, the New Jersey Department of Education reported an 80 percent graduation rate in the Abbott districts, up from 73 percent four years earlier. According to the Education Law Center’s 2006 report on the Abbott districts, an independent study found a much lower rate – 68 percent in 2004 versus 57 percent in 1998 – but still a significant improvement.

Another success of the Abbott-district reforms is in access to preschool. In 1999, only 11 percent of the Abbott districts’ 53,000 pre-kindergarten age children attended preschool. In 2006, the number had risen to 80 percent. According to the National Institute for Early Education Research at Rutgers University, initial data indicates that children who attend the Abbott Preschool Program end up four months ahead of other children in vocabulary growth. In addition, children who attend the program have better math skills, know more letters, know more letter-sound associations, and are more familiar with words. Furthermore, the quality of preschool instruction in the Abbott districts is also getting better. In 2004, 92 percent of preschool teachers met Abbott certification requirements. Before 1998, only 35 percent even had bachelor’s degrees.

Success can be measured in other areas as well. According to the Education Law Center report, elementary class school sizes – widely agreed upon as one of the most important factors in student learning – has decreased from 24 to 19 in the past decade. A 2006 report by Elaine Walker, Charles Achilles, and Carol Frances of Seton Hall University also looked at a range of data from Abbott district schools. One of their findings was that Abbott district schools have seen a steady drop in the number of Class B offenses – such as simple assault, gang fighting, robbery, drug dealing, harassment and bullying – since 2001.

When we judge the achievements and failures of students based only on a few selected test scores, we lose sight of real measures of success in education and do ourselves and our children a disservice. If we want to grade our schools, we should do it based on figures that matter – graduation rates, preschool enrollment, or other real measurements of the quality of education schools are providing. In these areas, states around the country have seen marked improvement, and that is something worth talking about.

Prepared by Matthew Samberg, March 15, 2006