Lies, Damned Lies, Statistics, and Rankings

Lies and College Ranking, Lies and High School Rankings, HS Rankings

Mark Twain had a saying about lies and statistics. We feel it applies to high school rankings.

The magazines that rank our best colleges, graduate schools, boarding schools, hospitals, health plans, doctors, nursing homes, and high schools do so because they’re in the business of selling magazines. And, of course, we buy into this craze. Rankings have an important place in our American culture. Yet once you learn what is actually being measured, you might see how unimportant some, though not all, of these rankings are because the criteria that is used can in some cases be rather subjective.

This week, Newsweek published its annual list of the best high schools in the United States. Since 1998, Newsweek has ranked public high schools based on the “Challenge Index,” which is calculated by the number of Advanced Placement (AP), International Baccalaureate (IB), and Cambridge tests administered each year divided by the number of graduating seniors. Note that exams that are taken by all students, not just seniors, count towards the tally. Public high schools whose students took as many tests in 2009 as the number of graduates that year achieved the “all important” cut-off ratio of 1. According to Newsweek’s report, “Just over 1,600 schools — only 6 percent of all the public schools in the U.S. — made the list.” The actual number of public high schools in the U.S. is approximately 27,000. Only 1,625 of these schools achieved the ratio and thereby made the list.

Some may then wonder if your high school is on this list, do you have a better chance of gaining acceptance to the college of your choice? We would argue that this is not the case because we can’t imagine that any college values this data. As an example, we have concerns with this list because it only measures a single factor. What it doesn’t measure is how well students perform on this test. If the majority of students who take these exams score a 1 or a 2, the high school can still make the list and so in this case, quantity trumps quality. If the rankings were to measure a quantifiable factor, possibly the results of those tests, then the high schools would be ranked based on a measure that shows how well the students have been prepared for college level work. The way it’s currently designed, this list doesn’t even take into account drop-out rates at a particular school. Additionally, magnet schools that so often attract the best and the brightest students are not included.

Interestingly enough, the list is called “Best High Schools” but it only applies to public high schools. It seems to us that this list should be called “Best Public High Schools.” Highly selective boarding / prep schools that are often considered feeder schools to the Ivies are also not on this list. And what about high schools that don’t allow students to self-select AP or IB courses? Obviously, high schools that base admission to these courses on teacher recommendation, class performance, and qualifying test results may not make this list because fewer students are permitted to take these courses. And what about high schools that don’t offer these courses altogether?

In the fall of 2007, Scarsdale High School, a highly competitive school in the affluent Westchester County suburb of Manhattan, replaced AP courses with what they call Advanced Topic (AT) courses. As stated on Scarsdale’s website, their rationale in doing so was to “encourage higher levels of thinking beyond rote learning in a rigorous yet lively learning environment.” As a result of this innovative idea, Scarsdale ranked 1,011 in Newsweek’s “Best High Schools” list this year as opposed to 748 in 2009, 283 in 2008, 247 in 2007, and 174 in 2006. Yet even with Scarsdale’s rank plummeting these last few years, for the Class of 2008, 49% of seniors were accepted to the most competitive colleges in the U.S. — up from 45% in 2007. So obviously, Scarsdale’s low “Challenge Index” ranking held no significance in the highly selective college admissions process.

But even though a college might dismiss the value of this list, the competitiveness of the high school absolutely does count! In reviewing applicants, college admissions officers without a doubt consider the student’s high school as a significant factor. Thus, a high school that offers AP and IB courses and has impressive numbers of students who score well on these exams — rather than just take the exams — is likely to give students who attend these high schools a more competitive edge in college admissions than students who attend high schools that lack these more rigorous courses. However, no matter how competitive a high school may be, in the end, the most important criteria is that a student takes the most rigorous courses offered and has earned exceptional grades in those courses.

According to Jay Matthews, a columnist for The Washington Post and the inventor of the “Challenge Index,” “Many educators say they think my list is dumb, narrow and deceptive because a high school’s efforts to teach its students cannot be summed up in a number.” We agree with the critics, Mr. Matthews. We agree with the critics.


You are permitted to use (including the content of the Blog) for your personal, non-commercial use only. You must not copy, download, print, or otherwise distribute the content on our site without the prior written consent of Ivy Coach, Inc.


Tags: , , , ,

Leave a Reply

Your email address will not be published.