The Secret Sauce of the US News Rankings

A new episode of Malcolm Gladwell’s Revisionist History focuses on the US News rankings.

Curious to learn more about the all-important US News & World Report college rankings? As our loyal readers know, one of Malcolm Gladwell’s greatest fascinations has long been the college rankings and the pop psychologist is out with a new Revisionist History podcast episode called “Lord of the Rankings” that is one of his very finest (his next episode will also focus on the rankings). The podcast episode zeroes in on how the US News college rankings came to be, how they’re calculated, and how they’re viewed within the realm of academia. So let’s dive into what he shared with his audience about the US News rankings. But we should first warn you…spoiler alert!

Gladwell first interviews the editor who came up with the rankings. At the time, there were three major weekly magazines in circulation in the United States: TIME, Newsweek, and US News & World Report. US News was a distant third in the magazine to its competitors. They were under new ownership — the ownership of a Canadian billionaire who had dreams of being a media baron — and they needed to find a way to compete with TIME and Newsweek. So in the middle of the 1980s, the “sleepy little publication in Washington, DC” rolled out what was really a promotional gimmick: a ranking of colleges across America based on a secret ranking methodology that ranked them from 1-100 (or mythology as its then-editor Peter Bernstein would say on the podcast in a Freudian slip of the tongue). As Gladwell says, “So they did something audacious, something no educated parent could possibly ignore, a scheme to examine every American university and rank them from top to bottom.” And that they would do — for many years to come. In fact, long after the heyday of TIME, Newsweek, and US News & World Report, the rankings endure. They remain as important today — if not more important — than years ago.

So what’s the secret sauce of the US News & World Report college ranking algorithm. Well, therein lies the rub. You see, US News has long been open about the variables in their algorithm (e.g., graduation rates). But the publication has not been particularly open about the weight it assigns those variables to arrive at its annual ranking. So, naturally, Gladwell tries to better understand the secret sauce of the algorithm by seeking out people who have tried to, well, hack the algorithm. And hack it they did at Reed College, a school ranked 90th by US News that refuses to report data to the magazine and, as such, feels the publication holds it against them when determining its ranking (its ranking plummeted once the school stopped supplying the data). Statistics students at Reed basically came up with an algorithm using the variables US News factors in to reasonably predict where a school will rank in the annual US News rankings. As Gladwell eloquently puts it: “They cracked the code at Reed College because US News dissed their school.”

When Gladwell asks about the most important variable, it’s no surprise that it’s the Reputation Score assigned to each university, long at the heart of the algorithm. And how is that Reputation Score calculated? Each year, college presidents, provosts, and enrollment managers or deans of admissions are sent three different surveys with lists of universities (three different people at each university rank the schools). They’re then asked to rank each school. But, of course, what makes the provost at, say, Cornell University qualified to rank Reed College? Has the provost even heard of Reed College? As Gladwell puts it in a question to the current wizard of the US News ranking, Robert Morse, “What does the rabbi who runs Yeshiva University know about Brigham Young?” It’s a great point. Morse, described amusingly by Gladwell as “an older man, white hair, very preppy, nice if a bit lugubrious” fumbles in his answer, suggesting succinctly that academics are experts in their field. As Gladwell puts it perfectly, “Heavy hangs the head that wears the crown.”

The best part of the episode comes next when Gladwell interviews a dean of admissions at a prestigious university that is not named — under a disguised voice of course so US News doesn’t take the dean’s words out on the school in its September’s ranking — and asks him what he knows about, say, the University of Washington. He once walked on the campus there. Or William and Mary University. He likes Colonial Williamsburg. So, basically, the dean at the prestigious university knows about as much about the schools that he’s ranking as just about anyone else. He’s no expert in their academics. In some cases, he’s hardly heard of the schools he’s ranking. Malcolm then jokingly ranks Syracuse University, where he once stopped for coffee. He observed the weather. He has a cousin that went there. “I’d probably give them a 4 or 5,” he says.

And yet it’s not such a joke because the Reputation Score is a major factor in the US News rankings. Through the humor, Gladwell is very effectively showing just how ridiculous they may be. But leave it to Gladwell to land his point. When interviewing the President of Rowan University, long a favorite institution of Gladwell’s, he learns that the man sends hot sauce to all the university presidents annually to help improve their rankings. Hot sauce! He makes it himself from seeds. As Gladwell sums it up, “”In their haste to compete with TIME and Newsweek way back when, maybe all US News did was create a system that allowed the presidents and deans of every college in the country to assign a number to their prejudice, to disguise mythology as methodology.” Well, yes, that’s precisely what US News did.

So why do these college presidents, provosts, and admissions leaders fill out these surveys each year? If they all just stopped filling out the surveys, at least the Reputation Score, which is incredibly subjective, would not influence the annual US News ranking. The answer? Because they’d lose their jobs. Their jobs depend on the annual US News ranking no matter how loudly and how vociferously they tell you they don’t care about their rankings. They can’t risk their school taking the reputational hit that Reed College took when they stopped supplying the magazine with the requested data. So, yes, as much as admissions leaders may tell you they wish to make the highly selective college admissions process less stressful for all, here they are contributing to the craze. Shocking, we know.

Have a listen to the episode!

 
 

You are permitted to use www.ivycoach.com (including the content of the Blog) for your personal, non-commercial use only. You must not copy, download, print, or otherwise distribute the content on our site without the prior written consent of Ivy Coach, Inc.

Categories:

Tags: , , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *