Tyler Foggatt & Rachel Siegel
January 12, 2015
The Obama administration’s first draft of a federally backed college performance rating system was released Dec. 19, but the Department of Education admitted that limited data and the project’s complexity mean there is still work to be done.
In August 2013, President Barack Obama announced a new system to review colleges on issues like accessibility to low income students, employment rates for graduates and affordability. The administration also plans to link federal student aid to these rankings, meaning that schools that score highly will receive more funding.
Despite general support for Obama’s focus on increasing college accessibility for low-income students, the idea of such a rating system has not been universally embraced. Associate Vice President for Federal and State Relations Richard Jacob said the program draft proves how difficult it is to develop an effective college rating system.
“The department acknowledges that the existing data are limited, and that many of the statistics are available only for students who receive federal financial aid, which raises questions about whether measures based on those sources will be representative,” Jacob said. “The Department notes that it is not sure how to group schools for purposes of comparison.”
Jacob added that higher education experts have wondered whether developing formulas to compute scores for institutions would be an improvement over existing resources. For example, the College Scorecard — a tool used by the Department of Education’s College Affordability and Transparency Center — simply presents statistics such as net price, graduation rate and loan default rate, Jacob said.
According to the Department of Education press release, the college ratings will be broken up into three categories based on performance, with most colleges falling in the middle grouping, somewhere between high-performing and low-performing.
The press release also noted that the first version of these ratings will largely rely on federal administrative data collections. The Department said it is currently weighing the available metrics, such as the percentage of students receiving Pell Grants, the number of first-generation college students and average net price of each university, for use in the new rankings.
But many colleges and higher education associations are questioning whether the government is equipped to compare schools that appear vastly different on paper.
In a January 2013 letter addressed to Richard Reeves, a program director at the National Center for Education Statistics, the Association of American Universities blatantly stated that it did not endorse the new rating system.
The letter said Obama’s rating system would likely draw most of its data from the Integrated Postsecondary Education Data System, an existent data source initially developed to track trends in higher education.
“Over the years [IPEDS] has become a tool for enforcing institutional compliance and assessing institutional effectiveness, with mixed results,” the letter said, adding that any new rating system would have to find ways to deal with missing or limited data. “Overall, while IPEDS data can be informative for parents and students, they not provide the whole picture, and require contextualization to be most useful.”
Barry Toiv, the AAU’s vice president for public affairs, said Congress did not specifically provide funds for the Obama administration to implement a rating system when it passed the omnibus appropriations bill for fiscal year 2015 in December.
“Whether [members of Congress] will act to prevent the Administration from moving forward, whether in the appropriations process or in a bill reauthorizing the Higher Education Act, which could be taken up this year, is hard to know at this time,” Toiv said.
Another gap in existing data is the transfer and graduation rates among college students. The AAU letter indicated that graduation rates may make sense for some institutions but not for others, noting that in 2011, only 27 percent of four-year institutions reported transfer-out students to IPEDS.
Bev Taylor, the founder of Ivy Coach — a New York-based college consulting firm — said that even if the Department of Education were to produce accurate ratings, it is unlikely that they would have a significant effect on the college choices of certain groups of students.
“There’s always going to be students and parents who look at the prestige of a university, and not the cost,” Taylor said. “And so if a school is going to be more highly rated because it has a lower sticker price, it’s not going to matter to this group of applicants.”
Taylor added that applicants not focused on cost effectiveness will probably default to using the U.S. News and World Report college rankings as a resource, since the U.S. News rankings are currently the most widely read.
Additionally, despite questions about the fundamental data, some are asking whether the government is in any position to rate American colleges to begin with.
Robert Kelchen, assistant professor in the Department of Education Leadership, Management and Policy at Seton Hall University, told the News that intense opposition to the new system may mean the ratings are doomed even before they come out. In a Dec. 18 op-ed for Politico Magazine, Kelchen pointed to a 2013 poll released by Gallup and Inside Higher Ed which found that only 16 percent of 675 surveyed college presidents said the Department of Education’s rating system was a good idea, compared to 65 percent who said they were against it.
Even the American Council on Education, which represents much of the nonprofit higher education community, released a statement earlier this year saying “most believe it is nearly impossible for the federal government [to rate colleges] with any degree of reliability or validity.”
Kelchen said the first step in improving the rating system would be improving the data it is founded upon. He added that the current data makes it impossible to rate most colleges at the department of program level, but that even with new metrics and reexamined data, it would be hard to produce ratings by this fall.
“Releasing ratings by fall 2016 seems more feasible, but that gets into an election year, and the likelihood of actions like these becomes less likely,” Kelchen said.