New players, new approaches to ranking colleges
AP Education Writer
(AP) – U.S. News & World Report may still be the 800-pound gorilla of college rankings. But with a formula that rarely changes, the latest edition _ out Wednesday _ looks pretty much the same as a decade ago, with very few exceptions.
More interesting are a pair of newer players to the rankings game. Both have shortcomings, but both produce a top-colleges list that looks somewhat different from the magazine’s (where Princeton and Harvard share the top spot, just like last year). And neither relies on information provided by the colleges themselves; more and more schools have been caught fudging the numbers they give to U.S. News.
More broadly, the new players offer data intriguing even to those who don’t buy the idea that colleges can be ranked like their football teams.
The first, now in its second year, comes from a company called Parchment.com, whose main business is serving as electronic middleman between students at 7,500 high schools and the colleges where they send transcripts. But through a college search function, the site has collected valuable and otherwise hard-to-come-by data: It knows the names of the colleges where 200,000 students nationwide applied, got accepted, and chose to attend.
Ranking automobile companies is fairly straightforward; each basically makes as many cars as it can sell and the results are clear enough. But college is often a two-way selection. The most prestigious ones have limited slots and regularly turn down business. So Parchment looks at what the customers with options choose.
The “Student Choice” rankings _ based on a model developed by economist Caroline Hoxby about a decade ago _ make no effort to measure a college’s “inputs” such as average freshman SAT scores. They do nothing to measure “outputs” like learning outcomes or alumni salaries.
Instead, they reveal only the collected wisdom of students choosing among multiple colleges. If, for example, 75 out of 100 students accepted to both Yale and the University of Virginia choose Yale, then Yale moves up relative to Virginia in the model. The choices of every student are played out in a kind of gigantic computerized tournament, and eventually the results settle into ranking similar to how chess players are ranked.
The list that emerges contains many of the familiar names, with Harvard, Stanford and Yale on top (few students accepted to those schools choose others instead). But distinctive colleges with a particular mission fare well, like the Air Force Academy, which comes in No. 10.
The system has weaknesses. The pool of students is large but somewhat weighted regionally. But it does some things others can’t. To the extent students reward value, the rankings do, too (tuition-free Cooper Union in New York City scores an impressive 43rd). And it allows schools of all sizes to be ranked on the same list, unlike U.S. News, which separates national universities and liberal arts colleges. That’s helpful because in practice, students are often choosing between a big and a small school, said general manager Brent Pirruccello, who helped create the rankings. Amherst College, No. 2 on the U.S. News liberal arts list, is the top liberal arts school in the Student Choice rankings, at No. 9 overall.
The second new system, which debuted Monday, focuses on the other end of college _ alumni satisfaction. Until now, individual colleges have surveyed their alumni, but surveying enough people across colleges to compare the results has been too complex.
A new company called The Alumni Factor is trying, claiming it’s surveyed and interviewed 42,000 alumni of 450 colleges over the last four years, and is now publishing the results on 177 well-known schools where it says it has enough data to be statistically reliable (at least around 200 alumni per school). The surveys try to pin down objective results on 15 attributes. Among them: intellectual and social development, friendships made and even overall happiness of graduates. Other attributes are purely financial, such as percentage of graduates earning over $150,000.
A generic ranking blending the survey results in those 15 category equally also produces some interesting results: Well-respected but not widely famous Washington & Lee University in Virginia finishes No. 1, followed by Yale, Princeton, Rice and _another surprise _ the College of The Holy Cross in Massachusetts. Some well-known names do less well. Harvard, for instance, scores spectacularly in the financial success categories, but relatively poorly on indicators like overall happiness and whether alumni would recommend the college to others. Overall it’s 37th, between Carleton College in Minnesota and Vanderbilt.
Alumni Factor (a subscription site that will charge either $3.95 or $5.95 per month depending on access level) then adds a feature even rankings critics will probably like: It will let users generate their own rankings, using sliding scales on its website to give various factors more or less weight. Students who care mostly about money can tilt the formula’s emphasis toward the financial outcomes and Harvard moves up from 37. But tweak the dials to indicate you care more about happiness and less about money, and it falls to No. 82 (some of the sorting options aren’t yet available on the website).
Still, Alumni Factor has at least one big problem. Despite a promise of transparency, it says little about how it finds the people it surveys and how it questions them, other than to offer assurances the methods are vetted by a Georgia Tech statistician.
Executive editor and CEO Monica McGurk did offer a few details in a telephone interview, such as the minimum of about 200 alumni surveyed per school. Also, she said the questions cover so many issues that respondents don’t realize they’re being surveyed about their college experience, so they won’t cheerlead for their alma mater.
But McGurk said she couldn’t provide more details without compromising the method’s objectivity. Without knowing how the company finds the alumni, there’s no way for outsiders to judge if they’re really tapping a representative group, or one that’s, say, more financially inclined and successful than the average graduate (the company hopes to sell some of the information back to colleges themselves).
All of which is to say, the rankings game may not be getting much more credible, but it is getting more interesting. Students who (wisely) take the horse-race aspect with a grain of salt nonetheless have some interesting new data to explore in their college search.
____
Online:
http://www.parchment.com/c/college/college-rankings.php
http://www.usnews.com/rankings
___
Follow Justin Pope at
http://www.twitter.com/JustinPopeAP
(Copyright 2012 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.)
