New players, new approaches to ranking colleges

Sep 12, 2012, 1:05 PM

AP Education Writer

(AP) – U.S. News & World Report may still be the 800-pound gorilla of college rankings. But with a formula that rarely changes, the latest edition _ out Wednesday _ looks pretty much the same as a decade ago, with very few exceptions.

More interesting are a pair of newer players to the rankings game. Both have shortcomings, but both produce a top-colleges list that looks somewhat different from the magazine’s (where Princeton and Harvard share the top spot, just like last year). And neither relies on information provided by the colleges themselves; more and more schools have been caught fudging the numbers they give to U.S. News.

More broadly, the new players offer data intriguing even to those who don’t buy the idea that colleges can be ranked like their football teams.

The first, now in its second year, comes from a company called Parchment.com, whose main business is serving as electronic middleman between students at 7,500 high schools and the colleges where they send transcripts. But through a college search function, the site has collected valuable and otherwise hard-to-come-by data: It knows the names of the colleges where 200,000 students nationwide applied, got accepted, and chose to attend.

Ranking automobile companies is fairly straightforward; each basically makes as many cars as it can sell and the results are clear enough. But college is often a two-way selection. The most prestigious ones have limited slots and regularly turn down business. So Parchment looks at what the customers with options choose.

The “Student Choice” rankings _ based on a model developed by economist Caroline Hoxby about a decade ago _ make no effort to measure a college’s “inputs” such as average freshman SAT scores. They do nothing to measure “outputs” like learning outcomes or alumni salaries.

Instead, they reveal only the collected wisdom of students choosing among multiple colleges. If, for example, 75 out of 100 students accepted to both Yale and the University of Virginia choose Yale, then Yale moves up relative to Virginia in the model. The choices of every student are played out in a kind of gigantic computerized tournament, and eventually the results settle into ranking similar to how chess players are ranked.

The list that emerges contains many of the familiar names, with Harvard, Stanford and Yale on top (few students accepted to those schools choose others instead). But distinctive colleges with a particular mission fare well, like the Air Force Academy, which comes in No. 10.

The system has weaknesses. The pool of students is large but somewhat weighted regionally. But it does some things others can’t. To the extent students reward value, the rankings do, too (tuition-free Cooper Union in New York City scores an impressive 43rd). And it allows schools of all sizes to be ranked on the same list, unlike U.S. News, which separates national universities and liberal arts colleges. That’s helpful because in practice, students are often choosing between a big and a small school, said general manager Brent Pirruccello, who helped create the rankings. Amherst College, No. 2 on the U.S. News liberal arts list, is the top liberal arts school in the Student Choice rankings, at No. 9 overall.

The second new system, which debuted Monday, focuses on the other end of college _ alumni satisfaction. Until now, individual colleges have surveyed their alumni, but surveying enough people across colleges to compare the results has been too complex.

A new company called The Alumni Factor is trying, claiming it’s surveyed and interviewed 42,000 alumni of 450 colleges over the last four years, and is now publishing the results on 177 well-known schools where it says it has enough data to be statistically reliable (at least around 200 alumni per school). The surveys try to pin down objective results on 15 attributes. Among them: intellectual and social development, friendships made and even overall happiness of graduates. Other attributes are purely financial, such as percentage of graduates earning over $150,000.

A generic ranking blending the survey results in those 15 category equally also produces some interesting results: Well-respected but not widely famous Washington & Lee University in Virginia finishes No. 1, followed by Yale, Princeton, Rice and _another surprise _ the College of The Holy Cross in Massachusetts. Some well-known names do less well. Harvard, for instance, scores spectacularly in the financial success categories, but relatively poorly on indicators like overall happiness and whether alumni would recommend the college to others. Overall it’s 37th, between Carleton College in Minnesota and Vanderbilt.

Alumni Factor (a subscription site that will charge either $3.95 or $5.95 per month depending on access level) then adds a feature even rankings critics will probably like: It will let users generate their own rankings, using sliding scales on its website to give various factors more or less weight. Students who care mostly about money can tilt the formula’s emphasis toward the financial outcomes and Harvard moves up from 37. But tweak the dials to indicate you care more about happiness and less about money, and it falls to No. 82 (some of the sorting options aren’t yet available on the website).

Still, Alumni Factor has at least one big problem. Despite a promise of transparency, it says little about how it finds the people it surveys and how it questions them, other than to offer assurances the methods are vetted by a Georgia Tech statistician.

Executive editor and CEO Monica McGurk did offer a few details in a telephone interview, such as the minimum of about 200 alumni surveyed per school. Also, she said the questions cover so many issues that respondents don’t realize they’re being surveyed about their college experience, so they won’t cheerlead for their alma mater.

But McGurk said she couldn’t provide more details without compromising the method’s objectivity. Without knowing how the company finds the alumni, there’s no way for outsiders to judge if they’re really tapping a representative group, or one that’s, say, more financially inclined and successful than the average graduate (the company hopes to sell some of the information back to colleges themselves).

All of which is to say, the rankings game may not be getting much more credible, but it is getting more interesting. Students who (wisely) take the horse-race aspect with a grain of salt nonetheless have some interesting new data to explore in their college search.

____

Online:

http://www.parchment.com/c/college/college-rankings.php

https://www.alumnifactor.com/

http://www.usnews.com/rankings

___

Follow Justin Pope at
http://www.twitter.com/JustinPopeAP

(Copyright 2012 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.)

United States News

Rep. Juan Ciscomani (R-AZ) delivers remarks in the House Chamber during the third day of elections ...
Kevin Stone

Rep. Juan Ciscomani of Arizona to give GOP Spanish rebuttal to Biden speech

Freshman U.S. Rep. Juan Ciscomani of Arizona will deliver the Republican Spanish rebuttal to President Joe Biden's State of the Union address Tuesday.
12 hours ago
(AP Photo/J. Scott Applewhite, File)...
Associated Press

Sen. Kyrsten Sinema’s split from Democrats shows party discord in Arizona

The Democrats' winning formula is in jeopardy ahead of the 2024 election after Arizona Sen. Kyrsten Sinema’s divorce from the party.
12 hours ago
(AP Photo/Steve Helber, File)...
Associated Press

US officials offer Congress briefing on Biden, Trump papers

The Biden administration has offered to brief congressional leaders on its investigation into the classified documents.
2 days ago
FILE - Mega Millions lottery tickets and a wager slip are displayed, Friday, Jan. 6, 2023, in Derry...
Arizona Sports

Drawing nears for $700M Powerball prize, 10th biggest in US

Another huge lottery jackpot will be on the line Saturday for players willing to put up $2 vs. daunting odds of actually winning the prize.
3 days ago
Chinese spy balloon flies above in Charlotte NC, United States on February 04, 2023. The Pentagon a...
Associated Press

US downs suspected Chinese spy balloon off the Carolina coast

The United States on Saturday downed a suspected Chinese spy balloon off the Carolina coast after it traversed sensitive military sites.
3 days ago
A high altitude balloon floats over Billings, Mont., on Wednesday, Feb. 1, 2023. The huge, high-alt...
Associated Press

Chinese balloon soars across US; Blinken scraps Beijing trip

A Chinese balloon sailing across the U.S. drew severe Pentagon accusations and sent excited or alarmed Americans outside with binoculars.
4 days ago

Sponsored Articles

...
Day & Night Air Conditioning, Heating and Plumbing

Company looking for oldest air conditioner and wants to reward homeowner with new one

Does your air conditioner make weird noises or a burning smell when it starts? If so, you may be due for an AC unit replacement.
(Desert Institute for Spine Care photo)...
DESERT INSTITUTE FOR SPINE CARE

Why DISC is world renowned for back and neck pain treatments

Fifty percent of Americans and 90% of people at least 50 years old have some level of degenerative disc disease.
(Photo via MLB's Arizona Fall League / Twitter)...
Arizona Fall League

Top prospects to watch at this year’s Arizona Fall League

One of the most exciting elements of the MLB offseason is the Arizona Fall League, which began its 30th season Monday.
New players, new approaches to ranking colleges