The Rankings Obsession

Welcome to Money Buff! Feel free to check out my other posts, and follow me on Twitter for more regular thoughts. For fresh insights into all things finance, subscribe here:

Starting in 1996, Northeastern University orchestrated one of the most dramatic turnarounds in higher education. At the time, it was a commuter school struggling to stay afloat. The former school president Richard Freeland is credited with saving the school thanks to an obsessive focus on one list: U.S. News & World Report’s “Best Colleges” ranking. This single list, Freeland determined, had the power to make or break a school.

Students pay close attention to these rankings because high rankings attract employer attention. Increased employer attention means better jobs. And better jobs mean both the students and schools can command more prestige. A virtuous cycle.

The “Best Colleges” rankings weigh several characteristics to determine a school's rank. Freeland targeted key areas:

Class size: He capped Northeastern class sizes to 19 students because the rankings formula rewards classes under 20 students. Foreign language and music course offerings, which typically have a low student-to-faculty ratio, were expanded to offset larger courses.

Selectivity: Northeastern launched an ambitious recruitment campaign. By increasing the number of applicants, the university could turn away more students, which made it appear more selective.

Retention: Retention rate carries significant weight in the rankings formula. Northeastern found studies showing that students who live on campus are more likely to stay enrolled, so they poured millions into the construction of dorms.

Peer assessment: As part of the ranking formula, academic leaders rate the quality of peer institutions on a scale of 1 to 5. Freeland embarked on a tireless campaign to influence the perception of Northeastern among his peers. He traveled extensively, attending conferences and meeting with influential academics in an effort to boost Northeastern's results.

The crown jewel of Northeastern’s rankings manipulation is the N.U.in Program. The university began enticing less-qualified affluent applicants to spend their first college semester abroad. The kicker? They're categorized as transfer students, so their GPA, test scores, and acceptance are excluded from admission statistics. The school gets to accept more people while maintaining a selective appearance.

Thanks to an unwavering commitment to the rankings, Northeastern has skyrocketed from #162 to #44 in less than 20 years.

Gaming U.S. News is part of the university’s DNA. And it’s more common than you’d think. Tons of universities have been caught doing whatever it takes to climb the rankings, which largely rely on the honor system.

Columbia, USC, Claremont McKenna, George Washington, Emory, and Temple have all been accused of reporting false data to U.S. News. Baylor offered scholarships to freshmen for retaking the SAT. Harvard is often accused of ‘recruiting to reject’ practices. The list goes on.

Goodhart’s Law

What we’re witnessing is a textbook example of Goodhart’s Law: When a measure becomes a target, it ceases to be a good measure. Acceptance rate, for example, used to just be a number; the total number of acceptances divided by the total number of applicants. As soon as it was redefined as a proxy for educational quality, it became a target.

There’s an urban legend that demonstrates this phenomenon:

With each passing year, Goodhart’s Law has a stronger grip on the college admissions process. Every administrator can see the exact metrics used to evaluate how their college is ranked. Those administrators know that families will consult that report in deciding where to apply. It was inevitable that rankings would become the gospel.

But does going to an elite college lend to greater educational value? Outside of certain specialties, probably not. After all, calculus is calculus no matter where you learn it; Harvard isn’t doing anything extraordinarily unique in its classrooms.

In fact, a survey found that 85% of economics professionals believe half or more of the value of an education is signaling, not human capital formation.

The increasing desperation around getting into top schools suggests that Harvard classes should have tighter security than the Federal Reserve. But anyone off the street can waltz into a lecture hall and get an Ivy League education for free. I grew up 15 minutes from Harvard and have most definitely sat in the upper balcony during CS50. The fact that more folks aren't doing this suggests that most people intuitively understand the state of affairs.

To the best of my knowledge, no studies have explored whether rank has any bearing on a university's effectiveness at imparting knowledge and critical thinking skills. It’s hard to imagine that the version of me who went to Harvard would've been better at my eventual job. But that job might've been higher paying.

Prestige opens doors — most doors, actually. It screams, "I did something only 4% of applicants could do”. That's the true value of attending an elite college. Students know it, colleges know it, employers know it. And how do employers calibrate prestige? Rankings.

Colleges aren't the only ones falling victim to Goodhart's Law. More than ever, the admissions process is a box-checking frenzy. Check out this Bloomberg headline from last week:

These consultants begin developing "strategic" plans for clients starting in 9th grade. Everything from course selection to extracurriculars is meticulously crafted in this all-consuming pursuit of a coveted acceptance letter. There's a perverse incentive where people are more focused on gaming the system than anything else.

College: “We’re elite!”

Student: “Ok, how so?“

College: “Because we only accept people who…”

Parent: “Consultant, you know what to do.”

The lengths parents will go is reminiscent of the Varsity Blues scandal, where many spent millions and risked prison to ensure their child had a spot at a top university.

Ultimately, your acceptance depends on your ability to achieve some level of what schools are looking for. Applicants attempt to signal that they have what it takes but it’s hard to know exactly, so they play a handful of safe bets:

1. Academics: Colleges are very keen to avoid students flunking out (remember, retention). They need assurance that applicants will be able to keep up and the most reliable way is to meet their minimum academic qualifications.

2. Money: Applicants who come from money represent a potential windfall for the institution. Maybe they're not “buy a building” rich, but athletic coaches can be bought for the price of a year’s tuition.

3. Success: It’s good for the school when alumni do things like start a successful company, win a Nobel Prize, write a bestseller, enroll in a top grad program, or get a job at a well-regarded company. To demonstrate their potential, applicants might, for example, volunteer at a hospital if med school is their plan.

4. Extracurriculars: Colleges benefit when students make the campus more attractive to others through extracurriculars. Candidates join extracurriculars in high school to show they are capable of this. Specifically showing that they have the potential to be a key member of that group (think captain or president).

Taken to its full extent, we’re left with a bunch of kids molding themselves to be who the elite college wants them to be. Captain of this. Honors in that. Their adolescence is curated for college readiness.

There’s arguably nothing inherently wrong with this. But is that fencing national championship really the pinnacle of your childhood passion? Let’s hope not.

One can argue whether helping the homeless, being on the fencing team, or teaching the homeless how to fence are good qualities for applicants to have. But it’s pretty clear the tail is wagging the dog. Goodhart's Law reigns supreme.

Incentives run the world, and higher education is no exception. Applicants and colleges alike focus on the areas in which they are measured, leading to this double-sided optimization game.

Applicants check boxes for the college, and the college checks boxes for U.S. News. Each party reverse engineers itself to fit a set of criteria. As a result, you have colleges mailing spurious recruitment letters and students signing up for oboe lessons.

The perfect encapsulation of Goodhart’s Law.

-Sam

If you liked this, add your email below to receive new posts!