How about a college degree?
Certain selective colleges have managed to distinguish themselves as "worth more." Parchment has an innovative method for divining applicants' perception of schools' worth. They treated each applicant's decision as votes. For example, a student who got into Columbia, Duke, and Stanford and chose Stanford votes for Stanford and against Columbia and Duke. Parchment compiled all these votes using an Elo method to determine which colleges have distinguished themselves in applicants' minds.
How the schools managed to distinguish themselves is a great question. Many did it through their age - our oldest colleges are often the most esteemed. Others did it through the reputation of their graduate schools. Sports catapulted other schools onto the scene. However, selectivity in admissions is the key variable. Maybe it is because U.S. News and World Report's college ranking method weights selectivity so highly, but even without the U.S. News rankings, selectivity would definitely affect people's perceptions. (Side note: the rankings seriously distort college's behavior.) People assume hard-to-obtain goods are worth more.
Are these schools, in fact, worth more?
In terms of the content of the courses, there is probably little difference. For the most part, a course in differential equations at Yale covers about the same topics at about the same pace as one at Ohio State. It is important to understand that most college courses are not special snowflakes but (cough) commodities. Of course, college professors do invent new courses, and there are programs unique to an individual school. But many courses are commodities. One may get a better or worse teacher, but because schools don't place much weight on teaching in professors' evaluations, teacher quality and school reputation don't have much correlation.
Of course, course content and teaching is not the only variable that matters when talking about institution educational quality. Two colleges might teach similar courses but at differing levels of effectiveness. Good institutions have professors who keep standards for student work high; good institutions give robust support to weaker students; and good institutions develop new programs. Furthermore, due to the enormous endowments highly selective colleges have, they have a lot more money to spend per student - although much of the extra funding goes to facilities like dorms, athletic buildings, and student recreation centers that have little impact on the quality of instruction and to research facilities that may have only a small impact on undergraduate instruction. However, institutional quality hardly seems to justify the hysteria.
One could argue that there are intangible benefits to going to a high-reputation school like being surrounded by motivated, smart students and professors. While this has makes intuitive sense, the best evidence does not really support this argument. The C.L.A., the Collegiate Learning Assessment, shows little pattern between college attended and student learning. Some learn a lot at lower reputation schools; some learn little at high-reputation schools. One can discuss Shakespeare with other smart, eighteen- and nineteen-year-olds, but the discussion could be more enlightening if it includes a working mom who's back in college, a soldier who's back from war, ... you get my point. The student body argument cuts both ways; diversity is important, too. The C.L.A. results show that neither way is intrinsically superior. How much or how little students learn has everything to do with them and little to do with the college itself.
A giant meta-analysis entitled How College Affects Students wrote:
"The great majority of postsecondary institutions appear to have surprisingly similar net impacts on student growth. If there is one thing that characterizes the research on between-college effects on the acquisition of subject matter knowledge and academic skills, it is that in the most internally valid studies, even the statistically significant effects tend to be quite small and often trivial in magnitude."
Quoted from the New York Times. And the New York Times continues:
The whole apparatus of selective college admissions is designed to deliberately confuse things that exist with things that don't. Many of the most prestigious colleges are an order of magnitude wealthier and more selective than the typical university. These are the primary factors driving their annual rankings at or near the top of the U.S. News list of "best" colleges. The implication is that the differences in the quality of education they provide are of a similar size. There is no evidence to suggest that this is remotely true. When college leaders talk about academic standards, they often mean admissions standards, not standards for what happens in classrooms themselves.
Of course, that's about learning. Let's talk about earning.
Advantages of reputation
This leaves reputation alone as the way in which high-reputation colleges are worth more. Reputation means whether the degree will open the door to good entry-level jobs in a field and get a person off to a great start. And the evidence is that the path to many elite jobs runs through high-reputation colleges almost exclusively. Why are many elite employers so enamored of a few colleges?
Let's admit that the undergraduate degree itself does not convey much information about what a person learned. We may assume that a computer science major covered certain basics in the course of earning his or her degree. But that's about it. The degree provides low-quality information about how deeply that person learned in college. (In fact, it is basically impossible to fail out of a high-reputation college - they don't want to ruin their statistics.) So why do businesses care about the undergraduate institution? The simple answer must be that the key information is about college admission. Businesses must believe that high-reputation colleges do a good job selecting the smartest and hardest-working students.
In some fields like law, the college (and law school) a person attended are always crucially important to hiring decisions. In other fields like computer science, the potential employers care far more about work samples and portfolios. While it is hard to make generalizations, for most fields, the reality is more like law. For many entry-level jobs, employers would be hard-pressed to come up with suitable work samples recent college graduates could submit, thus employers default to college reputation. Especially for the entry-level jobs that lead to elite jobs, employers recruit heavily - almost exclusively - from high-reputation schools, many going so far as to have dedicated H.R. teams for each school or special recruiting events. There are substantial employment advantages to going to an elite college in the person's initial job search that could have life-long effects. Once someone is shut out of this kind of entry-level job, it is hard to gain the experience to ever be considered for the culminating elite job.
The reputation of a college helps with starting a person out on the career path. A good start could have long-term financial benefits, so this might actually justify the reputation of some colleges as worth more. But my question is a different one. If what businesses are really getting is admission information, is this useful information? Are businesses right to think colleges are doing a good job selecting students?
Admission decisions
On the one hand, one can say colleges are selecting those students who are smart and hard-working - good traits for employers to seek. Let's stipulate that employers want to maximize both as much as possible; they want new employees with loads of content knowledge who can think flexibly. I am not going to engage with any question about the social implications of affirmative action or other admission policy, important though those questions are. I am merely addressing the question: Would employers be right to assume that the better the reputation of the school, the smarter and harder working its graduates?
After reading Mitchell Steven's book, Creating a Class, I realized that the defining fact for college admission officers is the lack of information. Despite S.A.T. and A.P. scores and other objective information, a lot of learning that students do is invisible. Can the student learn on their own, or do his or her scores hide heavy tutoring? Softer skills - like managing intellectual disagreements and debates, grit, research skills, and integrity - are hidden. Letters of recommendation only go so far to fill in the information gap. Smart, hard-working students at schools with overworked teachers and college counselors are at a disadvantage because they may not get high-quality letters. As a result, admission officers may revert to proxies, such as the reputation of the high school. (If employers are relying on the reputation of the college as a proxy, and colleges are relying on the reputation of the high school as a proxy...) This is what Shaun Harper found: great students at weak schools are overlooked.
To be fair to admission officers, students at weaker high school might never write an analytical essay, while students at great high school write one or more a week. The high school program does matter, but my point is that there are some students who would be capable but are not given the opportunity because of their high school's weak curriculum. While standardized tests are not great equalizers, without them, admission to selective colleges would be even more skewed to students who go to the best high schools. S.A.T. scores and A.P. scores give colleges some assurance that a student is exceptional despite attending a weak high school - but not enough to level the field. Colleges are not scooping up many hidden gems because they simply lack the information to do so. On top of this, of the smart, hard-working first-generation college students and minority students who do get in, many do not end up matriculating. So, these students also lack information. The bottom line is that college admission is not only about intellectual and personal capabilities but also about social capital.
Steven Pinker, the celebrity linguist at Harvard, points out facts about college admissions at selective schools that should unnerve everyone. Selective schools use holistic selection including academics, extracurriculars, and character. This disadvantages very smart but poor students who cannot afford to be well-rounded. Furthermore, it means that the student body, once at Harvard or other selective schools, spends a lot of its time in the same extracurriculars that helped them get in, and not as much on academics as one might expect. As Dr. Pinker heard a Harvard admission officer point out, their goal is not to train future academians but future leaders. (Is the fact that so many Harvardians go into finance -- the lucrative but well-beaten path -- an indication of the admission office's failure?) It is hard to believe that the extracurriculars are really a great proxy of leadership. Not to pick on any one activity here, but would an employer actually care that a person is an outstanding rower? singer?
And the other shocking issue is students from China. Most high schools there do not have college counselors, so a third-party system of packagers help get students into colleges. And the degree of fraud is truly shocking: 90% fake recommendations, 70% fake essays, and 50% fake high school transcripts. Check out the huge five-part expose in Reuters about cheating on the international S.A.T. test dates. Despite this, U.S. colleges continue to admit Chinese students in mass without demanding changes to the system. They could require gao kao scores. They could demand video interviews to prove English skills. Given that they do not make such demands, and given that colleges know the fraud problem, then do you have much faith in any part of the admission process? Colleges admission work is not so precise and thorough to justify business faith in college graduates absent other data. In light of this, college admissions officers' rhetoric that they are skilled at picking the best and brightest should make us incredulous -- and the effect of it on students is especially insidious. The data are just not that trustworthy to justify such boasts. (I would also add in the T. M. Landry scam -- which had to be at least very, very suspicious to college admissions officers, yet no one even went through the cursory steps of checking up on their data.)
This is not a plea to go to a fully objective system where only standardized test scores count. One only needs to look at the gao kao to see the dangers. I think it is fine for colleges to have subjective opinions about potential students, just like it is fine for students to have subjective opinions about which colleges they like the best! This is why I argued in a previous post for a matching system. Instead, my plea is for two things: college admissions should drop the rhetoric of infallibility. Just admit that the college is looking for students who clear a certain benchmark and who fit. Second, businesses should recognize that college admission is a fuzzy science at best. Sure, one might value candidates from highly selective colleges more than those from semi-selective colleges, but making distinctions between Harvard grads and Vassar grads is folly.
Businesses have created a self-fulfilling prophecy. By esteeming some undergraduate institutions far more than others, they have reinforced the reputation of those schools and created an admissions rush for those schools. The weak link in the system is that college admission cannot make the sort of fine distinctions it is presumed to. One way to break the cycle would be to prohibit employers from campus recruiting events and from asking about a candidate's undergraduate institution, but I doubt that would catch on. However, it is interesting to think about how employers would be forced to deal with evaluating 22 year olds if they only knew that he or she had an undergraduate degree in a chemistry but could not ask about the institution. Would they ask more questions about what the candidate had actually learned?
These problems all stem from the relative paucity of information about college outcomes. At the top end, the lack of information creates a mad scramble for a few schools of sterling reputation. In the middle, many students at flagship states schools and solid private colleges have their excellence overlooked. At the bottom end, many students who are capable of getting into more selective schools do not bother. Nor do they realize how much their decisions matter financially, or even how to seek out financial aid successfully. Enter the Obama administration's proposal to create college rankings. Will that help combat the information desert that currently exists?
Obama's college ratings
Obama has made a push for college ratings. They will largely eschew measures of quality, as it is so difficult to measure, and focus on graduation rates, affordability, and job prospects. One problem is that the schools that serve minority students, first-generation college students, and working students will by definition have lower graduation rates and unfortunately weaker job prospects. (While individual students might be improving their own prospects substantially, students at these schools as a group have weaker prospects than those for students attending more selective schools - a college degree knocks down some but not all barriers.) These schools may be doing a good job serving their students but get punished in the ratings, creating perverse incentives to not admit higher-risk students. It is worth including a social mobility score (i.e., SES diversity) in the rankings, which can help ensure colleges are compared to like-schools.
Graduation rates are relatively simple to compare. Instead of reporting the percentage of students who are done after six years, colleges should report median years until graduation (maybe also the 75th and 90th percentile years until graduation). Schools could be lumped into categories based on social mobility scores (perhaps six categories or so - all the most selective schools would go into one category, by the way) and compared against peer schools. Students could see whether one school is dramatically worse than its peers - maybe indicating that the school puts too little effort into counseling. Or the more mathematically accurate way would be to use a regression. Based on a college's student population demographics, schools could be ranked on the difference between what is the expected from the regression vs. the actual graduation rates.
What about affordability? First of all, the actual price, not the sticker price, ought to be used, so scholarships and other discounts ought to be factored in, plus the length of time it actually takes to complete the degree, and books. The other complication is boarding. Commuter schools need to be separated into a separate category. However, it is well worth the effort: giving simple prices to students would be a huge help to many first-generation college students! The hard part is that what people really care about is not the total cost of the degree but the cost compared to the expected earnings - so now we are talking about a combined metric. How about time to repay student loans at median earnings? This factors in the actual price of the degree and expected earnings, combining them into a number people can easily process: how long one will spend repaying loans, if the total cost is entirely borrowed. Twenty years repaying loans will be a sobering number for a lot of prospective students.
How is this data on earnings to be collected? Presumably, the federal government could track this through taxes (I would not trust the colleges too), although there are privacy issues with this tracking. The bigger statistical headache is people who are not working because of illness or injury, marriage, or graduate school. Skipping over these people could distort the earnings data considerably for some schools! This leads us to the other, biggest headache: different majors. Perhaps earnings data should be broken out by school by major or maybe by professional field people eventually go into: (1) math, computer science, natural science, and engineering; (2) business; (3) education, social work, and counseling; (4) humanities, journalism, and arts; (5) medicine; and (6) law. The school ratings might list years to repay full cost for graduates working in each of those fields. Job satisfaction and ability to find jobs in their desired field could be given a score too. There could be separate but similar questions for students who go to graduate school about whether they are happy with the graduate school they got into.
Social mobility / mission
Of course, the social mobility score is necessary to rate schools properly in different categories. And the federal government spends so much money on subsidizing loans, many loans should be going to schools that help students move up in society. It is worth it to report it explicitly. And it is worth reporting explicitly what fields graduates go into. Frankly, it is embarrassing how many Ivy League students go work on Wall Street. (Disclaimer: I went to an Ivy League undergraduate school.) If these students represent the best and the brightest, I would hope to see more in academic research, political leadership, social activism, education, and so on. It is not that surprising, but maybe seeing that fact given an explicit score, on a government webpage, will make some colleges recruit a little harder for people with an activist and not acquisitionist mindset. And it might make some qualified applicants who have no interest in consulting or finance think twice about going to an Ivy League school.
Of course, the federal government spends so much money on student loan subsidies that it could just decide to make every public university free. I think this might have a fascinating effect on the whole system judging from how university systems work in other countries. If the U.S. government changed direction 180 degrees and cut money for loans and grants and simply made public universities free, many highly qualified applicants - especially those from the middle class - would start to pick public universities over selective private schools. I do not believe that Ivy League schools would be hurt much, but other private schools would. They would see the quality of applicants and matriculants especially decline and their reputations suffer, while public universities would see theirs soar. In many countries where public universities are free, private schools have the weaker reputation. In the U.S., the Ivy League schools have so much money and prestige that their position is more or less secure, but flagship public universities and second-tier private schools might swap places in the reputation hierarchy.
An interesting take from Oliver Lee is to starve the system of money and let the most predatory schools collapse.
Conclusion
More or less, colleges have turned their selectivity into their competitive advantage: being hard to get in means their graduates must be desirable employees, and because employers seem to agree, the cycle is only reinforced as the next generation of students apply to elite schools in even greater numbers.
But basing reputation only selectivity is special kind of insanity. We do it only because education quality is hard to assess. There needs to be external verification to make the system fairer for students at every college, so that smart, hard-working students at any institution can get their due. While standardized tests are not perfect, they do help make college admission a bit fairer. Perhaps a dose of the same kind of medicine would help college graduates. I doubt college graduates will ever face a version of A.P. tests, but there is another option: digital learning badges. The idea is simple: any organization can serve as an external validator, certifying discrete skills that can be stacked into broader competencies. The source of the learning - selective college, community college, MOOC, self-taught, on-the-job learning - is irrelevant to the validator. Anyone can review the badge holder's work. If badges were to catch on, of course, graduates of selective schools would do well at acquiring them. But many students from less selective schools would do well, too. Maybe even people taking MOOCs would do well. Badges would have a profoundly democratizing, leveling force in college education because they provide a reliable source of data on what a person actually knows and has learned.
Postscript
In Dr. Pinker's article, I ran across this perfect description of what a good education should accomplish. My only reservation about badges is that it too many people might seek specific competencies and not a broad education as described below.
It seems to me that educated people should know something about the 13-billion-year prehistory of our species and the basic laws governing the physical and living world, including our bodies and brains. They should grasp the timeline of human history from the dawn of agriculture to the present. They should be exposed to the diversity of human cultures, and the major systems of belief and value with which they have made sense of their lives. They should know about the formative events in human history, including the blunders we can hope not to repeat. They should understand the principles behind democratic governance and the rule of law. They should know how to appreciate works of fiction and art as sources of aesthetic pleasure and as impetuses to reflect on the human condition.
On top of this knowledge, a liberal education should make certain habits of rationality second nature. Educated people should be able to express complex ideas in clear writing and speech. They should appreciate that objective knowledge is a precious commodity, and know how to distinguish vetted fact from superstition, rumor, and unexamined conventional wisdom. They should know how to reason logically and statistically, avoiding the fallacies and biases to which the untutored human mind is vulnerable. They should think causally rather than magically, and know what it takes to distinguish causation from correlation and coincidence. They should be acutely aware of human fallibility, most notably their own, and appreciate that people who disagree with them are not stupid or evil. Accordingly, they should appreciate the value of trying to change minds by persuasion rather than intimidation or demagoguery.
More about admissions here and here.
