Super Crunchers
Authors: Ian Ayres
CONTENTS
Introduction: The Rise of the Super Crunchers
1
                           Who's Doing Your Thinking for You?
3
                           Government by Chance
5
                           Experts Versus Equations
6
                           Why Now?
7
                           Are We Having Fun Yet?
8
                           The Future of Intuition (and Expertise)
For John Donohue and Peter Siegelman,
who were there when I needed them most
INTRODUCTION
The Rise of the Super Crunchers
Orley Ashenfelter really loves wine: “When a good red wine ages,” he says, “something quite magical happens.” Yet Orley isn't just obsessed with how wine tastes. He wants to know about the forces behind great and not-so-great wines.
“When you buy a good red wine,” he says, “you're always making an investment, in the sense that it's probably going to be better later. And what you'd like to know is not what it's worth now, but what it's going to be worth in the future. Even if you're not going to sell itâeven if you're going to drink it. If you want to know, âHow much pleasure am I going to get by delaying my gratification?' That's an endlessly fascinating topic.” It's a topic that has consumed a fair amount of his life for the last twenty-five years.
In his day job, Orley crunches numbers. He uses statistics to extract hidden information from large datasets. As an economist at Princeton, he's looked at the wages of identical twins to estimate the impact of an extra year of school. He's estimated how much states value a statistical life by looking at differences in speed limits. For years he edited the leading economics journal in the United States, the
American Economic Review
.
Ashenfelter is a tall man with a bushy mane of white hair and a booming, friendly voice that tends to dominate a room. No milquetoast he. He's the kind of guy who would quickly disabuse you of any stereotype you might have that number crunchers are meek, retiring souls. I've seen Orley stride around a classroom, gutting the reasoning behind a seminar paper with affable exuberance. When he starts out his remarks with over-the-top praise, watch out.
What's really gotten Orley in trouble is crunching numbers to assess the quality of Bordeaux wines. Instead of using the “swishing and spitting” approach of wine gurus like Robert Parker, Orley has used statistics to find out what characteristics of vintage are associated with higher or lower auction prices.
“It's really a no-brainer,” he said. “Wine is an agricultural product dramatically affected by the weather from year to year.” Using decades of weather data from France's Bordeaux region, Orley found that low levels of harvest rain and high average summer temperatures produce the greatest wines. The statistical fit on data from 1952 through 1980 was remarkably tight for the red wines of Burgundy as well as Bordeaux.
Bordeaux are best when the grapes are ripe and their juice is concentrated. In years when the summer is particularly hot, grapes get ripe, which lowers their acidity. And, in years when there is below-average rainfall, the fruit gets concentrated. So it's in the hot and dry years that you tend to get the legendary vintages. Ripe grapes make supple (low-acid) wines. Concentrated grapes make full-bodied wines.
He's had the temerity to reduce his theory to a formula:
Wine quality = 12.145 + 0.00117 winter rainfall + 0.0614 average growing season temperatureâ0.00386 harvest rainfall
That's right. By plugging the weather statistics for any year into this equation, Ashenfelter can predict the general quality of any vintage. With a slightly fancier equation, he can make more precise predictions for the vintage quality at more than 100 Châteaus. “It may seem a bit mathematical,” he acknowledges, “but this is exactly the way the French ranked their vineyards back in the famous 1855 classifications.”
Traditional wine critics have not embraced Ashenfelter's data-driven predictions. Britain's
Wine
magazine said “the formula's self-evident silliness invite[s] disrespect.” William Sokolin, a New York wine merchant, said the Bordeaux wine industry's view of Ashenfelter's work ranges “somewhere between violent and hysterical.” At times, he's been scorned by trade members. When Ashenfelter gave a wine presentation at Christie's Wine Department, dealers in the back openly hissed at his presentation.
Maybe the world's most influential wine writer (and publisher of
The Wine Advocate
), Robert Parker, colorfully called Ashenfelter “an absolute total sham.” Even though Ashenfelter is one of the most respected quantitative economists in the world, to Parker his approach “is really a Neanderthal way of looking at wine. It's so absurd as to be laughable.” Parker dismisses the possibility that a mathematical equation could help identify wines that actually taste good: “I'd hate to be invited to his house to drink wine.”
Parker says Ashenfelter “is rather like a movie critic who never goes to see the movie but tells you how good it is based on the actors and the director.”
Parker has a point. Just as it's more accurate to
see
the movie, shouldn't it be more accurate to actually
taste
the wine? There's just one catch: for months and months there is no wine to taste. Bordeaux and Burgundies spend eighteen to twenty-four months in oak casks before they are set aside for aging in bottles. Experts, like Parker, have to wait four months just to have a first taste after the wine is placed in barrels. And even then it's a rather foul, fermenting mixture. It's not clear that tasting this undrinkable early wine gives tasters very accurate information about the wine's future quality. For example, Bruce Kaiser, former director of the wine department at auctioneer Butterfield & Butterfield, has claimed, “Young wines are changing so quickly that no one, and I mean no one, can really evaluate a vintage at all accurately by taste until it is at least ten years old, perhaps older.”
In sharp contrast, Orley crunches numbers to find the historical relationship between weather and price. That's how he found out that each centimeter of winter rain tends to raise the expected price 0.00117 dollars. Of course, it's only a tendency. But number crunching lets Orley predict the future quality of a vintage as soon as the grapes are harvestedâmonths before even the first barrel taste and years before the wine is sold. In a world where wine futures are actively traded, Ashenfelter's predictions give wine collectors a huge leg up on the competition.
Ashenfelter started publishing his predictions in the late eighties in a semiannual newsletter called
Liquid Assets
. He advertised the newsletter at first with small ads in
Wine Spectator,
and slowly built a base of about 600 subscribers. The subscribers were a geographically diverse lot of millionaires and wine buffsâmostly confined to the small group of wine collectors who were comfortable with econometric techniques. The newsletter's subscription base was just a fraction of the 30,000 people who plunked down $30 a year for Robert Parker's newsletter
The Wine Advocate
.
Ashenfelter's ideas reached a much larger audience in early 1990 when the
New York Times
published a front-page article about his new prediction machine. Orley openly criticized Parker's assessment of the 1986 Bordeaux. Parker had rated the '86s as “very good and sometimes exceptional.” Ashenfelter disagreed. He felt that the below-average growing season temperature and the above-average harvest rainfall doomed this vintage to mediocrity.
Yet the real bombshell in the article concerned Orley's prediction about the 1989 Bordeaux. While these wines were barely three months in the cask and had yet to even be tasted by critics, Orley predicted that they would be “the wine of the century.” He guaranteed that they would be “stunningly good.” On his scale, if the great 1961 Bordeaux were 100, then the 1989 Bordeaux were a whopping 149. Orley brazenly predicted that they would “sell for as high a price as any wine made in the last thirty-five years.”
The wine critics were incensed. Parker now characterized Ashenfelter's quantitative estimates as “ludicrous and absurd.” Sokolin said the reaction was a mixture of “fury and fear. He's really upset a lot of people.” Within a few years
Wine Spectator
refused to publish any more ads for his (or any other) newsletter.
The traditional experts circled the wagons and tried to discredit both Orley and his methodology. His methodology was flawed, they said, because it couldn't predict the exact future price. For example, Thomas Matthews, the tasting director of
Wine Spectator,
complained that Ashenfelter's price predictions “were exactly true only three times in the twenty-seven vintages.” Even though Orley's “formula was specifically designed to fit price data” his “predicted prices are both under and over the actual prices.” However, to statisticians (and to anyone else who thinks about it for a moment), having predictions that are sometimes high and sometimes low is a good thing; it's the sign of an unbiased estimate. In fact, Orley showed that Parker's initial ratings of vintages had been systematically biased upward. More often than not Parker had to downgrade his initial rankings.
In 1990 Orley went even further out on a limb. After declaring 1989 the vintage of the century, the data told him that 1990 was going to be even better. And he said so. In retrospect we can now see that
Liquid Assets'
predictions have been astonishingly accurate. The '89s turned out to be a truly excellent vintage and the '90s were even better.
How can it be that you'd have two “vintages of the century” in two consecutive years? It turns out that since 1986 there hasn't been a year with below-average growing season temperature. The weather in France has been balmy for more than two decades. A convenient truth for wine lovers is that it's been a great time to grow truly supple Bordeaux.
The traditional experts are now paying a lot more attention to the weather. While many of them have never publicly acknowledged the power of Orley's predictions, their own predictions now correspond much more closely to his simple equation results. Orley still maintains his website, www.liquidasset.com, but he's stopped producing the newsletter. He says, “Unlike the past, the tasters no longer make any horrendous mistakes. Frankly, I kind of killed myself. I don't have as much value added anymore.”
Ashenfelter's detractors see him as a heretic. He threatens to de-mystify the world of wine. He eschews flowery and nonsensical (“muscular,” “tight,” or “rakish”) jargon, and instead gives you reasons for his predictions.
The industry's intransigence is not just about aesthetics. “The wine dealers and writers just don't want the public informed to the degree that Orley can provide,” Kaiser notes. “It started with the '86 vintage. Orley said it was a scam, a dreadful year suffering from a lot of rain and not high enough temperatures. But all the wine writers of the day were raving, insisting it was a great vintage. Orley was right, but it isn't always popular to be right.”
Both the wine dealers and writers have a vested interest in maintaining their informational monopoly on the quality of wine. The dealers use the perennially inflated initial rankings as a way to stabilize prices. And
Wine Spectator
and
The Wine Advocate
have millions of dollars at stake in remaining the dominant arbiters of quality. As Upton Sinclair (and now Al Gore) has said, “It is difficult to get a man to understand something when his salary depends on his not understanding it.” The same holds true for wine. “The livelihood of a lot of people depends on wine drinkers believing this equation won't work,” Orley says. “They're annoyed because suddenly they're kind of obsolete.”
There are some signs of change. Michael Broadbent, chairman of the International Wine Department at Christie's in London, puts it diplomatically: “Many think Orley is a crank, and I suppose he is in many ways. But I have found that, year after year, his ideas and his work are remarkably on the mark. What he does can be really helpful to someone wanting to buy wine.”
The Orley Ashenfelter of Baseball
The grand world of oenophiles seems worlds away from the grandstands of baseball. But in many ways, Ashenfelter was trying to do for wine just what writer Bill James did for baseball.
In his annual
Baseball Abstracts,
James challenged the notion that baseball experts could judge talent simply by watching a player. Michael Lewis's
Moneyball
showed that James was baseball's herald of data-driven decision making. James's simple but powerful thesis was that data-based analysis in baseball was superior to observational expertise:
The naked eye was inadequate for learning what you need to know to evaluate players. Think about it. One absolutely cannot tell, by watching, the difference between a .300 hitter and a .275 hitter. The difference is one hit every two weeksâ¦. If you see both 15 games a year, there is a 40 percent chance that the .275 hitter will have more hits than the .300 hitterâ¦. The difference between a good hitter and average hitter is simply not visibleâit is a matter of record.
Like Ashenfelter, James believed in formulas. He said, “A hitter should be measured by his success in that which he is trying to do, and that which he is trying to do is create runs.” So James went out and created a new formula to better measure a hitter's contribution to runs created: