The Long Tail (18 page)

So bottom line: Human attention is more expandable than money. The primary effect of the Long Tail is to shift our taste toward niches, but to the extent we’re more satisfied by what we’re finding, we may well consume more of it. We just won’t necessarily pay a lot more for the privilege.

SHOULD PRICES RISE OR FALL DOWN THE TAIL?

I’m often asked about the effect of the Long Tail on pricing. Should prices go down with demand as you travel down the Tail? Or should they rise, as more specific and narrowly focused goods appeal more strongly to their niche audiences?

The answer is that it depends on the product. One way to look at it is to distinguish between “want” markets and “need” markets, each of which has different implications for pricing.

Need markets are those in which customers know what they’re looking for and just can’t find it anywhere but, say, online. Take, for instance, a relatively hard-to-find nonfiction book on a topic of keen interest to you. When you find it, you’re probably going to be relatively price insensitive. You can see this effect writ large in the discounting policy at Amazon. The online bookseller discounts best-sellers by 30 to 40 per
cent, gradually reducing the discount until it’s around zero for the books with sales ranks in the hundred thousands.

By comparison, music and other forms of entertainment are typically “want” markets. For the right price, you can be encouraged to try something new, venturing down the Tail with diminished risk of wasting your money. Thus, many music labels have experimented with discount pricing for their older titles and more obscure new acts.

The ultimate manifestations of this would be dynamic variable pricing, where prices for music would automatically fall with popularity. That is, in fact, what Google does with its automatic auctions for keyword ads, and what eBay’s similar auctions do for everything else. The more demand there is, the higher the price goes.

A really efficient variable pricing market would presumably lead to a more gradual sales decay, and a flatter demand curve overall. But for music, at least, the adoption of such a model runs up against the advantages of single-price simplicity (as in iTunes’ fixed-price $0.99 model) and the perils of dreaded “channel conflict” with CD retailers who cannot so easily change their prices. As the music industry gets more desperate it will probably grow more bold in its search for new business models. And then we’ll have better data with which to answer this question.

“MICROSTRUCTURE” IN THE LONG TAIL

One of the features of powerlaws is that they are “fractal,” which is to say that no matter how far you zoom in they still look like powerlaws. Mathematicians describe this as “self-similarity at multiple scales,” but what it means is that the Long Tail is made of many mini-tails, each of which is its own little world.

When you look closely at the data, you can see that the big powerlaw curve of, say, “music” is really just the superposition of all the little powerlaw curves formed by each musical genre. Music is made up of thousands of niche micromarkets, miniature ecosystems that, when smooshed together into an overall ranking, look like one Long Tail. But look closer and each has its own head and tail.

As an example, I’ve broken out the Long Tail of music on Rhapsody by genre (just the A’s), plotting the average track rank in terms of downloads for each genre on the plot below. I then broke out one—Afro-Cuban Jazz—even further, showing the curve of track popularity
within
that genre.

What you can see is that the genre averages themselves make up a Long Tail, and within each genre there is another Long Tail of individual tracks. And so it goes for the entire music universe, which appears to be one big popularity curve but is actually curves within curves within curves.

The same is true for other markets, from books to blogs. Peter Hirshberg, an executive at the blog search company Technorati, describes the emergence of “topical Long Tails” that the company has been tracking, showing the popularity powerlaw for such categories as cooking blogs and parenting blogs. “As when you apply a prism to white light, there is a spectrum of individual long tail communities in the blogosphere,” he says. Rankings are most meaningful
within
such communities, not across them.

Why does this matter? First, because it suggests that the filtering is often most effective at the genre level rather than across the entire market. And second, because it explains an apparent paradox of the Long Tail. The characteristic steep falloff shape of a popularity powerlaw comes from the effect of powerful word-of-mouth feedback loops that amplify consumer preference, making the reputation-rich even richer and the reputation-poor relatively poorer. Success breeds success. In network theory such positive feedback loops tend to create winner-take-all phenomena, which is another way of saying that they’re awesome hit-making machines.

Compounding matters, today’s filters make word-of-mouth even more powerful by measuring so much more of it from so many more people and for so many more products. Shouldn’t that then have the effect of making the powerlaw even steeper,
increasing
the gap between hits and niches rather than having a leveling effect?

In other words, why don’t network-effect recommendation systems, which are essential in driving demand down the Tail, actually do the opposite: drive content
up
the Tail, further amplifying hit/niche inequality? That’s what you’d expect with more powerful network effects, yet what we actually see in Long Tail markets is a flattened powerlaw, with
less
of a difference between hits and niches.

The explanation, it turns out, is that these filters and other recommendation systems actually work most strongly at the niche level, within a genre and subgenre. But
between
genres their effect is more muted. There are breakout hits that rise to the top of a genre and then go on to become mainstream hits, topping the overall charts. Yet they’re the exception. More common are titles that use their genre popularity to break into the middle of the overall charts, at which point they have to compete with many other hits from other genres and thus tend to not rise much farther.

Thus the most popular “ambient dub” artist at the very head of the ambient dub popularity curve can hugely outsell the others in that category, but that doesn’t mean that artist will snowball and tear up the charts to knock 50 Cent out of the top ten. The lesson from this microstructure analysis is that popularity exists at multiple scales, and ruling a clique doesn’t necessarily make you the homecoming queen.

THE LONG TAIL OF TIME

Why are some things less popular than others? So far we’ve been focusing mostly on the depth and breadth of certain items’ appeal—how mainstream or niche they are, or how high or low their quality. But there is another factor that influences popularity: age. Just as things of broad appeal tend to sell better than things of narrow appeal, new things tend to sell better than old things.

When you look at a basic demand curve, the reasons why some things sell less well than others are lost in the merged rankings. But popularity is actually multidimensional: Factors that determine an album’s rankings, for instance, can include not just the quality of the music but also its genre, its release date, the fame and/or nationality of the band, similarity to other artists, and so on. Yet it’s all blended into the single dimension of a best-seller list, which obscures all those factors in a mushed-together mélange.

If you think about it, today’s hit is tomorrow’s niche. Almost all products, even hits, see their sales decay over time.
Twister
was the number two movie of 1996, but its DVD version is now outsold two-to-one on Amazon by a 2005 History Channel documentary on the French Revolution.

Einstein described time as the fourth dimension of space; you can think of it equally as the fourth dimension of the Long Tail. Both hits and niches see their sales slow over time; hits may start higher, but they all end up down the Tail eventually. The research to quantify this conclusion is continuing, but conceptually the picture looks like the graph on the next page.

What’s particularly interesting about time and the Long Tail is that Google appears to be changing the rules of the game. For online media, like media anywhere, there is a tyranny of the new. Yesterday’s news is fishwrap, and once content falls off the front page of a Web site, its popularity plummets. But as sites find more and more of their traffic coming from Google, they’re seeing this rule break.

Google is not quite time-agnostic, but it does measure relevance mostly in terms of incoming links, not newness. So when you search for a term, you’re more likely to get the
best
page than the
newest
one.
And because older pages have more time to attract incoming links, they sometimes have an advantage over the newer ones. The result is that the usual decay of popularity for blog posts and online news pages is now much more gradual than it was, thanks to the amount of traffic that comes via search. Google is in a sense serving as a time machine, and we’re just now being able to measure the effect this has on publishing, advertising, and attention.

THE TRAGICALLY NEGLECTED ECONOMICS OF ABUNDANCE

Broadly, the Long Tail is about abundance. Abundant shelf space, abundant distribution, abundant choice. How awkward, then, that one of the definitions of economics given by Wikipedia is:

eco-nom-ics:
n
The social science of choice under scarcity.

There are other definitions (“the allocation of scarce resources to satisfy unlimited wants,” and so on), but many share the same troublesome element: a focus on scarcity, especially on how to allocate scarce resources. In an age of abundance in the form of everything from Moore’s law (the observation that computer price/performance doubles every eighteen months) to its equivalents in storage and bandwidth, this is a problem.

It’s hard to overstate how fundamental to economics the notion is that you can’t have it all for free—the entire discipline is oriented around studying trade-offs and how they’re made. Adam Smith, for instance, created modern economics by considering the trade-off between time, or convenience, and money. He discussed how a person could live near town, and pay more for rent of his home, or live farther away and pay less, “paying the difference out of his convenience.” And since then, economics has been all about how to divide finite pies.

That’s just the way it is. Neoclassical economics explicitly does not deal with abundant inputs. It doesn’t deny that oxygen is free when you’re trying to light a fire; it just doesn’t include that in its equations. It leaves that to other disciplines, such as chemistry.

But we are entering the era of effectively infinite shelf space. Two of the main scarcity functions of traditional economics—the marginal costs of manufacturing and distribution—are trending to zero in Long Tail markets of digital goods, where bits can be copied and transmitted at almost no cost at all. Surely economics has something to say about that?

Clearly abundance (also known as “plentitude”) is all around us, especially in technology. Moore’s Law is a classic example. What Carver Mead, the semiconductor pioneer and Caltech professor, recognized in 1970 when he encouraged his students to “waste transistors” was that transistors were becoming abundant, which is to say effectively free. The shift in thinking from making the most of scarce computing resources to “wasting” cycles by, say, drawing windows and icons on the screen led to the Mac and the personal computing revolution. To say nothing of the scandalous profligacy—a supercomputer used for fun!—of an Xbox 360.