The Everything Store: Jeff Bezos and the Age of Amazon (30 page)

After O’Reilly’s visit, Bezos convened a meeting with Rick Dalzell, Neil Roseman, and Colin Bryar, the head of Associates at the time, to talk about the issue. Dalzell pointed out that there was already something like this under way inside the company and told Bezos about a young engineer named Rob Frederick whose mobile commerce startup, Convergence, Amazon had acquired in 1999. Frederick’s group was working on APIs that would allow non-PC mobile devices like phones and PalmPilots to access the Amazon store. After that meeting, Bezos invited O’Reilly to speak to a group of engineers, and later at an Amazon all-hands meeting, about lessons from computer history and the importance of becoming a platform.

Bezos added Frederick’s team to the Associates group under Colin Bryar and tasked them with creating a new set of APIs to let developers plug into the Amazon website. Soon other websites would be able to publish selections from the Amazon catalog, including prices and detailed product descriptions, and use its payment system and shopping cart. Bezos himself bought into the Web’s new orthodoxy of openness, preaching inside Amazon over the next few months that they should make these new tools available to developers and “let them surprise us.” The company held its first developer conference that spring and invited all the outsiders who were trying to hack Amazon’s systems. Now developers became another constituency at Amazon, joining customers and
third-party sellers. And the new group, run by Colin Bryar and Rob Frederick, was given a formal name: Amazon Web Services.

It was the trailhead of an extremely serendipitous path.

Amazon Web Services, or AWS, is today in the business of selling basic computer infrastructure like storage, databases, and raw computing power. The service is woven into the fabric of daily life in Silicon Valley and the broader technology community. Startups like Pinterest and Instagram rent space and cycles on Amazon’s computers and run their operations over the Internet as if the high-powered servers were sitting in the backs of their own offices. Even large companies rely on AWS—Netflix, for example, uses it to stream movies to its customers. AWS helped introduce the ethereal concept known as the cloud, and it is viewed as so vital to the future fortunes of technology startups that venture capitalists often give gift certificates for it to their new entrepreneurs. Various divisions of the U.S. government, such as NASA and the Central Intelligence Agency, are high-profile AWS customers as well. Though Amazon keeps AWS’s financial performance and profitability a secret, analysts at Morgan Stanley estimate that in 2012, it brought in $2.2 billion in revenue.

The rise of Amazon Web Services brings up a few obvious questions. How did an online retailer spawn such a completely unrelated business? How did the creature that was originally called Amazon Web Services—the group working on the commerce APIs—evolve into something so radically different, a seller of high-tech infrastructure? Early observers suggested that Amazon’s retail business was so seasonal—booming during the holiday months—that Bezos had decided to rent his spare computer capacity during the quieter periods. But that explanation is widely debunked by Amazon insiders, in part because it would require Amazon to kick developers off its servers every fall.

The shift to offering these infrastructure services actually began with the transition to Gurupa and a more reliable technology infrastructure, a process that gathered momentum in 2003. While
Amazon’s internal systems had been broken down into more durable individual components, Amazon’s technical staff was still organized conventionally as a single team, headquartered in a separate office building downtown near Seattle’s Union Station. This group strictly controlled who could access Amazon’s servers, and various teams inside the company had to plead for resources to try out their new projects and features. The process slowed down and frustrated many Amazon project managers. “You had a set of folks running these machines who were the priesthood of hardware, and the rest of us were railing against it,” says Chris Brown, a software-development manager at the time. “We wanted a playground where we could go to freely try things out.”

Bezos was getting annoyed as well. The company had improved on its pick-to-light system in the FCs, and its infrastructure had been successfully recast into component services, but the provisioning of computer resources remained a bottleneck. It got so dysfunctional that project leaders would present the S Team with their six-page narratives and then in the discussion afterward admit they had been unable to actually test their projects. Rick Dalzell recalls a particularly significant meeting when Matt Round, the head of Personalization at the time, complained that he didn’t have resources for experimentation. “Jeff finally just exploded at me,” Dalzell says. “I always handled Jeff’s outbursts pretty well, but to be honest about it, he had a right to be angry. We were stifling the flow of creativity. Even though we were probably faster than ninety-nine percent of companies in the world, we were still too slow.”

At the same time, Bezos became enamored with a book called
Creation,
by Steve Grand, the developer of a 1990s video game called Creatures that allowed players to guide and nurture a seemingly intelligent organism on their computer screens. Grand wrote that his approach to creating intelligent life was to focus on designing simple computational building blocks, called primitives, and then sit back and watch surprising behaviors emerge. Just as electronics are built from basic components like resistors and capacitors, and as living beings spring from genetic building blocks, Grand wrote that
sophisticated AI can emerge from cybernetic primitives, and then it’s up to the “ratchet of evolution to change the design.”
⁹

The book, though dense and challenging, was widely discussed in the book clubs of Amazon executives at the time and it helped to crystallize the debate over the problems with the company’s own infrastructure. If Amazon wanted to stimulate creativity among its developers, it shouldn’t try to guess what kind of services they might want; such guesses would be based on patterns of the past. Instead, it should be creating primitives—the building blocks of computing—and then getting out of the way. In other words, it needed to break its infrastructure down into the smallest, simplest atomic components and allow developers to freely access them with as much flexibility as possible. As Bezos proclaimed at the time, according to numerous employees: “Developers are alchemists and our job is to do everything we can to get them to do their alchemy.”

Bezos directed groups of engineers in brainstorming possible primitives. Storage, bandwidth, messaging, payments, and processing all made the list. In an informal way—as if the company didn’t quite know the insight around primitives was an extraordinary one—Amazon then started building teams to develop the services described on that list.

In late 2004, Chris Pinkham, head of the company’s IT infrastructure, told Dalzell that he had decided to return with his family to their native South Africa. At this point, A9 had taken root in Palo Alto, and Dalzell was busy establishing remote developer centers in Scotland and India, among other places. Dalzell suggested to Pinkham that instead of leaving Amazon, he open an office in Cape Town. They brainstormed possible projects and finally settled on trying to build a service that would allow a developer to run any application, regardless of its type, on Amazon’s servers. Pinkham and a few colleagues studied the problem and came up with a plan to use a new open-source tool called Xen, a layer of software that made it easier to run numerous applications on a single physical server in a data center.

Pinkham took colleague Chris Brown along with him to South Africa and they set up shop in a nondescript office complex in
Constantia, a winemaking region northeast of Cape Town, near a school and a small homeless encampment. Their efforts would become the Elastic Compute Cloud, or EC2—the service that is at the heart of AWS and that became the engine of the Web 2.0 boom.

EC2 was born in isolation, with Pinkham talking to his colleagues in Seattle only sporadically, at least for the first year. The Constantia office had to make do with two residential-grade DSL lines, and during the hot summer of 2005, one of the country’s two nuclear reactors went offline, so engineers worked amid rolling brownouts. Pinkham later said that the solitude was beneficial, as it afforded a comfortable distance from Amazon’s intrusive CEO. “I spent most of my time trying to hide from Bezos,” Pinkham says. “He was a fun guy to talk to but you did not want to be his pet project. He would love it to distraction.”

The dozen engineers concurrently developing what would become Amazon’s Simple Storage Service, or S3, did not have that luxury, despite their best attempts to keep to themselves. They worked in an office on the eighth floor of Pac Med, ate lunch together every day for nearly two years, and often played cards together after work. Neither they nor their manager, Alan Atlas, a veteran of crosstown digital-media startup Real Networks, were able to hide half a world away.

Bezos was deeply interested in the evolution of Web services and often dived into the minutiae of S3, asking for details about how the services would keep up with demand and repeatedly sending engineers back to the drawing board to simplify the S3 architecture. “It would always start out fun and happy, with Jeff’s laugh rebounding against the walls,” Atlas says. “Then something would happen and the meeting would go south and you would fear for your life. I literally thought I’d get fired after every one of those meetings.”

Atlas said that while working on the S3 project, he frequently had difficulty grasping just how big Bezos was thinking. “He had this vision of literally tens of thousands of cheap, two-hundred-dollar machines piling up in racks, exploding. And it had to be able
to expand forever,” Atlas says. Bezos told him, “This has to scale to infinity with no planned downtime. Infinity!”

During one meeting, Atlas blundered by suggesting they could figure out how to keep up with any unexpected growth after the service launched. That triggered a Bezos nutter. “He leaned toward me and said, ‘
Why are you wasting my life?,
’ and went on a tirade about Keystone Cops,” Atlas says. “That was real anger. I wasn’t keeping up with him. There were a number of times like that. He was so far ahead of us.”

For the launch of Simple Storage Service, Atlas had commemorative T-shirts made up for his colleagues; he used the design of Superman’s costume but with an
S3
rather than an
S
on the chest. Naturally, he had to pay for the shirts himself.

As their ambitions for Web services expanded between South Africa and Seattle, Bezos and Dalzell began to consider who would lead the effort. Bezos suggested Al Vermeulen, Amazon’s chief technology officer, but Al V. commuted to Seattle by plane every day from Corvallis, Oregon, and said he didn’t want an administrative job. He demoted himself to an engineer working on S3 with Alan Atlas. So Dalzell recommended Andy Jassy, who had inauspiciously started his career at Amazon so many years ago by hitting Jeff Bezos with a kayak paddle during the company’s first game of broomball.

If the new era in high-tech was indeed to be the age of computer science PhDs, then Jassy would prove to be a conspicuous anomaly. A graduate of Harvard Business School with a passion for buffalo wings and New York sports teams, Jassy seemed unlikely to fit in at a geeky technology startup. Perhaps as a result of this, his path at Amazon had been meandering and sometimes difficult. It was Jassy who presented the original business plan in 1998 for Amazon to enter the music business, but then he watched in disappointment as another executive was selected to lead the charge. A few years later, in a companywide reorganization, Jassy was chosen to oversee the personalization group, but then the engineers in that department
objected to being led by someone they viewed at the time as nontechnical.

So Jassy was given a unique opportunity. Bezos asked him to become his first official shadow—a new role that would entail Jassy’s following around the CEO and sitting with him in every meeting. Other technology companies like Intel and Sun had similar positions, and Bezos had tried this before with executives who were new to the company—including a D. E. Shaw engineer named John Overdeck and Accept.com founder Danny Shader—but it had never been a full-time job, and many of those previous shadows had subsequently left the company. Jassy was conflicted about the proposal. “I was flattered by the offer to work closely with Jeff but wasn’t initially excited about it because I’d seen the way it had gone down before,” he says. “I asked Jeff what success would look like. He said success would be if I got to know him and he got to know me and we built trust in each other.” Jassy agreed and spent much of the next eighteen months by Bezos’s side, traveling with him, discussing the events of each day, and observing the CEO’s style and thought process. Jassy would define the shadow role as a quasi chief of staff, and today the position of Bezos’s shadow, now formally known as technical adviser, is highly coveted and has broad visibility within the company. For Bezos, having an accomplished assistant on hand to discuss important matters with and ensure that people follow up on certain tasks is another way to extend his reach.

As Jassy’s tenure as shadow ended, he became a natural candidate to step in as the new head of AWS. One of his first jobs was to write a vision statement; he had to tinker with the margins to get it under six pages. The paper laid out the expanded AWS mission: “to enable developers and companies to use Web services to build sophisticated and scalable applications.” The paper listed the primitives that Amazon would subsequently turn into Web services, from storage and computing to database, payments, and messaging. “We tried to imagine a student in a dorm room who would have at his or her disposal the same infrastructure as the largest companies in the world,” Jassy says. “We thought it was a great playing-field leveler
for startups and smaller companies to have the same cost structure as big companies.”