Authors: Luke Harding,David Leigh
And there was another problem. How was Davies to get the Afghan material back to the
Guardian
in London? He could, of course, save it on a memory stick, but this ran the risk that British officials might confiscate it at customs control. Assange,
the hacking prodigy, offered the answer: he would transfer the material in encrypted form to a special website. The website would only exist for a short period before disappearing.
Reopening his netbook, Assange typed away and then circled words on the Hotel Leopold napkin. They were the password to decrypt data downloadable from the temporary website he would set up, encrypted in GPG (also known by its generic name, Pretty Good Privacy or PGP). Without the password, the website would be virtually uncrackable unless an opponent happened to stumble on the two large prime numbers which generated the encryption. Armed with the password,
Guardian
staff would soon be able to access the first tranche of data – the Afghan war logs. The three other promised “packages” were to follow.
The two men agreed on other precautions: Davies would send Assange an email saying that no deal had been agreed. (Written on 23 June, it read: “I’m safely back at base. Thanks for spending time with me – no need to apologise for not being able to give me what I’m after.”) The idea was to throw dust in the eyes of the Americans. Assange and Davies parted.
Davies grabbed a pastry and a cup of railway station coffee the following dawn and took the first train back to London. In the office he bumped into Rusbridger. “I’m going to tell you a secret,” he said. According to Davies, the owlish Rusbridger’s reaction was, as ever, understated. But he clearly appreciated the implications. By 9.30am he had agreed to ring Bill Keller, his
New York Times
counterpart, as soon as he woke across the Atlantic.
Heading back to his home in Sussex, Davies waited for news from Assange. Mid-morning on 24 June an email arrived directing Davies to the website. He downloaded the huge file, but was unable to disentangle the procedure required for GPG decryption. He phoned his local computer specialist, who was unable to help. Frustrated, Davies put the still-encrypted data on to a memory stick, and deleted Assange’s email. Soon afterwards the website
ceased to exist. Davies traveled back up to London and handed the stick to Harold Frayman, systems editor at the Guardian Media Group. Frayman easily downloaded the contents as a decrypted spreadsheet. “It wasn’t actually a terribly difficult thing to do at all. We knew what the password was,” Frayman said calmly.
So by that evening the
Guardian
had the Afghan database – an unprecedented hour-by-hour portrait of the real, harsh war being fought in the mountains and dusty streets of the Hindu Kush. But it didn’t look like it at the time: for the first five or six days the Afghan record proved almost impossible to read. “It was a fucker,” Davies said. “The spreadsheet was terribly difficult to extract information from, slow and difficult.” Nonetheless, he sent a triumphant email back to Assange. It read: “The good guys have got the girls.”
Fourth floor, the
Guardian
, Kings Place, London
July 2010
“
It felt like being a kid in a candy shop
”
D
ECLAN
W
ALSH, THE
G
UARDIAN
In the small, glass-walled office on the
Guardian
’s fourth floor, maps of Afghan and Iraqi military districts were stuck with magnets on to a whiteboard. Alongside them, the journalists were scrawling constantly updated lists of hitherto unknown US military abbreviations. “What’s EOF?” a reporter would shout? “Escalation of force!” someone would answer. HET? Human Exploitation Team. LN? Local national. EKIA was the body count: enemy killed in action. There were literally hundreds of other jargon terms: eventually the paper had to publish a lengthy glossary alongside its stories.
The discreet office, well away from the daily news operation, had become a multinational war room, with reporters flown in from Islamabad, New York, and eventually Berlin to analyse hundreds of thousands of leaked military field reports. They jostled with London-based computer experts and website specialists. A shredder was installed alongside the bank of six computer
screens, and the air of security was intensified by the stern notice stuck on the door: “Project Room. Private & Confidential. No Unauthorised Access.”
Nick Davies was so fixated by secrecy that he initially even refused to tell the
Guardian
’s head of news, deputy editor Ian Katz, about the project. He was dismayed to discover how quickly word spread that he was involved in a top-secret story. Another colleague, Richard Norton-Taylor, the
Guardian
’s veteran security editor, soon asked Davies about his “scoop”. Davies refused to tell him. A couple of hours later Norton-Taylor encountered Davies again, and teased him gleefully: “I know all your secrets!” A newspaper office is a bad place in which to try and keep the lid on things for very long.
The paper’s staff did do their best, however. Declan Walsh, the
Guardian
’s Pakistan-based correspondent, was recalled in conditions of great secrecy. Meeting round a table in the editor’s office, the
Guardian
’s team chewed over the technical difficulties. David Leigh was cantankerous: “It’s like panning for tiny grains of gold in a mountain of data,” he complained. “How are we ever going to find if there are any stories in it?” The answer to that question set the
Guardian
’s old hands on a steep learning curve as they got to grips with modern methods.
First they discovered, embarrassingly, that their first download, the Afghan spreadsheet, did not contain 60,000 entries, as they had spent several days believing. It contained far more. But the paper’s early version of Excel software had simply stopped reading after recording 60,000 rows. The real total of hour-by-hour field reports – the war logs – amounted to 92,201 rows of data. The next problem was greater still. It transpired that a spreadsheet of such enormous size was impossibly slow to manipulate, although it could theoretically be sorted and filtered to yield reams of statistics and different types of military event. The Iraq war logs release dumped another 391,000 records into their laps, which quadrupled the data problems.
Harold Frayman, the technical expert, solved those problems: he improvised at speed a full-scale database. Like Google, or sophisticated news search engines such as LexisNexis, the Frayman database could be searched by date, by key word, or by any phrase put between quotation marks. Declan Walsh recalls: “When I first got access to the database, it felt like being a kid in a candy shop. My first impulse was to search for ‘Osama bin Laden’, the man who had started the war. Several of us furiously inputted the name to see what it would produce (not much, as it turned out).” Leigh, too, began to cheer up: “Now this data is beginning to speak to me!” he said.
Leigh was introduced to another
Guardian
specialist, Alastair Dant: “Alastair’s our data visualiser,” he was told. Leigh: “I didn’t know such a job existed.” He was soon brought up to speed. The WikiLeaks project was producing new types of data. Now they needed to be mined with new kinds of journalism. Dant explained that he could convert the statistics of the thousands of bomb explosions recorded in the Afghan war logs into a bespoke moving graphic display. He could use the same basic template with which the
Guardian
had formerly developed a popular interactive map of the Glastonbury festival. That had been a nice bit of fun for music fans. The viewer had been able to move a pointer over a map of the festival field, and up came the artists playing at that spot, at that particular time.
Now, with Afghanistan, the viewer would be able similarly to press a button, but this time a much more chilling display would start to run. It would reveal, day by day and year by year, the failure of the US army to contain the insurgents in Afghanistan, as literally thousands of “improvised explosive devices” blossomed all around the country’s road system. The viewer could see how the vast majority of the roadside bombs were slaughtering ordinary civilians rather than military opponents, and how the assaults ebbed and flowed with changes in political developments. It was
a rendering that made at least something comprehensible, in an otherwise scrappy and ill-reported war.
The key online expert proved to be Simon Rogers, the
Guardian
’s data editor. “You’re good with spreadsheets, aren’t you?” he was asked. “This is one hell of a spreadsheet,” he said. After working on those spreadsheets, he concluded: “Sometimes people talk about the internet killing journalism. The WikiLeaks story was a combination of the two: traditional journalistic skills and the power of the technology, harnessed to tell an amazing story. In future, data journalism may not seem amazing and new; for now it is. The world has changed and it is data that has changed it.”
One obvious opportunity was to obtain genuine statistics of casualties for the first time. The US military had asserted, disingenuously, that at least as far as civilians and “enemies” were concerned, there were no figures available. In fact, the journalists could now see that the war logs contained highly detailed categories that were supposed to be filled in for every military event, breaking them down into US and allies, local Iraqi and Afghan forces, civilians and enemy combatants, and classing them in each case as either killed or wounded. But it wasn’t so simple. Rogers and his reporter colleagues had to grapple with the realities on the military ground: those realities made apparently enticing data sets into dirty and unreliable statistics.
At its simplest, a person listed as “wounded” at the time might have actually died later. More sweepingly, the casualty boxes were sometimes not filled in at all. The reporters felt sympathy with exhausted soldiers, after a day of fighting, being confronted with forms to input that required the filling in of no fewer than 30 fields of bureaucratic information. Some units were more meticulous than others. Early years of the wars saw sketchier information gathering than later, when systems were better organised. When there was heavy urban fighting, or when bodies were carried away, casualties were hard to count. Some units had a penchant for writing
down improbably large numbers of purported “enemy killed in action”. Sometimes, more sinisterly, civilians who were killed were recorded as “enemy”. That avoided awkward questions for the troops. All the figures were in any event too low, because some months and years were missing. So were details from the special forces, who operated outside the normal army chains of command. And many of the clashes involving British, German and other “allies” were apparently not recorded on the US army database.
So it was a tricky task to produce statistics that could be claimed to have real value. That highlighted once again the inescapable limitations of the purist WikiLeaks ideology. The material that resided in leaked documents, no matter how voluminous, was not “the truth”. It was often just a signpost pointing to some of the truth, requiring careful interpretation.
Assange himself eventually flew into London from Stockholm late one night in July 2010. He arrived in the
Guardian
office with nothing but his backpack and a shy smile, like one of the Lost Boys out of
Peter Pan
. “Have you anywhere to stay?” asked Leigh. “No,” he said. “Have you had anything to eat?” Again the answer was no. Leigh walked him down the road to the brasserie which was still open at St Pancras station and presented him with the menu. Assange ate 12 oysters and a piece of cheese, and then went to stay the night at Leigh’s flat in nearby Bloomsbury.
He spent several days there, sleeping in the day and working on his laptop through the night. Then he moved to a nearby hotel, spent the World Cup final weekend at Nick Davies’ Sussex home (but, says Davies, “He wasn’t the slightest bit interested in football”) and settled for a while at the Pimlico townhouse of Gavin MacFadyen, the City University professor and journalist. Assange brought with him only three pairs of socks. But he swiftly charmed the MacFadyen household, borrowed poetry books from the shelves, and patiently explained the Big Bang, complete with
mathematical formulae, to some wide-eyed visiting children. The only uncomfortable moment came over a meal of risotto, cooked by Sarah Saunders, a gourmet caterer and the daughter of MacFadyen’s wife, Susan. Typically, Assange would tap at his laptop throughout meals; other WikiLeaks volunteers who came and went did the same thing. On this occasion Saunders told him to turn his laptop off. Assange, to his credit, instantly complied.
A month later, he was provided with a bigger base for his growing organisation at the journalists’ Frontline Club in west London. Something about the wandering Assange made a succession of people he encountered want to look after him and protect him – even if that sentiment was not always enduring.
The team flowing in and out of the
Guardian
war room was also growing in size. The
Guardian
’s two distinguished veterans of the Iraq conflict, Jonathan Steele and James Meek, were co-opted. The executive editor of the
New York Times
, Bill Keller, sent over Eric Schmitt, his highly experienced war correspondent. Schmitt, whose knowledge of the military background was helpful, was able to report back that the war logs seemed authentic. He put them on a memory stick and flew home to start the process of building a database in New York.
The German contingent, too, were able to make a crucial contribution to the verification process. As the broker of the original deal with the
Guardian
and the
New York Times
, Nick Davies had not at first been entirely pleased with the arrival of
Der Spiegel
– a prospect that had only been tentatively mentioned at the Brussels meeting by his colleague Ian Traynor. Assange told him that lunch with
Der Spiegel
was taking place in Berlin. Then, in a phone call from a man calling himself Daniel Schmitt – actually Assange’s then No 2, Daniel Domscheit-Berg – he was told not only
Der Spiegel
but also a German radio station would be full “media partners” on the war logs. “I felt very confused. My first instinct was to say no,” Davies recalled. “A deal is a deal. Security is very important.
I felt: ‘You can’t come in.’” Davies eventually agreed that while German radio was out,
Der Spiegel
could be in. Their reporters John Goetz and Marcel Rosenbach flew over to the war room.