Read The Bletchley Park Codebreakers Online
Authors: Michael Smith
The Iraqi attack against Kuwait on 1 August 1990 was arguably an event which GCHQ had been waiting for for three decades. The historic tensions in the area, together with Britain’s defence relationship with Kuwait, ensured that Britain had long devoted herself to deterring an Iraqi attack and also to providing some warning of such an attack if deterrence failed. In January 1965, Leonard Hooper took over from Clive Loehnis as Director of GCHQ. One of his first priorities was to discuss the future of a small team of GCHQ staff in Kuwait, whose task was to constantly sweep the airwaves for signs of an Iraqi attack. ‘While the team was a reasonable insurance factor in providing timely warning of an external attack,’ he explained to the Joint Intelligence Committee, ‘it could do little or nothing to give Sigint warning of internal unrest or a coup.’ Hooper was pressed for resources elsewhere and wondered whether this task could be transferred to the British facilities in Cyprus. But by monitoring from Cyprus a good deal of local Sigint would be lost and the Joint Intelligence Committee decided the GCHQ team should stay in place, not least because they did not want to suffer the ‘loss of tactical Sigint information on the Basra brigade’ of the Iraqi army.
During the Gulf War of 1990, one of the first warnings of the Iraqi attack was provided by satellite detection of Iraqi radar activity. Soviet-constructed, long-range radar units, codenamed ‘Tall King’, became very active on 29 June 1990, having been silent for some months. By mid-July 1990 a considerable programme of satellite monitoring
of Iraq – mostly through imagery – had been developed, which was expected to provide somewhere between twelve and twenty-four hours’ notice of an attack. But despite warnings of troops massing on the Kuwait border, the prevailing thinking in the White House was that this was a bluff. Once the war began in earnest it became clear that US training offered to Iraq in the 1980s had facilitated very secure communications, including the extensive use of landlines with fibre-optic cables and a great deal of inbuilt communications channel redundancy. It also appears that the Soviets may have given the Iraqis information about the timetable of US satellites passing overhead. British and American Sigint experts then became involved in complex discussion about how much of the Iraqi signals infrastructure should be left intact to permit continued monitoring, perhaps with the intention of locating Saddam Hussein, and also taking into account the additional factor of integrating deception operations which were distributing false orders to Iraqi battalions. The Iraqis were also adept in the field of sophisticated electronic deception, including maintaining silence on key military channels prior to an attack. British forces conducted a successful operational deception of their own, superintended by a unit codenamed ‘Rhino Force’. British transmissions from the First Armoured Division were recorded during the weeks of desert exercise before the final invasion. When the final attack was launched and the First Armoured Division moved west to join the US Seventh Army Corps, the messages were replayed in a manner that suggested the Division was moving east. Sigint showed that the Iraqis had bought this deception in its entirety.
The war in Yugoslavia, which followed hard on the heels of the Gulf War, raised very different kinds of questions for GCH Q. In a war which has seen the first head of state arraigned before an international war crimes tribunal, the intelligence services have taken a lead in preparing evidence for these unprecedented proceedings. In the mid-1990s, considerable debate ensued in Whitehall and Westminster as to the extent to which the capabilities of GCH Q should be revealed in the course of bringing war criminals to justice. To present the most compelling types of evidence would also be to reveal the minutiae of what British monitoring could achieve. In the event, a high-level decision was taken to allow some material to be presented before The Hague.
During the 1980s and 1990s, the Sigint agencies of most developed states had a tough time. They were confronted by the increasing use of
fibre-optic cables, which provided greater security for communications by states. They also had to address the important issue of public key cryptography by non-state actors and individuals. This is the system that allows most computers to send secure messages to each other without first exchanging ciphers, the basis of today’s commercially available cryptography packages and also an essential part of the security in systems for controlling nuclear weapons. The widespread use of increasingly powerful desk-top computers, combined with the availability of 128-bit encryption packages, has increasingly rendered the exponentially growing e-mail traffic impenetrable to the major Sigint agencies. During 1997, GCH Q put forward a complex solution to the problem of public key encryption involving escrow, but this collapsed after protests by private individuals and major corporations alike. In September 1999, the British Government announced that GCH Q would be a partner in a new project designed to address the increasing sophistication of e-mail encryption by criminal users including drug-runners.
There is no doubt that Downing Street is taking the threat of international organized crime very seriously. In early December 1999, Francis Richards, the director of GCHQ, joined the heads of MI6 and MI5 in an extended meeting at Number 10 on the ‘crime emergency’ facing Britain and how to tackle what Whitehall increasingly refers to as the ‘Red Mafia’. GCHQ will work closely with MI5 and the National Criminal Intelligence Service in a new unit, initially designated the Government Telecommunications Advisory Centre, intended to deal with the growing use of e-mail by organized crime. In July 2000 this was reinforced by the passing of the Regulation of Investigatory Powers Act. This involves GCHQ more closely in the complex and controversial territory of domestic monitoring, which is increasingly inseparable from international communications. It also blurs domains hitherto occupied primarily by the police, the post office and customs and excise.
It now appears that the concept of Public Key Cryptography, perhaps the most important development in secure communications for several centuries, was invented by James Ellis at GCHQ in the 1970s. Some would rank Ellis’s discovery alongside the contribution that Alan Turing made to the development of computing. But Ellis’s achievement was ahead of its time and so neither GCHQ nor the NSA could initially find any workable method of implementing the idea until a young GCHQ mathematician, Clifford Cocks, made the necessary
breakthrough, which was confirmed by Malcolm Williamson, a friend who joined GCHQ later. Shaun Wylie, one of GCHQ’s top cryptologists, raised the issue during a visit to Washington, but it did not seem practicable to anyone at the time. Later, Dr Whitfield Diffie discovered public key cryptography independently at a time when the development of desk-top computers rendered it a breakthrough development. Inevitably, some have wondered aloud whether the NSA and GCHQ simply wished to delay the advent of Public Key Cryptography for as long as possible. The advent of Public Key Cryptography and powerful personal computers is now reflected in the fact that one of the most significant developments at Cheltenham in recent years has not been material but organizational. The advent of ‘e-commerce’ and talk of a knowledge-based economy has heightened the importance of communications security, not only for Government departments but also for major industries and financial institutions in Britain. This has led to the expansion of GCHQ’s sister organization, the high-profile Communications Electronics Security Group.
The future shape of GCHQ is round – or to be more precise – bagel-shaped. By 2003 all of GCHQ’s activities will be located on the original Benhall site, mostly within a vast new circular building with an open centre. This will provide a state-of-the-art environment for the 4,500 staff of GCHQ, brought together on one site for the first time for many decades. This is the largest government private finance initiative and as a result GCHQ will not own, but instead will lease, its new quarters, the construction of which is costed at £330 million. Capital and running costs over thirty years are estimated at £800 million. This structure will be immense when completed and constitutes not only the largest building project ever initiated by the British Government but also the largest construction currently in progress in Europe. An underground road will be used to service the main building and computer halls and also to allow for the secure movement of documents. Above ground, an inner corridor will lead off the main entrance which will be surfaced with sixteen miles of carpet. When completed it will provide more than a million square feet of office space. Francis Richards described this arrangement as offering ‘maximum efficiency, better working conditions and excellent value for money’. However, given that the new MI6 and MI5 headquarters each cost more than three times their original estimated price, the projected figures might be looked on with some scepticism.
The vast space-age GCHQ building at Cheltenham and the expansion of the NSA presence at sites such as Menwith Hill and Digby are not best calculated to reassure all of Britain’s European allies. Menwith Hill has been at the centre of charges by the European Parliament about Anglo-American economic espionage in Europe. The French-led complaints about the UKUSA alliance’s so-called Echelon electronic surveillance operations culminated in a front-page article in
Le Monde
. This featured a cartoon of Britain as a mad cow, electronically eavesdropping from inside the European Union on behalf of her Anglo-Saxon intelligence allies. It is hard to resist the observation that French accusations mingle some naivety with a certain hypocrisy. Most states have openly asserted the importance of intelligence-gathering for the purpose of securing their economic well-being over a period of several decades and so these ‘revelations’ should come as no surprise. Moreover, the French have been as active as anyone else in this important field. France deploys its own GCHQ-type organization that is no less sophisticated than that to be found in Britain – it is located at Domme in the Dordogne Valley. Ironically, some of its more modern equipment has been supplied by the United States. The future appears to be one in which there will be more communications and more complex alliances, which in turn will mean more rather than less work for the new GCHQ.
As several chapters in this volume have demonstrated, Ultra was an Allied rather than a purely British triumph. Bletchley Park was the centre of a great Imperial and Commonwealth Sigint network which spanned the globe. The British–American Sigint alliance forged during the war has remained ever since the most special and the most secret part of the ‘special relationship’. Remarkably, Bletchley Park collaborated more closely with the US military and naval Sigint agencies than those agencies collaborated with each other. At the end of the war, Telford Taylor, the head of the Special Branch (US military Sigint) mission at Bletchley Park, expressed ‘pride at the ease, goodwill and success’ with which his mission had worked with the British, but reported that the problems of US
inter-service
Sigint rivalry ‘have not been solved by this war’. They were to recur in an acute form at the beginning of the Korean War and were not adequately resolved until the foundation of the National Security Agency (NSA) in November 1952 established roughly the same level of co-ordination achieved by the (admittedly much smaller) British GC&CS a generation earlier and continued after the war by GCHQ.
The first step in the transformation of wartime Sigint collaboration into a peacetime alliance was a tour of Imperial, Commonwealth and US Sigint stations in the spring of 1945 by Sir Edward Travis, head of Bletchley Park, his assistant Harry Hinsley, Rear-Admiral Edmund Rushbrooke, the Director of Naval Intelligence, and Commander Clive Loehnis (a future Director of GCHQ) of the Admiralty Operational Intelligence Centre. Travis, Hinsley and Loehnis visited Washington towards the end of their tour to put proposals for a peacetime Sigint alliance to representatives of the War, Navy and State Departments. Though ignorant of intelligence and deeply suspicious of peacetime espionage, Harry S. Truman, who became President on Roosevelt’s death, shortly before the British delegation reached Washington, was impressed by the insights given him by Sigint into the final months of the war against Japan. On 12 September 1945 he signed a top-secret one-sentence memorandum, providing for the peacetime continuation of the wartime Sigint alliance:
The Secretary of War and the Secretary of the Navy are hereby authorized to direct the Chief of Staff, U.S. Army and the Commander in Chief, U.S. Fleet, and Chief of Naval Operations to continue collaboration in the field of communication intelligence between the United States Army and Navy and the British, and to extend, modify or discontinue this collaboration, as determined to be in the best interests of the United States.
The details of the postwar Sigint alliance were embodied in the classified British–American agreements of March 1946 and June 1948 (the latter known as the UKUSA agreement), also involving Australia, Canada and New Zealand, which divided the world up into spheres of Sigint influence assigned to each of the five powers, and provided for extensive sharing of the intelligence obtained. British personnel continued for some years to play leading roles in the main Commonwealth Sigint agencies. The balance of power in the transatlantic Sigint alliance, however, which during the war had centred on Bletchley Park, passed decisively from Britain to the United States. A global Sigint network based on huge banks of advanced computers (which included both the first IBM mainframes and, later, the first Cray supercomputers), intercept stations around the world and Sigint personnel running eventually into six figures was simply too expensive for postwar Britain to afford, save as a junior partner.
Though Truman’s interest in Sigint faded rapidly after the end of the war in the Pacific, Ultra had turned his successor, General Dwight D. Eisenhower, into a lifelong enthusiast. Soon after his arrival in Britain as commander of American military forces in June 1942, Ike had been briefed personally on Ultra by Churchill during a visit to Chequers. So committed was Eisenhower to the intelligence alliance with Britain that he insisted on having a British rather than American officer as his G-2 (intelligence chief), who oversaw his supply of Ultra. When he became Supreme Commander of Allied Forces in Europe at the end of 1943, the War Office attempted to remove his G-2, Brigadier (later General Sir) Kenneth Strong. Eisenhower appealed direct to Churchill and Strong was allowed to stay. ‘The best time in a man’s life’, enthused Strong later, ‘is when he gets to like Americans.’ At the end of the war Eisenhower declared that Ultra had been ‘of priceless value’ to him and sent his ‘heartfelt admiration and sincere thanks’ to British cryptanalysts at Bletchley Park ‘for their very decisive contribution to the Allied war effort’. Eisenhower’s enthusiasm for Bletchley Park helps to explain his committed support as President for the NSA. Founded on the day of his election victory, the NSA received unprecedented resources during his two terms in the White House.
Although the total volume of Sigint available to the superpowers during the Cold War was much greater than during the Second World War, Soviet high-grade cipher systems – based on the theoretically unbreakable one-time pad – were far less vulnerable than those of Nazi Germany and Imperial Japan. In the immediate aftermath of the Second World War, the failure to produce a Soviet Ultra led to what a later top-secret US official inquiry described as ‘a sense of frustration and anti-climax’. Both American and British cryptanalysts, however, had some successes against Soviet traffic. Though most of these are still classified, one of them, the Venona operation, was declassified in the mid-1990s. Venona was the final codename given to approximately 3,000 Soviet intelligence and other classified telegrams intercepted during the period 1940 to 1948, which – as the result of errors in Soviet cipher production – used the same one-time pads more than once and thus became vulnerable to cryptanalytic attack. Most were decrypted, in whole or in part, by American code-breakers with some assistance from the British in the late 1940s and early 1950s.
Though the decrypts provided important information on Soviet espionage in regions of the world as far apart as Scandinavia and
Australia, the most numerous and important revelations concerned intelligence operations in the United States. Venona revealed that over 200 Americans were working as Soviet agents during, and sometimes after, the Second World War, and that the leadership of the American Communist Party was hand-in-glove with the KGB. Every section of the wartime administration of Franklin D. Roosevelt, from the foreign intelligence agency, the Office of Strategic Services (OSS) to the atomic programme, had been penetrated by Soviet intelligence. Had the terminally ill Roosevelt died in 1944, the penetration would have been worse still. In 1944, Roosevelt would have been succeeded not by Truman but by his then vice-president, Henry Wallace. In preparation for this eventuality Wallace had already selected some of his future administration. The men he had chosen to be his Secretary of State and Secretary of the Treasury, Larry Duggan and Harry Dexter White, were both Soviet agents, codenamed respectively Frank and Jurist. Venona also identified agent Homer as the British diplomat Donald Maclean, one of the ‘Magnificent Five’ young Cambridge graduates recruited by Soviet intelligence in the mid-1930s. The unmasking of Maclean led eventually to the identification of the other four and to the belated discovery that the ‘Fifth Man’, John Cairncross, had worked as a Soviet agent inside Bletchley Park at the turning point of the war on the Eastern Front in 1942–3.
The legacy of Ultra influenced the handling of Venona in two ways. First, it was treated with at least the same level of extreme secrecy. Basking in the reflected glory of Ultra, Anglo–American cryptanalysts found it much easier than before the war to persuade those with access to their intelligence to keep secret both their past successes and current operations. Secondly, as with Ultra during the Second World War, collaboration on Venona during the early Cold War worked better within the transatlantic Special Relationship than within the US intelligence community. GCHQ liaison officers at the postwar US military Sigint service, the Army Security Agency (ASA), were informed of the initial Venona breakthrough even before the FBI. By 1948, according to a recent CIA/NSA study, ‘there was complete and profitable US–UK cooperation’ on the project. As well as collaborating closely with GCHQ, the chief American cryptanalyst working on the Venona project, Meredith Gardner, had regular meetings in the later 1940s with the representatives in Washington of MI6 and MI5, Peter Dwyer and Geoffrey Patterson, both of whom provided information
on individuals referred to in the Soviet telegrams to assist the process of decryption. GCHQ also regularly briefed MI6 and MI5 on Venona.
But while ASA, and later the Armed Forces Security Agency (AFSA), kept the British intelligence agencies informed of progress on Venona, they concealed the entire project from the Central Intelligence Agency (CIA), founded in 1947. Ironically, therefore, as MI6 liaison officer in Washington, Peter Dwyer had to be careful not to mention the decrypts in his meetings with CIA officers. The initial decision to keep the CIA in the dark seems to have been taken by the ASA at the urging of J. Edgar Hoover, the autocratic, long-serving director of the Federal Bureau of Investigation (FBI), which was responsible for internal security. Hoover regarded the CIA as a dangerous upstart, which prevented him achieving his ambition of extending Bureau operations into the field of foreign intelligence, and also wrongly suspected it of being, like its wartime predecessor the OSS, penetrated by Soviet intelligence. General Omar Nelson Bradley, who became Chairman of the Joint Chiefs of Staff in 1949, probably at Hoover’s prompting, also opposed sharing Venona with the CIA. Bradley regarded the new Agency as insecure and as an unwelcome threat to the service intelligence agencies; he also resented the fact that, unlike the OSS, it was not placed under the authority of the Joint Chiefs. Most remarkably of all, Truman also appears not to have been informed of Venona – probably for fear that he would mention it to the Director of Central Intelligence (DCI, head of the CIA) at his weekly meetings with him. As a result the President remained confused about the nature and reality of Soviet intelligence penetration of the United States. Significantly, in over 1,200 pages of presidential memoirs, Truman never mentioned the Rosenbergs, Alger Hiss or anyone else publicly accused of being a Soviet agent during his presidency. The CIA was not briefed on Venona until Eisenhower (who had probably been briefed at the outset) was elected to succeed Truman in November 1952.
Even though the relevant British files have yet to be declassified, there is no reason to doubt that the British Prime Minister, Clement Attlee, and at least a few of his senior ministers were briefed on Venona. One remarkable consequence of the British–American Sigint alliance forged during the Second World War was that until November 1952 the British Government and intelligence community were better informed on both the achievements of American cryptanalysts and
Soviet espionage in the United States than the President and the DCI. So was Joseph Stalin. In 1950, the AFSA was shocked to discover that one of its employees, William Weisband, had been a Soviet agent ever since he joined the wartime army Sigint agency in 1942. Thus it was that the Venona secret was communicated to Moscow almost six years before it reached either the President or the CIA. Although Weisband was arrested in 1950, Philby, who succeeded Dwyer as MI6 liaison officer in Washington in 1949 and was fully briefed on Venona, continued to inform Moscow until his recall in 1951.
The attempt to keep the Ultra secret continued for over a quarter of a century after the Second World War. The British Chiefs of Staff agreed on 31 July 1945 that ‘It is imperative that the fact that such intelligence was available should NEVER be disclosed.’ This demand for perpetual secrecy was based chiefly on the conviction that revelation of the past Sigint successes would alert foreign powers to the possibility that their current ciphers were being broken by GCHQ and cause them to introduce cipher systems which would be difficult, if not impossible, to crack. GCHQ adduced one further, shorter-term reason for keeping the Ultra secret. If the Germans knew that their codes had been broken and that this had hastened their defeat, they might come to believe, as after the First World War, that they had not been ‘well and fairly defeated’ and succumb once again to a variety of the ‘stab in the back’ myth which had been exploited by Adolf Hitler.
GCHQ feared from the outset, however, that it might prove impossible to preserve the Ultra secret: ‘[T]he comparing of the German and British documents is bound to arouse suspicion in [historians’] minds that we succeeded in reading the enemy ciphers.’ It now seems astonishing that for over a quarter of a century the great majority of historians suspected no such thing.
With the gift of hindsight, some of the clues now seem remarkably obvious. The fact that American cryptanalysts had broken the main Japanese diplomatic cipher in 1940 was extensively publicized during the Congressional inquiry into Pearl Harbor at the end of the war. It was also common knowledge that British cryptanalysts had broken German codes during the First World War; indeed, one well-publicized German decrypt – the Zimmermann telegram – had hastened American entry into the war. But, until the revelation of Ultra in 1973, almost no historian even discussed the possibility that German ciphers had been extensively broken during the Second World War as well
as the First. The minority of academic historians who had served at Bletchley Park, or had been ‘indoctrinated’ into Ultra while writing official histories, were thus in the remarkable position of knowing that colleagues in their university departments who wrote about the Second World War had misunderstood an important aspect of the war, and of being forbidden by the Official Secrets Act to discuss this with them.