The Secret War: Spies, Codes and Guerrillas 1939–1945

William Collins

An imprint of HarperCollinsPublishers

1 London Bridge Street

London SE1 9GF

WilliamCollinsBooks.com

This eBook first published in Great Britain by William Collins in 2015

Copyright © Max Hastings 2015

The author asserts the moral right to

be identified as the author of this work

A catalogue record for this book is

available from the British Library

Cover photographs © Topham/Picturepoint (figure, body); Sovfoto/Getty Images (figure, head)

Every effort has been made to trace copyright holders and to obtain their permission for the use of copyright material. The publisher apologises for any errors or omissions in the above list and would be grateful if notified of any corrections that should be incorporated in future editions of this book.

All rights reserved under International and Pan-American Copyright Conventions. By payment of the required fees, you have been granted the non-exclusive, non-transferable right to access and read the text of this e-book on screen. No part of this text may be reproduced, transmitted, down-loaded, decompiled, reverse engineered, or stored in or introduced into any information storage and retrieval system, in any form or by any means, whether electronic or mechanical, now known or hereinafter invented, without the express written permission of HarperCollins.

Source ISBN: 9780007503742

Ebook Edition © September 2015 ISBN: 9780008133023

Version: 2016-02-23

Dedication

For

WILLIAM and AMELIE

the next generation

Contents

Cover

Title Page

Copyright

Dedication

Introduction

1 Before the Deluge

1 SEEKERS AFTER TRUTH

2 THE BRITISH: GENTLEMEN AND PLAYERS

3 THE RUSSIANS: TEMPLES OF ESPIONAGE

2 The Storm Breaks

1 THE ‘FICTION FLOOD’

2 SHADOWING CANARIS

3 Miracles Take a Little Longer: Bletchley

1 ‘TIPS’ AND ‘CILLIS’

2 FLIRTING WITH AMERICA

4 The Dogs that Barked

1 ‘LUCY’S’ PEOPLE

2 SORGE’S WARNINGS

3 THE ORCHESTRA PLAYS

4 THE DEAF MAN IN THE KREMLIN

5 Divine Winds

1 MRS FERGUSON’S TEA SET

2 THE JAPANESE

3 THE MAN WHO WON MIDWAY

6 Muddling and Groping: The Russians at War

1 CENTRE MOBILISES

2 THE END OF SORGE

3 THE SECOND SOURCE

4 GOUREVITCH TAKES A TRAIN

7 Britain’s Secret War Machine

1 THE SHARP END

2 THE BRAIN

3 AT SEA

8 ‘Mars’: The Bloodiest Deception

1 GEHLEN

2 AGENT ‘MAX’

9 The Orchestra’s Last Concert

10 Guerrilla

1 RESISTERS AND RAIDERS

2 SOE

11 Hoover’s G-Men, Donovan’s Wild Men

1 ADVENTURERS

2 IVORY TOWERS

3 ALLEN DULLES: TALKING TO GERMANS

12 Russia’s Partisans: Terrorising Both Sides

13 Islands in the Storm

1 THE ABWEHR’S IRISH JIG

2 NO MAN’S LAND

14 A Little Help from Their Friends

1 ‘IT STINKS, BUT SOMEBODY HAS TO DO IT’

2 AMERICAN TRAITORS

15 The Knowledge Factories

1 AGENTS

2 THE JEWEL OF SOURCES

3 PRODUCTION LINES

4 INFERNAL MACHINES

16 ‘Blunderhead’: The English Patient

17 Eclipse of the Abwehr

1 HITLER’S BLETCHLEYS

2 ‘CICERO’

3 THE FANTASISTS

4 THE ‘GOOD’ NAZI

18 Battlefields

1 WIELDING THE ULTRA WAND

2 SUICIDE SPIES

3 TARNISHED TRIUMPH

19 Black Widows, Few White Knights

1 FIGHTING JAPAN

2 FIGHTING EACH OTHER

3 THE ENEMY: GROPING IN THE DARK

20 ‘Enormoz’

21 Decoding Victory

Picture Section

Acknowledgements

Notes and Sources

Bibliography

Index

Also by Max Hastings

About the Publisher

Introduction

This is a book about some of the most fascinating people who participated in the Second World War. Soldiers, sailors, airmen, civilians had vastly diverse experiences, forged by fire, geography, economics and ideology. Those who killed each other were the most conspicuous, but in many ways the least interesting: outcomes were also profoundly influenced by a host of men and women who never fired a shot. While even in Russia months could elapse between big battles, all the participants waged an unceasing secret war – a struggle for knowledge of the enemy to empower their armies, navies and air forces, through espionage and codebreaking. Lt. Gen. Albert Praun, the Wehrmacht’s last signals chief, wrote afterwards of the latter: ‘All aspects of this modern “cold war of the air waves” were carried on constantly even when the guns were silent.’ The Allies also launched guerrilla and terrorist campaigns wherever in Axis-occupied territories they had means to do so: covert operations assumed an unprecedented importance.

This book does not aspire to be a comprehensive narrative, which would fill countless volumes. It is instead a study of both sides’ secret war machines and some of the characters who influenced them. It is unlikely that any more game-changing revelations will be forthcoming, save possibly from Soviet archives currently locked by Vladimir Putin. The Japanese destroyed most of their intelligence files in 1945, and what survives remains inaccessible in Tokyo, but veterans provided significant post-war testimony – a decade ago, I interviewed some of them myself.

Most books about wartime intelligence focus on the doings of a chosen nation. I have instead attempted to explore it in a global context. Some episodes in my narrative are bound to be familiar to specialists, but a new perspective seems possible by placing them on a broad canvas. Though spies and codebreakers have generated a vast literature, readers may be as astonished by some of the tales in this book as I have been on discovering them for myself. I have written extensively about the Russians, because their doings are much less familiar to Western readers than are those of Britain’s Bletchley Park, America’s Arlington Hall and Op-20-G. I have omitted many legends, and made no attempt to retell the most familiar tales of Resistance in Western Europe, nor of the Abwehr’s agents in Britain and America, who were swiftly imprisoned or ‘turned’ for the famous Double Cross system. By contrast, though the facts of Richard Sorge’s and ‘Cicero’s’* doings have been known for many decades, their significance deserves a rethink.

The achievements of some secret warriors were as breathtaking as the blunders of others. As I recount here, the British several times allowed sensitive material to be captured which could have been fatal to the Ultra secret. Meanwhile, spy writers dwell obsessively on the treachery of Britain’s Cambridge Five, but relatively few recognise what we might call the Washington and Berkeley five hundred – a small army of American leftists who served as informants for Soviet intelligence. The egregious Senator Joseph McCarthy stigmatised many individuals unjustly, but he was not wrong in charging that between the 1930s and 1950s the US government and the nation’s greatest institutions and corporations harboured an astonishing number of employees whose first loyalty was not to their own flag. True, between 1941 and 1945 the Russians were supposedly allies of Britain and the United States, but Stalin viewed this relationship with unremitting cynicism – as a merely temporary association, for the narrow purpose of destroying the Nazis, with nations that remained the Soviet Union’s historic foes and rivals.

Many books about wartime intelligence focus on what spies or codebreakers found out. The only question that matters, however, is how far secret knowledge changed outcomes. The scale of Soviet espionage dwarfed that of every other belligerent, and yielded a rich technological harvest from Britain and the United States, but Stalin’s paranoia crippled exploitation of his crop of other people’s political and military secrets. The most distinguished American historian of wartime codebreaking told me in 2014 that after half a lifetime studying the subject he has decided that Allied intelligence contributed almost nothing to winning the war. This seems too extreme a verdict, but my friend’s remarks show how scepticism, and indeed cynicism, breed and multiply in the course of decades wading in the morass of fantasy, treachery and incompetence wherein most spymasters and their servants have their being. The record suggests that official secrecy does more to protect intelligence agencies from domestic accountability for their own follies than to shield them from enemy penetration. Of what use was it – for instance – to conceal from the British public even the identities of their own spy chiefs, when for years MI6’s* most secret operations were betrayed to the Russians by Kim Philby, one of its most senior officers? The US government repudiated a bilateral intelligence exchange agreed with the NKVD* by Maj. Gen. William Donovan of OSS, but official caution did little for national security when some of Donovan’s top subordinates were passing secrets to Soviet agents.

Intelligence-gathering is not a science. There are no certainties, even when some of the enemy’s correspondence is being read. There is a cacophony of ‘noise’, from which ‘signals’ – truths large and small – must be extracted. In August 1939, on the eve of the Nazi–Soviet Pact, a British official wrung his hands over the confused messages reaching the Foreign Office about relations between Berlin and Moscow: ‘We find ourselves,’ he wrote – using words that may be applied to most intelligence – ‘when attempting to assess the value of these secret reports, somewhat in the position of the Captain of the Forty Thieves when, having put a chalk mark on Ali Baba’s door, he found that Morgana had put similar marks on all the doors in the street and had no indication which was the true one.’

It is fruitless to study any nation’s successes, its pearls of revelation, in isolation. These must be viewed in the context of hundreds of thousands of pages of trivia or outright nonsense that crossed the desks of analysts, statesmen, commanders. ‘Diplomats and intelligence agents, in my experience, are even bigger liars than journalists,’ wrote the British wartime spy Malcolm Muggeridge, who was familiar with all three, and something of a charlatan himself. The sterility of much espionage was nicely illustrated by František Moravec of Czech intelligence. One day in 1936 he proudly presented his commanding officer with a report on a new piece of German military equipment, for which he had paid an informant handsomely. The general skimmed it, then said, ‘I will show you something better.’ He tossed across his desk a copy of the magazine Die Wehrmacht, pointed out an article on the same weapon, and said dryly, ‘The subscription is only twenty crowns.’

In the same category fell the Abwehr transcript of a December 1944 US State Department message appointing a new economic affairs counsellor to the Polish exile government in London. This read, in part: ‘His transportation expenses and per diem, Tunis to London, via Washington, DC, transportation expenses and per diem for his family and shipment effects direct authorised, subject Travel Regulations.’ A page-long translation of this decrypt was stamped ‘Top Secret’ by its German readers. The man-hours expended by the Nazi war machine to secure this gem reflect the fashion in which intelligence services often move mountains to give birth to mice.

Trust is a bond and privilege of free societies. Yet credulity and respect for privacy are fatal flaws to analysts and agent-runners. Their work requires them to persuade citizens of other countries to abandon the traditional ideal of patriotism, whether for cash, out of conviction, or occasionally because of a personal bond between handler and informant. It will always be disputed territory, whether those who betray their society’s secrets are courageous and principled heroes who identify a higher loyalty, as modern Germans perceive the anti-Hitler Resistance, or instead traitors, as most of us classify Kim Philby, Alger Hiss – and in our own times Edward Snowden. The day job of many intelligence officers is to promote treachery, which helps to explain why the trade attracts so many weird people. Malcolm Muggeridge asserted disdainfully that it ‘necessarily involves such cheating, lying and betraying, that it has a deleterious effect on the character. I never met anyone professionally engaged in it whom I should care to trust in any capacity.’

Stalin said: ‘A spy should be like the devil; no one can trust him, not even himself.’ The growth of new ideologies, most significantly communism, caused some people to embrace loyalties that crossed frontiers and, in the eyes of zealots, transcended mere patriotism. More than a few felt exalted by discovering virtue in treason, though others preferred to betray for cash. Many wartime spymasters were uncertain which side their agents were really serving, and in some cases bewilderment persists to this day. The British petty crook Eddie Chapman, ‘Agent ZigZag’, had extraordinary war experiences as the plaything of British and German intelligence. At different times he put himself at the mercy of both, but it seems unlikely that his activities did much good to either, serving only to keep Chapman himself in girls and shoe leather. He was an intriguing but unimportant figure, one among countless loose cannon on the secret battlefield. More interesting, and scarcely known to the public, is the case of Ronald Seth, an SOE agent captured by the Germans and trained by them to serve as a ‘double’ in Britain. I shall describe below the puzzlement of SOE, MI5, MI6, MI9 and the Abwehr about whose side Seth ended up on.

Intelligence-gathering is inherently wasteful. I am struck by the number of secret service officers of all nationalities whose only achievement in foreign postings was to stay alive, at hefty cost to their employers, while collecting information of which not a smidgeon assisted the war effort. Perhaps one-thousandth of 1 per cent of material garnered from secret sources by all the belligerents in World War II contributed to changing battlefield outcomes. Yet that fraction was of such value that warlords grudged not a life nor a pound, rouble, dollar, Reichsmark expended in securing it. Intelligence has always influenced wars, but until the twentieth century commanders could discover their enemies’ motions only through spies and direct observation – counting men, ships, guns. Then came wireless communication, which created rolling new intelligence corn prairies that grew exponentially after 1930, as technology advanced. ‘There has never been anything comparable in any other period of history to the impact of radio,’ wrote the great British scientific intelligence officer Dr R.V. Jones. ‘… It was the product of some of the most imaginative developments that have ever occurred in physics, and it was as near magic as anyone could conceive.’ Not only could millions of citizens build their own sets at home, as did also many spies abroad, but in Berlin, London, Washington, Moscow, Tokyo electronic eavesdroppers were empowered to probe the deployments and sometimes the intentions of an enemy without benefit of telescopes, frigates or agents.

One of the themes in this book is that the signals intelligence war, certainly in its early stages, was less lopsided in the Allies’ favour than popular mythology suggests. The Germans used secret knowledge well to plan the 1940 invasion of France and the Low Countries. At least until mid-1942, and even in some degree thereafter, they read important Allied codes both on land and at sea, with significant consequences for both the Battle of the Atlantic and the North African campaign. They were able to exploit feeble Red Army wireless security during the first year of Operation ‘Barbarossa’. From late 1942 onwards, however, Hitler’s codebreakers lagged ever further behind their Allied counterparts. The Abwehr’s attempts at espionage abroad were pitiful.

The Japanese government and army high command planned their initial 1941–42 assaults on Pearl Harbor and the European empires of South-East Asia most efficiently, but thereafter treated intelligence with disdain, and waged war in a fog of ignorance about their enemies’ doings. The Italian intelligence service and its codebreakers had some notable successes in the early war years, but by 1942 Mussolini’s commanders were reduced to using Russian PoWs to do their eavesdropping on Soviet wireless traffic. Relatively little effort was expended by any nation on probing Italy’s secrets, because its military capability shrank so rapidly. ‘Our picture of the Italian air force was incomplete and our knowledge far from sound,’ admitted RAF intelligence officer Group-Captain Harry Humphreys about the Mediterranean theatre, before adding smugly, ‘So – fortunately – was the Italian air force.’

The first requirement for successful use of secret data is that commanders should be willing to analyse it honestly. Herbert Meyer, a veteran of Washington’s National Intelligence Council, defined his business as the presentation of ‘organized information’; he argued that ideally intelligence departments should provide a service for commanders resembling that of ship and aircraft navigation systems. Donald McLachlan, a British naval practitioner, observed: ‘Intelligence has much in common with scholarship, and the standards which are demanded in scholarship are those which should be applied to intelligence.’ After the war, the surviving German commanders blamed all their intelligence failures on Hitler’s refusal to countenance objective assessment of evidence. Signals supremo Albert Praun said: ‘Unfortunately … throughout the war Hitler … showed a lack of confidence in communications intelligence, especially if the reports were unfavourable [to his own views].’

Good news for the Axis cause – for instance, interceptions revealing heavy Allied losses – were given the highest priority for transmission to Berlin, because the Führer welcomed them. Meanwhile bad tidings received short shrift. Before the June 1941 invasion of Russia, Gen. Georg Thomas of the WiRuAmt – the Wehrmacht’s economics department – produced estimates of Soviet weapons production which approached the reality, though still short of it, and argued that the loss of European Russia would not necessarily precipitate the collapse of Stalin’s industrial base. Hitler dismissed Thomas’s numbers out of hand, because he could not reconcile their magnitude with his contempt for all things Slavonic. Field-Marshal Wilhelm Keitel eventually instructed the WiRuAmt to stop submitting intelligence that might upset the Führer.

The war effort of the Western democracies profited immensely from the relative openness of their societies and governance. Churchill sometimes indulged spasms of anger towards those around him who voiced unwelcome views, but a remarkably open debate was sustained in the Allied corridors of power, including most military headquarters. Gen. Sir Bernard Montgomery was a considerable tyrant, but those whom he trusted – including his intelligence chief Brigadier Bill Williams, a peacetime Oxford don – could speak their minds. All the United States’s brilliant intelligence successes were gained through codebreaking, and were exploited most dramatically in the Pacific naval war. American ground commanders seldom showed much interest in using their knowledge to promote deceptions, as did the British. D-Day in 1944 was the only operation for which the Americans cooperated wholeheartedly on a deception plan. Even then the British were prime movers, while the Americans merely acquiesced – for instance, by allowing Gen. George Patton to masquerade as commander of the fictitious American First US Army Group supposedly destined to land in the Pas de Calais. Some senior Americans were suspicious of the British enthusiasm for misleading the enemy, which they regarded as reflecting their ally’s enthusiasm for employing guile to escape hard fighting, the real business of war.

GC&CS, the so-called Government Code and Cipher School at Bletchley Park, was of course not merely the most important intelligence hub of the conflict, but from 1942 Britain’s outstanding contribution to victory. Folk legend holds that Alan Turing’s creation of electro-mechanical bombes exposed Germany’s entire communications system to Allied eyes by breaking the Enigma’s traffic. The truth is far more complex. The Germans employed dozens of different keys, many of which were read only intermittently, often out of ‘real time’ – meaning insufficiently rapidly to make possible an operational response – and a few not at all. The British accessed some immensely valuable Enigma material, but coverage was never remotely comprehensive, and was especially weak on army traffic. Moreover, an ever-increasing volume of the Germans’ most secret signals was transmitted through a teleprinter network which employed an entirely different encryption system from that used by Enigma. The achievement of Bletchley’s mathematicians and linguists in cracking the Lorenz Schlüsselzusatz was quite distinct from, and more difficult than, breaking the Enigma, even though recipients in the field knew the products of all such activities simply as ‘Ultra’.* Bill Tutte, the young Cambridge mathematician who made the critical initial discoveries, is scarcely known to posterity, yet deserves to be almost as celebrated as Turing.

Ultra enabled the Allied leadership to plan its campaigns and operations in the second half of the war with a confidence vouchsafed to no previous warlords in history. Knowing the enemy’s hand did not diminish its strength, however. In 1941 and into 1942, again and again the British learned where the Axis intended to strike – as in Crete, North Africa and Malaya – but this did not save them from losing the subsequent battles. Hard power, whether on land or at sea or in the air, was indispensable to the exploitation of secret knowledge. So, too, was wisdom on the part of British and American commanders and their staffs – which proved conspicuously lacking at key moments during the 1944–45 north-west Europe campaign. Intelligence did, however, contribute importantly to mitigating some early disasters: young R.V. Jones’s achievement in showing the path towards jamming the Luftwaffe’s navigational beams significantly diminished the pain inflicted by the Blitz on Britain. At sea, Ultra’s pinpointing of German U-boats – with an alarming nine-month interruption in 1942 – made it possible to reroute convoys to evade them, an even more important contribution to holding open the Atlantic supply line than sinking enemy submarines.