22 May 2013

If you asked my uncle Charlie what he did in the Second World War, you got some evasive boilerplate about working for Army Intelligence. He'd tell you that during the Bulge, say, his unit searched abandoned German command posts for compromising material, and sometimes it was touch and go, because the battle lines shifted back and forth, but he was generally close-mouthed about it, and made his service out to be pretty much routine duty. He did in fact have an old Third Army sleeve insignia, a pin with the white A on a blue field, circled in red, so there might have been some truth to that Battle of the Bulge story. If there was, it was a very small part of the truth, because he was actually in on one of the biggest secrets of the war.

It was called ENIGMA, and the product was code-named ULTRA.

Enigma machine

ENIGMA was an encipherment system, used by the German military and diplomatic services. Polish intelligence did the initial heavy lifting, reverse-engineering captured German equipment, and passed their results on to the Brits in 1939. British code-breakers set up shop at Bletchley Park, north of London, and began reading Luftwaffe and Army traffic.

To simplify enormously, the Enigma machine was a transposition cipher device with a typewriter keyboard. There were three rotors inside, each with twenty-six characters. When you struck a key, the first rotor advanced one position, until it reached twenty-six, and then the next rotor advanced, like an odometer. In other words, any given letter was substituted with another, but the possible combinations were twenty-six to the power of three. The rotor settings were predetermined, but they changed every day, in theory. One weakness Bletchley Park exploited was that the German operators didn't always change the settings daily. Another was that messages were sometime sent in plaintext, for redundancy, and you could compare the coded transmission to the uncoded one. If not for German security breaches, the coded traffic might well have proved unreadable.

Rotors

Then they hit a bottleneck. German naval security had always been more rigorous than that of the Luftwaffe and Wehrmacht, and the Kriegsmarine introduced a machine with four rotors. The number of possible character substitutions multiplied, and the traffic went dark.

In the North Atlantic, the convoys that were Britain's lifeline had no effective air cover or escorts, in 1940 and '41, and crossed two thousand nautical miles of unpatrolled open ocean. Here the wolfpacks hunted. The loss of Allied tonnage was crippling. Bletchley Park needed to break the U-boat codes or the resupply would founder.

Alan Turing

The guy who probably deserves the most credit was an eccentric mathematics done from Cambridge named Alan Turing, who'd been recruited by the Government Code and Cypher School even before the war began. Turing designed an analytical machine, a numb-cruncher, in effect one of the earliest computers, a bombe, so-called. With it, they "unbuttoned," Turing's word, the German naval ciphers, and shortened the Battle of the Atlantic. It's no exaggeration to suggest they shortened the war.

Model of the Bombe

At its peak, Bletchley Park was reading 4,000 messages a day. The decrypts tipped the balance in every campaign from North Africa to D-Day. (They were never shared with the Russians, however. Churchill's mistrust ran deep.) The people who worked there didn't talk about it, then or later. They maintained their habit of silence, and the whole story didn't break until thirty years afterwards. It was a better-kept secret than the Manhattan Project.

Alan Turing died in 1954. He was queer, and MI-5 hounded him, as a security risk. He underwent chemical castration, and was eventually driven to suicide, his contribution to the war effort unrecognized at the time of his death. Some thirty years later, Hugh Whitemore's play "Breaking the Code" opened in the West End, with the astonishing Derek Jacobi as Turing. Sodomy hasn't been criminally prosecuted in Great Britain since the repeal of the gross indecency acts in the late '60's. A legislative motion was introduced in Parliament to grant Turing a statutory pardon, just this past year.

I don't want to wade into the question of gay civil rights, although it seems to me obvious that without legal protection, homosexuals are still fair game. Turing was blackmailed by the law. His reputation doesn't deserve just rehabilitation. This has already happened. The computer library at King's College, Cambridge, for example, is now named after him, and he's widely accepted as a pioneer in Artificial Intelligence (the Turing test), and secure speech, the Delilah program. I'm saying that he deserves a posthumous knighthood, or an Order of Chivalry, at the least. This odd, cranky-pantsed fairy did as much to beat Hitler as any divisions in the field.

Churchill was later to say: "ULTRA won the war."

Editor's note: Software developer Terry Long has created a free Enigma Simulator. If you happen to use a Macintosh, download it and try it out.

13 February 2013

by David Edgerley GatesHerbert Yardley was never a household name, but among his peers, he was almost godlike. He was, in effect, the father of American codebreaking. (He was also, it happens, one hell of a poker player. You do the math.)Yardley started out as a code clerk at the State Dept., in 1912. His first significant coup de theatre came when he intercepted a coded message to President Wilson from Wilson's close personal aide, Colonel House. This was before America entered the European warm and House had been sent to meet the Kaiser: here was his confidential report. On a dare, we might say, Yardley broke the encrypted traffic in two hours, and realizing just how vulnerable American diplomatic cipher systems were, he took the results to his boss. The fuse was lit. In 1917, with America now in the war, Yardley got a commission and went to work for the War Dept., heading up MI-8, codes and ciphers, and eventually turned it into the first real U.S. cryptographic intelligence operation. 1918 found him at Versailles for the peace conference, and his shop encoded American traffic, while secretly decoding those of their Allies. Of course, the French and the British were doing the same thing, and Yardley by this time was no innocent in duplicity.

The war over, Yardley headed home, assuming he was out of a job. But meanwhile, Military Intelligence and the State Dept. had decided to pool their resources, and establish a full-time clandestine eavesdropping organization, with a black budget, hidden from the Comptroller General. Yardley was given the mandate. They were up and running by May of 1919, and in December, Yardley hit the jackpot, when he cracked the Japanese encipherment protocols, and opened up their coded military and diplomatic cables. This was a big deal, and it gave the American negotiators at the 1921 disarmament conference an enormous advantage. The object of the conference, between the five major naval powers, the U.S., Britain, France, Italy, and Japan, was to stabilize the ratio of seapower tonnage. Basically, each navy would agree to a limit on warships, relative to the navies of the other countries. Japan was aggressively pursuing a higher limit for the Imperial Navy, and even at this stage, U.S. and British military strategists were disturbed by Japanese ambitions in the Pacific. But what Japan's admirals said publicly at the negotiating table was undercut by their secret instructions from Tokyo, which conceded political realities. Yardley, knowing his way around a game of stud poker, compared this to knowing your opponent's hole card, and in the end, the Japanese caved. It was high-water mark for American intelligence capacities, and for Yardley, personally, who never minded the attention.There were, however, clouds on the horizon. Wartime cable censorship was over. Communications were supposed to be private. Yardley, or high-ranking military surrogates, approached the major telegraph companies, and strong-armed them into continuing to supply their cable traffic. This was, of course, completely illegal, unless you got a search warrant, and Yardley couldn't blow his cover by doing any such thing. His operation flew under the radar. He turned to the Signal Corps, but State spiked the idea of setting up Army listening posts. Yardley had started his operation with fifty people, and expanded with the heavy demand. By 1929, with the Depression, his staff was down to seven, and badly demoralized, their astonishing successes forgotten. Times had changed. Hoover was president, now. Yardley gambled it all on one last throw of the dice. He went to the Secretary of War, the newly-appointed Henry Stimson, and put his chips on the table. Here, for example, are the recent Japanese decrypts, he told Stimson. And perhaps as a joke, or just to show off, Yardley said he could read the Vatican's private communications. Exaggeration for effect? We don't know. The joke apparently fell flat. Yardley had bet into a stronger hand.We imagine a moment of stony silence.Stimson then comes up with a next to legendary line, in the clandestine world. He looks at Yardley, and says---wait for it---"Gentlemen don't read other gentlemen's mail." And with this, Yardley is put out to pasture, his operation dismantled, and their efforts ignored, if not disgraced.Yardley, in his uppers, writes a book, THE AMERICAN BLACK CHAMBER. Published in 1931, it's a sensation. Washington hunkers down. William Friedman, another big-time cryptologist, now chief of Codes and Ciphers, is in a fury, because Yardley's book gives up sources and methods. He has a point, since the Japanese immediately change their encipherment programs. Friedman won't break the Purple Code until late in WWII. But the government can't embargo Yardley's book. He's not in violation of any existing security laws. And it's something Yardley's thought about. He himself wonders if he's letting the genie out of the bottle. But, he decides, men like Stimson have their heads in the sand. He goes ahead with publication, in spite of his own doubts, and the ship pushes slowly back against the iceberg. Inertia bows to necessity. Yardley works for the Canadian government, and later the Nationalist Chinese. He writes another book. Suppressed, for whatever reason, by the U.S. government, but afterwards declassified. Not exactly a victim, like Alan Turing, but sort of forgotten. Yardley was a tireless self-promoter, a guy who never shunned the limelight, and maybe took credit for other people's labors, but for all that, he's still the man behind the curtain.

NSA is the largest of the American intelligence agencies, dwarfing CIA. National Reconnaissance had the bigger budget, because they put satellites in orbit, but Ft. Meade has the personnel, and the brute mainframes, and the black budget. They can suck the air out of a room. This is perhaps Herbert Yardley's legacy. Not that he thought of it that way. I doubt if he imagined a world where they can read all our mail.He would have preferred to read all the cards.[Many of the specifics here are taken from David Kahn's book THE CODEBREAKERS, and James Bamford's THE PUZZLE PALACE, two excellent resources.]