Finding the American Black Chamber in New York

Herbert Yardley and the American Black Chamber

The U.S. Government's code-breaking operation was
based in midtown Manhattan from 1918 until 1921.
This was MI-8, or Military Intelligence, Section 8,
the special bureau handling all aspects of cryptology.
They created U.S. codes and ciphers and work to break
those of other nations.
The "American Black Chamber was run by
Herbert O. Yardley, who based it in some
brownstones on 37th Street and 38th Street.
Let's visit New York and see what can be seen today
of the American Black Chamber's operation.

Herbert O. Yardley

Herbert Yardley
was born in Worthington, Indiana in 1889.
His father was a railroad station master and telegrapher,
and he taught
telegraphy
to Herbert.

After graduating from high school in 1907,
Yardley worked as a telegraph operator for the railroad
while teaching himself poker,
concentrating especially on the probability involved.

Now this was
real poker
and not the silly "Texas Hold-em" card game based on bluffing
and guessing when the other players are bluffing.
Yardley's game required serious understanding of probability.

He applied his poker winnings toward further education,
and in 1912 he passed the civil service exam and was
hired as a telegraph operator for the federal government.
He began as a code clerk in the State Department,
where he discovered that he was able to break
the current U.S. government codes.

Yardley was startled when he easily broke the code used by
President Woodrow Wilson
and then was even more surprised to learn that
the code had been in use for over ten years.
This extreme cryptographic weakness while
World War I
raged in Europe
(the U.S. would not get directly involved until April 1917)
led Yardley to write a hundred-page paper
"Solution of American Diplomatic Codes".
He submitted it to his superior, who became quite angry
as he had been the developer of most of these weak codes.

The U.S. entered the war in April 1917,
and Yardley was sent to the War Department and then on to the
U.S. Army Signal Corps.
He was given a Reserve commission as Lieutenant,
and named the head of
MI-8, or Military Intelligence, Section 8.
MI-8 was a special bureau dealing with cryptology,
both the creation of U.S. codes and ciphers,
and the
cryptanalytic attack
on those of other nations.

The small MI-8 under Yardley's leadership had broken almost
all German diplomatic and military codes within a few months.
One prominent success was the decryption of a message found
on
Lothar Witzke,
a German saboteur who was arrested after
entering the U.S. from Mexico.
Witzke had a 424-letter cryptogram sewn into the left
upper sleeve of his jacket.
The resulting cleartext proved that he was a German saboteur,
one of those responsible for sabotage on
Mare Island Naval Shipyard
near San Francisco,
and probably the
"Black Tom" explosion
in New York harbor in June 1916.
Witzke was tried and convicted, sentenced to death,
the only German agent to be so sentenced,
although President Calvin Coolidge issued a reprieve in
November 1923 just before the sentence was to be carried out.

The
Zimmermann Telegraph
as it was relayed as ciphertext
by Western Union from Washington to Mexico City.

By then promoted to major, partly thanks to the Witzke case,
Yardley went to Britain in August 1918.
There he met Vernon Kell, head of the U.K.'s
MI5,
and Admiral William Reginald Hall, head of British naval
intelligence and the director of the Room 40 group.
Room 40 had broken the Zimmermann Telegram
which finally brought the U.S. into the war.
From there he traveled on to France and met with
the French cryptography bureau, known as the
Chambre Noire or Black Chamber.

The war ended 11 November 1918, and there was call for
MI-8 to be disbanded.
However,
General Marlborough Churchill,
head of U.S. Army Intelligence
and a distant relative of Britain's Winston Churchill,
had read Yardley's post-war report
"Code and Cipher Investigation and Attack".
General Churchill was determined that the United States
must not foolishly throw away a valuable resource.

Churchill managed to persuade State Department officials
to fund a covert and deniable cryptanalysis operation.
It would analyze and attack foreign codes and ciphers,
but for legal reasons it was to be located in New York,
not Washington.

Or at least Yardley's explanation was that it was some
legal constraint — Yardley himself later said it
was get out of Washington as it was "overrun with spies",
and it's also possible that Yardley chose New York
because his close friend and head of MI-8's Shorthand
Subsection, Franklin Allen, was familiar with New York
from operating there during the war.

Whatever the reason for the New York location,
MI-8 operated there covertly.
Ties to the government were cut.
Employees were paid in cash from a secret payroll and
had no civil servant status.
Official mail from Washington was received at a post office
box, at least some of the time at box number 354,
Grand Central Station.
They established the Code Compiling Company as a cover,
a producer of commercial code books.
One of them, the "Universal Trade Code", sold in fairly
large numbers.

Yardley organized a team of cryptanalysts
largely based on his MI-8 staff.
Yardley named his operation the
American Black Chamber
in honor of the French Chambre Noire.
It was also known as the
Cipher Bureau.
This was the United States' first peacetime cryptanalytic
organization, responsible for breaking the codes of
other nations.
It was a predecessor, although not a direct one for
reasons explained below, of the
National Security Agency.

The State Department and War Department had special
arrangements with commercial telegraph companies,
including at least the Western Union
and Postal Telegraph companies.
During the first year the work focused on Latin American
countries (remember the Zimmermann Telegram
and its Mexican destination) plus Germany, Japan, and Spain.

The Black Chamber soon broke the codes of the
Чека
(Cheka),
the Bolshevik secret police that had replaced
the Czar's
Охрана
(Okhrana)
and which eventually evolved into the KGB and today's
ФСБ or FSB.
It was then asked to attack the Japanese diplomatic codes.

Given the quick break of the Japanese ciphers, the
Black Chamber was able to contribute significantly
to the partial containment of Japan at the 1921
Washington Naval Conference,
often referenced as one of the Black Chamber's greatest
accomplishments but perhaps the cause of its eventual
disbanding.

Russian cruiser Pallada under fire at
Port Arthur, Manchuria, now Lüshun, PRC,
before being seized as war booty by Imperial Japan.

This post-World-War-I international conference was intended
to reduce tension and make major war less likely by
limiting nations' total warship tonnage.
Japan had won significant naval victories over Russia's
Pacific Fleet in the
Russo-Japanese War of 1904-1905.

The Black Chamber decrypted the intercepted messages
between the Japanese emissaries and their home government.
What was even more sensitive was that they were also breaking
the messages of American allies.
The U.S. negotiators had a huge advantage at that conference.

The U.S. position called for a 10:6 ratio,
for the total tonnage of Japanese military warships
to be limited to 60% of that of the U.S. fleet.
The Japanese negotiators insisted on a 10:7 ratio,
70% rather than 60% of U.S. tonnage.
However, the Black Chamber discovered that the
Japanese government in Tokyo was telling its delegation
to eventually compromise at 10:6 or 60% if needed to
avoid a confrontation.
Given that knowledge, the U.S. side simply stood firm
until the Japanese delegation accepted 10:6.

Despite its successes and obvious utility, the State Department
significantly reduced the Black Chamber's funding in 1924.
Then, in 1929,
Henry L. Stimson
was named Secretary of State.
Stimson was offended by espionage and any type of covert action,
and he imperiously and naïvely issued the infamous
directive "Gentlemen do not read each other's mail."
The American Black Chamber was shut down on 1 November 1929.

Black Chamber locations

The American Black Chamber was first established at
3 East 38th Street, just off Fifth Avenue.

This is a telephoto view down Fifth Avenue from 38th Street
late in the day.
The Empire State Building,
filling the block between 33rd and 34th Streets,
is the building with the flags on the right side of the Avenue.

The location at 3 East 38th Street was a
four-story brownstone building at the north-east corner
of the intersection, just across the avenue from the
Lord and Taylor department store.

The American Black Chamber was established here in the
summer of 1919, but just operated out of this location
for about a year.

The department store is still there and is still operated
by Lord and Taylor, but the brownstone is gone.
It has been replaced by the modern building seen in
the second picture above.
That view is toward the west along 38th Street to the
intersection with 5th Avenue.

Presumably at least some of these buildings seen
below were neighbors of the brownstone.
The building housing the Butterfield 8 bar is the first
building to the east of Fifth Avenue not built recently.
Butterfield 8 now has the address 5 East 38th Street,
so if the street numbering has not changed, it is the first
address to the east of the original Black Chamber.

The lease to the brownstone was sold in 1920,
and the American Black Chamber moved a short distance
to another brownstone between Lexington and
Third Avenues
at 141 East 37th Street on 1 July 1920.
That brownstone is seen above and below.

By now the Japanese codes and ciphers were the main
priority, including Japanese diplomatic, military,
and naval attaché codes.

The State Department passed along its 1924
budget cuts in the form of
greatly reduced support to MI-8,
and Yardley was forced to cut more than half his staff.
MI-8 was down to just six employees including Yardley.

The greatly reduced MI-8 was moved in 1924
to an office building at 52 Vanderbilt Avenue.
That is seen in the first picture below as the first building
on the right side of the avenue with a visible near corner.
It is the building beyond the white truck.
Vanderbilt is a relatively short north-south avenue running
along the west side of Grand Central Station.

They had access to less material, apparently because
the telegraph companies were becoming more and more
reluctant to pass along copies of their traffic.
Yardley wrote, in a later book discussed below,
of being "forced to adopt rather subtle methods" of
obtaining traffic, methods not always supported by those
to whom he reported, presumably illegal methods such
as break-ins, theft, and bribery.

Meanwhile Yardley received another
Distinguished Service Medal in 1926,
and through the mid and late 1920s MI-8 was
successfully attacking and exploiting cryptosystems of
Japan, China, Mexico, Nicaragua, and Peru.

Yardley's operation basically ceased operation in June
or July of 1929, when the State Department announced
the coming end of funding.
The office was not officially closed until
the first of November 1929.
William F Friedman
arrived in New York in October to transfer the files and
records to Washington, where they soon were used by
the U.S. Army's newly formed
Signal Intelligence Service.
The SIS eventually led to the
National Security Agency.

The government soon offered Yardley a temporary position
at far less pay,
and later another position at half his previous pay,
apparently both times in the expectation that he would
not accept the offers.
In 1930 he sought a position as an instructor for the
U.S. Navy's cryptanalytic research section, but was turned
down, probably on the grounds that he had no experience
with current Japanese naval cryptosystems.

Yardley was out of a job and unable to find another.
This wasn't surprising, given that the few organizations
looking for his set of skills had mostly been targets
of his cryptanalysis.
He then wrote a book that became immensely popular
and simultaneously brought him harsh criticism lasting
to this day,
The American Black Chamber.
He published three articles in the
Saturday Evening Post, then the leading mass
circulation weekly.
"Secret Inks" on April 4, 1931;
"Codes" on April 18, and "Ciphers" on May 9.
On June 1, 1931,
The American Black Chamber
was released.

The second of those magazine articles included a
photograph of a code book page, showing that
U.S. military intelligence was working on
British diplomatic codes in 1921.
The book exposed much more.

The American Black Chamber
was denounced by the
U.S. government as having given away American secrets.
Not the secrets of how to perform cryptanalysis,
but the secrets about how and when it had been applied
— not only against likely adversaries
such as Japan, but also against U.S. allies.
This led to Public Law 37, signed by Franklin D.
Roosevelt in 1933, criminalizing any such
future exposure.
Further publication of
The American Black Chamber was
halted, although it was later
allowed back into publication.

Yardley next wrote a book titled
"Japanese Diplomatic Codes: 1921-1922",
but the government cited Public Law 37 and seized the
manuscript before it was printed.
The manuscript was classified until 1979,
when it was finally declassified.

By 1932 the U.S. Army had resumed cryptanalytic work,
although only on a part-time and very limited scale.

Yardley went to China in 1938 and worked until late 1940
with Chiang Kai-shek's intelligence bureau.
Another book of his memoirs,
The Chinese Black Chamber,
was declassified and published in 1983.

In early 1941 he was in Canada helping establish a
cryptanalysis bureau.
After the U.S. entry into World War II in December 1941,
Yardley returned to the U.S. and was given a government job
shuffling meat inspection forms for the
Office of Price Administration.
Meanwhile,
William F Friedman
and others were breaking German and Japanese ciphers,
using Yardley's unpublished manuscript
of "Japanese Diplomatic Secrets"
at least for some initial guidance.

He wrote and co-wrote several magazine articles and
some thrillers,
"The Blonde Countess",
"Red Sun of Nippon",
and
"Crows Are Black Everywhere",
featuring fictionalizations of his exploits overseas
and some of the world of his Black Chamber,
although it is still debated as to how much he
actually wrote of some of these.
See the journal article,
"Who Wrote 'The Blonde Countess'?
A Stylometric Analysis of Herbert O. Yardley's Fiction",
Cryptologia, 33:2 (April 2009), pp 108-117.
The movie
Rendezvous
was based very loosely on "The Blonde Countess".

Well after the war, in 1957, Yardley published
"The Education of a Poker Player",
describing a powerful system for playing real poker
where an understanding of probability is required.
Sales were strong, but Yardley died in 1958.