Posted
by
CmdrTaco
on Monday January 31, 2005 @02:04PM
from the terces-egassem-seog-ereh dept.

sebFlyte writes "From the introduction of this document, the U.S. Army's field manual guide to Cryptanalysis: 'This manual presents the basic principles and techniques of cryptanalysts and their relation to cryptography. Cryptanalytics is the art and science of solving unknown codes and ciphers.'"

Private: Sir! I found this, it may be a clue. Should I
consult the field manual to cryptopgraphy? Sergeant:"Gur
jrncbaf bs znff qrfgehpgvba ner va Fnqqnz'f Onfrzrag"? [crumpling
paper] These are obviously the scribblings of a madman, Private. Get back to
your patrol!

I've always enjoyed the road sign on the GW Parkway to the "George Bush Center for Intelligence".
This is, of course, Bush the elder. His son has his own view of the world:
"I just want you to know that, when we talk about war, we're really talking about peace." -- George W. Bush/Mark

A sneak peek at chapter 7:
Solution of polygraphic substitution systems polyalphabetic substitution systems
and that's just the title. Pack a lunch for this one...

A good read along with this would probably be Between Silk And Cyanide: A Codemakers War, which gets into, among other things, creation of more secure codes during WW II. An excellent read (I currenly have The White Rabbit on order (story of Yeo-Thomas' activities in France, capture, interrogation and imprisonment))

I can also highly recommend Between Silk And Cyanide [amazon.com] (no referral tags in that link.) Marks is a brilliant writer, and it reads far more like an entertaining work of fiction than a historical narrative.

My literary forays into the field have been limited, but perhaps one of my favorite reads to date has been Simon Singh's 'The Code Book' (which has [slashdot.org] been [slashdot.org] mentioned [slashdot.org] before [slashdot.org]).

It's not terribly detailed by way of 'how to', but the history of cryptography/cryptanalysis it offers is fantastic. It's also pretty well known for the contest in the back of it, wherein Mr. Singh offers a reward for the solution to all 10 of the codes (I believe a Swedish team eventually won the prize, shortly after the deadline had p

ok that was in jest. But seriously, how much good is a field manual going to do you when its possible for handheld computers to encrypt data to such a strong degree that it's theoretically impossible to decrypt with any likelihood of success that's indistinguishable from zero in the lifetime of the universe?

I mean even if the guys at the NSA use different theories from the rest of us, I can only imagine that the methods they use still require vast amounts of hardware...

This document is also 15 years old. Let's think about computing power available 15 years ago. Yes, there were computers more than powerful enough to do handle brute force decryption, not to mention more sophisiticated means. In terms of portability, however, there was nothing. Computing power has become so inexpensive and widespread now that more advanced forms of cryptography have (natrually) replaced the older, hand driven cyphers of old.
Let's also think about the types of encryption that were being used back then. The mathematics that it takes to drive many of these algorithms was simply not practical in 1990.
This document is serves more as a historical artifact now rather than a practial guide to decrypting like the government.

This manual brings back old memories. Everybody who laughs at this FM seem to know very little about the history of the military and the NSA. I joined the Army in 1975 and was a member of ASA which was the Army Security Agency. I was a traffic analyst MOS (98C) and had add on modifiers for Korean language, Chinese Mandarin language , and T9 which was a code breaking. I went to school to break hand generated codes vs the guys who went to school for machine generated codes i.e. computer generated codes. I worked directly with NSA on a daily basis in the performance of my job. We where the arms, legs, and ears for the NSA in foreign countries. In the 70's I would say we were the bulk of the data collection for the NSA. Ask anyone if they remember the elephant cage in Thailand or a similar structure in Germany. It was a large antenna field that was in a circle that was nicknamed the elephant cage. It was amazing what the military did with them 30 years ago. Also this is an unclassified manual and no where near what the military actually taught just an introductory source of information. Most/. probably aren't aware that the military taught people how to break codes that were written in a foreign language that you didn't know. The military is very good at training no matter what people think of the intelligence level of the members of the military. Not everyone in the world has access to computers in the field and I am willing to bet there are still a lot of hand generated codes used by different militaries in the world. This manual may or may not be of historical value but there were a lot of morse code traffic through the 70's and early 80's using these type of codes. A morse code operator could send traffic around the world with very little wattage. There use to be automated "radio stations" that sent out nothing but endless streams of what appeared to be random numbers and letters for hours at a time. Ever wonder if someone sent a 3 hour stream of traffic and the only portion of any intelligence value was at 2 hours 5 mins 30 seconds for 12 seconds. I wonder how long it would take for a computer to decode the message? Better yet do the same thing except the 12 seconds of intelligence is a book code where it tells you what words in a specified book are the correct ones. How long would it take to brute force that? All an operative needs is a shortwave radio and one of these simple codes to receive information. Don't tell the Dept of Homeland Security that their computers won't help catch an operative that receives instructions by this low tech means. Just because something is low tech doesn't mean it isn't still of a value.

This document is also 15 years old. Let's think about computing power available 15 years ago. Yes, there were computers more than powerful enough to do handle brute force decryption, not to mention more sophisiticated means. In terms of portability, however, there was nothing. Computing power has become so inexpensive and widespread now that more advanced forms of cryptography have (natrually) replaced the older, hand driven cyphers of old. Let's also think about the types of encryption that were being used back then. The mathematics that it takes to drive many of these algorithms was simply not practical in 1990. This document is serves more as a historical artifact now rather than a practial guide to decrypting like the government.

I can attest that your assertion is exactly right. I was a Signal Intelligence Analyst in the US Army from '87 to '91, and most of what we saw was pretty crude. Remember, the Army doesn't generally intercept diplomatic comm's encrypted with sophisticated devices locked in embassy basements. It's probably more sophisticated now, but back then we mostly got stuff encoded by drafted soldiers and sent via morse code! I was trained in basic cryptanalysis, but most of what we saw was (Soviet) Red Army code table stuff. Morse transmissions would come in as a bunch of 3-digit numbers. The first two digits correspond to the X and Y axes of a 10x10 grid. Each square in the grid would contain 3 to 9 numbered code "snippets", and the 3rd digit of the 3-digit number refers to which. These snippets could be anything-- "weather report", "infantry", "battalion", "heading", a single number, a single letter, etc-- that might make up part of a message. Codes like this are tough to break when used properly, but of course they weren't. Some red army private would send "225 171", and the guy on the other end would say "huh? say again?" because he was holding his code table upsode down or something. They'd go back and forth five or six times before the first guy would just lose his shit and say "GIVE ME A BALLISTIC WEATHER REPORT, YOU STUPID TARD!" and then we'd know that "225 171" meant "REQUEST" and "BWX(ballistic weather report)".

But at about the time of the fall of the Soviet Union, all that started to change. The russkies were gone, and most of the "warsaw pact interoperability" tendency for all their client states disappeared with 'em. A prime example of a military with excellent COMSEC was the Iraqi army, and they did it very simply as well. Instead of using radio, they ran wire and used field telephones for nearly EVERYTHING. When we were deployed for DESERT SHIELD we found the airwaves almost dead. The days of morse code and ciphers are pretty much gone.

The other thing I loved about the Soviet Army was that they were so inflexible and tightly regulated. I don't know if they still do this, but for months one of the units that we were listening to requested the same information at the same time, and in the same order.

If you ask for the BWX every morning in your third transmission, your COMSEC is shot to hell no matter how often to change the cods.

The very point I would have made...this stuff was declassified, [if it was ever classified] long ago. Its main significance in being published is only to tell us that the arms race between cryptographers and cryptanalysts has escallated so far beyond what is in the manual as to render it harmless.

Elonka gave an interesting talk about cryptography at Defcon this past year. Nowadays, to me anyway, it seems as though cryptography-by-hand is more of an intellectual challenge; rather than something you would ACTUALLY attempt on something like a 4096 bit PGP encrypted o-mi-god problem.

Because few people bother to use them properly. The Confederacy had access to ciphers (Vigere) which were practically unbreakable at the time, but they didn't use them, and so it was well worth the North having codebreakers as they got some pretty useful intel from them. Similarly, although J. Terrorist could use PGP and be safe, he could just as easily be using Vigere or something weaker, so codebreaking certainly has its place.

But seriously, what happens when, out in the field, your equipment is broken. or stolen. or there weren't enough to go around. or you're captured and held as a POW, and the escape plan is encrypted and scratched into a tree behind a chunk of bark? bet you'll wish you read that manual then.

Of course the problem in Iraq is that they don't use the English alphabet or language. The frequency analysis we depend on for the shift cipher or Vigenere cipher doesn't work for Arabic.

The techniques still work. You just have to use a different set of language statistics. You don't even have to understand the language, although it helps. There are precomputed lists of letter frequencies, initial and final letters, digraphs, trigraphs, etc. for all common languages.

"Theoretically impossible to decrypt...in the lifetime of the universe" you say?

That only applies if you're taking a brute force approach to cracking it - something that should be the absolute last resort.

Far more often, the code is broken by exploiting some mathematical weakness in the algorithm (or bug in the software implementation). If that's your aim, then it obviously pays to have a thorough understanding of the field, the various cyphers that have been devised, and how they've been broken in the pa

ok that was in jest. But seriously, how much good is a field manual going to do you when its possible for handheld computers to encrypt data to such a strong degree that it's theoretically impossible to decrypt with any likelihood of success that's indistinguishable from zero in the lifetime of the universe?

Because voice messages and on-the-fly manual encryption still exist. I've only been out of the U.S. Army for a few years, but we all learned (in my field, anyway) manual coding techniques, because you

Nice troll. It has more to do with the fact that one expect the government that's supposed to protect him to be able to keep secrets from the rest of the world. Any other government has the same prerogative, and the people those governments are intended to protect may have the same objections to their governments' secrets being publicly available.

Actualy the automobile was either invented by Nicolas-Joseph Cugnot [source [wikipedia.org]] (a Frenchman) or by Frederick William Lanchester [source [wikipedia.org]](a Brit) depending on what you count as an "automobile"

While Ford's model T was certainly the first affordable auto, European models predate the Model-T by as much as 60 years. [source [wikipedia.org]]

Right, because the Germans invented DES which started the rush of crypto algorithms while IBM (an American company) was still using polyalphabetic substitution cipers. No, wait, it's the other way around, stupid ass. You're not only wrong, but stupid.

One thing Capitalism does very well is foster innovation, both in invention and improvement of other inventions. We didn't invent the rocket, but we made it better. We invented the atomic bomb. We made serious cryptography.

I agree. An astute observation came from a member the OPFOR, the "Red Army" used at the Fort Irwin, California National Training Center, in briefing materials. Too many commanders used improvised codes instead relying on the tactical codebook or the CEOI; they were very easy to break.

I didn't read The Codebreakers but it seems a little too expensive. I really like The Code Book [amazon.com] on this subject. Much cheaper, and it was hard for me to put it down. It goes all the way into quantum cryptography.

It's an interesting book from an academic standpoint. I'm not sure how practical it is, though. It's all about cryptanalysis the old fashioned way (i.e. before computers). Still, I suppose it is good to acknowledge that the enemy may surprise us by taking a low-tech approach.

FIELD MANUALNO 34-40-2 HEADQUARTERSDEPARTMENT OF THE ARMYWashington, DC, 13 September 1990

The original for this came from <a href=http://www.atsc-army.org/cgi-win/$atdl.exe/fm/34-40-2/default.htm>here</a> on Tue Dec 17 01:21:11 EST 1996.

This thing is 14 years old and has been public for over 8 years (at least) and somebody thinks that it is worth putting on slashdot. Thanks.
(P.S. - note that the link they used for "here" doesn't even work./.ed maybe?)

On the other hand, the people we are likely to fight in the near future are probably using fairly low tech methods.

Back when I was doing SigInt for a living in the late 80s, we used all kinds of stuff like burst transmission, line of sight radio relays, and encryption computers. At the same time, the Russians and Czech units we were listening to were using fairly basic cyphers up to batallion and even brigade level. The one I remember most were fairly simple extensions of Polybios squares that encrypted pa

I'm not sure how practical it is, though. It's all about cryptanalysis the old fashioned way (i.e. before computers). Still, I suppose it is good to acknowledge that the enemy may surprise us by taking a low-tech approach.

You mean if they surprise us by doing exactly the same thing we would have to do if the computers weren't available, right? You would be surprised how practical low-tech methods can be.

"It's all about cryptanalysis the old fashioned way (i.e. before computers)."

Last time I checked computers where available 15 years ago.I seem to remember that the British used computers for cryptanalysis all they way back in WWII.I would bet the most nations used computers for cryptanalysis for many decades. Many of the methods in the book could be converted into a program and uses to verify that new encryption systems can not be solved using these old methods.

15 years ago, computers weren't widely available at the battalion level. Furthermore, power supplies were unreliable--you can't run a generator tactically, and you never have enough batteries--so hand encryption made sense.

This field manual (no. 34.40.2) seems to have a Distribution Restriction placed on it as of March 5, 1990. The index page of the manual features a prominent warning about its restricted nature and a banner at the bottom of the page reads, "For Official Use Only".

Is this document classified or are these just standard warnings with no teeth? Is our dissemination of this 15-yr-old document criminal?

I'm not sure if an army manual can be distributed openly like this. What exactly does DA Form 12-11E say about distribution of such manuals, can someone from the Army who knows the details explain the legal aspect?

Generally something like this manual would have a classification of FOUO (For Official Use Only). When I took the crypto correspondence course, all of the courseware was FOUO. So, about the only person(s) that could get in trouble would be those who gave it to be posted (or more accurately the last one in the DOD chain before it hit a civilian's hand). It is quite possible this was gained via a FOIA request (Freedom Of Information Act).

I'm now a "former" Marine, but in January of '03 I found myself shipped to Kuwait, and eventually wound up in Iraq. I had it a bit lucky. I worked as an "Intel Analyst" for the 6th Engineers.

In the COC (Combat Operations Center, center of confusion, or simply Circle of Cocksuckers), we had many little toys, ranging from Toshiba toughbooks to proxima projectors, etc. We used microwave relay to keep in touch with group and make sure our batallion commander was seeing the same operational picture that 1st FSSG was seeing.

That was all done via an electronically encrypted network. Which is fine and dandy when you have:

Electricity

Computers

A network

For forward units and combat units in the field the only thing they have that comes close is the field radio. While the encryption on these things is very advanced, the radio's are bullet, shock and explosion proof. Yes, the guy carrying your map, and perhaps a list of checkpoints might not be around forever. That is why field and forward units still have to employ non-electronic means of deciet and encryption. Even if it's as simple as one guy having the map, and the other guy having a clear piece of plastic with lines drawn on it.

If U.S. Marines and soldiers are still using "old fasioned" techniqies such as this, one could surmise that our enemies are doing the same.

I was in a unit which replicated Marxist/Viet Cong style guerrillas, and we were able to use methods like this to great effect. Since we were replicating low-tech guerrillas, most of our radios were Vietnam-era, with controls like Fisher Price's My First Radio (PRC-77 for those interested). However, we were able to confound our opponents (the regular Army) on a regular basis using very simple codes, while at the same time penetrating their networks almost as regularly.

We had a pre-defined encryption scheme that radio operators were required to memorize. Mostly it was just simple word substitution, along with a simple way of encoding numbers. The key was that we all new each other and used knowledge common to all that the enemy had no way of knowing. We would avoid giving out locations more detailed than "300m South of that place we had lunch last week".

The reason these methods worked was twofold. First, the information was only useful for a limited amount of time. So even if you figured out that "Beaker plus one, minus 5, Donkeypunch plus 3 plus 1 Boomhauer minus 6 plus 2" was really grid VQ 606 419, it wouldn't do you much good because we weren't there anymore. Second, the people who were actually capable of figuring this stuff out were way in the rear, and the overhead of getting the information to the grunts (or crunchies as we always called them) on the ground was so much that it basically never happened.

Just for anyone who cares, this document is marked for official use only (FOUO) which means it contains sensitive material that should not be passed around (especially on/.). Though this is one of the lowest forms of classifications, it is still a classified document.

This is the manual I used when I trained as a 98C (Signals Intelligence Analyst = SigInt) some 10 years ago. This is *still used* now.

FOUO classification means it shouldn't have been published at all. Just because it's common knowledge does NOT declassify a document. The document can only be declassified by the originating authority (the people who wrote it, and classified it to begin with). You'll see "DECL:OADR" on these docs a lot - "Declassify on Originating Authority Directive".

This FM is meant to teach the basics of cryptology to ASVAB-passing recruits. We run through the whole thing. Some very smart people go into Intel. Some pretty dumb ones do too:).

Everyone is expected to pass the final after this is taught, which consists of 4 days worth of simulated "traffic" being passed between target stations. We've reference books for traffic pattern types, run locational analysis, crack subsitution ciphers - it's romping good fun.

The encryption methods taught are still used in the field, though less and less thanks to the Internet, crypto-secured frequency-hopping radios, and whatnot, mostly for Meteo and Logistics.

Brings back some nostalgia, reading though this. I hope they don't get into too much trouble for posting it.

I hope that no-one gets into trouble, because this stuff is found in many, many books on crypto-analysis. To think that this is in anyway unique is very, very naive. I mean, I've read books for beginners that even explained the enigma code breaking system. That's way beyond substitution etc. etc.. And academic articles go way beyond even that. That said, it is probably very easy to use in the field, so I'll keep it handy.

I've read Singh's "The Code Book". I liked a lot of the examples given and found it an interesting read.

I'd like to know if there is a good "exercise" book or website with many puzzles of increasing difficulty, including several that may need a computer. I'd much rather do this as an intellectual exercise than crosswords.

Any ideas? The select few web sites I've found tend to have one simple exercise (monoalphabetic cypher) and then suddenly change to really complex ones.

I've read several comments here about removing this from the/. site because it's secret, but in reality it's anything but. I teach cryptography in college, and the ciphers explained in the document and the codebreaking techniques are strictly old school. Nobody even uses these anymore. Old-style ciphers like Playfair, Hill, and even Vigniere have been crackable by computer in a ridiculously small time for the past 20 years.
The areas of interest for codebreakers are in advanced symmetric ciphers and pu