This is why we will perish

101010

OK, we’ll all perish (no one gets out of here alive), but there are really interesting sign posts on the way to our general demise. Forget about climate change for a minute; this one is the courtesy of the computer world, which is encapsulated in the “Underhanded C Contest.”

In this devilishly (!) clever contest, a human programmer is tasked with doing something “underhanded” but at the same time, looking “innocent” and unable to be picked up by even more serious study/analysis. Perhaps, with enough analysis, you’d find the glitch, but these little programming “bon mots” are frighteningly clever, and are probably the tip of the iceberg when it comes to backdoors, both accidental and planned.

Because computers have become hideously complex, most of the time, we don’t have the time to manually verify everything ourselves. We rely on computer tools and compilers to check things out; and if those tools don’t catch devious tricks and errors, we can be in a world of hurt.

Some of the underhanded things done in this contest have been:

“Fingerprinting” images that aren’t supposed to be fingerprinted

Making some computer operating systems look bad

Fiddling with file encryption, so a small portion of files aren’t really encrypted

Messing with a luggage tracking system

Spoofing a hypothetical nuclear weapons monitoring program

Yes, these were all theoretical exercises, but they really opened my eyes as to how complex and sneaky some folks can be in both coming up with interesting problems, *and* how to sneakily perform the required tasks. The website notes it is “The official perfectly innocent web page for law-abiding good guys,” and the FAQ makes some good points about why this kind of stuff is important.

For me, the larger view is that when these sorts of technologies can’t even be checked by humans, or even the tools that they’ve built to check them, and that’s where we start to seriously lose control. These ideas and contests are probably the tip of the iceberg.

Questions:

What other technologies have this sort of glitch? Could (or has?) an airplane (or spacecraft) designer come up with a subtle flaw (shades of Rogue One) that would make something incredibly vulnerable?

Is this a problem or a predicament? Is a solution possible, or is it too complex?