I am working towards building a format-compliant encryption system for pictures. The aim of it is to be able to obscure specific areas of a picture (i.e. faces, car license numbers...) while keeping the rest unaltered. The aim would be, for instance, being able to publish a picture on any given social network, making the "sensitive" regions visible only to a limited number of people, probably a subset of your friends, whom have been granted permission.

I had some (arguable) ideas regarding pixel permutations and BMP, but I have stumbled face-first on the fact that most social networks use exclusively JPEG, and any other format is compressed, essentially ruining the scheme.

So now I am stuck trying to find a new encryption scheme. Ideally, it should have the following properties:

Format-compliant: the image should be still a valid JPEG, openable and viewable.

Unobtrusive: as much as possible, the encrypted image should not have a chunk of randomly colored pixels on the encrypted areas, but keep somewhat close to what it "should" look like - something like blurring or pixelation, but reversible. The rest of the image should be unaltered.

Secure: aside from the crypto perspective, from a perceptual point of view the image should be obscured enough so as to make, e.g. a face unrecognisable without decoding.

2 and 3 are somewhat opposite, so my aim is finding some acceptable middle ground between them.

I have been trying to work with Droogenbroeck and Benedett's "Techniques for a selective encryption of uncompressed and compressed images", which proposes, for every 8x8 square, keeping the DC (aka most significant) coefficient of the DCT, and encoding the rest. However, it works pretty badly for mid-large images: since encryption is done on small squares, and the DC coefficient -which holds most of the energy- is left unaltered, the larger the image is, the "smaller" those 8x8 become in proportion, eventually becoming useless for images over about 500x500.
Other than that, most papers I have found use JPEG2000 - which apparently would make everything much easier, but basically isn't supported by anything, so it doesn't seem a reasonable solution for now.

I know this is a bit on the open-ended-questions side, so specifically, my question is: Is there previous work, or any already discussed solution, for this problem or some other similar one? Do you have any ideas about how to approach this?

Thank you for your time, and please let me know if there is I can clarify - ideally I should link pictures of what I mean, but I don't think I'm allowed below a certain rep threshold :(

EDIT

As fgrieu noted, this has an added problem [please see below]. For example, Facebook apparently decompresses and recompresses the image on uploading, which means an additional, unavoidable quality loss. That means "fine-grained" tricks with coefficients (especially the least significant ones) would not work.
Also thanks to Ilmari Karonen for linking the paper!

RE-EDIT

The problem pointed by fgrieu is avoidable in other social networks - for instance, Google+ does not recompress the image, so no problems there.

Nice question. Are you starting from a JPEG, or a non-lossy encoding? Do you care about preserving moreless the same size as a JPEG? About preserving with pixel-for-pixel accuracy (w.r.t. normal JPEG) the non-blurred area? An idea: IIRC, JPEG has comments fields that perhaps could contain the original image (or the blurred fraction thereof), enciphered classically.
–
fgrieuFeb 13 '13 at 19:41

I would like it to be able to work both with lossy and non-lossy files. Worst case scenario, just JPEG would be enough, since e.g. a BMP could be compressed to JPEG and then encrypted. Size, ideally, should not grow dramatically, but small variations would be OK. In the non-blurred area, pixel-for-pixel accuracy is not required, as long as it still "looks good" to the eye.
–
CarlosFeb 14 '13 at 8:37

I thought about that approach too, with the EXIF fields (I think those are what you mean, please let me know if I'm wrong!), but they are somewhat limited: they can't grow over 64KB in total -which is a rather small threshold for an arbitrarily large list of regions-, and I'm not really sure if maybe some social networks remove some or all of them on uploading... I'm going to look into that now.
–
CarlosFeb 14 '13 at 8:48

1

Would you care to check if in the process of removing the EXIF, Facebook (or whatetever thing post-processes your encoded images) decompreses and recompresses the image data? That would make your task much more difficult.
–
fgrieuFeb 14 '13 at 11:10

1

Your extra info tells that the image you encoded is converted to bare pixels per the JPEG standard, then re-encoded using JPEG; and you still want to restore the original using data in the image (rather that in some other database, which would be quite easy). You'll have to use techniques similar to high-density 2-D bar-codes with color, in order to encode the enciphered (portions of) the original image, perhaps to some degree in the blurred areas, and (since that wont fit entirely) in other areas, like borders. And you'll be vulnerable to the quality settings and algorithms of the re-encoder.
–
fgrieuFeb 14 '13 at 11:45

1 Answer
1

Selective format-compliant JPEG encryption as you are trying to do it is a great idea, but it won't work... not like this.

To keep the reasons short and simple:

JPEG uses lossy compression (and even lossier recompression). If you really want to create a format-compliant implementation, you'll have to take care that you're independent of any "(re)compression" issues, which might destroy your valuable, encrypted data.

This means you'll have to solve the problem that you can not store as much data in your image as you're encrypting... in simpler words: your algorithm will be lossy by nature too and you will not be able to recover the "encrypted" information back to it's original.

In fact, it's a bit like a reverse pidgeon-hole problem: you will have to see how many pidgeons you can recover after someone closed an unknown number of pidgeon-holes. Whatever you do, it will not be the same number of pidgeons as your original population. Getting back to your image question this means that you will not be able to recover as many pixels as you've encrypted... that's a result of JPEG's lossy compression.

The only alternative option would be to create a new image format which fits your needs. Yet, then the question arises if anyone will adopt your new image format. Looking at what happened to aPNG (animated PNGs), I'm not very optimistic that you'll have a chance - unless some social network(s) push forward and start supporting it. And then you'll still have to see if it really spreads throughout the several programs and tools out there (think: Photoshop support etc.).

Please don't get me wrong: I love the idea you're having there - but as with many good ideas, I'm not seeing it happen for technical reasons. Sorry to say so.

Hello, e-sushi! Thank you for your reply, I was not really expecting any, anymore :) Still, I can't say I agree with what you're saying, sorry! If you mean compressing from e.g. BMP to JPEG, yeah, I'm with you - it has to be lossy. But if the compression is JPEG->JPEG, it doesn't have to. Think (as in the article linked above) of a procedure consisting in a traditional encryption (e.g. Caesar's encryption, just to keep it simple) on the JPEG coefficients. That would neither lose quality, nor increase the size of the message, would it? Maybe I'm not getting exactly what you mean?
–
CarlosMay 20 '13 at 14:32

I was referring to recompression and conversion issues. In an ideal world you would embed a recovery record as a failsafe for recompression and conversion cases. The failsafe would help recover crypto-data to it's original, in case it gets modified. Now the RealLife™ logic: you only have as much byte-space as the to-be-crypted pixels use (that's only 4 BytesPerPixel in an RGB32). So there's no space for any recovery record. As soon as someone converts or recompresses your crypto-image, the crypted data will be broken or lost. Simply relying on Google not to change is not an option… for a pro.
–
e-sushi♦May 20 '13 at 18:51

You can encode data in the alpha channel for most photographs which don't use that channel (if the format permits it), which lets you introduce some redundancy, maybe saving the scheme. You can also insert encrypted metadata in most formats without breaking compliance, which may or may not be compressed. So I'm not too satisfied with this answer. There is more to it than just "you can't do it because there's not enough space". There is plenty of space in many image file formats to write a bunch of stuff (though obviously it will be very file-format-centric, not a general solution).
–
ThomasOct 18 '13 at 14:12

@Thomas In case you missed it: OP wants it to work with social network websites… which tend to recompress images using lossy algorithms. The problem OP and you seem to ignore: you have no control on how social networks will handle your images, so you have no way to be sure your precious data isn't destroyed along the way. Facebook and Twitter recompress and dump unused alpha-channels, resulting in loss of data. That's why OP's way and your arguments, with the social networks recompressing images in the middle, won't work. No matter if you're satisfied or not, I can not change those hard facts.
–
e-sushi♦Oct 18 '13 at 14:26