Month: May 2018

The third week of coding is nearing its end and I’m quite happy with how my project turned out so far.

The last two days I was ill, so I haven’t got anything done during that period, but since I started my work ahead of time during the boding period, I think I can compensate for that :).
Anyway, this week I created a second Manager class as another entry point to the API. This one is specifically targeted at the Instant Messaging use-case of XEP-0374. It provides methods to easily start encrypted chats with contacts and register listeners for incoming chat messages.

I’m still not 100% pleased by how I’m handling exceptions. PGPainless so far only throws a single type of exception, which might make it hard to determine, what exactly went wrong. This is something I have to change in the future.

Another thing that bothers me about PGPainless is the fact, that I have to know, how an OpenPGP message is constructed in order to process it. I have to know, that a message is encrypted and signed to then decrypt and verify it.
XEP-0373 does not specify some kind of marker that says “the following message is encrypted and signed” which is a design decision which was made in order to counter certain types of attacks. So I have to modify PGPainless to provide a method that can process arbitrary OpenPGP messages and which tells me afterwards, whether the messages was signed and so on.

Compared to last years project I spent way more time on documenting my code this time. Nearly every public method has a beautiful green block of javadoc above its signature documenting what it does and how it should be used.
What I could do better though are tests. Last year my focus was on creating good JUnit and integration tests, while this time I only have the bare minimum of tests. I’ll try to go through my API together with Florian next week to find rough edges and afterwards create some more tests.

It has been a week since my last blog post, so it is time for an update.

I successfully tested my OX client against an experimental Gajim plugin written by Philip Hörist. Big thanks for his help during the testing 🙂

My implementation can now backup the users secret key in a private PubSub node, as well as restore it from there. This was vastly useful during testing, as I don’t have a persistent store implementation yet.My next steps will be to implement a solution to persisting keys, as well as some kind of trust management. Florian suggested to implement the TOFU (trust on first use) trust model.

PGPainless has a key selection strategy which selects keys based on the UID. I will have to change this to use key fingerprints instead, as I noticed that a user mallory@malware.sys could publish a key with her own uid, as well as the uid of juliet@capulet.lit. In that case my implementation would encrypt the message to mallorys key as well, as it also has juliets uid. Going with fingerprints instead makes the system more secure.

XEP-0373 had some typos and was missing some examples, for which I submitted fixes. One change I made is a breaking change, so we have to see, whether it will be merged in the next days, or delayed to be merged together with later breaking modifications.

BouncyCastle

The mystery has been solved! I finally found out, why the OpenPGP keys I generated for my project had a broken format. Turns out, there was a bug in BouncyCastle.
Big thanks to Heiko Stamer, who quickly identified the issue in the bug report I created for pgpdump, as well as Kazu Yamamoto and David Hook, who helped identify and confirm the issue.

The bug was, that BouncyCastle, when exporting a secret key without a password, was appending 20 bytes of the SHA1 hash after the secret key material. That is only supposed to happen, when the key in fact is password protected. In case of unprotected keys, BouncyCastle is supposed to add a two byte checksum instead. BouncyCastles wrong behaviour cause pgpdump to interpret random bytes as packet tags, which resulted in a wrong key id being printed out.

-If the string-to-key usage octet is zero or 255, then a two-octet
checksum of the plaintext of the algorithm-specific portion (sum
of all octets, mod 65536). If the string-to-key usage octet was
254, then a 20-octet SHA-1 hash of the plaintext of the
algorithm-specific portion.

Working OX Test Client!

I spent the last night to create a command line chat client that can “speak” OX. Everything is a little bit rough around the edges, but the core functionality works.
The user has to do actions like publishing and fetching keys by hand, but encrypted, signed messages can be exchanged. Having working code, I can now start to formulate a general API which will enable multiple OpenPGP back-ends. I will spend some more time to polish that client up and eventually publish it in a separate git repository.

EFAIL

I totally forgot to talk about EFAIL in my last blog posts. It was a little shock when I woke up on Monday, the first day of the coding phase, only to read sentences like “Are you okay?” or “Is the GSoC project in danger?” 😀
I’m sure you all have read about the EFAIL attack somewhere in the media, so I’m not going into too much detail here (the EFF already did a great job *cough cough*). The E-Fail website describes the attack as follows:

Is EFAIL applicable to XMPP?Probably not to the XEPs I’m implementing. In case of E-Mail, it is relatively easy to prepend the image tag to the message. XEP-0373 however specifies, that the transported extension elements (eg. the body of the message) is wrapped inside of an additional extension element, which is then encrypted. Additionally this element (eg. <signcrypt/>) carries a random length, random content padding element, so it is very hard to nearly impossible for an attacker to guess, where the actual body starts, and in turn where they’d have to insert an “extraction channel” (eg. image tag) to the message.

In legacy OpenPGP for XMPP (XEP-0027) it is theoretically possible to at least execute the first part of the attack made in EFAIL. An attacker could insert an image tag to make a link out of the message. However, external images are usually shared by using XEP-0066 (Out of Band Data) by adding an x-element with the oob namespace to the message, which contains the URL to the image. Note, that this element is added outside the body though, so we should be fine, as so the attack would only work if the user tried to open the linkified message in a browser 🙂

Another option for the attacker would be to attack XHTML-IM (XEP-0071) messages, but I think those do not support legacy OpenPGP in the first place. Also XHTML-IM has been deprecated recently *phew*.

In the end, I’m by no means a security expert, so please do not quote me on my wild thoughts here 🙂
However it is important to learn from that example to not make the same mistakes some Email clients did.

I noticed that my blog posting frequency is substantially higher than last year. For that reason I’ll try to keep this post shorter.

Yesterday I implemented my first prototype code to encrypt and decrypt XEP-0374 messages! It can process incoming PubkeyElements (the published OpenPGP keys of other users) and create SigncryptElements which contain a signed and encrypted payload. On the receiving side it can also decrypt those messages and verify the signature.

I’m still puzzled about why I’m unable to dump the keys I generate using pgpdump. David Hook from Bouncycastle used my code to generate a key and it worked flawlessly on his machine, so I’m stumped for an answer…

I created a bug report about the issue on the pgpdump repository. I hope that we will get to the cause of the issue soon.

Changes to the schedule

In my original proposal I sketched out a timeline which is now (that I already making huge steps) a little bit underwhelming. The plan was initially to work on Smacks PubSub API within the first two weeks.
Florian suggested, that instead I should create a working prototype of my implementation as soon as possible, so I’m going to modify my schedule to meet the new criteria:

My new plan is, to have a fully working prototype implementation at the time of the first evaluation (june 15th).
That prototype (implemented within a small command line test client) will be capable of the following things:

Storing keys in a rudimental form on disk

automatically creating keys if needed

publishing public keys via PubSub

fetching contacts keys when needed

encrypting and signing messages

decrypting and verifying and displaying incoming messages

The final goal is still to create a sane, modular implementation. I’m just slightly modifying the path that will take me there 🙂

The Coding Phase has begun! Unfortunately my first (official) day of coding was a bad start, as I barely added any new code. Instead I got stuck on a very annoying bug. On the bright side, I’m now writing a lengthy blog post for you all, whining about my issue in all depth. But first lets continue where we left off in this post.

In the last part of my OpenPGP TL;DR, I took a look at different packet types of which an OpenPGP message may consist of. In this post, I’ll examine the structure of OpenPGP keys.

Just like an OpenPGP message, a key pair is made up from a variety of sub packets. I will now list some of them.

Key Material

First of all, there are four different types of key material:

Public Key

Public Sub Key

Secret Key

Secret Sub Key

It should be clear, what Public and Secret keys are.OpenPGP defines a way to create key hierarchies, where sub keys belong to master keys. A typical use-case for this is when a user has multiple devices, but doesn’t want to risk losing their main key pair in case a device gets stolen. In such a case, they would create one super key (pair), which has a bunch of sub keys, one for every device of the user. The super key, which represents the identity of the user is used to sign all sub keys. It then gets locked away and only the sub keys are used. The advantage of this model is, that all sub keys clearly belong to the user, as all of them are signed by the master key. If one device gets stolen, the user can simply revoke that single key, without losing all their reputation, as the super key is still valid.

I still have to determine, whether my implementation should support sub keys, and if so, how that would work.

Signature

This model brings us directly to another, very important type of sub packet – the Signature Packet. A signature can be seen as a statement. How that statement is to be interpreted is defined by the type of the signature.There are currently 3 different types of signatures, all with different meanings:

Certification SignatureSignatures are often used as attestation. If I sign your PGP key for example, I attest other users, that I have to some degree verified, that you are the person you claim to be and that the key belongs to you.

Sub Key Binding SignatureIf a key has such a signature, the key is a sub key of the key that made the signature. In other words: The signature is a statement, that the key that made the signature owns the signed key.

Direct-Key SignatureThis type of signature is mostly used to bind additional information in form of Signature Sub Packets to a key. We will see later, what type of information that may be.

Issuing signatures shall not be taken too lightly. In the end, a signature is a statement, which will be interpreted. Creating trust signatures on keys without verifying their authenticity for example may seriously harm ecosystems like the Web of Trust.

Signature Sub Packets

What’s the purpose of Signature Sub Packets?Signature Sub Packets are used to bind information to a key. Examples are:

Creation and expiration dates.It might be useful to know, how long a key should be in use. For that purpose the owner of the key can set an expiration date, after which the key should no longer be used.It is also possible to let signatures expire.

Preferred AlgorithmsThe user can state, which algorithms (hashing, compressing and symmetric encryption) they want their interlocutors to use when creating a message.

Revocation statusIt might be necessary to revoke a key that has been compromised. That can be done by placing a signature on it, stating that the key should no longer be used.

Trust SignatureIf a signature contains a trust signature packet, the signature is to be interpreted as an attestation of trust in that key.

Primary User IDA user can specify the main user id of a key.

User ID

User IDs are stating, to which identity of a user a key belongs. That might be a real or a companies name, an email address or in our case a Jabber ID.Sub keys can have different user ids, that way a user can differentiate between different roles.

Trust

Trust can not only be expressed by a Trust Signature (mentioned before), but also by a Trust packet. The difference is, that the signature is cryptographically backed, while a trust packet is merely an indicator.Trust packets are mostly used by a user to keep track of which keys of contacts they trust themselves.

There is currently no XEP specifying how trust decisions of a user are synchronized across multiple devices in the context of XMPP. #FutureWork? 🙂

Bouncycastle and (the not so painless) PGPainless

As I mentioned in my very last post, I was able to generate an OpenPGP key pair using GnuPG (using the “–allow-freeform-uids” flag to allow the uid format used by xmpp). The next step was trying to generate keys on my own using Bouncycastle. Bouncy-gpg (the library I forked into PGPainless) does not offer convenient methods for creating keys, so thats one feature I’ll add to PGPainless and hopefully upstream to Bouncy-gpg. I already created some basic builder structure for creating OpenPGP key pairs using RSA. In order to generate a key pair, the user would do this:

Using the above code, I’m trying to create a key pair which is constructed equally as a key generated using GnuPG. I do this mainly to make sure that I don’t have any errors in my code. Also GnuPG is an implementation of OpenPGP with a lot of reputation. If I do what they do, chances are that I might do it right ;D

Unfortunately I’m not quite sure, whether I’m successful with this method or not. To explain my uncertainty, let me show you the output of pgpdump, a tool used to analyse OpenPGP keys:

Unfortunately the output indicates an unknown packet tag and it looks like something is broken. I’m not sure what’s going on, but I suspect either an error in my implementation, or a bug in Bouncycastle. I noticed, that the output of pgpdump is drastically changing if I change the first boolean value in any of the hashedSubPackets setter function calls from false to true (that boolean represents, whether the set value is “critical”, meaning whether the receiving implementation should throw an error in case the read property is unknown). If I do set it to true, the output looks more disturbing and broken, since strange unicode symbols start to appear, indicating a bug. Unfortunately my mail to the Bouncycastle mailing list is still unanswered, although I must add that I wrote it only seven hours ago.

It is a real pity, that it is so hard to find working example code that is not outdated 🙁 If you can point me in the right direction, please let me know!! You can find contact details on my Github page.

My next steps debugging this will be trying whether an exported key can successfully be imported both back into PGPainless, as well as into GnuPG. Apart from that, I will spend more time thinking about an API which allows different OpenPGP backends.

Yesterday I got my first results encrypting and decrypting OpenPGP messages using PGPainless, my fork of bouncy-gpg. There were some interesting hurdles that I want to discuss though.

GnuPG

As a first step towards working encryption and decryption, I obviously needed to create some PGP keys for testing purposes. As a regular user of OpenPGP I knew how to create keys using the command line tool GnuPG, so I started up the key creation by typing “gpg –generate-key”. I chose the key type to be RSA with a length of 2048 bits, as those settings are also the defaults recommended by GnuPG itself. When it came to entering user id information though, things got a little more complicated. GnuPG asks for the name of the user, their email address and a comment. XEP-0373 states, that the user id packet of a PGP key MUST be of the format “xmpp:juliet@capulet.lit”. My first thing to figure out was, if I should enter that String as the name, email or as a comment. I first tried with the name, upon which GnuPG complained, that neither name, nor comment is allowed to contain an email address. Logically my next step was to enter the String as the users email address. Again, GnuPG complained, this time it stated, that “xmpp:juliet@capulet.lit” was not a valid Email address. So I got stuck.

Luckily I knew that Philipp Hörist was working on an experimental OX plugin for Gajim. He could hint me to a process of “Unattended Key Generation“, which reads input values from a script file. This input would not be validated by GnuPG as strictly as when using the wizard, so now I was able to successfully create my testing keys. Big thanks for the help 🙂

Update: Apparently GnuPG provides an additional flag “–allow-freeform-uid”, which does exactly that. Allowing uids of any form. Using that flag allows easy generation and editing of keys with freely chosen uids. Thanks to Wiktor from the Conversations.im chat room 🙂

Bouncy-gpg

As a next step, I wrote a little JUnit test which signs and encrypts a little piece of text, followed by decryption and signature validation. Here I came across my next problem.

Bouncy-gpg provides a class called Rfc4880KeySelectionStrategy, which is used to select keys from the users keyring following a certain strategy. In my testing code, I created two keyrings for Romeo and Juliet and added their respective public and private keys like you would do in a real life scenario. The issue I then encountered was, that when I tried to encrypt my message from Juliet for Romeos public key, I got the error, that “no suitable key was found”. How could that be? I did some more debugging and was able to verify that the keys were in fact added to the keyrings just as I intended.

To explain the cause of this issue, I have to explain in a little more depth, how the user id field is formatted in OpenPGP.RFC4880 states, the following:

A User ID packet consists of UTF-8 text that is intended to represent
the name and email address of the key holder. By convention, it
includes an RFC 2822 [RFC2822] mail name-addr, but there are no
restrictions on its content.

A “mail name-addr” follows this format: “Juliet Capulet (The Juliet from Shakespear’s play) <juliet@capulet.lit>”.
First there is the name of the key owner, followed by a comment in parentheses, followed by the email address in angle brackets. The usage of brackets makes it unambiguous, which value is which, so all values are optional.
“<juliet@capulet.lit>” would still be a valid user id for example.

So what was the problem?
The user id of my testing key looked like this: “xmpp:juliet@capulet.lit”. Note that there are no angle brackets or parentheses around the text, so the string would be interpreted as name.
Bouncy-gpg’s Rfc4880KeySelectionStrategy however contained some code, which would check, whether the query the user entered to search for a key would be enclosed in angle brackets, to follow the email address format. In case it doesn’t, the code would add the angle brackets prior to executing the search. Instead of searching for “xmpp:juliet@capulet.lit”, the selection strategy would look out for keys with the user id “<xmpp:juliet@capulet.lit>”.
My solution to the problem was to create my own KeySelectionStrategy, which would leave the query as is, in order for it to match my keys user id. Figuring that out took me quite a while 😀

Conclusions

So what conclusions can I draw from my experiences?
First of all, I’m not sure, if it is a good idea to give the user the ability to import their own PGP keys. GnuPGs behaviour of forbidding the user to add user ids which don’t follow the mail name-addr format will make it very hard for a user to create a key with a valid user id.(Update: Users can use the flag “–allow-freeform-uid” to generate new keys and edit existing ones with unconventional uids.) Philipp Hörist suggested that implementations of XEP-0373 should instead create a key for the user on the first use and I think I aggree with him. As a logical next step I have to figure out, how to create PGP keys using Bouncycastle 😀

I hope you liked my little update post, which grew longer than I expected 😀

OpenPGP

OpenPGP (know as RFC4880) defines a format for encrypted and signed data, as well as encryption keys and signatures.

My main problem with the specification is, that it is very noisy. The document is 90 pages long and describes every aspect an implementer needs to know about, from how big numbers are stored, over which magic bits and bytes are in use to mark special regions in a packet, to recommendations about used algorithms. Since I’m not going to write a crypto library from scratch, the first step I have to take is to identify which parts are important for me as a user of a – lets call it mid-level-API – and which parts I can ignore. You can see this posting as kind of an hopefully somewhat entertaining piece of jotting paper which I use to note down important parts of the spec while I go through the document.

Lets start to create a short TL;DR of the OpenPGP specification.
The basic process of creating an encrypted message is as follows:

The session key then gets encrypted for each recipients public key and the resulting block of data gets prepended to the previously encrypted message

As you can see, an OpenPGP message consists of multiple parts. Those are called sub-packets. There is a pretty respectable number of sub-packet types specified in the RFC. Many of them are not very interesting, so lets identify the few which are relevant for our project.

Public-Key Encrypted Session Key Packets
Those packets represent a session key encrypted with the public key of a recipient.

Signature Packets
Digital signatures are used to provide authenticity. If a piece of data is signed using the secret key of the sender, the recipient is able to verify its origin and authenticity. There is a whole load of different signature sub-packets, so for now we just acknowledge their existence without going into too much detail.

Compressed Data Packets
OpenPGP provides the feature of compressing plaintext data prior to encrypting it. This might come in handy, since encrypting files or messages adds quite a bit of overhead. Compressing the original data can compensate that effect a little bit.

Symmetrically Encrypted Data Packets
This packet type represents data which has been encrypted using a symmetric key (in our case the session key).

Literal Data Packets
The original message we want to encrypt is referred to as literal data. The literal data packet consists of metadata like encoding of the original message, or filename in case we want to encrypt a file, as well as – of course – the data itself.

ASCII Armor (not really a Packet)
Encrypted data is represented in binary form. Since one big use case of OpenPGP encryption is in Email messaging though, it is necessary to bring the data into a form which can be transported safely. The ASCII Armor is an additional layer which encodes the binary data using Base64. It also makes the data identifiable for humans by adding a readable header and footer. XEP-0373 forbids the use of ASCII Armor though, so lets focus on other things instead 😀

Those packet types can be nested, as well as concatenated in many different ways. For example, a common constellation would consist of a Literal Data Packet of our original message, which is, along with a Signature Packet, contained inside of a Compressed Data Packet to save some space. The Compressed Data Packet is nested inside of a Symmetrically Encrypted Data Packet, which lives inside of an OpenPGP message along with one or more Public-Key Encrypted Session Key Packets.

Each packet carries additional information, for example which compression algorithm is used in the Compressed Data Packet. We will not focus on those details, as we assume that the libraries we use will already handle those specifics for us.

OpenPGP also specifies a way to store and exchange keys. In order to be able to receive encrypted messages, a user must distribute their keys to other users. A key can carry a lot of additional information, like identities and signatures of other keys. Signatures are used to create trust networks like the web of trust, but we will most likely not dive deeper into that.

Signatures on keys can also be used to create key hierarchies like super keys and sub-keys. It still has to be determined, if and how those patterns will be reflected in my code. I can imagine it would be useful to only have sub-keys on mobile devices, while the main super key is hidden away from the orcs in a bunker somewhere, but I also think that it would be a rather complicated task to add support for sub-keys to my project. We will see 😉

During preparations for my GSoC project, I’m finding first traces left by developers who dealt with OpenPGP before. It seems that Florian was right when he noted, that there is a lack of usable higher level Java libraries as I found out when I stumbled across this piece of code. On the other hand I also found a project which thrives to simplify OpenPGP encryption as much as possible. bouncy-gpg – while apparently laying its focus on file encryption and GnuPG compatibility – looks like a promising candidate for Smacks OX module. Unfortunately its code contained some very recent Java features like stream semantics, but I managed to modify its source code to make it compatible down to Androids API level 9, the version Smack is currently targeting. My changes will eventually be upstreamed.

While my next target is now to create a very basic prototype of OX encryption, I’m also reading into OpenKeychains OpenPGP API. It would be very nice to create a universal interface that allows for OX encryption using multiple backends – BouncyCastle on pure Java systems and SpongyCastle / OpenKeychain on Android.

During my work on OX providers for Smack, I stumbled across an interesting issue. When putting a body element as a child into an signcrypt element, the body did not include its namespace, as it is normally only used as child of the message element. Putting it into a signcrypt element made up a special edge case. When a Provider tries to parse the body element, it would falsely interpret the missing namespace as the one of the parent element. Florian provided the solution to this problem by modifying the “toXML()” method of all elements to require an enclosing namespace. Now the body is able to include its namespace in the XML in case the enclosing namespace is different from “jabber:client”.