Two weeks ago I got the task to establish TLS secured connections via certificates to a service endpoint.
I thought it’s not a big deal, because the endpoint already uses an EV certificate from a trusted CA (SwissSign) in Switzerland. Therefore I shouldn’t have to worry that the certificate would be considered as untrusted so I don’t have to import it to the trusted certs in the Java key store etc.
FAIL! I’ve got a security exception, cert is not trusted. Same problem when I visit the website with the browser. Ok, that’s bad, SwissSign is not such a big player like thawte, so, it needs some time till it will be added to the android trusted CA list. But, when I visit thawte.com, their cert is also not trusted by android. WTF?
Windows Phone and iPhone trust my SwissSign CA and don’t complain.

So, let’s ask google, stackoverflow and the blogosphere. Found a lot of solutions how to disable certificate checking entirely.
Yeah, great, this will solve my problem, my connection will be “secure” and everyone will be able to intercept my connection and inject his own certificate. But I finally found the solution with help from other sites and some testing and debugging.

The Solution

The following main steps are required to achieve a secured connection from trusted Certification Authorities.

Grab all required certificates (root and any intermediate CA’s)

Create a keystore with keytool and the BouncyCastle provider and import the certs

Load the keystore in your android app and use it for the secured connections

Don’t use the standard java.net.ssl.HttpsURLConnection for the secure connection. Use the Apache HttpClient (Version 4 atm) library, which is already built-in in android. It’s built on top of the java connection libraries and is, in my opinion, faster, better modularized and easier to understand.

Step 1: Grab the certs

You have to obtain all certificates that build a chain from the endpoint certificate the whole way up to the Root CA. This means, any (if present) Intermediate CA certs and also the Root CA cert. You don’t need to obtain the endpoint certificate.
You can obtain those certs from the chain (if provided) included in the endpoint certificate or from the official site of the issuer (in my case SwissSign).

Ensure that you save the obtained certificates in the Base64 encoded X.509 format. The content should look similar to this:

-----BEGIN CERTIFICATE-----
MIIGqTC.....
-----END CERTIFICATE-----

Step 2: Create the keystore

Download the BouncyCastle Provider and store it to a known location.
Also ensure that you can invoke the keytool command (usually located under the bin folder of your JRE installation).

Now import the obtained certs (don’t import the endpoint cert) into a BouncyCastle formatted keystore.
I didn’t tested it, but I think the order of importing the certificates is important. This means, import the lowermost Intermediate CA certificate first and then all the way up to the Root CA certificate.

With the following command a new keystore (if not already present) with the password mysecret will be created and the Intermediate CA certificate will be imported. I also defined the BouncyCastle provider, where it can be found on my file system and the keystore format. Execute this command for each certificate in the chain.

That’s it. Took me long to figure it out, hope this helps and saves you that time.

I really hope that the android platform will implement a better mechanism in future releases for defining which Certification Authorities should be trusted or not or just expand their own trusted CA list. If they don’t, I can’t believe they will get good acceptance from the business sector. Ok, you can control which certificates you want to trust in your app, but you still can’t add thawte as a trusted CA in the android keystore and your browser will always complain about an untrusted CA. The only way I know to eliminate this problem is to root your phone (very user friendly) and add your CA manually to the android keystore.

Hmm… I could reproduce your mentioned exception. Everything looks good with the certificates. I intercepted the HTTPS with Wireshark and saw something very interesting.

The certificates are sent in the following order from the battle.net server:
> *.battle.net
>> thawte Primary Root CA
>>> Thawte SSL CA (this is the intermedaite cert)

I really can’t believe that the Cert. Path validation is so bad implemented, that the order how the server sends the certificates is important.
But I’ve had a customer, whose server was sending the endpoint server certificate twice. And I got similar exceptions in android (IE, Chrome and Firefox weren’t complaining, they also didn’t show the duplicate in the chain). We removed the duplicate from the chain and everything worked fine.
It’s really possible that the Cert. Path validation really depends on the correct order, which in may opinion sucks.

I also tried in your case to import the thawte Root cert first and then the Intermediate cert into the keystore, but still got the same exception.

Maybe you can write to battle.net to change their order, or write your own CertPathValidator or just use the crappier HttpsUrlConnection from the standard java libraries. There you can turn off certificate validation off at all (but you loose a lot of security).

You should NOT place the *.battle.net certificate in your keystore. Your keystore should only contain the trusted issuers (the Root CA and any intermediate CA’s) and not the endpoint certificate itself.

The check if your endpoint certificate is valid is done with several other criterias, like the hostname verification (does the cert correspond to the presented site).

I like to know how google manage security certificate on Android. Reason i am asking is i came across a webcast by google for android and in that they mention storing certificate on Android device and using it instead of passing it everytime and accordingly this reduces data connection overhead.

@Vlad:
The error is indicating that your operation timed out. Does your app executes the request to the server? You could intercept the traffic between the emulator and server for example with Wireshark. Sometimes the reply from the server could also be helpful.

@swapnil:
Do you have the link or name of the mentioned webcast?
I think I didn’t understand your question. What do you mean with storing the certificate and not passing it every time? The certificates are just passed during the establishment of the secure connection. After the cert. exchange (when all certificates are considered as trusted and valid) session keys are generated to secure the actual data. The session keys are valid until the connection closes.

a very nice tutorial. Thank for that.
My question is regarding client authentication. As far as i saw, the Apache HttpClient cannot handle this situation. When using only Server Auth, it works perfect , but when setting the Server to authenticate the client it fails.Do you know something about it ?
I am trying it now the “Java way” with SSL Socket , but it fails too with ” Not trusted Server certifiate 🙁
PLZ drop me an email if you have knowledge about turials or articles dealing with that problem

Hi Antoine
i figured out the problem with the “Not Trusted server certificate”
It was the TrustManager. For some reason i do not know, The Trustmanager could not authenticate the certificate of the server. I have implemented my own.
Client authentication works now.
but if you have any idea how to do it with the Apache HttpClient, please drop me some line
Thanks
Gergo

Hi,
I found your post while looking for a solution to a similar problem. What I need is to connect to the remote server and download the cert for my app to use. Basically steps 1 and 2 but within the app itself. Any thoughts on how to do that?
Thanks

Hi,
very useful, but I’m getting stuck with a 404 reply in the response. Have you any ideas to the reason for this?
Wireshark shows a encrypted data load, length 288, which could corespond to the expected return from the Webservice

@Martin
Can you access your resource via browser (without 404)?
Maybe your HTTP headers are wrong and the server returns a 404 (but another error code would be much more appropriate in this case). In my opinion 288 bytes is not very much. Maybe the 404 error itself.

I think I was able to find a fix to the sporadic nature of this solution. For those who used Antoine’s solution, but get inconsistent behavior, try importing just the root cert into your keystore and not the intermediate cert. not sure if this only worked for our setup with our servers, but it worked for us. I noticed all the exceptions I got were because it was trying to compare the root cert against the intermediate. Hope that helps!

@pyko
I think that the order of the present certificates could still be the problem, especially when you are connecting to an endpoint, which is behind a web farm. One miss configured web server could send the certificates in the wrong order, another one not.
Check with Wireshark etc. if the order is correct.

If you are not able to change the certificate order of the webserver, you could change the order programmatically.

@allen:
AFAIK every involved certificate must be in the keystore. Never tried to remove the intermediate cert. But I could imagine, that if the order of the certificates is sent wrong from the webserver, for example the first entry is the Root CA and not the Intermediate, then the Bouncy Castle checks only against the one and only Root CA certificate in the keystore and completes the checking. But this is just an assumption. You can also look at the link that I posted to pyko.

Very pleased with this tutorial. In my app, I am trying to send a soap request using ksoap2, which is working fine in my test environment. If I am trying to connect to a secured site, getting ‘Not trusted server certificate’. I dont want to bypass the certificate check like it was mentioned in some of the blogs. Is it possible to implement your solution with soap. Here is how I am sending the request.

But I’ve got a weird one. I created a self-signed certificate. It works flawlessly for all clients over WiFi. However, it fails on SOME devices over 3G/GPRS with CONNECTION_TIMED_OUT or CONNECTION_REFUSED exception. Any pieces of advice?

@steff
Is the colleague using the same Android version as you? Did you try to establish the connection via the Apache HttpClient? If not, try it, because the java.net libraries have some difficulties in the reliability in some versions.

Antoine,
Thanks for the detailed steps. I’m trying to make a connection from an app to https://auth.startssl.com and it gives me the following error:
WARN/System.err(1736): java.io.IOException: SSL handshake failure: Failure in SSL library, usually a protocol error

Do you realize that your URL requires a client certificate authentication. This means, that not only the client expects a (valid) certificate from the webserver, also the webserver expects a certificate from the client for authentication.

I am facing a problem: I use tomcat as web server, but it only accepts JKS, PKCS11 and PKCS12; while on Android, only BKS is supported. So if I use keytool to generate a JKS format certificate and keystore, but it cannot be used by Android. Is there any way to do conversions? Like, to convert JKS to BKS? or another question is Android can use PKCS12 certificate? If so, how to convert JKS to PKCS12?

@elioncho
You need to import the lowermost intermediate certificate (in your case this is cert 2) first. The root ca (cert 3) must be imported at last.
You don’t need to import your wildcard certificate (cert 1) at all into the keystore.

But please note: As I can see, your root CA cert (cert 3) is probably not a root certificate at all, because the subject and issuer names are different. A root CA cert have the same subject and issuer name (because the Root CA issues a certificate for itself). Please check if you have the right Root CA cert.

@Antoine I did as you told me in your message, but I keep getting an error. The error messages are below ( By the way, when I was adding the root ca the following message was returned: “Certificate already exists in system-wide CA keystore under alias “.

I don’t understand all that issuer-subject relation, but I have fetched other sites certificates and I found its a common pattern to not match at all. BTW, I downloaded the root and intermediate certificate from Digicert.

Below are the error messages (TrustAnchor for CertPath not found error seems suspicious, any ideas?)

@elioncho
When you imported your Root CA cert into your keystore, you most probably specified an alias that is already assigned to a cert that is already in your keystore.
In your case, I would create a new keystore and just import the needed certificates (just specify another filename for your keystore).

About the Issuer/Subject pattern: The issuer is the authority, which issues a certificate to a subject. For example the “DigiCert High Assurance CA-3” issued a cert to “*.heroku.com” and the Root cert authority “DigiCert High Assurance EV Root CA” issued a cert for “DigiCert High Assurance CA-3”. So, there is a certificate chain from the lowest cert (which in your case is heroku.com) through all Intermediate CA to the Root CA. Only the Root CA cert does not match this Issuer/Subject pattern. The root cert contains the same value for the Issuer and Subject field, because it is the root (topmost instance) of the cert chain. Therefore no superior instance is available, which can issue the Root CA cert to the root instance. In this case, the Root CA instance issues a certificate to itself (so the subject field is identical with the issuer).

So, just create a new keystore and import the intermediate CA cert first and then the Root CA cert. Don’t import your wildcard (heroku.com) cert. The keystore is just used to specify which Intermediate and Root Cert Authorities you want to trust. When your wildcart cert matches to the WHOLE chain (up to the root cert) in your keystore, it is considered as trusted (if all other validations succeeded, like expriry date etc.) and you can create a secured connection.

NOTE: Please check, if your webserver is sending the certificates in the correct order. Further info on comment #25

Why the BouncyCastle KeyStore instead of the one built-in to Android (java.security.KeyStore), like the example code you linked to does?

If I could use the latter, that would be a HUGE gain, since the BouncyCastle provider is 1.6Mb of JAR file (I haven’t yet checked to see how much of that ends up in the final package, but that’s HUGE). I’ve already got 6Mb of app package, and I’d prefer not to make it any larger if I don’t absolutely have to.

I was wondering why I run into troubles with TLS by these days… Then I realized I used to work with a 2.2 VM and, as I’m using my old laptop while on vacation, there’s a 2.1 VM. The big difference is that StartSSL (my certs provider) appears to be a trusted provider on 2.2, not 2.1! I googled a bit and found this very useful post. As I’m used with keytool, cert chains and so on since many years, I simply go straight forward your explanation with the greatest of ease.

Thanks a bunch for this code. This works perfect. I have one more concern though. I need to keep the ssl session resuable. Is there anyway this could be converted to SSLCertificateSocketFactory instead of SSLSocketFactory so that the session becomes cached with SSLSessionCache.

Thanks for this, I have been looking around trying to find this solution for months. Only issue I had was exporting the certificates. I found the easiest way to export the certificates was to use firefox. Go to the https://www.TheURLYouWantToTrust.com in you want to trust in firefox then click then to the left of the URL to bring up the certificate information then click More Information, View Certificate, Detail Tab, Then selection the root and the intermediate certificate in the tree view to be exported and click export. I was using firefox 3.6, old but did the job. http://www.mozilla.org/en-US/firefox/all-older.html

First of all thank you very much for your post on how to import and use trusted certificates in an android app. I highly appreciate it. I had a question for you. What would happen when the imported certificates expire? Do we need to deploy a new version of the android app with new certificates?

@Chris
When the imported (Root and Intermediate) certs expire, they need to be replaced with newer ones.

But AFAIK you must not deploy a new app, you could also implement a mechanism in your app to fetch a new keystore file or certificates and “install” them on the app. But I do not recommend this option due to security reasons.

I am not sure where you are getting your information, but good topic. I needs to spend some time studying more or figuring out more. Thanks for fantastic info I used to be searching for this info for my mission.

Is there ANY reason you can think of that all this code would work in Android 4 but not in 2?? I am at a total loss. It works perfectly on my phone running Android 4 but breaks in emulators and phones running 2.3 or lower.

I keep thinking I must be doing something wrong, but then I remember that it works in 4. I’m not posting the code because I’m using your code! Also, I created a keystore with my site’s cert, loaded it into a trust manager, and that is all working – in 4, not 2.

Hi,
I am runtime exception after executing the code. creation of truststore and listing of certs in truststore are same including Bouncy castle version on device 1.34.

I’m using : Rooted Galaxy S-I900 with 2.2.x android

FATAL EXCEPTION: main
java.lang.AssertionError: java.io.IOException: Wrong version of key store.
at com.httpsurlconn.SecureHttpClient.newSslSocketFactory(SecureHttpClient.java:151)
at com.httpsurlconn.SecureHttpClient.createClientConnectionManager(SecureHttpClient.java:125)
at org.apache.http.impl.client.AbstractHttpClient.getConnectionManager(AbstractHttpClient.java:221)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:539)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:487)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:465)
at com.httpsurlconn.HttpsurlconnActivity.onCreate(HttpsurlconnActivity.java:55)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1049)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2627)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2679)
at android.app.ActivityThread.access$2300(ActivityThread.java:125)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2033)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:123)
at android.app.ActivityThread.main(ActivityThread.java:4627)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:521)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:858)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
at dalvik.system.NativeStart.main(Native Method)
Caused by: java.io.IOException: Wrong version of key store.
at org.bouncycastle.jce.provider.JDKKeyStore.engineLoad(JDKKeyStore.java:839)
at java.security.KeyStore.load(KeyStore.java:676)
at com.httpsurlconn.SecureHttpClient.newSslSocketFactory(SecureHttpClient.java:139)
… 19 more

Similar to some of the later comments from users, I am having a BKS keystore problem although so far it defies solution. The 1st error I got was the “wrong version of keystore”, error listed by WebnetMobile. I found other webposts indicating “don’t use v147” of BouncyCastle or default. So I tried using mentioned bcprov-jdk16-146.jar, but I cannot get keytool to recognize this jar. I have tried placing the jar in $JAVA_HOME/lib/ext and adding “security.provider.3=org.bouncycastle.jce.provider.BouncyCastleProvider” to java.security, w/o success. I have tried specifying a path in the keytool command:
“keytool -genkeypair -v -alias androiddebugkey -keyalg RSA -keysize 2048 -validity 10000 -keypass android -keystore /Users/djames/dropbox/bc146keystore/debug.keystore -storepass android -storetype BKS -providerclass org.bouncycastle.jce.provider.BouncyCastleProvider –providerpath /Users/djames/dropbox/bc146keystore/bcprov-jdk16-146.jar”
but then I get this error: “keytool error: java.lang.RuntimeException: Usage error, ?providerpath is not a legal command”. Which is odd since when I run keytool -help, it lists -providerpath as a legal command. (Although when you look on oracle website {http://docs.oracle.com/javase/6/docs/technotes/tools/solaris/keytool.html}, keytool has no option for -provider path…???) I am using MacOSX with JavaSE6 (Vendor=Apple). I tried using alternate software for creating a BKS keystore (http://portecle.sourceforge.net/) which appears successful in creating the keystore (although not even sure which version of BC it used…). However, now Eclipse (Indigo SR2) will not recognize the keystore when packaging the android apk and throws a “java.io.IOException: Invalid keystore format”, and I’m not sure why. I have added the bcprov-jdk16-146.jar to the android build path. Has anybody using Eclipse had success in using the BouncyCastle BKS format keystore with android and have any tips?

In summary, the usual tricks working for others aren’t working for me, and I have no clue as to why. Any help greatly appreciated.

Thanks a lot for the article – it helps me a lot and save my time. The only thing that should be mentioned is Bouncy Castle provider library version: currently last Java version (it is 1.47) can not work with Andoid. One of the previous releases should be used. Instead you will receive java.lang.AssertionError: java.io.IOException: Wrong version of key store.

I have followed each and every step and have built the bks file also,have also done coding according to the details given by you but then also i am getting no peer certificate exception.What should i do now?Please help me to sort out this..