Okay, for those of you who’ve been waiting, this is the blog. For those who haven’t, welcome, we’re going to talk about https connections (specifically) on Android (TM) from within a native app and connecting to sites that Android views as possible security risks. Easy like learning to drive if your face was in your navel. But I’ve tried to make the issues and the code as clean and clear as possible for you. Hope it is of some assistance.

I’m going to take it slow and address all of the issues, even the seemingly unrelated ones. My goal with this blog is to help those who are just getting started on Android too, so I’ll use a lot of headings, giving you the chance to skip sections you are already familiar with. Otherwise, grab a cup of coffee (or a spot of tea, or a glass of water, whatever suits you), sit back, and read. Feedback is, as always, welcome. I’m not one of the Android designers, and had to admit long ago I wasn’t perfect, so anything to improve this blog is welcome.

Android Info

We all basically know what Android is, and if you’ve been developing for it for very long, you know the quirks. There are a few major ones that I’ll mention up-front so simple gotchas and oddities later on don’t catch you off guard when I discuss them. The first, and one that is very important to the general version of this solution, is pretty straight-forward. You cannot stop execution of code and wait for user input. Purists would argue with me, and I’ve even seen the core Android developers claim that the system supported modal dialogs because of some design features, but modal as in “stop and wait for input” Android does not even pretend to support. It is a cooperative multitasking environment, and you’re not allowed to hog a ton of time idling while other apps are crying for cycles. If your app takes too long to do anything, Android will close it. If you try things like (important for our solution) database queries or network connections, Android will close the app. Remember that, because it comes up again later.

Ignoring the “cannot stop execution” part, and focusing on the “long running processes” part, we then are forced to move all long-running things like queries and network connections out of the core application thread and into a separate one. Android did a great job of making this relatively painless. There is good documentation, everything works as expected, and there are several different ways to do it based upon your needs. You can just spawn a Java thread, and let it go, you can use an AsynchTask, which allows limited interaction with the UI thread (meaning you can directly post updates relevant to the background task), and there are services. The thing is that both AsynchTask and Thread are stopped when your application is stopped. For some uses, that’s okay. For most, it is not. If you want that long-running SSL communication task to complete before exiting, then a Service is your tool. Service has an interface you can use to send it commands, and when it finishes a command, it exits on its own. This gives you data consistency, even if the application is terminated for some reason (and trust me if you don’t already know, apps are “terminated” for some pretty superfluous reasons). In our code, this is the way we went, but the source I am introducing here works with any of these approaches, simply because it isn’t the execution mechanism that matters, it is the SSL code that matters.

SSL Info

SSL has been around forever, and Java support for SSL has been too. Every time a new platform comes along though, you get people asking how to implement SSL on the platform, and you get people suggesting various ways to disable SSL on the platform. This blog is not one of those. It is one way that you can make SSL act like it does in your browser, without compromising security. Note: Security geek friends, the following is far from a definitive treatise on SSL, it is aimed at helping developers use it more effectively. Do not pick at my generalizations, I’m aiming at more secure Android apps, not training security peeps.

When an SSL connection is made, one of the first things that occurs is the SSL handshake.

This includes the server coughing up its certificate for the client to determine if the server certificate is valid. This is the point in most Java developers’ careers that their hair starts to turn grey. The process of validating a certificate is baked into the java.net libraries, and at the level we need to care about consists of three steps. First is to get an instance of javax.net.ssl.HostNameVerifier, and use it to check if the name on the certificate matches the hostname advertised through DNS. A mismatch of these two items may imply a Man-In-The-Middle attack, so the check is there. The thing is, nearly every test environment on the planet also has a host name mismatch, because they’re temporary. In the browser, a dialog pops up and says “Host Name Mismatch, continue? (Y/N)” and you choose. In code, you cannot have the browser do that for you, so you’ll have to handle it – and we’ll see how in a bit.

Aside from the Host Name Verifier, the certificate has to be checked for validity. There is a “store” (generally an encrypted file) on the Android device (and really any device using Java for SSL) that has a list of valid “Root Certificates”. These are the certificates that are definitive and known good, there are relatively few of them in the world (compared to the pool of all certificates), and they are used to issue other certificates. It costs money to get a trusted cert for production use, so in test environments, most companies do not, so again, if you’re testing, you’ll see problems that might not exist in the real-world. A production certificate that you would use is “signed” by someone and theirs was “signed” by someone, etc, all the way up to one of the root certificates. If you can get back to a root by looking at the signatures, that’s called the certificate chain, and it means the certificate is good.

But there are two catches. First, Android has not consistently included the all of the worlds’ largest of the root servers’ certificates in the certificate store. “Android” devices being sold by many different companies – who can change the certificate store – doesn’t help. So sometimes, a valid certificate will trace up the chain correctly, but at the top of the chain, the root certificate is not in the store, so the entire chain – and by extension the server you’re connecting to – is “invalid” in Androids’ eyes. Second, as alluded to above, in a test environment, you can generate “self-signed” certificates. These are valid certificates that do not point to a Root, the chain ends with them. These are never deemed valid in Android, nor in most other SSL implementations, simply because anyone could self-sign and there is no way to track it back to a root that guarantees that signing. There are other things that are checked – is the certificate expired, has it been revoked by someone further up the chain, etc. – but the killer for Android developers is the “break in the chain”, either by lack of inclusion of a root certificate or self signing.

Android With SSL

So in the normal processing of SSL, you can implement the validation interfaces and change normal function – we’re going to stick with the two that we use in this source, but there are (can be) more. We’ll look at the problem in a couple of different ways, and in the end show you how to use the code developed to pop up a descriptive dialog, and ask the user if they want to continue. This is exactly what browsers (including the default Android browser) do for you. Looking around online there are several excellent examples of handling this problem with dialogs.

Android doesn’t support this model well at all. If you recall up above I pointed out that you can’t stop code execution on Android, meaning you can’t ask the user from within, say, HostNameVerifier.verify(), because you must return true or false from this routine. False says “don’t allow this connection”, and true say “yes, allow this connection. But in Android, since you cannot use a modal dialog to stop execution and wait for an answer, if you do throw up a dialog, the code continues, and whatever the last value of your return variable was, that’s what the answer will be. This is not likely to meet your design goals.

At this point, since we’ve hit the first fiddly bit we have to solve, I’ll say point blank that those people who are suggesting that you simply always return true from this routine are not the ones to listen to. This protection is there for a reason, do not lightly take it away from your users, or you will eventually regret it.

A very similar scenario exists with the certificate validation classes – in our case, X509Certificates – you either throw an exception, or the act of exiting the routine implies acceptance and allows the connection to go through. Needless to say, in Android, before the dialog to ask a users’ preference is drawn, the routine would have exited.

So What to Do?

There are several decent work-arounds for this problem. We’ll discuss three of them that cover common scenarios.

Solution #1: If you are only ever going to hit the same servers all the time (like you would in a VPN), and have access to the tablets/phones that will be doing the connecting, then you should look up how to add certs to the Android device. This will give you a solid solution with zero loss of security and essentially no code complexity. I will not show you how to achieve this one in this blog, simply because it is the best documented of the three solutions.

Solution #2: If you can ask the user ahead of time (this is my scenario) if they want to ignore errors for a given server, then you can save it in a database and use that information to tell the routine how to behave.

Solution #3: If you cannot ask the user ahead of time, and want to be able to dynamically go to many sites, then you’ll have a little more work to do – notably subclassing the verification classes you’ll need, and installing them as the defaults for your kind of communication (TLS in our case). I’ll detail this below, but the overview is (1) Return false or throw an exception. (2) Send a message to the UI loop to say “connection failed, bad cert, retry”. (3) In the main UI loop you can pop up a dialog, and while it will not be blocking, you will be notified when the user clicks okay or cancel (assuming you’re the listener), if the notification is “yes, proceed”, then you can tell your background thread to retry and return true from HostnameVerifier, while not throwing exceptions in X509 processing.

I loathe this solution, because it causes you to build the entire connection twice, which is always bad form, but doubly so on phones with data plans, which is a huge chunk of the Android market. But short of writing extensive code that could become a project on its own, this is the best solution for dynamic access to websites available at this time. It essentially adopts the blocking dialog model to the Android model.

Source – Solution #2

The key to this solution and the next is the over-ridden classes for host name verification and certificate validation. We’ll delve into them first, then how to install them, then into how to set the up to give the SSL processing code the correct answers. We’ll also need a database, but we’ll discuss that afterward, because in theory you could use other mechanisms to implement that bit, depending upon the rest of your architecture.

HostnameVerifier is the easier of the two. The Interface in the javax.net.ssl library has only one routine – verify. We’ll implement that routine with a few assumptions, and I’ll talk about the assumptions afterward.

So here is the source for our host name verifier. It is really pretty simple, we’re just implementing the interface and using our own criteria to determine if a host name mismatch should cause us to abandon the connection…

I mentioned caveats, well as you can see, we set whether this should be accepted or not outside of the class. That makes nothing to the class. But we have to have it or the standard rule of “host names don’t match, let’s get out of here” will apply. As mentioned in the comments, you’ll want to get a pointer to the default handler before this one is installed, so you can use it for most validations – except in cases (like ours) that only need to know what the user said and don’t have to care about the normal case where hostnames match. A call in the constructor that constructs an instance of DefaultHostNameVerifier and saves the created object to a global (just as we did in X509) will then give you a reference to use to check normal processing before returning the value of bAccepted.

The original iteration of this code had the determination if we should accept the mismatch or not right in the verify method. The reason it had to move out of here is simple – connections have timeouts. We were wasting so much time doing that determination inside this class and the X509 class that the SSL connection would time out and close. Not exactly what we wanted, so I did it this way, which conveniently for blogging purposes is also easier to follow. The determination is made and we set the accepted variable before the call to connect (which eventually calls this classes verify()), More on that after this next class…

CIO, CTO & Developer Resources

The X509TrustManager handles validating certificate chains. It cares not what you’re doing, merely validates that the certificate is valid. We need to add a caveat that says “If the default Trust Manager doesn’t think this is valid, we should try to validate it with our specialized parameters.” And that is indeed what we do. Again, we use a boolean to tell us how to handle this scenario, which keeps the code both fast in execution and clean for learning.

The first thing we do (in the constructor) is use the KeyStore (where the recognized trusted certs are stored) and TrustManagerFactory (the class that instantiates and manages Trust Managers for different types of keys. We seek out the default, which is TLS on most systems. Some would tell you that this is unsafe and you should explicitly ask for TLS, but I am of the opinion that we want the default for X509, since we don’t much care beyond that. So we grab it. We store a class level reference to the default trust manager for use later. That really is the whole point of overriding the constructor.

It is save to ignore the implementation of checkClientTrusted(). We have to implement it, but don’t need it (unless you’re doing client connections…), so it simply returns the default trust manager implementation of this class.

SetAccepted() is called from our source before the connect call is made, to tell this class whether the user has agreed to accept certificates that aren’t perfect. It simply sets a class-level variable that we use in the next, most important, routine.

checkServerTrusted() is the key routine in this class. It tries to validate with the default trust manager, and if that fails, it then uses the value set with accepted() to determine if it should let the connection go through. If the default trust manager threw an exception (finding fault with the cert or its chain), and the user said “No, don’t accept bad connections” when we asked, this routine throws a certificate exception that stops us from connecting. In all other scenarios, it returns normally and allows the connection to go through.

getAcceptedIssuers() is another routine that the Interface requires, but we didn’t want to modify, since our accepted/not accepted is not based on adding new certificates. You might want to modify this routine if you are going to manage the certificates you (or the user) have explicitly trusted.

Okay, that’s the worst of it. Now we have two implementations of the interfaces that will do what we want. But first, we have to tell the system to use our classes instead of the ones currently in use. That’s relatively easy, here’s the method to do it (this comes from our connection class, and is called before HttpsUrlConnection.connnect() is called).

private void trustOurHosts() throws Exception {

// Create a host name verifier mhnv = new ModifiedHostNameVerifier(this);

This simply tells the SSL context associated with TLS that when its socketfactory creates a new socket, it should use our socketfactory (in sc,.init), then tells HttpsURLConneciton that when it opens a socket, it should use the modified socketfactory. It then tells HttpsURLConnection that it should use our hostname verifier. In the event of an exception, we’re unable to connect reliably and need to get out, so we pass the exception on.

Finally, we have to call this routine, tell our implementations whether or not to accept errors, and actually create an https connection. Almost there, here’s the code.

if(mod509 == null) trustOurHosts();

What we don’t want to do is waste a bunch of time creating and destroying the handling classes, so we create them and keep a reference to them, only recreating them if that reference is null.

Then each time we need to start a new connection with the device, we call setAccepted on both of our classes. For our purposes, we could lump both host name mismatch and invalid cert into a single question, you might want to have two different values for users to check.

And finally, in our case, we use HttpTransportBasicAuth from kSOAP2 Android to connect. We have also tested this solution with the more generic SSL connection mechanism:

HttpsURLConnection https = new HttpsURLConnection(ourURL);

https.connect();

And it worked just fine.

Last but not least

Okay, so this solution works if you already know how the user is going to respond, but what if you don’t, and you want your behavior to mimic the browser, but for whatever reason cannot use the browser with Intents to achieve your goals…?

Well, I have not implemented this, but knew looking into how to solve the problem that more people would be interested in this solution. Here’s my answer.

1. Do all of the steps above except set “accept” to false for both class implementations.

2. Make the connect call, in the cases where there are problems, it will throw an exception.

4. In the UI thread, respond to this message by throwing up an “Allow connection?” dialogfragment. with your calling class as the onclicklistener

5. in OnClick, if they said “no, do not trust”, just move on with whatever the app was doing. If they said “Yes, do trust”, use their answers to set up a new call to the thread to connect(). You now have actual values for the setAccepted() calls.

As I said above, I don’t like it because all the overhead of establishing a connection occurs twice, but it is pretty clean code-wise, and short of stopping execution, this is the best solution we have… For now. One thing Android has been great at is addressing issues in a timely manner, this one just has to impact enough developers to raise it up as an issue.

And that, is that.

There’s a lot here, both for you to consume and me to write. Feel free to contact me with questions/corrections/issues, and I’ll update this or lend a hand as much as possible. It certainly works like a champ for our app, hopefully it does for yours also. The people out there offering all sorts of insecure and downright obfuscated answers to this problem could send you batty, hopefully this solution is a bit more concise and helps you not only implement the solution, but understand the problem domain also.

My caveat: None of us is an island any more. I wish I had a list of references to the bazillion websites and StackOverflow threads that eventually lead me to this implementation, but alas, I was focused on solving the problem, not documenting the path… So I’ll offer a hearty “thank you” to the entire Android community, and acknowledge that while this solution is my own, parts of it certainly came from others.

Don MacVittie is founder of Ingrained Technology, A technical advocacy and software development consultancy. He has experience in application development, architecture, infrastructure, technical writing,DevOps, and IT management. MacVittie holds a B.S. in Computer Science from Northern Michigan University, and an M.S. in Computer Science from Nova Southeastern University.

In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential.
Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...

The now mainstream platform changes stemming from the first Internet boom brought many changes but didn’t really change the basic relationship between servers and the applications running on them. In fact, that was sort of the point.
In his session at 18th Cloud Expo, Gordon Haff, senior cloud strategy marketing and evangelism manager at Red Hat, will discuss how today’s workloads require a new model and a new platform for development and execution. The platform must handle a wide range of rec...

When building large, cloud-based applications that operate at a high scale, it’s important to maintain a high availability and resilience to failures. In order to do that, you must be tolerant of failures, even in light of failures in other areas of your application. “Fly two mistakes high” is an old adage in the radio control airplane hobby. It means, fly high enough so that if you make a mistake, you can continue flying with room to still make mistakes. In his session at 18th Cloud Expo, Lee A...

Lori MacVittie is a subject matter expert on emerging technology responsible for outbound evangelism across F5's entire product suite. MacVittie has extensive development and technical architecture experience in both high-tech and enterprise organizations, in addition to network and systems administration expertise. Prior to joining F5, MacVittie was an award-winning technology editor at Network Computing Magazine where she evaluated and tested application-focused technologies including app secu...

Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...

Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...

Using new techniques of information modeling, indexing, and processing, new cloud-based systems can support cloud-based workloads previously not possible for high-throughput insurance, banking, and case-based applications. In his session at 18th Cloud Expo, John Newton, CTO, Founder and Chairman of Alfresco, described how to scale cloud-based content management repositories to store, manage, and retrieve billions of documents and related information with fast and linear scalability.
He addresse...

SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY.
DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.

Discussions of cloud computing have evolved in recent years from a focus on specific types of cloud, to a world of hybrid cloud, and to a world dominated by the APIs that make today's multi-cloud environments and hybrid clouds possible.
In this Power Panel at 17th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the importance of customers being able to use the specific technologies they need, through environments and ecosystems that expose their APIs to make true ...

In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...

Data center, on-premise, public-cloud, private-cloud, multi-cloud, hybrid-cloud, IoT, AI, edge, SaaS, PaaS... it's an availability, security, performance and integration nightmare even for the best of the best IT experts.
Organizations realize the tremendous benefits of everything the digital transformation has to offer. Cloud adoption rates are increasing significantly, and IT budgets are morphing to follow suit. But distributing applications and infrastructure around increases risk, introduces complexity and challenges availability at every turn.
To embrace DX and to come out on top, there are four underlying principles that should guide you. Understanding these four essentials along with their relevance and impact will elevate you to DX Hero status now. Jonathan will provide a high-level overview of these principles and how some of his organization's clients have embraced them with resounding success.

DevOps has long focused on reinventing the SDLC (e.g. with CI/CD, ARA, pipeline automation etc.), while reinvention of IT Ops has lagged. However, new approaches like Site Reliability Engineering, Observability, Containerization, Operations Analytics, and ML/AI are driving a resurgence of IT Ops. In this session our expert panel will focus on how these new ideas are [putting the Ops back in DevOps orbringing modern IT Ops to DevOps].

Data center, on-premise, public-cloud, private-cloud, multi-cloud, hybrid-cloud, IoT, AI, edge, SaaS, PaaS... it's an availability, security, performance and integration nightmare even for the best of the best IT experts.
Organizations realize the tremendous benefits of everything the digital transformation has to offer. Cloud adoption rates are increasing significantly, and IT budgets are morp...

DevOps has long focused on reinventing the SDLC (e.g. with CI/CD, ARA, pipeline automation etc.), while reinvention of IT Ops has lagged. However, new approaches like Site Reliability Engineering, Observability, Containerization, Operations Analytics, and ML/AI are driving a resurgence of IT Ops. In this session our expert panel will focus on how these new ideas are [putting the Ops back in DevOps...

Atmosera delivers modern cloud services that maximize the advantages of cloud-based infrastructures. Offering private, hybrid, and public cloud solutions, Atmosera works closely with customers to engineer, deploy, and operate cloud architectures with advanced services that deliver strategic business outcomes. Atmosera's expertise simplifies the process of cloud transformation and our 20+ years of ...

Moving to Azure is the path to digital transformation, but not every journey is effective. Organizations that start with a cohesive, well-planned migration strategy can avoid common mistakes and stay a step ahead of the competition. Learn from Atmosera CEO, Jon Thomsen about the opportunities and challenges found in three pivotal phases of the journey to the cloud: Evaluation and Architecting, Mig...

Every organization is facing their own Digital Transformation as they attempt to stay ahead of the competition, or worse, just keep up. Each new opportunity, whether embracing machine learning, IoT, or a cloud migration, seems to bring new development, deployment, and management models. The results are more diverse and federated computing models than any time in our history.

SUSE is a German-based, multinational, open-source software company that develops and sells Linux products to business customers. Founded in 1992, it was the first company to market Linux for the enterprise.
Founded in 1992, SUSE is the world's first provider of an Enterprise Linux distribution.

Intel is an American multinational corporation and technology company headquartered in Santa Clara, California, in the Silicon Valley. It is the world's second largest and second highest valued semiconductor chip maker based on revenue after being overtaken by Samsung, and is the inventor of the x86 series of microprocessors, the processors found in most personal computers (PCs). Intel supplies pr...

CloudEXPO has been the M&A capital for Cloud companies for more than a decade with memorable acquisition news stories which came out of CloudEXPO expo floor. DevOpsSUMMIT New York faculty member Greg Bledsoe shared his views on IBM's Red Hat acquisition live from NASDAQ floor. Acquisition news was announced during CloudEXPO New York which took place November 12-13, 2019 in New York City.
Our Si...

Artifex Software began 25-years ago with Ghostscript, a page description language (PDL) interpreter software prevalent in printing and related applications requiring rendering and/or conversion from one software language to another. Founded by renowned computer scientist Dr. L. Peter Deutsch, our company has thrived on the basis of our sharp focus on this area of expertise, a zealous commitment to...

In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential.
Manish Dixit is VP of Pr...

The graph represents a network of 1,329 Twitter users whose recent tweets contained "#DevOps", or who were replied to or mentioned in those tweets, taken from a data set limited to a maximum of 18,000 tweets. The network was obtained from Twitter on Thursday, 10 January 2019 at 23:50 UTC.
The tweets in the network were tweeted over the 7-hour, 6-minute period from Thursday, 10 January 2019 at 1...

FinTech is a disruptive innovation that denotes the adoption of technologies that have changed how traditional financial services work. While FinTech is now embedded deeply into the financial services ecosystem, the rise of digital age has paved way to FinTech 2.0 - which is rolling out innovative solutions through emerging technologies at a disruptive pace while maintaining the tenets of security...

Microservices Journal focuses on the business and technology of the software architecture design pattern, in which complex applications are composed of small, independent processes communicating with each other using language-agnostic APIs.

Cloud computing budgets worldwide are reaching into the hundreds of billions of dollars, and no organization can survive long without some sort of cloud migration strategy. Each month brings new announcements, use cases, and success stories.