Should we move to an all HTTPS web?

There was a bit of tweeting in the SEO community today because Bing introduced an HTTPS version of their site and people thought that would mean they’d lose their keyword data. That’s not true, if you take the right precautions. I thought I’d write a bit of an intro in how all this works so you can make an informed decision on what to do and I’ll tell you what we will do.

Referrer data and keywords

When you click from http://example.com to http://yoast.com, your browser tells the website you went to (yoast.com in this case), where you came from. It does this through an HTTP header called the referrer. The referrer holds the URL of the previous page you were on. So if the previous page you were on was a search result page, it could look like this:

http://example.com/?q=example+search

If you clicked on that search result, and came to yoast.com, I could “parse” that referrer. I could check whether it holds a q variable and then see what you searched for. This is what analytics packages have been doing for quite a while now: they keep a list of websites that are search engines and then parse the referrer data for visits from those search engines to obtain the searched for keywords. So your analytics rely on the existence of that referrer to determine the keywords people searched for when they came to your site. And this is where a search engine moving to HTTPS starts giving some trouble.

HTTP, HTTPS and referrer data

What is HTTPS?

Normal HTTP website traffic is unencrypted. Every server that your traffic flows through on the way to the website’s server can read that data. If a website you visit uses HTTPS, the data is encrypted, so in theory only you and the website you visit can see what you’re doing on that website.

The HTTPS protocol is designed as such that if you go from an HTTPS page to an HTTP page, you lose all referrer data. That’s necessary because you’re going from an encrypted to an unencrypted connection and if you’d pass data along there, you’d be breaking the security. If you go from HTTP to HTTPS or from HTTPS to HTTPS, this is not the case and the referrer is thus kept intact.

So if all search engines were on HTTPS and your site wasn’t, you’d never get keyword data. The solution for that is simple though: move your website to HTTPS and you’d suddenly have all your data back. This is the case with Bing’s HTTPS implementation: if you search on it and go to an HTTPS page from their results, the keyword data is all there, as you’d expect.

Google’s not provided

“But, but, but” I hear you think: would moving to HTTPS get me all my Google keywords as well? No. Google is doing some trickery when you click on a URL, they actually redirect you through another URL so that the site you visit does get referrer data (showing that you came from Google). They hide the keyword though, as they say that’s private data. Even if you think they’re right that keywords are private data, the wrong bit about what Google is doing is that they are still sending your keyword data to AdWords advertisers. I’ve written about that before in stronger words. If they were truly concerned about your privacy they’d hide that data too.

I’d argue, in fact, that Google is breaking the web more than Bing here: even though I’m going from HTTPS to HTTP, Google is telling the website I visited that I came from Google. It shouldn’t. That’s just wrong.

Is this “right” in the first place?

I’ve been thinking a lot about this. Of course, as a marketer, I love keyword data. I love knowing what people searched for, I love being able to profile based on that. But is it right? Let’s compare it with a real world case: say that you’re shopping in a mall. You leave store A, and they put a sticker on your back. You enter store B and the shopkeeper there takes the sticker from your back and can see what you looked for in store A. You would argue against that, wouldn’t you? Now if you walk from section to section in a store and the shopkeeper can see that and help you based on that, there’s arguably not that much wrong with that.

Of course there’s more to this, in real life a shopkeep can see you, your clothes, your behaviour etc. And of course, shopkeepers target on that too. Targeting always happens, perhaps it’s just that people should be more aware of this. In quite a few cases, it might actually be deemed helpful by the user too.

I’m thinking the same is true for referrer data on the web: if you go from site A to site B, perhaps referrer data shouldn’t be passed along. Within a site though, it’s probably better if you do get that data. This is exactly what Aviator does, a browser that touts itself as the most secure browser on the planet. I think it’s an interesting concept. While as a marketer I’d hate losing all that data, as a person I think it’s the right thing to do.

Another thing I should mention here is EFF’s HTTPS everywhere project (of which I used the logo in the top of this post), which helps you use HTTPS on websites that have HTTPS for users but don’t default to it.

Should we all go to HTTPS with our websites?

Now that Bing has launched its HTTPS version (even though the vast, vast majority of their users still get the HTTP version by default as you have to switch to it yourself), it makes even more sense to move your website to HTTPS.

Here at Yoast.com we’ve always had every page that contained a contact form and our checkout pages on HTTPS and everything else on HTTP. The reason for this was that HTTPS was slower than HTTP and we’d rather not put everything on HTTPS because of that. Google’s recent work on SPDY actually negates most of that speed issue though, if your hosting party supports it. It was one of my reasons to switch to Synthesis a while back.

There’s another issue with mixed HTTP / HTTPS websites: they’re horrible to maintain when you’re on WordPress because WordPress mostly sucks at it. When you’re on an HTTPS page all internal links will be HTTPS and vice versa, which is annoying for search engines too.

So we’ll be changing, moving everything to HTTPS somewhere in the coming weeks. My suggestion is you do that too. If we’re all on HTTPS, we all get referrer data from each other (for now at least), we get keyword data from search engines like Bing that play nice and we get a more secure web. I’d say that’s a win-win situation. I’d love to hear what you think!

Post author: Joost de Valk

Joost de Valk is the owner and creator of Yoast.com. He's a WordPress / Web developer, SEO & and an Open Source fanatic.
He's also (and more importantly) the father of three sons called Tycho, Ravi and Borre, a daughter called Wende and the husband of the lovely Marieke, who also works at Yoast. Read all about Joost »

45 Responses

I think https everywhere is an extremely good idea. If enough people change, then browsers could start implementing warnings for when navigating to an insecure page, then the rest of the unwashed masses would need to switch to https to avoid scaring users who visit their pages.

Having said all that, I’m lazy and still haven’t bothered switching to https :/

More of a question than anything – don’t you need a dedicated iP address for HTTPS ? and the certs cost money? I mean surely we are adding another layer of cost to small businesses running websites for no real advantage to anyone for “flag waving” sites? the personal blogs etc… ?

It’s a common myth that SSL/TLS slows down your server much. Usually your server will have A LOT of extra CPU cycles to spare, and now in 2014, both servers and clients alike will not notice the small extra overhead of SSL/TLS. With SPDY enabled, your clients will probably see your site as faster.

SPDY is “one” of way to tackle speed issue. You can optimize ciphers list, turn on SSL connection cache (available on nginx) and tweak few more things to improve speed. I already shared link to our article dealing with it above.

By Sophieon 13 January, 2014

Nice article,
As we have been working with numerous client on SSL implementations, we found it very useful from both; users and business owners point of view.

Regarding to two different version, don’t you think that the search engine will considers them duplicate pages? I recommend keeping single version of website (that is https) rather than running two version because it will:
– avoid browser security warning when you are having https url in http page.
– improve website maintenance and management process

If you migrate correctly from HTTP into HTTPS using a combination of HTTP 301 permanent redirects and rel=”canonical” tags, Google should transfer the equity in your HTTP site over to the new HTTPS equivalent version.

This process would cause you to loose a small amount of equity in your site as there are loses going through a 301 redirect or cross URL rel=”canonical” tags but it’ll be negligible and every time I’ve done large site migrations – it hasn’t impacted the site mid-long term.

One thing I haven’t seen a lot of discussion about is support for devices that can’t/don’t support HTTPS.

Imagine the scenario where you migrate your site over to HTTPS using 301 redirects and a user agent accesses your site that doesn’t support HTTPS. The user agent will access the first HTTP URL and get redirected, then fail to load properly when the HTTPS URL kicks in.

If the migration is done using rel=”canonical” tags, requests for the HTTP version of the site will still work without any problems. A WordPress plugin could be written to change all internal links when browsing the site using HTTP to use the HTTP versions of all internal links, so as to maintain compatibility/accessibility with those user agents.

The problem still arises for traffic from search engines, in either the 301 redirect or rel=”canonical” tag migration process – search engines are going to return the HTTPS version of the URLs in search. If a user agent that doesn’t support HTTPS navigates to your site via search – that is going to fail.

There are processes in place for gracefully upgrading a devices connection, based on the logic that every internet connected device can support HTTP – but I don’t think there is a smooth way to transition down into HTTP from HTTPS if the device doesn’t support it.

Again, not relevant for most websites and businesses but it is food for thought and an interesting discussion point I think.

Do note that many of the low end SSL certs are not supported on mobile devices. If you did manage to make it responsive and optimized for a mobile user experience do make sure the SSL you get does support mobile, otherwise people will get a nasty warning when they visit you.

Steve Gibson covered HTTPS quite a bit on the Security Now podcast because of the security issues brought to light by Firesheep. He dealt with the question of slowness in episode #273 from November 2010 (transcript).

His conclusion was that there was no longer any significant computational burden for SSL. Not only are servers faster, but SSL can now cache the credentials. So once the id is established, further interaction with the server for that session does not require any more key calculation. He also quoted from Google engineers who worked on transitioning some of Google’s services to HTTPS saying that it “is not computationally expensive anymore.”

For Google Search the keyword data is still available via Google Webmaster Tools along with position in search results, number of impressions, CTR and % change. It even shows keywords that website ranks for but no one clicks them, so i guess not only AdWords and it is not so bad or i am missing something?

Good point Henk,
But how many people that are not working in our industry really understand the difference between http and https? Very few. And the average mom looking to buy shoes online does not care. I doubt customer visits will increase from earning trust through https, The truth of it is, https levels the playing field. And that’s worth every penny.

By Elsie Whitelockon 14 January, 2014

I believe that your point regarding SSL indicating that the data is encrypted is incorrect; only the tunnel is encrypted i.e. as per the name Secure Sockets Layer. Encrypting the actual data is a very different process. Enjoyed this article and thanks.

Very nice article and good content. I think If the search engine does consider them duplicate, i think we should choose either one from the beginning. But SSL is not a best way for a popular webiste (don’t need a high security) … (smile)

I’m not quite sure on the bit about https -> http not passing referrer data. You definitely still see this data in Google paid search referrers when going from a https serp to a http site. Why is this?

We made the jump to Synthesis to host our WordPress Site. I have to say very impressed and find it much better than VPS… Our next step would be to change over to https, the only question I have is that our website is a real estate website and for our listing and home search data we get the information form a 3rd party IDX Broker, they had us set up a CNAME record that forwards the information to a subdomain on our site. If we set up https will the subdomain of our site display as https too?

The company’s website does not use transactions of money, not security requirements to access so, the HTTP is the best option. There is no reason to choose HTTPS to reduce page load speed when not needed.

I have heard that your Google serps affect if you remove the HTTPS SSL certificate.
Its true. Because, I had inserted SSL certificate for one of my blog and when I removed it, after couple of weeks, it started affecting it.

By Markon 23 January, 2014

Yoast,
When you do make the move, would you consider writing an article showing the various steps you took when converting for others to follow please?

By Gregon 26 January, 2014

I second that. An article outlining how to migrate to HTTPS would be fantastic!