the fast, reliable localhost tunneling solution

Today we decided to jump on the bandwagon with all the other cool kids, and start accepting the Bitcoin electronic currency as a way to pay for PageKite service! We have set up two different Bitcoin addresses:

Important: Once you have made a transfer, please get in touch via e-mail and let us know, including information about which account is yours! This is necessary because not all Bitcoin clients correctly pass along the identification code which will let us link the transaction to your account.

Note that fulfillment is currently manual, so it may take up to 2 business days for your account to be updated. If this Bitcoin proves popular, we will of course streamline this and automate.

Edit 2012-12-28: Updated prices, Bitcoin ain't worth what it used to be!

We are currently in the process of upgrading our SSL certificates to use the SHA-2 algorithm instead of SHA-1. This should be a routine upgrade, but if anyone notices anything funky, please do let us know!

We have rolled the new certificate out to about half our relays, for .pagekite.me names only. If everything looks good, the certificate will be rolled out to the entire fleet and all PageKite supplied TLDs on the weekend.

There is one known downside to this - it breaks compatibility with some obsolete versions of Windows. The support for these was already marginal, and we don't feel we can compromise on security for legacy systems forever. Hopefully this won't cause too much bother for anyone.

PageKite would like to hire a Free Software hacker for the summer! We have a couple of tasks in mind (applicants should specify which they are interested in). If all goes well there may be an opportunity for longer term part-time employment.

In all cases, applicants must be able to work independently and without a fixed office. We will use e-mail, Etherpad, IRC, Skype and Github to communicate and share work.

Our budget should roughly pay for one full-time programmer for the entire summer, but if you're flexible then so are we - the sooner you can, start the better. Part time work is fine. Remote work is also fine, although we do have a slight preference for folks based in Reykjavík, Iceland (folks from outside Iceland must be able to work as contractors, handle their own taxes and send invoices for their work). We encourage mothers, students and folks with funny hair to apply.

All code will be released as Open Source (the Apache 2.0 license) and the programmer retains full copyright to his or her work.

During the first few days of 2014, there was a DNS-related outage that affected many PageKite users, causing both difficulty flying new kites and difficulty connecting to the kites that were already flying.

This is the most significant outage we have experienced since launching the PageKite.net service and we are very sorry about any inconvenience it may have caused. This post is a full "postmortem", a document which explains what happened, what the impact was, how the issue was resolved and what steps were and will be taken to prevent the problem from happening again.

Rough outage time-line

On January 1, the VPS server which was the master DNS for the pagekite.net zone had its IP address changed by our upstream provider.

Around January 2nd or 3rd, the DNS provider which ran the "slaves" for our zone stopped serving results for our zone, most likely as a result of being unable to make contact with the master. The exact timing of this event is unknown, due to insufficient monitoring and DNS caching effects.

On January 3rd, the configuration of the slave servers was updated, restoring partial availability.

On January 4th, all DNS services for the pagekite.net domain were moved to gandi.net, fully resolving the issue.

Impact

The domain pagekite.net and all subdomains were completely unresolvable for at least 12 hours around January 3rd, and service was degraded (DNS responses were slower or would fail) for the the first four days of the year. This impacted the following user-facing PageKite services:

This website, due to its use of the pagekite.net domain name

Dynamic DNS updates on up.pagekite.net and white-label customers using CNAME aliases of that

The b5p.us domain (and all subdomains) became unresolvable, preventing discovery of available front-end relays

The last two issues resulted in almost total service unavailability for a significant number of PageKite service users until they were resolved, as flying kites couldn't be resolved and new kites couldn't find relays to connect to.

Mitigating factors

Some white-label customers are using .net or .com domains for their kites. The root name servers for those top-level domains do serve glue records for nsX.pagekite.net, and thus kept those kites visible and available during the outage.

These customers may, depending on configuration, still have been unable to update their dynamic DNS records or discover new front-end relays which would have caused kite unavailability in some cases.

Analysis and lessons learned

Our monitors were insufficient to detect and report how serious the outage was. This was largely due to the effects of DNS caching - the monitors had access to cached information about the affected zones long after the problems had become visible to the wider Internet.

The slave DNS service we were relying on has been deemed unfit for use.

The pagekite.me and b5p.us problems were due to the fact that the root name servers for these top level domains do not serve glue records for the authoritative PageKite dynamic name servers (which reside on ns1, ns2 and ns3.pagekite.net). Further testing has revealed that this weakness is also shared by the .is top level domain. This is markedly different behavior from that of the root name-servers for .com and .net, and our assumptions about how these systems behaved were incorrect. This has implications not just for reliability, but also for performance, as looking up names under these domains will require more DNS requests to complete.

Our reliance on CNAME records in various white-label configurations needs to be reconsidered, as this increases the risk that DNS issues impacting one domain will impact others.

Steps taken

We have retired the old master DNS server and the 3rd party slave service and moved the pagekite.net zone to gandi's infrastructure which should be significantly more reliable.

Direct, un-cached monitoring of the pagekite.net root DNS servers has been enabled.

The monitoring server itself was moved to a new location.

Steps planned

The use of CNAMEs in infrastructure-related DNS records is being reconsidered and will be phased out wherever possible.

The TTL (caching lifetime) policy for infrastructure-related DNS records is being reviewed.

Epilogue

Once again, we thank you for your patience and understanding.

We hope the steps above will suffice to prevent this problem from reoccurring, and hopefully by sharing this document we can help others avoid making the same mistakes.

This is a minor update, mostly fixing bugs here and there, but also adding experimental native support for tunneling the Minecraft protocol. If you are a Minecraft player, check out our Minecraft Server How-To and let us know what you think!

This week we also updated the SSL certificate which is used to identify this website and provide automatic TLS encryption to our .pagekite.me users. The new certificate is valid for another two years and adds automatic SSL support to the .testing.is and .302.is domains which we provide to our subscription customers.

Fixed a memory leak which could cause the app (or even the phone) to crash

Fixed a problem where fast uploads could cause kites to disconnect

Upgrading is recommended, but note that it may take a little while for the Play store to list the new version. People who have the app installed should be notified automatically when it becomes available.

A while ago, one of our esteemed paying customers asked us if we could provide him with proper receipts for his purchases. Due to a series of misunderstandings, I had been convinced that this was actually already taken care of by our payment processor, which I eventually realized was not actually the case.

Oops. Rookie mistake!

So this week I spent some time teaching our order processing systems to generate nice PDF receipts and make them available for download from our users' account pages. As we have records of all our previous transactions, I will be generating receipts back in time for all of our customers and they should become available for download on Monday or Tuesday at the latest.

How it works

For the techies in the crowd, here is a brief summary of how the system works:

Our payment processor POSTs a notification to pagekite.net that a payment has been accepted.

Details of the order are written to a file as the account quotas are updated.

The receipt generating process notices the new file and uses the contents to customize an ODT template.

The customized ODT file is converted to PDF by unoconv.

The PDF is made available for download and the user's account history is updated.

The most interesting part of this process was writing the tool which customizes the ODT template. This turned out to be a relatively trivial task since the Open Document format is actually just a bunch of XML files in a ZIP container - both very easy to manipulate with code.

The most annoying part, was that unoconv seems to be relatively unstable and prone to randomly crashing. So much of the work involved was making sure the batch processing systems were robust enough to notice failures and retry. This is also the main reason there may at times be a slight delay between accepting an order and the receipt becoming available for download.

Please feel free to let us know if you feel the new receipts are inadequate in any way.

PageKite makes it easy for your Android device to become a public web server or SSH server, even if it is on 3G or trapped behind NAT or strict firewalls.

PageKite gives your droid a public name and establishes a connection between a server app of your choice and a public frontend relay server (a reverse proxy). You can either run the relay on a server of your own, or use the public relay service provided by pagekite.net.

This is a very exciting release for us, partially for technical reasons (this is the first app built on our high performance embeddable C library), but mostly because we think Android devices have massive untapped potential as personal servers and platforms for experimentation and innovation.

Google like to boast that over a million new Android devices are activated every single day. One of the interesting things about this incredible growth becomes apparent when you consider the average lifetime of each phone: in two years, over a million Android devices will be retired every single day due to broken screens, dead batteries or just because they have been replaced by something newer and shinier.

In most cases, these retired devices will still be quite capable little computers. Most will still have working WiFi and 3G networking, they will have working microphones, speakers and cameras. They will have working GPS receivers and accelerometers. And once they've been deemed "obsolete", for whatever reason, they will become so cheap as to be effectively free.

There are lots of these retired phones out there already and we think they deserve a second life.

Every single one of them could be a personal web server. A live web-cam. A baby monitor. A remote-controlled music player. A tracking device for your car. Or all of the above.

All it takes is a USB charger and the right apps. While testing PageKite for Android, we spent 10 dollars and 10 minutes to turn a retired old HTC G1 into a live web-cam at our office, using PageKite and a free IP camera app we found on Google Play. The hardest part was to figure out how to keep the phone from falling over...

So we're excited. :-)

Please, try the app and let us know if you do something cool with it. Tell your friends! Together, we can recycle all the phones!