Engineers Do for $1.00 What Any Fool Can Do for $2.00

Guest post by Wade Billings, VP Operations, Kynetx

When we started Kynetx, we had a very limited IT budget due to fact that the company was bootstrapped with the proceeds of selling the founders Piper Turbo Arrow airplane. From the beginning, Kynetx was built upon the conviction to be initially self funded and very strong beliefs regarding system performance and stability, so from day one, we lived inside a paradox: how do you maintain a high level of reliability, accessibility and stability when you have little in the way of resources or capital to expend.

Had we started Kynetx a scant five years ago, this paradox may well have been an unassailable barrier to our success as we would not be able to afford all of the infrastructure required to build out a scalable and stable platform. Just take a moment to think about all the equipment required to build highly scalable, available and performant systems these days. There are routers, switches, firewalls, IPS and IDS systems all of which are in a redundant mesh. Then there are the compute farm servers, network operating systems and backend storage units, all of which need to be redundant and configured to fail over in the event of a failure. All of these elements must be able to exist along two distinct dimensions represented by the X and the Y axes. The X is the horizontal axis which signifies the number of "boxes" you must scale out in order to meet your load projections. The Y axis, or the vertical axis, signifies the ability to scale up a single machine. Each dimension has unique considerations and challenges, and as a seasoned capacity planner, I take all of them into account with forming capacity plans.

To provide a point of comparison to Kynetx, when I worked for Lowermybills.com, I had the opportunity to re-architect the production platform from scratch. It was a much different experience as the capital budget I had to work with was just over 2 million dollars. When the project was complete, my team had built out over ten racks of gear and two hundred square feet of data center cage space to meet the capacity and scaling projections for the next three years. Not only did this build out provide ample headroom, resiliency and availability, it still serves as a production site for the company with very little in the way of upgrades, even seven years later.

Given our reality at Kynetx, we had to think and plan differently but end up at the same place which was a platform which would provide 99.9% (less than 8 hours of downtime a year) uptime and would be performant for our users who are spread out across the entire country. Thankfully today there exist options which allowed us to build out infrastructure without spending large amounts of capital or man hours. I am speaking specifically of the cloud computing services that are springing up all over the Internet.

Our platform, called the Kynetx Network Services or KNS platform, makes use of multiple open source virtualization technologies as well as the the Amazon EC2 cloud platform. We went virtual in order to ensure we are able to scale along both the X and Y axes with little delay or investment. Being a cloud computing based company allows us to meet demand on an "as needed" basis without raiding the company coffers. It also allows me to provide very detailed usage metrics and a quick return on investment (ROI), which makes our CFO and CTO very happy.

As a practical example of why we choose a virtual over physical infrastructure, I offer the following real life scenario. Recently we had a Kynetx platform based application, Hover.me, "go viral" after it was featured in TechCrunch. This produced an unexpected increase in demand on our platform in a very short amount of time. Had we not made the planning and development investment in an virtual and elastic platform, this development would have meant long hours spent scrambling to keep up with the demand. Instead, it turned out to be a "non event" as our platform auto scaled to meet the demand just as it was designed to do.

Had the author of the application been required to build out his own physical infrastructure, he would not have been able to provide the experience he wanted his users to have at a reasonable cost. In a word, he would have been screwed. In comparison, the overall KNS cloud based platform usage was just under 50% of available resources, which provided plenty of headroom to handle spikes. Meanwhile, the Kynetx staff was able to enjoy the holidays with family, safe in the knowledge that the platform was performing as designed.

At the end of the day, in a successful startup you have to make the best of what you have been given to work with. As the title states, engineers do for $1.00 what any fool can do for $2.00. This is not just a pithy idiom, but a guiding principal for how we operate at Kynetx.