Author: aboyd

When it comes to storage, one of the most common assumptions is that cloud is the best option for keeping data safe and accessible. And while it does have its benefits, we recently sat down with Allan Boyd, COO of AetherWorks, a software research and venture development firm specializing in distributed systems and with patented technology in software-defined storage and fog computing, to learn more about storage beyond just cloud solutions.

Q. Why does everyone think cloud for backup?

A. The availability of cloud storage has greatly expanded our options for keeping data safe and accessible. Before the cloud, options were much more limited. You could either choose to write to tape and send it via truck to a remote location or buy more hardware to meet your backup needs. Now that the cloud is better understood among mainstream audiences, it’s become the “go-to” storage solution for ease and convenience. Using the cloud means not having to worry about purchasing or managing additional hardware (or tape, for that matter), saving time and money.

Q. What are the benefits of the cloud?

A. Cloud adoption continues to accelerate, and for good reasons. The cloud makes data accessible from anywhere with an internet connection, which is great for remote working, and its usability, flexibility, and accessibility profiles are quite strong. Cloud isn’t necessarily the cheapest option, but when you factor in all of the costs of acquiring, installing, powering, maintaining, and upgrading your own hardware – that’s when cloud can start to look like a good idea.

Q. What is the problem with cloud?

A. As great as the cloud is, it’s not a magic bullet. There are certainly accessibility and cost issues. For example, if you don’t have an Internet connection, you cannot access your data. Even if you do, cloud recovery times can be prohibitively slow, as evidenced by the number of cloud providers that offer to write your data onto a disk and ship it to you. In terms of cost-effectiveness, there are hidden costs beyond just data storage. For example, Amazon S3 retrieval and requests cost extra, as does expediting retrieval, which is still slow. To address these issues, many large companies offer to sell you more hardware that you can use onsite. However, more hardware creates more problems, including added costs and a negative impact on the environment.

Q. What’s a better solution?

A. I think that a better solution is locally pooled storage with a cloud tier. Locally pooled storage takes advantage of spare hard drive space in an office environment, combining it to create secure drives that are distributed across many existing workstations and servers. In this way, your available storage grows with your primary infrastructure. It enables users to create a “private cloud” for onsite backup, without the costs of purchasing and maintaining additional hardware.

With this approach, you can stick to the 3-2-1 golden rule of data backup. This means having 3 copies of your data, across 2 different storage types, and 1 of those should be offsite.

Our product, AetherStore, is software that complements the cloud and provides the local component for faster reads, and better availability. AetherStore is a more reliable, flexible and cost-effective solution. It will be there whenever you need it and your data will always be immediately recoverable.

Q. Why does this matter?

A. Locally pooled storage is replicated across many machines so there’s no central point of failure. The data is encrypted and chunked before being distributed across the network, so if a computer in the system is lost or stolen, there’s no discoverable data on any one machine and the backup automatically recovers to your desired replication level. Similarly, if a chunk is corrupted for any reason, the system identifies it and a non-corrupt version is re-replicated.

The local storage pool offers high availability, fast local recovery times, and intelligent self healing, while reserving cloud for rare disaster recovery events. It’s not dependent on any third-parties so your data is always there (so long as your office is there). Plus, it is easily downloadable and only takes a few minutes to get the system up and running.

Q. What makes locally pooled storage better? For example, how does AetherStore help?

A. Recovery time objective (RTO) is one of the most important metrics in backup and recovery. AetherStore enables you to meet vastly improved RTOs, and does so without incurring the added expense of a backup server. By making use of existing hardware, AetherStore is better for the environment and better for your wallet. There’s no downtime waiting for data recovery, which can drain valuable time and resources. And it provides peace of mind– our users can sleep at night knowing that the copies of their data are exactly where they should be.

So, the next time you need to think through your storage solution, remember, there are options beyond the cloud, and locally pooled solutions may be exactly what you need.

About the Author

Allan Boyd is Chief Operating Officer at Aetherworks LLC and is Co-Founder of AetherStore and ActiveAether, deep tech solutions for distributed storage and fog computing. Based at AetherWorks HQ in NYC, Allan oversees operations and strategic planning for the company across its verticals, which include development of original high tech software solutions, investment in early stage companies, and patenting of new technologies.

Allan joined AetherWorks following his ascent through the ranks at Kelvin Connect in Glasgow, Scotland. During his time at Kelvin, he saw the company through its acquisition by Airwave (now Motorola), successfully managing the transition and executing the company’s post-merger workforce integration strategy.

Allan graduated from the University of St Andrews with Honors in Computer Science. Prior to completing his degree, he was awarded “Best Growth Potential” and “Best Overall Business” from the Scottish Institute for Enterprise for his first company, IT Onsite. Allan is an accomplished musician, having performed with the Royal Northern Philharmonic, and is a supporter of children’s music education through the Fife Horn Union, a charitable organization based in Scotland, and through an annual scholarship he sponsors at the Ingenium Music Academy in the UK.

IT outages happen so often these days it can be hard to keep track. Most recently, a typo at Amazon “didn’t quite break the internet”, but caused headaches for hundreds of thousands of websites across the US. Another well-reported blunder occurred in late January when a Delta Air Lines outage prompted it to cancel 280 flights. This followed an outage in August that cost Delta $150 million and another at Southwest Airlines that cost that airline $177 million last July.

Data center downtime costs are always expensive. Last year, a study by the Ponemon Institute found that unplanned outages at data centers cost $8,851per minute. Unfortunately, current backup methods to minimize data downtime have shortcomings across the board. Backup hardware can be prohibitively expensive to purchase and run, and cloud backups are often so slow to restore that, in the event of data loss, the cost of downtime is crippling.

Locally pooled storage has emerged as the best alternative for getting your data back, when you need it.

More Hardware vs. the Cloud

One logical safeguard to minimize your risk is simply to double up on your existing hardware. That way, if one of the systems goes down, there’s still a backup. This makes intuitive sense and is the same reason we all have more than one house key.

The issue is that computer hardware is a lot pricier and more complex than a key. In addition to the upfront capital cost of purchase and setup, there are requirements for additional physical space, ongoing maintenance, and hardware updates every three to five years. Figuring out capacity can also be a challenge. Buy too little and you impact your company’s ability to function. Buy too much and you’re wasting money.

Being on-premise also opens up the hardware to all types of vulnerabilities. Employee accidents pose a constant hazard. For instance, it is ‘speculated’ that a lone worker’s soda spill may have taken downBloomberg’s terminals across the country for two hours in 2015.

The common alternative to on-premise hardware is the cloud. While you won’t have the same hardware related issues in the cloud, you will have a tough time getting your data back in a timely manner. Cloud providers reassure users that cloud is now an option for tier 1 backup, but relying on 3rd party telecoms to transmit high volumes of data as and when required is a high-risk strategy. Even in urban areas with high-speed connections, recovery time for cloud-based systems are often so long that it can be quicker to physically mail hard drives than to restore data from cloud. Unreliable in the best connected environments, cloud as a viable tier 1 recovery method is met with outright dismissal in many rural areas of the US and in most of the rest of the world.

Hidden charges of the cloud are also often forgotten. Basic cloud storage might look cheap but be aware of your access patterns and the recoverability you are required to have. Reads, edits, deletes, and geographic redundancy all come at an additional cost.

Given these two options, the leading strategy is a hybrid approach that aims to reduce the risk of failure posed by the weaknesses and pitfalls each presents individually.

However, piecing two problematic solutions together to combat each other’s risks is a suboptimal solution at best.

So what is optimal?

Locally Pooled Storage in Hybrid

A smarter alternative to on-premise physical, cloud-based, or the combination hybrid backup is a solution that manages to combine the best pieces of each whilst eliminating the disadvantages: locally pooled storage with a cloud tier.

Locally pooled storage takes advantage of spare hard drive space in an office environment, combining it to create secure drives that are distributed across many existing workstations and servers. Thus, your available storage grows with your primary infrastructure. It enables users to create a “private cloud” for onsite backup, without the costs of purchasing and maintaining additional hardware. Many businesses don’t realize that they have terabytes of space already paid for and never used.

Locally pooled storage is replicated across many machines so there’s no central point of failure. The data is encrypted and chunked before being distributed across the network, so if a computer in the system is lost or stolen, there’s no discoverable data on any one machine and the backup automatically recovers to your desired replication level. Similarly, if a chunk is corrupted for any reason, the system identifies it and a non-corrupt version is re-replicated.

Best of all, installation is simple: It only takes a few minutes to get a locally pooled storage system up and running.

Tiered to a cloud layer for offsite backup, this model is a smarter core infrastructure than is currently available. The local storage pool offers high availability, fast local recovery times, and intelligent self healing, while reserving cloud for rare disaster recovery events.

Dodging a Bullet

We are in a dynamic environment in which game-changing tech solutions come around quickly. As downtime continues to threaten vibrant businesses and onsite hardware continues to require too much TLC, it’s time to consider an alternative to the status quo. A solid backup infrastructure and executable, tested plan is a form of insurance that can save companies millions of dollars. But make sure your system actually works. Will yours be ready when you need it?

A call, a dodgy website, an impressively themed pop-up – most of us know how tech support scams work, and most of us probably think we’ve got it covered. So how did I end up on the phone to LifeLock, IdentityForce, Experian, Equifax, my banks and my immigration lawyer, in an ordeal so frustrating I’m dedicating an entire blog post to it?

My Mum.

Turns out, our digital risk extends further than our own machines. There are more questions we should be asking, including, “How many people do we know that might be susceptible to digital theft?” And , “How would this affect us?”

Most people I’ve spoken to initially told me they don’t store any private files on their machines. A little probing reveals they do, because they recently renewed their passport, lost their driving license, or confirmed their ID with the bank.

My Mum got caught out by a ‘Microsoft’ pop up. She gave their technical team a call on the number given, and by the time she worked out she wasn’t speaking to Microsoft, they had ran through a barrage of tasks, installed some software and deleted a few bits and pieces. Of course, they charged her for it.

“At best, the software is worthless or available elsewhere for free. At worst, it could be malware — software designed to give criminals access to your computer and your personal information.” FTC

I found out about the mistake several days later in an email containing a list of documents of mine that were on the computer in question. Shortly after, I received a second email as the list increased in size. From forwarding a bank related letter via email, to scanning my birth certificate for my green card application, all my info was on there for one reason or another.

“Tech support scams are a million-dollar industry and have been around since 2008. Every single day, innocent people are tricked into spending hundreds of dollars on non-existent computer problems.” MalwareBytes

It is impossible to fully protect ourselves and others against these attacks, even if you think you personally would never fall for them. As we continue to churn out solutions and educate inexperienced users, there is no doubt the bad guys will come up with more innovative ways of tricking our friends and family.

But that shouldn’t stop us from trying.

Our software has always been designed to use spare space on existing computers to provide highly secure and self healing drives in minutes. While businesses typically deploy it across many machines for these reasons, the encryption, locking and invisibility features could be just as vital to home users with just one or two computers. AetherStore can be downloaded, installed, set up and ready to go in under 5 minutes. If you don’t believe us, watch this 2-minute setup video.

There are a number of other companies and open source projects that provide a service like this, but I believe we have the most straightforward solution to understand, setup, and maintain. We follow the core AetherStore principles: give your drive a name, size, drive letter, and enter a password. Then if there’s anything you’d like to have secured in your computer, store it in your AetherStore.

We’ve decided to offer a 25GB store for free so everyone (and their friends and relatives) can take advantage of this. You can download AetherStore 2.0 here.

It seems that the most popular analogy on the web for having an encrypted drive on your machine is of a house having re-enforced front doors. But looking at this situation, I don’t see it like that. This is an extra line of defense. You might make a mistake and let a dodgy salesperson into your house, but would you open your safe?

Recently LinkedIn added a feature that allows you to endorse people for certain skills. Simply put, this is the ability to ‘plus 1’ a skill for any connection.

Most would agree that the ability to gauge someone’s perceived expertise at a glance is useful. My problem with endorsements is that the lack of restrictions on handing them out undermines the value of the process. For example, I have one colleague, who shall remain anonymous, who simply endorses everyone for everything (more on this later). The consequence of this approach is that people are happy about receiving endorsements and then feel obliged to endorse him/her back. My colleague is now a perceived expert in an astonishing number of areas!

This guy is good!

Problem 1 – Reciprocal Endorsements

There is no cap on the amount of endorsements one person can give. With the general encouragement LinkedIn gives to ‘pass endorsements forward,’ we arrive at a situation where it is beneficial to endorse everyone for everything in hopes the favor will be returned. Since there is no way of viewing how many endorsements any particular user has given, the theory suggests a heavy endorser will likely be made to look far more impressive than they actually are – the most effective strategy is to endorse indiscriminately. As this trend continues, we tend toward a scenario where honest endorsements are marginalized.

Should you ‘pay it forward?’

The obvious preventative solution is to limit the number of endorsements a user can give. This would limit the range of the tool, but increase the value of each endorsement, while still using a metric that is easy to understand.

If they didn’t limit endorsements, LinkedIn could display how many recommendations a user has given next to how many that user has received. A user viewing a profile can then gauge how meaningful someone’s received endorsements are. The downside to this approach is that the metric cannot pin-point reciprocal endorsements and thus leaves a large margin of error for different types of users. Interestingly enough, LinkedIn already does this for the written recommendations; although as the amount of these is far fewer, the metric is easier to assess for users.

Personally, I think the best solution would be a dynamic cap. One endorsement per connection added on your profile, with no restrictions on their use. Make people consider what their use case is and how to disperse them in a meaningful way.

Problem 2 – Suggested Skill Box

In a bid to encourage endorsements, LinkedIn displays an endorsement box on all profile pages. It proposes four suggestions:

Who do you want to endorse?

First, the skills generated in the skill box are not always the skills specified on the relevant user’s profile. The LinkedIn algorithm often suggests skills that are completely irrelevant. Overnight you can become a perceived expert in something you have no experience in at all. My personal favorites this week were Bash and Python, for which I received 3 endorsements each. I have never, in my life, held a developer/engineer position or written a line of Python!

Second, these questions are far too suggestive. Being offered a name and an associated skill is simply leading the user down a path, and as discussed above, this is often the wrong path. “Who would you recommend for Java?” would be a far better question. The user should at least have to think about the skill set and their connections. Even better, how about limiting the suggested endorsements to those that you have noted as your own skills? Maybe this wouldn’t be enforced if the user were to go to a connection’s page to endorse, as this requires more effort than a skill box endorsement.

Finally, the interface for the endorsement box just encourages inaccuracies. The most prominent (and highlighted button) is, of course, ‘Endorse all 4’. If you close one recommended endorsement, another appears. If you endorse one connection, another will appear. If you ‘endorse all 4’, you can do four more! You can endorse 100 connections in your network for a random skill in less than 30 seconds. Now that’s efficiency. I have been told by my anonymous source that if you continue to ‘endorse all 4’ the suggestions do eventually dry up…

Wrap Up

I’ve had quite a lot of fun on LinkedIn this week, adding new connections and endorsing people I know well, a little, and although I must confess to being part of the system I’m complaining about, really not at all. I’ve had a variety of responses, from thank you emails and return endorsements, to the dreaded no response (disclaimer – I have since deleted all the extra skills that were created and the related endorsements!).

As attractive as it is to have endorsements coming out your ears, with the current approach LinkedIn is taking to try and encourage its use, it is only a matter of time before people will disregard what could be an incredibly useful feature. You can see the inevitable happening already – try it.

New York is pushing hard in tech start-up world right now. Bloomberg’s WeAreMadeInNY campaign received a lot of press attention last week and hopefully this continuing support by the city, alongside the success stories that are published with chiming regularity, will encourage the best grads and experienced pros in our industry to flock here. But it doesn’t appear to be happening just yet. Not for us anyway.

So, who are we — a small-scale start up — competing against for talent?

One idea is that we are pitted against big corporate. We see a lot of resumes filled with banking development experience. Multiple jobs lasting 18 months to 2 years. Boredom seems to set in and candidates go searching for the next chapter in their career, invariably another bank. The trend goes on…

I get the impression that these candidates are asking themselves the wrong questions. It has been mentioned to me several times that going to a large company aids career progression in a way that can’t be achieved by going to a start-up. Well what kind of career progression? Better salary? Maybe. Better title? Possibly. More responsibility? I don’t think so! I understand different types thrive in different environments, but from a computer science perspective and someone working in software I’d encourage anyone to read James Currier’s open letter to his computer science alma mater at Princeton. And just in case you don’t, below is a short quote that captures the essence of Currier’s message.

“Big companies teach you how to work through layers of bureaucracy and how to solve problems in very risk-averse ways — in short, how to make something happen in their organization. A big company is not the safe career choice. It’s the risky choice. It risks your mind and your life.”

Here at AetherWorks, we have a very talented group who are all involved in the work and decisions that push the company forward. As a result, we all have a collective responsibility to learn, to achieve, and to progress. We are a team. There are opportunities that present themselves here that I believe would represent valuable experience to anyone at any stage of life, let alone new grads. Sit in on design meetings, take part in product planning, generate new sales strategies for our latest products, pick a topic for our weekly research meetings. We have an environment in which anyone can come forward. We all have our areas of specialty, but we also have the opportunity to learn and assist across the breadth of our business. The best bit? If we do something fantastic, then we’re all going to reap the benefits.

Great, right?

We do need people to take the initial leap of faith though. Our latest recruitment drive has been very challenging and it’s made me aware of a few areas we need to work on to change perceptions.

The resumes pile up.

One notable disadvantage we have, versus large corps and more established start-ups, is that we are less embedded in the systems of recruitment that can help attract talent. AetherWorks has had little time to integrate into the NYC community, we do not have a live product just yet (but we will very soon), we have had limited (but very positive) press, we have had little opportunity to create useful internships that may lead to permanent roles, and maybe more than anything, our alumni network connects us to the UK rather than the US.

We continue, however, to strive to address all of these issues. We are using multiple sources to advertise and post our openings and we continue to raise our profile in the city by attending meet ups & conferences, writing our blog and distributing press releases. On top of that, Fay (Director of Business Development) has been sharing her experiences in schools around the northeast, and Gus (Director of Research) is planning on opening up our research sessions to the public once a month (for more info email us).

It’s a long and competitive game, but hopefully we will see some good results soon. Of course if you happen to be searching for a software engineering role, you know what to do!

For the first of my posts on the operations of AetherWorks, where better to begin than our first real challenge: finding the perfect office for our company.

When we started our search in 2010, we had three options:

1) Setting Up Within an Incubator. The tech scene in NYC is booming and there are plenty of options here. If we had applied and been successful, an incubator would have provided managed office space alongside a range of other services. These vary from organization to organization but often include access to mentors/advisors and strategic help with partnering and access to loans and grants. Being in an environment focused on the development and longevity of start-ups and having bundled expertise on hand was a very attractive proposition. See Dogpatch Labs as an example.

2) Fully Managed Facilities. These facilities are fully furnished and equipped offices that are already in move-in condition. They provide dedicated support staff and are available on short or long-term leases. In addition to reducing your upfront costs, work can begin immediately in Fully Managed Facilities. This is where we were based when we started our search. See Regus as an example.

3) Traditional/Custom Office Space. With traditional office space you must first decide on the grade of building you are willing to settle in. Grading’s are based on attributes from location strengths (public amenities, transportation etc.) through the building efficiencies (energy management, exit routing, parking etc) and can give a quick overall assessment of the quality of the building. The grading gives a clear idea of how much you will spend per square foot. If you can find a suite, room or floor that has been occupied in the past, and suits your needs, your calculations can stop there.

Alternatively you may find a raw space within a building you like that requires a build out. The landlord may offer to do this for you, with some degree of flexibility over the space, or you can opt to take care of this yourself. If building out your space, there are significant costs for contractors as well as the delay in entering the space while work is ongoing. There is also a great deal of setup involved for services that can be taken for granted in the other options. Leases are standardly 5+ years and as a young company with limited credit there can be large down payments, returned at the conclusion of the lease. But, this does allow any company to completely tailor the space for their specific needs. See Abramson Brothers as an example.

We found the perfect raw custom office space at 501 Fifth Avenue. It would involve a comprehensive build-out, requiring a significant amount of input from a number of our employees. But, having long-term investment, we felt that long-term planning was the way forward. With a clear vision of where we wanted to be, this would ensure we had the perfect space to operate from and expand into. Little did we know everything else that this would entail. In April’s blog I’ll write in detail about the build out and everything you’ll need if you ever consider going through with one!

Requirements

The only way we could successfully tackle this project was to focus on who we were, what our goals were and, what we needed to achieve those goals. As an R&D firm, we felt we had a lot of default requirements for an office space that would not be as highly prioritized within a more standard software development company. Yes, we still had to ensure that each individual felt comfortable in their space, had the room they needed and the tools to assist them, but we also needed to consider the research in R&D. Our longevity as a company hinges on not only designing and delivering software of the highest quality, but also continuing to create intellectual property and ensuring our research cycles are efficient and rewarding.

Our office before the build-out.

There are many ideas that claim to contribute to random stimulation and creativity within your office or work space: visual stimulation through artwork and colors, having magazines and journals lying around, providing games and toys for breaks, but here, we had the chance to go a step further. We had the opportunity to create the perfect environment where all these extras would add to the creativity within our space. To this end we knew the pressure was on to build a great space for creative cooperative working and within this requirement, we had to cater for research groups that would often vary in size. We had to ensure that both large and small groups would feel motivated and comfortable enough to spend prolonged periods of creative time (occasional overnights if necessary…) dealing with hard problems – all within our space restrictions in NYC.

With the walls of the office up, all that’s left to do is make the space our own.

Being rather inexperienced in the world of build-outs, we had yet to discover how important it was to have designers and contractors that bought into our ethos and fully understood what we needed. We got lucky… the second time.

The finished article. Lots of natural light and open spaces.

As I mentioned before, I’ll save the specifics of the design and construction, which inevitably went on far longer than planned and caused an incredible amount of stress, for another post.

Was it the right option?

Building out our space was a lengthy process; the duration from our initial viewing through absolute completion was nearly 18 months. But for all the phone calls, emails, drama, irritation and stress, we now have a fantastic, light, open, creative office that suits us perfectly. We do have a long term lease, and we do have to manage our own services, but having the freedom and flexibility to create our own environment was far more important to us. Everyone now has their own personal space, we have ample room for our research activities, and we also have a great environment for welcoming customers and visitors. Most importantly, if you speak to any of our employees, one of the first things they will say about working here is that we have a phenomenal space to do our thing.

We knew what AetherWorks was, we built what it needed, and now we have an office that shows people who we are. That’s a result.

Welcome to the new blog. I’ll use this first post to give you a brief idea of who we are and what we do.

At our heart we are a software R&D firm with a particular expertise in distributed systems. We were founded in 2010 with the goal of developing intellectual property assets for the enterprise, and have since focused our efforts on bringing our first product, AetherStore, to market.

We are now a seven person team, built from a core of experts with the knowledge and experience needed to make AetherStore a reality.

Over time everyone will be contributing their perspective to the blog, sharing their experiences at AetherWorks. Personally, I’m looking forward to contributing my own experiences as COO in the coming weeks and months.