Archive

Contegix turned me on to daemontools which is a great way to manage services you need to keep alive. A great use for it is managing multiple Apache Tomcat instances which can each be easily be configured to run under different users with different options.

I was recently running into a problem where a webapp running in Tomcat didn’t have access to a particular directory mounted via fuse. I could verify that the user Tomcat was running under had access to the directory by logging in as that user then creating a file there. WTF?

Turns out the run script being used for the Tomcat service was using the daemontools program setuidgid, i.e.

The Tomcat user’s access to this particular directory is in fact granted based on a secondary group membership and in this case should really stay that way. setuidgid only allows a single user argument that’s used for both the user and group so I couldn’t change the running group and there’s no option to keep the user’s secondary groups.

Instead I opted to use su instead of setuidgid which leaves the secondary groups intact and gave the desired access to the directory:

If you find the need to outsource some or all of your server environment during your bid to take over the world, remember that there’s a huge difference between a hosting provider that merely gives you access to hardware, be it physical or virtual, and one who truly manages those servers with you. Here’s a few traits to consider that can prove invaluable not only in day to day operations but when it comes time to upgrade, expand, or add to your deployment:

Level of Competence

No single provider can excel in every area for which you might need a server. An asterisk VoIP deployment has very different requirements from a LAMP app for example. On the other hand, most apps one might deploy these days require knowledge of quite a few technologies, and most providers will probably tell you they can do whatever you need, so it’s important that you vet the provider’s competency in the areas that will benefit you the most.

Of course the level of competence in any area can vary from complete jackassery, to being able to solve a problem after exhaustive research, to being real innovators in a space, on the forefront of the subject, even disseminating their knowledge through blogging or other means.

A Good Trouble Ticket System

Phone support is often overrated and inefficient, especially when it concerns servers. A phone call to support is almost always going to spawn a support ticket at some point, requiring follow-ups, attachments, etc. so it just makes sense to initiate it there.

A good trouble ticket system will allow you to email support with your issue, identifying your account with your email address, and will maintain any CCs you included on the original request.

Seamless Handoff Between Techs

When an issue reaches a support tech on the providers end who might not be the best qualified to handle it, the transition from that tech to another is a crucial step in the problem solving process. The customer shouldn’t be responsible for bringing each new tech assigned to the ticket up to speed.

Great support departments handle that transition so well that you have to double check the ticket to see who you’re replying to.

Specialization Rather Than Escalation

Most of us have had trouble tickets rise through tiers of support where the first is really just gathering information or digging through their knowledgebase for a previous answer, the second has some problem solving skills, and the third or higher are some form of ‘engineering’.

It’s fun to screw with first tier techs and tell them whatever they need to hear to escalate the ticket, but more fun when you get the sense that all of your provider’s support staff are ‘engineers’, and if a ticket does get handed off to another tech it’s most likely due only to a shift change or because the second tech really knows the crap out of the subject and the support team thinks that person would be the best to tackle it.

Response Time

Whether the issue is handled via email, ticket system, instant message, or other medium, response time is key.

Most deployment servers are mission critical (for at least someone’s mission) and downtime of more than a few hours can cause rapid hair loss.

Even if an issue isn’t resolved immediately, some indicator that the provider is looking into the issue gives great piece of mind.

Monitoring

It’s no secret that you and your support staff love dealing with angry users, but it’s often beneficial to have trouble brought to your attention before your users notice. Maintaining resource monitoring and alert systems is a job in itself and a good provider will let you know not just when your server or app has been down for a while but when it’s getting close to consuming its allocated resources, is under some sort of malicious attack, or otherwise exhibiting odd behavior.

When done right the level of monitoring is so good that you feel like a jerk when you forget to tell the provider you’re going to work on your own application and will be taking it down.

They Take Action

Being notified that something isn’t working is one thing, but having a provider that takes obvious first steps to solve the problem, like restarting a service, can save all sorts of time, and may circumvent the need for you getting involved at all.

They Call You

No one likes to be woken up in the middle of the night because an app has crashed and can’t be brought back up, but if the provider’s team has done everything they can without interaction from you and you’ve selfishly closed your eyes for a few hours without a mechanism to poke you for each new email received, a phone call may be the only recourse, and it’s certainly better than waking up to find that a critical service that has been down for hours.

Trust

It’s only when you really trust your provider that you can realize the true value of their services.

When you trust them to install and setup a third party application rather than tackling it yourself you save the inevitable hours searching through documentation and forums to resolve some stupid issue unique to your deployment environment.

When you trust their recommendation on which app to use to fulfill some need you can save hours of additional research time.

When you trust that their network, power, and most importantly people infrastructure can handle your app when it becomes the mega success you’ve dreamt it will be, you’re free to dream of the next.

If trust could be quantified and put on a feature list we’d certainly have an easier job of comparing these folks, but as we all know, this one can only be earned over time.

An Unsolicited Shout Out

I can say from experience that Contegix excels in all of these areas. I’m in no way affiliated with them, just a satisfied customer.