Posted
by
samzenpuson Friday August 08, 2014 @06:34AM
from the with-my-eyes-closed dept.

jfruh writes Not everyone has a job like Homer Simpson, who's been replaced at various times by a brick tied to a lever and a chicken named Queenie. But many IT workers have come up against mind-numbing, repetitive tasks that probably could be automated. So: what do you do about it? Well, the answer depends on how much power you have in an organization and how much your bosses respect your opinion.

But that's usually the easiest part of the whole process. They rarely look beyond it, to the maintenance phase.

The maintenance phase, of course, often lasts far longer than the implementation phase. It often outlasts the people who pushed for the automation in the first place, and the people who initially implemented it.

No automation is perfect, and no surrounding environment is static. Things will break, or change will eventually be needed. And this is where automation can often fall flat on its face.

I can't count the number of times I've seen companies with scripts or apps that perform some simple operation, but it only saves a few minutes each day. Yet at some point something with the automation breaks or needs to be changed, but the original developers are long gone, and now some other developer has to investigate.

This poor developer will end up needing hours, days, weeks, or even months in some cases in order to find out where the fuck the script or app is running, where the hell the most recent version of the source code is, how to get it running on a development system, and how it works, all before being able to make the fix or the change. Then it takes time to fix it or make the change, plus some time for testing, and then it needs to be redeployed, and finally it needs to be monitored for some time.

So the automation saved maybe a few dollars a day. Yet just a single fix or change to the automation can end up costing hundreds, thousands, or even tens of thousands of dollars once all is said and done. Merely one small fix or change can literally wipe out any cost savings that the automation has ever brought in the past, and then wipe it out for the next few years!

It's all rainbows and roses to claim that "documentation" or "training" or "best practices" will solve these problems, but even when those are in place and actually working, they rarely reduce the actual cost of maintenance.

So do some real analysis before screaming, "JUST AUTOMATE IT!" The cost of even simple or minor automation can quite often far, far exceed the benefits it'll ever bring. Maybe it's better to have the intern or low-paid employee just do the work manually for a few minutes each day.

Please mod parent up! There are some decent points in this posting, regardless of it coming from an AC.

I've spent a good 15+ years doing process automation. And process automation itself is indeed a process. It's significantly more than some server monkey slapping together a script together during lunch to do something. Don't get me wrong; such an approach is fine if it's going to sit on an admin's machine and be used every once in a while. But once enshrined in day-to-day operations, the game changes.

What are the specifics that the script is going to do?What is the "Happy Path" operation of the script?How will common / uncommon errors / exceptions be handled?How will the script handle unknown or unexpected errors (ie. is it written to be resiliant)?How will the script be monitoried (e.g. snmp stuff) to ensure it hasn't choked?Where are these scripts being maintained and managed (ie, groupings of P.A. scripts in version control)WHERE IS THE DOCUMENTATION ABOUT THE SCRIPT?

Stuff like that.

Do some planning. Be a software developer about it. Just because it's a [Perl | Python | Groovy | Bash | DOS ] script does not mean it should be treated differently than compiled source or a more complex app.

Do you need to turn this into a full-blown project, with 8x the overhead of just writing the damn script and sticking it in place? No.Do you just write the damn script and hope for the best? No, not if you want the system to die a horrible, disfiguring death.

So, put a bit of planning into what's being done. Prove the script out first. Test it (remember that?). Manage it.

Scripts are generally more accessible, but that's about it when it comes to comparison with other languages. That means you get some pretty bad scripts, but... well, there's no real reason to not use decent coding practice. The core one being 'get everyone involved a working knowledge of the language'.

Not just user documentation, but also system documentation. Good commenting of the procedure can also help.

Without the documentation you can't pass on the procedure (or support).

Now, even without documentation, it becomes just your baby... Maybe it helps you do your job, but you better have SOME documentation so you will know what it does, and how to change it when you HAVE to change it in the future.

The maintenance phase, of course, often lasts far longer than the implementation phase. It often outlasts the people who pushed for the automation in the first place, and the people who initially implemented it.... No automation is perfect, and no surrounding environment is static. Things will break, or change will eventually be needed. And this is where automation can often fall flat on its face.... I can't count the number of times I've seen companies with scripts or apps that perform some simple operation, but it only saves a few minutes each day. Yet at some point something with the automation breaks or needs to be changed, but the original developers are long gone, and now some other developer has to investigate.... This poor developer will end up needing hours, days, weeks, or even months in some cases in order to find out where the fuck the script or app is running, where the hell the most recent version of the source code is, how to get it running on a development system, and how it works, all before being able to make the fix or the change. Then it takes time to fix it or make the change, plus some time for testing, and then it needs to be redeployed, and finally it needs to be monitored for some time.

Which is a social and cultural problem, not a problem of automation itself. If people were using open environments similar to Smalltalk or Oberon, half of those problems would go away. As ESR put it in TAoUP, there are simply systems hostile to casual programming. Now he of course draws the comparison between IBM's MVS and Microsoft Windows on one side, and Unix on the other side, but I'd actually stack MVS and Windows AND Unix against Smalltalk and Oberon. (Mind you, those two systems are far from being pe

I wasn't talking about languages, more about environments (both that I mentioned are not merely a syntactic and semantic definitions of a language, they're complete software artifacts, with OS kernels, runtimes, compilers, tools, and conventions). And actually, this is the part I actually had in mind: the documentation tools suck. These days, I'd expect some sort of an inter-linked system, allowing you to not just jump between pieces of code in your editor, but, i.e., to find all relevant document referring

Why are you stacking operating systems against languages? Both Smalltalk and Oberon are available for MS Windows and Unix, if not MVS.

:
Also, what are the advantages of Smalltalk and Oberon over, say, Perl or Python, to counter the fact that it's easier to find Perl and Python people when something breaks? Carrying their own OSes along so they're harder to integrate into another environment? All you need in this case is good documentation and readable programs, and it's just as easy to do those in Perl

:DDD Yeah, right, because Seaside never made any impression on anyone, despite so many people trying to copy its basic design, and it didn't even work because there's obviously never been any libraries to make it work. You're an idiot.

Let me know when "non-dead languages" finally get the capabilites of the "dead" Smalltalk debugger. I'll go make myself a few million cups of coffee in the meantime.

It's all rainbows and roses to claim that "documentation" or "training" or "best practices" will solve these problems, but even when those are in place and actually working, they rarely reduce the actual cost of maintenance.

Oh, enough weasel words. You start off with a strawman of "it saves a few minutes a day" when in fact nobody automates systems that actually only take a few minutes a day - and it's probably your "few minutes a day" that's off, if they do (you're ignoring the costs of errors - the few minutes a day can often cost half a day if it's done wrong). Then you quote "documentation" as if it's some kind of mythical being and again maybe in your experience it is, but get real. Documentation solves all the problems you cite. Documenting your automated systems does reduce the actual cost of maintaining them. And I suspect if you'd ever seen it in place and actually working, you wouldn't be giving us this luddite anti-automation rant on the basis of a woeful misunderstanding of what constitutes a business case.

I feel your pain, but the area I work with in the company has been using trello to keep track of this kind of stuff, we have one card for each process/application/script running on our servers. These cards describe what each script does, where it runs, how often it runs and so on. It has been working great.

I can't count the number of times I've seen companies with scripts or apps that perform some simple operation, but it only saves a few minutes each day. Yet at some point something with the automation breaks or needs to be changed, but the original developers are long gone, and now some other developer has to investigate.

This poor developer will end up needing hours, days, weeks, or even months in some cases in order to find out where the fuck the script or app is running, where the hell the most recent version of the source code is, how to get it running on a development system, and how it works, all before being able to make the fix or the change. Then it takes time to fix it or make the change, plus some time for testing, and then it needs to be redeployed, and finally it needs to be monitored for some time.

So the automation saved maybe a few dollars a day. Yet just a single fix or change to the automation can end up costing hundreds, thousands, or even tens of thousands of dollars once all is said and done. Merely one small fix or change can literally wipe out any cost savings that the automation has ever brought in the past, and then wipe it out for the next few years!

If it's only saving a few minutes per day then when/if it breaks then you scrap it. Why would you spend "days, weeks, or even months" fixing a scriptthat is only saving you a few minutes per day?

This xkcd comic tells you when to automate: http://xkcd.com/1205/ [xkcd.com] but it applies equally to how long you can afford to spend fixing a script that breaks.This is across 5 years so IMHO you should probably cut the number in half so that 1) you are actually coming out ahead and 2) you have some roomfor updating/rep

You need a job scheduler to centralize the tasks and knowledge of the processes, so you can bring someone in to run all the automated tasks. Sure you still have a human in the equasion, but now you're down to 1 person, instead of a room of 20.

Absolutely, but DO NOT TELL ANYONE. honestly automation will not get you a raise or a promotion, it will just get you extra work. for the same pay.Automate all of it and keep your frigging mouth shut.Hell I used to automate emails to be sent at 2am so that management though I was working 24/7.

If you pull out the all the stops occasionally, you're a hero. If you do it routinely, you're taken for granted. It's hard enough to measure 'productivity' in IT anyway. Far better to automate your job, and 'pay yourself' to support it on an ongoing basis.

Absolutely, but DO NOT TELL ANYONE. honestly automation will not get you a raise or a promotion, it will just get you extra work. for the same pay.Automate all of it and keep your frigging mouth shut.Hell I used to automate emails to be sent at 2am so that management though I was working 24/7.

If you've automated your job, *shouldn't* you get new tasks to do? You're being paid to do the job to the best of your ability. You've done that by automating - but that leaves you on-the-clock time to do other productive tasks.

No. That would be wrong. A worker should maximize efficiency by discovering the best way to achieve maximum pay with minimal work. That is what economists say the company should be doing and since companies are now people and workers are people that's what workers should do. In fact doing it any other way flies in the face of the "Free Market" and therefore maximizing efficiency is both an ethical and moral imperative.

Well, it would be okay for you to get additional tasks to do as long as the pay increased in proportion (which of course, would not actually happen). So the GP was not necessarily wrong, just unrealistic.

No. That would be wrong. A worker should maximize efficiency by discovering the best way to achieve maximum pay with minimal work. That is what economists say the company should be doing and since companies are now people and workers are people that's what workers should do. In fact doing it any other way flies in the face of the "Free Market" and therefore maximizing efficiency is both an ethical and moral imperative.

This "free market" theory of yours works for a while until someone else discovers that they can automate the taskyou've already automated and you're out of a job. That's how the "free market" really works. If you were the onlyfarmer with a combine then you would make a ton of money but eventually everyone else gets one too and yourcompetitive advantage goes away. If you really do come up with a "time-saving" device, the best way to get richwith it is to either hide it so noone knows about it or license i

I wouldn't consider a combine, a washing machine, or turbotax a "business method patent".Those all automate tedious tasks. If you were the only one who had a washing machineor tax software then you could charge the same as everyone else yet do work multipletimes faster. Historically this has been tried many times. Some times the person takesthe secret to their grave and noone ever discovers it but most of the time the competitiveadvantage doesn't last too long as someone else figures out a way to copy it

As a dirty libertarian capitalist, I agree with this. Management's job is to set objectives and make sure they're accomplished, within budget. If one of their employees can meet these objectives in 20 hours a week by being a genius with the computing speed of Data and the communication skills of Ronald Reagan, so be it. If they can meet them by being cunning, and working a few extra hours to automate tasks so that they can meet the objectives in 20 hours a week, so be it. Only a moronic manager (but I r

And, if the worker wants to, he should ask for a raise and more work. This would benefit him and the business financially; whether it would happen depends on the wisdom of his managers. Less efficient firms would be forced to innovate in kind or suffer competitive disadvantages.

If you've automated your job, *shouldn't* you get new tasks to do? You're being paid to do the job to the best of your ability. You've done that by automating - but that leaves you on-the-clock time to do other productive tasks.

If they want to pay me hourly, then yes, absolutely. As long as employers do their damnedest to push the limits of "exempt", however, then very much no. I get paid to perform certain tasks to the best of my ability. As long as my employer doesn't care whether that takes me 40 or 60 hours a week, then I don't care if it only takes me 20.

Note that I mean this somewhat in the abstract, in the sense that I refuse to work for someone who expects me to work more than 40 a week regulary. My current employer actually treats me pretty well, and as a result, yes, if I automate task X, I'll spend my newly-found time doing the rest of my work somewhat better (I wouldn't specifically say "picking up new tasks", because we all know that what we can do in 40 hours doesn't mean we should to produce the optimal result).

Yes, you have a moral duty to your corporate overlord to serve it to the best of your ability. You must strive to do more than the job that you are paid to do because that is your moral duty. In return, you can fully expect your corporate overlord to honor its moral duty to you (because it has no duty to you).

Absolutely, but DO NOT TELL ANYONE. honestly automation will not get you a raise or a promotion, it will just get you extra work. for the same pay.Automate all of it and keep your frigging mouth shut.Hell I used to automate emails to be sent at 2am so that management though I was working 24/7.

If you've automated your job, *shouldn't* you get new tasks to do? You're being paid to do the job to the best of your ability. You've done that by automating - but that leaves you on-the-clock time to do other productive tasks.

Perhaps, but with the understanding that part of your time will now be devoted to maintaining the automation you created.

I guess it depends on the company. I work for a small company with just 2 developers. We have A LOT of processes automated. If we didn't we would only be doing 1/4 the amount of work we currently do. Yes you know what sometimes automation breaks but we fix it. Also any automation that is done that only saves 10 minutes per day the person that was originally in charge of that work still is required to know how to do it manually so that if it eventually breaks it won't effect them other than opening a tick

Absolutely, but DO NOT TELL ANYONE. honestly automation will not get you a raise or a promotion, it will just get you extra work. for the same pay.

And if you worked for me and I found you doing this I would fire you on the spot. You are being paid to perform a certain number of hours of work, not to sit on your ass and collect a paycheck. What you are suggesting is fraud, plain and simple.

Actually, nobody's watching my hours, although I put in about 40 a week (it varies). I get evaluated based on the work I do. As long as I get enough stuff done with good quality, everybody's happy and I get a good annual review and pay increase to match.

No. GP post suggested automating tasks and seeking no additional work while collecting the same paycheck for less work. If you can automate a task which saves time then you should go seek out a new task to fill the time. You are not paid to sit on your ass and admire your handiwork. There is a difference between making yourself more efficient at your job so that more gets done and pretending to do work that you have automated to collect a paycheck. The former is worthy of promotion, the later is fraud

What a lovely Protestant Work Ethic you have there! I hope your jobless great-grandchildren are just as proud of it.

What a lovely Entitlement Complex you have there! If you think your ability to obtain a paycheck is dependent on you being as unproductive as possible without getting fired then you will be unemployed long before I will.

If you can automate a task which saves time then you should go seek out a new task to fill the time. You are not paid to sit on your ass and admire your handiwork. There is a difference between making yourself more efficient at your job so that more gets done and pretending to do work that you have automated to collect a paycheck. The former is worthy of promotion, the later is fraud.

So you are actively discouraging your employees from ever doing anything to improve quality of life or even improve accuracy. Interesting.

Learn how best to automate that task so you can start on other projects to automating other tasks.

Yup. But do it in secret and don't share the automation with the employer. Use your spare time to look for a new job.

IF you come to the point where know your job is going to evaporate, it's better not to make a lot of waves (and that includes positive ones) until you are ready to go anyway. Your employer is already NOT paying attention and may not have a full understanding of what you do already. You'll be facing "the Bob's" in no time.

There is a reason they call it "work." Boring and repetitive comes with that. Brush up on your Zen skills and deal with it. And FIND ANOTHER JOB.

There is a fine line between a dead mans switch and an unnecessarily complicated script.

But always remember, unfireable is unpromotable.

However, on the gripping hand, being unpromotable is not an issue if your employer doesn't promote anyhow. Being unpromotable is not the same thing as being tied to you job. Go ahead and make yourself unpromotable, but then find a new place and really screw over the assholes.

Well, automate them, off course. That is how I started my programming career. I started as a technical draftsman using AutoCAD, and I was "actively lazy": when I had to type something 10 times, I wrote a little program to do that for me. When my bosses noticed that my computer was better configured than that of my colleagues, I started getting programming assignments as well.

Lots of people 'play spreadsheets' all day, which is working really inefficiently. Some companies prefer this, because they operate on billable hours in the first place. You being efficient costs them money. Some companies know better, and will value your skills. You should go and work for those. Getting laid off sucks a bit, but it's far from the worst thing that'll ever happen to you.

Considering most of our job is writing, I'm amazed at how poorly most people understand Microsoft word. I am not an expert, but if you don't even know what style sheets are or how to insert a cross reference, you are creating SO much more work for yourself as you try to "clean-up" the document later.

But as attorneys who are getting paid by the hour, there can be short term benefit (more billable hours) to this inefficiency.

I've espoused the doctrine of proactive laziness since I started sysadminnning. I figure I'm doing my very best work when there's nothing that I need to be doing, and I can be spending my time fiddling with the next thing.
That means applying appropriate automation and scripting. (Don't overdo it - not all scripts need to be gold plated).
Decent documentation. (Which is easier: explaining or fixing a problem, or saying 'RTFM' and waving a hand dismissively - if TFM is up to scratch, they won't come back a

If your design documentation couldn't be directly executed by the computer and tested, then we cannot say that it was even remotely complete or correct.

That's not true - it's perfectly possible to give a specification that's complete and comprehensive, and yet is not executable by a computer.

For instance, you could specify a "sort" function by saying that (1) it must return a list that contains a rearrangement of the items passed in, (2) that list must be in ascending order, (3) the time taken must be less than K*n*log(n), where n is the number of items passed in.

I've given that specification in English for brevity, but you could equally specify it in a mo

Automating simple/repetitive tasks represents most of my self assigned projects (in between larger/outside funded development projects which are assigned) the most important thing is to always be working on something... or at the very least always appear to be working on something.

I've ended up creating a few solutions where I think I'd rather spend three hours doing something creative than one hour doing it mindnumbingly dumb and repetitive. Often the maintenance of tweaking it eats up the savings.

But even doing the maintenance and eating up the savings is fine if it doesn't come out taking longer overall. You spent the same amount of time working, but didn't have to do anything dull or repetitive.

The "Is it worth the time?" comic is illustrative, but I think it's built on false premise. I don't automate parts of my job (just) to save time, I do it because repetitive tasks, that I know can and should be automated, drive me insane. Even if there is no net time savings, I'd rather take the creative route.

Most of the IT jobs (emphasis on the "jobs" part) that I see, cannot be automated - or if they can be automated, the automation needs a level of oversight and constant tweaking that it is not economically viable to automate the process.Almost without exception, an IT "job" can be split into discrete "tasks", where some of the tasks can be and should be automated for various reasons, but in terms of the W.W.W.W.H. (What, Where, When, Why, How) aspects, the reason for automating (Why) has a significant bearing on whether it would be a good idea to even try automating.Automating the tasks which can be automated within a job makes sense in many cases - relieving the employee of the trivial and repetitive tasks to tackle the more high-value elements of the job. From a commercial perspective, if you are spending most of your time on the high value tasks, you are probably earning more money for your company or providing better value. As long as the boss recognizes that fact, your job should be more secure and your pay packet should, at some point, see an increase to recognize the higher value that you represent. ok, you might need to leave the company and parlay that higher value experience at a new employer to see the increase in your salary, but if your CV can show a successful sequence of task automation leading to higher productivity, then you will probably be more in demand.

If you have either a role that can be automated to the point where you are irrelevant, or a manager who thinks that your role can be automated to the point where you are irrelevant, then my advice would be to start looking for a new job where either you are more stretched or your manager appreciates your contributions more.

You are right a lot of jobs just plain cannot be automated away but I bet if you ask any seasoned sys admin they will show you a boat load of scripts that they run and watch while drinking their {insert caffeinated beverage} and reading/.

I have gigs of scripts I've collected over the years that do all kinds of creating/disabling/moving/modifying AD accounts and custom reporting for ldap and various applications, even telephony related scripts. {gedi master?}

Automate it and find something else to work on. At no place I've ever been has there been a shortage of work.

Only the lazy and incompetent fear automating themselves out of a job. If worst comes to worst, you'll end up maintaining all those scripts you created, fighting fires, and dealing with the "one off" situations that the scripts can't handle.

I'd tend to agree with you, although the problem here is often not the employee and his/her laziness or incompetence. The fear often comes from uncertainty that management recognizes the value in what you've done for them.

For example, I had a buddy who used to work for a company that installed and maintained point of sale systems. He found all sorts of time consuming processes that he could automate, by virtue of knowing how to code in languages like PHP or Perl, and did so wherever possible.

One of the tasks here is the daily generation and collation of statistics across our various platforms to present to the clients. It contains things like the state of the tickets, number of tickets raised/closed, peak data across the platforms, systems metrics for utilisation and other similar things.

Currently these metrics are gathered manually be people eyeing up graphs and manually reading the data from them or running scripts that pull out metrics they need, it went from a half hour task to a 120 minute task, daily, as we've been growing.

I created a scaleable system that polled all the required statistics, from all of the various different systems in place, directly accessing the RRD files that generate graphs, polling our ticketing systems API to access the data, etc etc and then pushed them in to a database which could be polled easily for historic data as well as interrogated via a front end to generate the reports. It even allowed you to alter text in the report and export it to PDF to email to the clients.

After 2 years it's still sat, waiting to be rolled out, because of politics. The guys that run the reports just work from a process document and aren't really good enough to do much else. With the additional free time they have it would be too transparent that they can't do anything other than follow process but we do need them for various other tasks that can't be automated. At least by doing the reports manually they look really useful and the clients are happy..

You are doing bug tracking by hand??? We used to that back in the 1990's. I was given a three day task to sort, reorder, prioirtize about 200 open tickets in a single text file. Wrote a script in six hours to do this automatically, and had the report completed in minutes. Today, we would just use bug-tracking software like Jira

If you have the desire and/or ability to use your computer properly and automate tasks, and your job title is "______ Assistant", your boss will likely not respect you enough to permit automating anything. Therefore, you should do it as quietly as possible, and do not expect any pats on the back for mysteriously having perfect reports in your boss's inbox every morning at 8:31AM, or data requests completed before he/she even has time to walk back to his/her desk. Your boss may "expect" perfection, but will not actually know what to do about a subordinate who is actually capable of delivering it.

Expect to have only Microsoft VBA at your disposal. And amuse yourself daily with this image [imgur.com] which sums up your situation perfectly!

After reading a few responses I thought I'd share my observations on workers, and the "Three Kinds of Lazy:"

The Plugger: Doesn't like the drudgery, but rationalizes it with victimization and anxiety.
The Troll: Doesn't like the drudgery, does a barely passable job. Jellously guards all knowledge of how to do said task under the idiotic assumption that this will make them indispensable and thus impossible to eliminate.
The Neckbeard: Hates drudgery. Will spend enormous amounts of time learning whatever s

I've been in IT for over 30 years and I've seen a lot of changes. My first program was coded onto punch cards and read into the system that way. Nowadays I'm doing some traditional programming and SQL, but also working with some new tools like SAP's Data Services and Dell's Boomi. These newer platforms are very visual in how you hook up your components, yet still offer the flexibility to write special modules in languages like Java or ABAP. This, I think, is the future of programming, where a lot of the

Automate what you can, but even with automation in place, someone who understands what's being automated and how automation works still needs to be around to monitor and improve the system. Automation should only ever replace tasks with are safe and repetitive, to free up time for people, such as IT staff, or engineers, to focus on much more important tasks.

The way I would spin it to a boss ( and have before ), is this way: "I made a spread sheet and pie chart outlining where my time is spent each wee

There will always be something in my experience. I do it all the time, though not part of my primary job description. It is usually because of some wonky legacy system still in production that has been terribly maintained because the organization doesn't have the money to fix properly, and certainly doesn't have the money to replace it. Usually patches are applied, which despite UAT ends up breaking some other unforeseen part of the application because it was apparently originally designed by blind monkeys

In one instance it wasn't my task, but the task of 2 employees that I was supervising. My boss had given them a task that took each of them 8 hours to complete (turning phone logs into readable reports, daily). One would start the process and hand her work off to the other which was then turned into management at the end of the second shift. Each day, processing the logs from the previous day. At best, the report data was 24 hours old - often indicating trends after it was too