Web/Tech

02/11/2010

I am trying to regain some perspective after splitting my days between startups charging along at a break-neck pace and corporate IT chugging along well-worn rails.

The startup IT world is compelling in its creative fecundity – new features and functions pouring forth from the latest open-source tools, operated by young, intelligent, energetic workers that slough off disappointments and set-backs with a shot of Red Bull.

But so much of the startup world’s data today is, frankly, trivial and disposable:Children’s games, reservoirs of transient messages, amusements and eye candy; “nice to have” information of all sorts, but very little of primary importance.In part, this is because very few enterprise software firms remain after a decade of consolidation, and those firms are so large, little enterprise software gets built at startups anymore.Even when a startup’s data is important (such as personal finances), web data is usually redundant to a “real” system of record in an enterprise somewhere.So much web data is therefore, ultimately, disposable.No wonder startup’s view testing as a troubling annoyance, and really love the idea that customers can and will test for them.

The corporate IT world is also compelling -- in its devotion to robust solutions.New feature requests are pored-over as carefully as the welds that hold a fighter plane together.More often than not, the workforce consists of graying baby-boomers devoted to finding a solution will hold fast even in a hundred-year storm, deeply fearful of failure, for the security of their jobs, if not for their customers.

The essence of corporate IT today is integration, the endless gluing of one application with another extant application.And all this plumbing is ceaselessly stressed by business process changes that test the limits of human adaptability.There is usually no practical alternative to endless cross-functional meetings where a dozen people grope for a common language and understanding, and too often end up with a fairly trivial application no matter how important the data, perhaps because that’s all they can find so few shared words to agree upon.

All this could mean nothing more than:

Delivered function points = Feature Requirements / Data significance.

If the data’s not important, then features flow into the code like water over Niagara Falls.If the data is important, then code experiences a much more painful birthing.

04/07/2009

An integrated development environment (IDE) may require fewer programmers, but they must be even smarter and more experienced than ever before.Here’s why:

Development has always involved tool selection:Modeling and requirements tools for the analysts; an integrated design & development environment for the programmers; test automation tools for QA; project management tools for the project managers; etc.The classic analogy was with building contractors, where developers, like carpenters and metalworkers, only required a good foreman and some practice using their equivalent of joiners, lathes or band-saws.

Unfortunately, unlike real joiners or lathes, the software equivalents have become amazingly complex.Why is this complexity necessary?IDE’s are an attempt to solve a long-standing software and system development management problem for corporate applications:If you allow analysts, programmers and testers to use separate tools, how do you ensure they will work together productively using a consistent process to achieve high quality results, especially in corporate IT development shops, which must constantly integrate new & old data, systems and networks?

I don’t doubt that the product managers of these integrated development environments (IDEs) added every feature for very good reasons, with the goal of making development more productive.Some of the products I’ve recently reviewed (such as IBM’s Rational tools) are truly a tour de force achievement, encapsulating a huge percentage of the “software body of knowledge” in their conception, design, and functionality.Just reading the help screens can give you a capsule PHD in the history of software engineering and management.Furthermore, today’s IDE’s even attempt to automate the business process of software development itself, requirements elicitation, a good deal of coding, almost every good software engineering idea we’ve had over the last forty years.

But IDE complexity distracts developers from understanding the business problem underlying any project.Every added feature drains attention from achieving a useful business goal.IDE’s require many weeks of training, many months of practice, and understanding why they work as they do is difficult for anyone with less than a decade’s work experience or a higher degree. This involves complexity well above and beyond learning any programming language.

Part of the reason for IDE complexity is an attempt to create more specialization of development roles.For large projects, this allows for more divisions of labor to allocate work most cost effectively.For example, developers are only required to understand a requirements formalization delivered by selected analysts.Of course, this means that the developers cannot be expected to apply much judgment to shaping their work products, and bad requirements will tend to propagate downstream.Probably even more severe is the impact on QA, who will now just test formal requirements, again without applying any good business judgment, and many defects deemed “obvious” by your customers will be discovered.

The most expensive defects are requirements defects, because their impact is felt throughout the development lifecycle.Pending some new methods for validating requirements, IDE’s will introduce new risks, with no guarantee that lifecycle cost reductions will be achieved.

If you believe, as I do, that an understanding of the business problem is essential to high quality results, corporate IT managers, once they decide to take the IDE-plunge, are left with two options:Use the IDE for the largest projects, and for smaller projects, only use IDEs when a sufficient staff of no-shit IDE experts is available.

The largest projects have greater ability to spread capital & labor & training costs around, especially offshore.Such projects must have a larger staff devoted to requirements analysis and dedicated to understanding the business problem.The project’s size helps amortize IDE training costs, via both formal training on the IDE tool and by providing an opportunity for employees to use the IDE to solve a real-world problems. However, the risk of requirements defects needs to be rigorously managed, via prototypes, constant customer contact, formal methods – whatever tools are at hand.

For projects with less than about 20 full time equivalents, I believe IDE’s should be used only when you have a staff that can operate in both your business and technology spaces, and who have mastered the IDEs already.Understanding a business problem and the use of technology to solve it requires developers to understand two different linguistic and social spaces, one business and the other technology, and such talent is no doubt hard to find or develop.But if you don’t use such a staff, there is too much risk that your business problem won’t be solved, through misunderstanding of requirements, or because the solution won’t be cost-effective, due to tool training costs and low productivity.

Parting thought:It may be that the rapidity of Internet application development, relative to corporate applications, is due to its simpler programming model and its “let a thousand tools bloom” attitude.Internet applications, of course, do tend to be much less integrated than corporate applications, and many of them are green-field.But it may also be true that the newest market for software development benefits from its more old-school, non-integrated development environment approach to software development.

04/02/2009

Any Wino Theory:Half of all technical problems can be resolved by patiently explaining it to any wino.

AWT, as we’ll call it, is very much a product of a certain period of my career, but has proved of enduring usefulness.Here the back story:

I was working in Berkeley, California, and living in San Francisco, two cities that had -- and have still -- an enduring affection for the homeless.Both cities developed a fascination with street culture in the 60’s, where the homeless were there by choice, mostly middle-class kids performing embarrassing acts of self discovery, and many of whom moved on to have excellent careers in technology.By the time we reach the Reagan era 80’s, though, most homeless are drunks & drug-addicts, the mentally ill, misfits and street criminals.However, there are always just enough victims of bad luck and economics among the homeless to justify advocacy implying their failings are purely due to evil landlords and Republican ideology.

It’s one of the oddities of the Left Coast that we combine the highest intellectual pursuits and wealth-producing business endeavors with policies that allow these people to live with no dignity whatsoever.Buddhist-like, we are happy to meet the homeless poor, because they present an opportunity to offer alms, thereby earning merit for our next reincarnation, which allows some of us to bear the stench of excrement with equanimity.Unfortunately, not much has changed in this regard over the last three decades.

Also during the 80’s, tech development practices and management techniques were still immature, to say the least (whether we’ve improved at all by now is a topic for another post).There was just so much damned money flowing to anyone who could credibly code, cable or calculate required amps; labor economics were really on the side of the Workers.All our work was time and materials contracts, too, so you were well-paid for fixing all your bugs & mistakes.Liberal arts degrees, or none at all, were as common as engineering degrees.“Process” was a dirty word that implied the squashing of creativity and higher thought.“Code reviews” smacked of Big Brother.“Change Control” was the stuff of Fascism.“Testing” was only performed by ungrateful customers.“Management” was a suit or skirt who talked only to non-technical customers.

This left me, as a development lead, with a couple dozen direct employees happily cranking out code by the kilo-line, oft times fueled by lines of cocaine, well-paid, but wrapped in black-leathered Berkeley attitude, and working whenever the spirit moved them over a twenty-four hour day.I felt, at times, like I was the only thread keeping the possibility of a useful software product alive, and the work hours devoured me.

Now, I enjoy managing people and being a good listener, but I badly needed a triage technique if I was going to survive & succeed while still having a life of my own.So AWT was born of necessity.When the queue outside my door started to snake, and someone asked for guidance that required more time or mental capacity than I possessed at that moment, I’d instruct them to go ask some random other person for help, whether I thought said person could be helpful or not.In short, I blew them off.

Not being a believer in content-free management techniques, I of course afterwards felt guilty.But when I circled back around later to provide the help I didn’t have time to give, I discovered that many of their problems were now solved.But not usually via anyone’s actual help.

It seems that forcing a clear problem statement can induce the seeker to recognize a solution, and it doesn’t really matter who the interlocutor is.I was simply suggesting psychotherapy’s “talking cure” to my coders, but by discarding the therapist, I was saving much time and money.Ever since, my directive has been “go ask any wino”.There are still plenty of winos around, and they’re just as good as any engineer, especially when the real problem is:You haven’t stated the problem clearly to yourself.

In closing, let me note that if I applied AWT to this blog, I am the troubled seeker looking for advice.Which makes you, my reader, the Wino.

(Intelligent comments entitle you to a bottle of Thunderbird, my treat.)

After years of procrastination, I’ve decided to start self-publishing short essays on topics concerning technology management that interest me.Based on my varied experiences over the last three decades in the business, I know my posts are probably going to be too long for the prevailing short attention spans, but what the heck, I just can’t bring myself to write in sentence fragments.

Part of the impetus for this blog is staying linguistically fit, but from a professional point of view, these essays will help me organize my thoughts and strategies.As those that know me will attest, I often “think by writing” and “lead by email”, so writing, leading, and problem solving are all the same for me.

Hopefully, my opinions will be of interest to any readers who find their way here, and I hope they will initiate some interesting conversation.Such conversations with my colleagues, clients and correspondents always keep me motivated and energized to learn.After all, working with smart people is *the* great reward for professional technology managers.(And by “managers”, I mean:Someone who gets someone else to do the work.)

I am about to enter my fourth decade in tech, and by “tech” I include the entire range of computer hardware, software and communications creations that have transformed the world so profoundly.If you look over my vitae, you’ll see I have worked as a developer and manager in all three disciplines, and I am a generalist to the core.For the most part, my posts will be at a fairly high business level, highly cross-disciplinary, and rarely a geeky deep-dive.

I am also writing from a more personal motivation.Most all the tech writing I read is really quite bad:Sterile, hyperbolic, narrow-cast, a-historical, with anything like an individual point of view vigorously scrubbed.There is little or no fun to be had reading the stuff, and most of it is only intended to be quickly scanned in combination with slideware, all in tune with an industry that has made attention deficit disorder a job requirement.

Bad tech writing is a tax on us all, because consistent reading about our field is a necessity.All-in research to prepare for a discussion in depth can take weeks.Our situation is similar to a programmer forced to read another’s poorly formatted and uncommented code; the sheer effort it takes to understand obscure new ideas and vocabulary via poor writing make you very crabby.

So I’m going to share a few discoveries and disappointments, amuse myself and hopefully any readers, sort out my thoughts on management issues and hopefully initiate some discussion.