Summary
Glenn Vanderburg believes that software engineering and craftsmanship are not mutually exclusive, and there is synergy between them, explaining how to combine them in the software development process.

Sponsored Content

Bio

Glenn Vanderburg is a consultant at Relevance focused on cutting-edge software development technologies and techniques, including Ruby, JavaScript, Ajax, and state-of-the-art development practices.

QCon is a conference that is organized by the community, for the community.The result is a high quality conference experience where a tremendous amount of attention and investment has gone into having the best content on the most important topics presented by the leaders in our community.QCon is designed with the technical depth and enterprise focus of interest to technical team leads, architects, and project managers.

I admit I have to respect anyone who takes the time to actually open up the NATO report.

Yet the conclusions drawn here are entirely off the ground sometimes.

It's a good idea to check what do other engineering disciplines do. Perhaps this should stop people claiming publicly that engineering documentation is mostly for workers. It'd be enough to just spend 8 work-hours at any architecture or civil engineering company to realize it's not the case.

(Also, saying that in traditional engineering, actual building is the highest price is entirely off the ground, especially as in some fields of architecture, the design and construction firms are separate, with one firm having 0 cost of construction as they don't move a single brick.)

So, in some ways, this presentation, I feel, is about supporting the presenter's view on that Agile is the holy way. It isn't, and it's not a preliminary form of something. This doesn't mean it doesn't have good parts of course.

Let's go back to that NATO discussion a bit. It's in 1968, and what's the driving force? Projects are getting late, they are unpredictable, and they don't bear the results their customers wanted.

And here's what we have missing in the entire one hour presentation: customer, end-user, and predictability. Nothing. Absolutely nothing is about them.

In order to have predictable results, they wanted people who have disciplines of how to build software. They wanted to have the ability to say, that 3 months from now, you'll have a working system which you just described and your problems will be solved.

I don't want to claim percentages, but a large part of engineering documentation is about making customers, 3rd party engineers (like, house registry officials) reassured that the system complies with their wishes. Some of this documentation is written in a format which is to provide a digestible model to uninformed people.

Anyone who thinks UML is about anything else should look at what Rational made a living from first. They were consultants. They made sure that the customer gets what she wants.

Also, saying that construction has 0 cost in software engineering - albeit I wouldn't call the 40-minute compile time of larger java projects w/o unit tests zero - misses a pretty important point: that if you put a not well-thought model into the production system, you can mess it up for the system's entire lifetime.

There's nothing wrong with prototyping; there's even nothing wrong with prototyping as close to the real system as it can be. It's just that making the system fully testable is costly, and that not having a clear distinction of when it was just "tried out" and when it is executed can and will bear disasterous results.

The IEEE definition of software engineering is as follows:

"The application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software"

Craftsmen don't have such: however, craftsmen don't have any formal education, and craftsmen sometimes fail. If you fail to do a bag, or a necklace, you messed up a single person's life a bit, who lost a few bucks. But if you mess up a system, as an engineer, you messed up the life of a community, and a multi-thousand dollar product: as an engineer you're always responsible for the community of your end-users.

That point is entirely missing somehow, and if nothing is about the customer and the end-user, if it's not about reassuring them at every single point that we are going to solve their problems, and that we'll solve the right problem in the right way, then, I believe, excuse me, but this presentation doesn't solve the right problem in the right way, as this is why that NATO conference was held, and this is why I claim I'm an engineer.

I'm an Engineer too (B. Eng, MIEE) and I haven't seen the presentation but I felt compelled to respond to your comment. I have also read the 1968 NATO Proceedings and the thing that struck mis is that Dykstra et al didn't know how to solve the crisis. They settled on Engineering as a metaphor, but they also spent half the report remonstrating over why the metaphor didn't quite fit the problem.

Software is software a new discipline with new challenges. The IEEE definition of Engineering suites problems where the reconfiguration costs are high. So if you build a multi million dollar bridge, you better get the design right before construction. Software just isn't like that. The reconfiguration costs are relatively low. You can change code/design at any time if you do it right.

We have learn't some since 1968 which has led people to alternative metaphors, craftsmanship is just one alternative. Back to the goals of an Engineer. I agree with you about making the customer central, but plently professions share this goal. For examples the goals of a Theatre company (one of my preferred metaphors for software) are just the same, produce a product that will please the customer.

The Engineered buildings with blue-prints and safety certificates that fall down killing their occupants during earth quakes, do they please their customers? The proof of the pudding is in the eating not the paper work.

For software we have learn't that the best way to give visibility to customers is not to show them plans or artefacts, but to show them working software, early and often. This is where the Agile movement starts.

It'd be interesting to dwell into Dyjsktra's biography, but it would dwell us too far as certainly Dijsktra is way too pedantic to our age. For example, he claimed that doing a bugfix release is just plain nonsense, you should've learnt how to do it right at the first time (Discipline in Thought).

I stick to the engineer metaphor as there's something, which is

a work of art (a catedhral is also a work of art in my view)

and based on hard science (a fast-fourier transformation is pretty much science)

is an infrastructure system (waterpipes, railroads, etc?)

done by changing the environment (parafemized definition of architecture)

to serve from hundreds to thousands of people who happen to be there (living structure)

for a limited, but relatively long period of time (buildings don't last forever, softwares last years)

for a crazy amount of money (10 programmers for a year is more than a million bucks)

I'm pretty sure you wouldn't trust a group of craftsmen and bring them one and half million dollars, just to make your 3000 employees suffer as they can't deliver reliably something which is an integral part of their everyday experience 40 hours a week - which otherwise happens quite often today.

They do it because this group claims they're engineers, that they can deliver reliably.

The reconfigurability prices are tremendously high: just try to move a 10 years old system from one SQL engine to the other and you'll realize that it'll be near to the price of a space shuttle launch, and much more expensive than building a house, or buying one, especially in the US.

I agree that sometimes showing working software early is the best thing, yet this can come with that price - I've seen it too often - that it gets binded to an SQL engine with pretty dirty code. Don't make me wrong: I'm not for Big Upfront Design, yet our current agile practices are really short-sighted.

And although the theory of Adaptive Programming is a good one, what I see at Fortune 500 companies is more about wasting millions of dollars per month on the simple fact that some developers were ... ehm... too agile.

Delivery is key. IMHO it's not what you call yourself that counts, but what you can do. I am an Engineer and I am a computer programmer. Engineering has probably moved on since I was designing military radios (circa 1990), but sometimes as a programmer I feel like I'm Engineering and at other times I don't.

These are just metaphors. Programming has unique characteristics that the world hasn't encountered before. None of these metaphors IMHO is a precise fit. Which is why I'm happy calling myself a Programmer :) Metaphors are useful though, because they come with inferences that can be used to change mindsets. The Craftsmanship metaphor is a backlash and comes with the inference of learning a craft (or trade) through practice. A useful message, but not the whole story.

We get into trouble when we take any single metaphor too far. The map is not the territory.

Where we agree is that incremental delivery done poorly can be an even bigger disaster then batch development done poorly. One caveat though. The customer gets visibility that you don't know what you are doing a lot earlier. So if they choose to keep payng after that, then I suppose they deserve what they get :)

It is definitely worth reading the 1968 Nato Report. The level of uncertainty over what to do to solve the "software crisis" is staggering. They didn't have a clue. They didn't think that Programming was Engineering, but they had to come up with something :). They were having the same debates about what programming is and is not that we are still having today. How about we stop trying to put Programming in some other professions box and accept that Programming calls for a unique set of skills never needed before and that the vast majority of practitioners lack many of these skills (especially the people and listening skills) and hence aren't very good at it :)

Calling yourself a Craftsman an Engineer or a Wizard doesn't tell me whether you are any good at programming. May be if we found ways to make the quality of our work more visible, we'd worry less over what we choose to call ourselves.

I'm pretty fine with your agreement. It's just that I finished a SEEK-compliant course, and read the SEEK material ( sites.computer.org/ccse/SE2004Volume.pdf ) and in retrospective, after years of practicing, it makes sense to me. Note, that there's still some uncertainity in that writing, more than 30 years after the conference.

I pretty much agree that Software Engineering is not a polished study - yet. But every attempt to not define our profession as an engineering profession just weakens it - that's why I'm against it.

I've worked with clients in urban architecture; I have a lot of books in architecture, and somehow I could always explain myself in their terms: the similarities are sometimes staggering, yet they have nothing to do with what Glenn presented up there.

My university teachers also weren't software engineers: most of them came from the field of electrical engineering, but some of them were mechanical engineers, as in the old days, even electrical engineering didn't exist as a separate field.

The reasons I'm for software engineering are above: it's a pretty expensive field, with softwares costing a fortune, and affecting thousands of people's life; I wouldn't give this task to anyone not formally trained to bear this. A bug which a single developer hunts for a week is comparable to a price of a car.

I also understand that people got disillusioned of the waterfall model, and the more formal and pedantic interpretation of RUP: but this is not what software engineering is about! Honestly, I believe that people simply misinterpreted RUP/Waterfall in the 90s, and misunderstood the role of UML, that's why the fuss.

For example, software engineering has nothing to do with batch delivery. Why would it? RUP never ever said they should do batch delivery.. As for waterfall, it's a model of much smaller tasks than what we do today. Back then, an algorhythm with a few thousand lines was the full system; nowadays I write such manually in a single week.

(Please note, that I don't have to use punch cards, and compilation doesn't take a full day, which makes development a lot faster I guess...)

As for documentation: most of the documentation engineers write are for customers. Sometimes a customer has to get funding, either financial or political; sometimes you have to be able to draw a baseline, "this is all we know now",and you can't do it in code quick enough. Sometimes you have to prove to 3rd parties, without a full implementation. Sometimes a prototype would be too expensive for the task, but anything non-formal would be inferior. Sometimes you don't want to have a luxury of having a document hanging in the air, being completely independent and telling nothing of the actual product.

What frightens me, is the complete lack of understanding in design, and lack of motivation to zoom just a bit further on what people are doing right now: I know it's officially not Agile's fault, yet I blame on them as Agile tries to defy SE values, and you're either part of something or being a complete counterpart.

And with that comes the complete disrespect for education, for formal methods; with that comes the continous reinvention of the wheel, even for those advances which were made by the founders of Agile (like, Kent Beck or Ward Cunnigham) but somehow didn't make it into the Agile canon.

I'm an engineer. With this I mean not just that I feel that having formal knowledge on what you're doing is important, it seems I also have to claim that current agile practices - not theories! - are wrong; that customer satisfaction is more important than feeling yourself good at workplace; that a bit of future proofness is more important than doing requests as quick as possible; that understanding each situation separately is much more important than following the scrum/agile canon.

So yes, despite being one of the first people to do scrum in 2006 in my country, it seems that I have to be the person who says no to it; not because it's wrong; but because its average implementation makes harm to the society, delivering softwares which are way too buggy, and makes it hard for the user.

Software is software a new discipline with new challenges. The IEEE definition of Engineering suites problems where the reconfiguration costs are high. So if you build a multi million dollar bridge, you better get the design right before construction. Software just isn't like that.

Depends on the system. The cost of getting it wrong are very different between building software that goes into a medical device and software behind a shopping cart. Even in commercial systems, I've worked in such systems where glitches literally costs ten of thousands of dollars per hour at least. It might not compare to multi-million dollar costs on a bridge built bad, but I'd be hard press to see a company bleeding that much money like that and not see this as critical.

The reconfiguration costs are relatively low.

Relative to what? Also, relative does not imply trivial or nominal. Costs are costs, and even if they are relative low compared to other types of engineering work, that does not make them trivial or non-critical. Ergo, companies that rely on such software systems, and those who are in charge of building them, must do better. Engineering maturity is the only way to go.

You can change code/design at any time if you do it right.

As anyone who has build non-trivial software for a living will ask you, how do you get it right?. What approaches do you use (or should/must use) to get it right? If it was trivial, or at least self-evident, we wouldn't have these type of conversations and problems, would we?

Just because software is malleable, that does not remove the challenge to actually build it right. We have to learn how to do engineering with software. At least you guys in the physical engineering disciplines have tools and physical models with which to analyze and verify ideas that guide your building of things. We don't. Ergo, a greater need to identify how to do engineering.

Bingo. The greatest bane of our industry is that people swing to extremes. We need to do engineering, which entails a firm rooting in scientific principles as well as pragmatic, on-the-ground concessions and trade-offs.

Instead we have people that, without discipline or deep thought, jump from one quasi-religious silver-bullet fad of the day, without any thought, in a all-or-nothing manner.

As a result, on the one hand, we have people calling themselves engineers w/o understanding what exactly that means. And on the other hand, we have self-proclaimed agilistas calling themselves craftmen when in reality they are just undisciplined hacks with a very myopic view of the software building landscape.

Software is software a new discipline with new challenges. The IEEE definition of Engineering suites problems where the reconfiguration costs are high. So if you build a multi million dollar bridge, you better get the design right before construction. Software just isn't like that. The reconfiguration costs are relatively low. You can change code/design at any time if you do it right.

Programming is a design activity so compare it to the design phase of any engineering project not the actual construction of the project and you'll see a lot more similarities (a software project is build by a compiler, so the construction is free). Iterative design wasn't invented by software development. It is used in most engineering disciplines.

Actually, the main difference between traditional engineering and software engineering is in a tradional project you have to deal with the physical world, so in the end you have to respect mother nature laws. A software on the other hand is virtual so you don't have to deal with physical laws, but you need to focus on things like extensibility. The amount of features is also much greater on a software than on any physical machine so testing is big concern. The main problem as I see it is that we have no way yet to measure or just quantify extensibility and things like that and that testing techniques are still in their infancy.

I agree, construction in software is free. This is my point. Iterative development when construction isn't free is VERY expensive. I was an Electronic Engineer and at most we built one or two prototypes. With software, I have a new version of the product every time I get a green bar.

It's not just the construction costs, it is any reconfiguration costs. Someone mentioned reconfiguring the SQL Engine. Well if the code is well maintained and clean, and there is a well defined I/O layer then swapping out one database implementation for another is relatively easy and quick. Try shifting from a MC68000 CPU to an Intel 20286, it means wripping up the mother board and starting again.

Design changes are expensive in most Engineering disciplines, which is why a lot of effort is spent planning and trying to get the design right upfront (BDUF). Engineers live and die by their plans. With software the design can emerge over time. What we call emergent design. This allows A LOT more experimentation and discovery as you go, and much less reliance on getting the plan right first.

I know of no Engineering discipline that has a similar reconfiguration cost. Software is different. Not to say that there aren't lessons that can be learned from Engineering, but we are lying if we say the two are the same. They knew as much in 1968, and the intervening 40 years have proven them right.

The software crisis is with us as much today as it was in 1968. All the talk about Engineering, and the efforts of the SEI didn't solve the crisis. In some ways the Engineering nomenclature set us back, only for us to reinvent some of what we lost with the Agile and Craftsmanship movements of today.

Software development is difficult. Calling it Engineering doesn't make it any easier. Calling it a Craft doesn't making it easier neither. It's neither and both plus much more at the same time.

It's very strange to see such an opinion. Craftsmen are nothing like undisciplined hacks. If you try following their practices for a couple of months you'll be astonished in exactly how much discipline is needed to perform their routine - TDD/BDD, code catas, clean code.

Talking about software (let's leave aside software that needs formal approaches, such as medical, avionics, etc.) as to engineering activity leads to more problems then benefits - especially in the area of customer relations. If what we do is engineering then we can use some formula/model and just calculate to the customer the two main things they need to know - what's the release date and what's the budget? And that's the root of all evils. you can't imagine how much time and energy software firms are investing in explaining why something would take exactly that amount of time, or why there are deviations from the original estimate.

It is not even close to engineering. My observations are that university graduates majoring Computer Science or Software Engineering are just not ready to do production-quality work. That's what "formal" training can achieve for the current moment. Then these guys start working - most likely their first job is going to be an apprenticeship, where the more senior developers are tuning their theoretical knowledge into the practical stuff of getting things finished in time and within budget.

Craftsmen are just filling in the gap of how a graduate can get to the next level through disciplined practice and apprenticeship.

But there's one thing I can agree on. The sheer amount of average developers no matter what they call themselves engineers, agilistas or craftsmen are just plain mediocre at what they do. This leads me to the conclusion that maybe our profession is closer to writing - there aren't too many good writers, are there?

It's not just the construction costs, it is any reconfiguration costs. Someone mentioned reconfiguring the SQL Engine. Well if the code is well maintained and clean, and there is a well defined I/O layer then swapping out one database implementation for another is relatively easy and quick.

in an ideal world, every chip has the same amount of legs, so it can be exchanged without too much ado.

In practice however, no code is clean enough that this would be an easy task, or you loose runtime stability (just like you can wire any chip in a logical analyzer leg-by-leg fashion, yet you don't want to do it for a production system).

Again, construction of software costs paid worktime. If people don't realize this, it just costs more and more time.

Most coders write 0 number of prototypes: they do the real system. It's like handing over the FPGA to the customer as the finished product. Please note: nowadays electronic engineering is also handing over the FPGA firmware. A cost for a new FPGA isn't zero, but the cost of having no formal approach can be a disaster.

What I claim here, is that the design activities aren't to be missed and that the craftmanship and most agile movement - despite every attempt of their inventors - are entirely dropping design and forward thinking on the base that construction is free anyway, if we did an error, we just go back and fix it.

While it's really nice to see the code catas, what it reminds me about is simply a cargo cult. While the question-answer system of design is a pretty good approach (see Alexander's Notes on the Synthesis of Form as an example), it is also widely known that a higher level of self-consciusness is possible (see also there.)

As for your comments on engineer graduates, I'm sorry to tell you that architect graduates cannot be used as full-blown architects, and this is usually even restricted by regional laws. It's easy to go from the Architect's wikipedia definition to this article: en.wikipedia.org/wiki/Professional_requirements...

It feels to me that most practiticioners of development who are arguing here have never ever looked after what do architects (or other engineers) really do, how are they educated and what do they deem important, but instead try to get it from a layman's perspective.

That formal training has a lot of good parts:for example, it tells you if a system is complete in certain senses. But it also tells you how to apply systematic approaches to requirement elaborating, testing, and any other elements of a software construction flow.

While I understand your frustration that these aren't "burned in" enough yet, the university has no intent to burn it in, exactly as to let people have some specialization by their first years of practice.

Its unfortunate, however, that those people "already out there" mostly don't know about these. My first question to junior developers about certain set of problems - like, typically, how to start elaborating requirements, or how to distill design from them - is the question: "What did you learnt in school for that?"

It turns out they got pretty good foundations, and can solve any problems systematically, once they really did understand how their knowledge should be applied. Without this however, the next generation will forget to use it too - to the level that now, much to my surprise, I'm standing nearly alone to defend certain values.

One last thought: Those things they teach in school didn't came while doing nasty things under an academic desk: they are stories and conclusions of real-life developments from the 60s to the 90s. Everything you see from UML diagram styles to development lifecycle styles were proven solutions for real-life problems before they got into the education realms.

I'm not into the logic that they were academic assholes who don't know how to solve today's problems. I feel such thinking is short-sighted, and its prevalence saddens me a lot.

What I claim here, is that the design activities aren't to be missed and that the craftmanship and most agile movement - despite every attempt of their inventors - are entirely dropping design and forward thinking on the base that construction is free anyway, if we did an error, we just go back and fix it.

Yep that's the main advantage and yet the main problem of software development. If a car has a design flaw, you need to recall it to fix it which cost a lot of money. As for a software you just send a patch which is almost free for the developers. But ask your customer how much a bug can cost him in lost time...

Don't get me wrong! I'm personally involved as a lecturer in the Sofia University. I do see value in education. I do see value in being theoretically prepared. I do see value in being well informed and went through at least some of the chapters of Ivor's Jackobson Software Engineering in Practice or something similar.

What you just told me is that academic curriculum is like that because it has to pass to the students only solid proven, research backed-up, real-life problems solving knowledge. This simply doesn't work! As an owner of a software company, I can tell you that 95% of my interviews are resulting in a rejection. And it doesn't matter whether the candidate is a fresh-out of the university, or with many years of experience, or with an impressive CV and experience that is far-more rich than mine for example. Why so many software guys are at that dire straights level of skill?

I'm sure that architecture or civic engineering graduates are in a pretty similar situation. It's just a speculation but I would bet that in the engineering field again 95% of the professionals are on the level of an intermediate beginner, while only 5% are truly capable of handling the messy process of Synthesis of Form.

It's logical to conclude that what they do is also a craft and not quite engineering. The only thing that makes it an engineering is the high level of standardization of the basic parts and the potential of vast multiplication of simple materials.

And the fact that they're just bit better at repeatably building successful arcs in a predictable fashion, right?

But when it comes to building something really, really unique, do we have enough data about how successful are architects and engineers? Or we're just assuming they're really good at that?

In software there aren't such really basic parts and materials - technologies and patterns are at a way more higher-level of organization than simple reusable parts. Another point to ponder is that it seems to me that all current software design principles and maxims are possible to be followed only by professionals with higher level of self-consciousness.

And how do you get to that level? Certainly not by academic training and certainly not by following software engineering books. And certainly not by imitating the way engineers from other fields are working.

Craftsmen practices are yet to be research backed-up and solid proven. But I see them working in the field and for me this empirical knowledge is a quite good approximation on which I can successfully rely.

Yep that's the main advantage and yet the main problem of software development. If a car has a design flaw, you need to recall it to fix it which cost a lot of money. As for a software you just send a patch which is almost free for the developers. But ask your customer how much a bug can cost him in lost time...

Maybe some developers work for free, but the companies I've heard of employ their employees with the full cost of between 5000 to 10000 EURs a month. that means, a single bug hunted for a week by a single person costs 1200 to 2400 EURs, which is half of the price of a whole car

Software construction is multiple cars and houses away from free, start to deal with it.

A craft is different from some kind of science (even engineering science) that it has some scientifical basis.

Software engineering has scientifical basis by nature: otherwise we wouldn't have operation systems or graphical representations.

However, most of the developers are just users of advances in fields of Computing Science and Software Engineering.

This is bad on one side. On the other side, a general practitioner doctor is also just a user of recent drugs, even if he is the one to answer on the field for unique needs; so is an architect selecting which kind of brick or window to use.

95% of my interviews are resulting in a rejection

I drop about 90% at a multinational development firm as an interviewer, due to time restraints ("We have to find guy who knows X by date Y").

standardization of the basic parts

The basic parts are standardized: they are language constructs and patterns. Also, you can employ a library for nearly anything.

You never had to think about wether you can use the "class" construct in a programming language, most of them has it built-in (with javascript as a well-known exception in some regards).

Our basic parts are concepts: the concept of a class, the concept of a drop-down select, the concept of a QR-Code reader. They are ready-made and available for a vast array of platforms.

they're just bit better at repeatably building successful arcs in a predictable fashion

The fact that they don't collapse usually makes them magnitudes better. However, in recent years there are also serious problems for sure.

Also, they do avoid collapses as it would cause physical damage to people and personal damage to themselves (architects literally go into prison after such, regardless of how much the customer insisted of a given feature).

something really, really unique

I don't think an accounting system is really-really unique I'm afraid. It is personalized to the customer's needs, of course, but so is an office building.

technologies and patterns are at a way more higher-level of organization than simple reusable parts

Since multiplication is easily done by using the 'new' operator in most languages, I guess a library is a pretty good candidate for a pre-made low-level reusable part.

all current software design principles and maxims are possible to be followed only by professionals with higher level of self-consciousness

Would you give 10 000 euros a month for anybody who isn't on a certain high level of self-consciousness?

And how do you get to that level?

By demanding academic knowledge and background check every single time a problem arises at the student's first workplace. It worked so far, all of my "students" as I called them became development team leads sooner or later. It also needed their talents of course.

Craftsmen practices are yet to be research backed-up and solid proven

While I have nothing against some practices - like, formal specification based development, which they call TDD - the term craft can be nothing more than cargo cult.

Once their practices get research back-up it should be a part of the standard software engineering curriculum, that's all.

So, it's not the practices, but the viewpoint which makes me worried.

So, why not trying out some other analogies?

Because a programmer costs a huge amount of money, and as long as we think of software construction as free, our employers and customers will loose multiple houses worth of cash every single month based solely on that assumption.

If something is based on science, even if it's closed into code libraries, and yet it has signs of an art, and has to be adopted for each environment, I call it as engineering.

Analogies are just as worth as the viewpoints we look it from.

I look the analogy of a craftsmen as the quality of things I see at a usual fair. I look at the analogy of engineering in terms of background and science.

I just freak out if I think of a future were formal education would be seen as non-essential, in case information technologies would be such important for our society as they are now.

I literally see the craftsmanship movement, that although their practices are well-intended, the pure analogy to compare software construction to craftmanship is defining darkness as standard, to quote the old Microsoft-joke. Their practices are welcome as a part of software engineering practices, though.

It's not just the construction costs, it is any reconfiguration costs. Someone mentioned reconfiguring the SQL Engine. Well if the code is well maintained and clean, and there is a well defined I/O layer then swapping out one database implementation for another is relatively easy and quick.

in an ideal world, every chip has the same amount of legs, so it can be exchanged without too much ado.

In practice however, no code is clean enough that this would be an easy task, or you loose runtime stability (just like you can wire any chip in a logical analyzer leg-by-leg fashion, yet you don't want to do it for a production system).

Adam,

From this response it is clear to me that you haven't done any real Engineering. For Electronics you would have to reconfigure the whole assembly line. That means new templates for the PCB's and new tooling. This is *VERY* expensive and is why Engineers invest so much time getting their design right *before* production. I've watched the video now and Glenn is spot on.

Which begs the question why are you so stuck on this single metaphor? It seems to me that for you the title of Engineer bestows greater credibility and kudos then that of Craftsman. Well if your code is crap it doesn't matter what you want to call yourself in my book :)

Which brings me to my next point. If you produce code where you can't swap out the persistence layer relatively easily then I would say that is pretty poor. A design goal should be a clear separation of concerns. Persistence is an orthogonal concern and should be addressed as such.

Spaghetti code where everything is horribly coupled to everything else is expensive to re-configure I agree. I wasn't assuming spaghetti code though. I was assuming clean code written by skilled professionals.

I agree that over 40 years of Software Engineering has left us with unskilled practitioners that produce crap code that is expensive to maintain.

Just one thing that you're totally missing. Building software is about creation, writing and it is rarely a construction. It's about emergence, shared conceptualization which happens not inside of a factory, but in a social setting. It's about language. You're leaving me with the impression that to you the emergence of human language had happened as a result of some sort of scientific research.

In Programming there's something more than engineering - there's some sort of spontaneity and a moment of illumination. A design activity strongly bond to language. I would never refer to language as to basic parts. Zeros and ones - these are the basic parts we can reuse.

I mostly deal with architecture theory nowadays through the works of Christopher Alexander (albeit I came from an architect/engineer family). He says the same things about architecture, and defines it as a language, and its conceptualization according to him should happen in a social setting. Basically everything you say about writing applies to architecture in his view.

If I'd claim anything about emergence is that builders of cathedrals had respect of their history, whereas in our current situation, the persistent notion is the "everything before was wrong" approach, which applies to Craftmanship as well as to Agile, and - funnily - applied to OOP vs Structured Programming before (its proponents seemingly didn't deal with the fact that the class construct appeared one of the first times in the Dijsktra-Dahl-Hoare book on Structured Programming).

I'd claim that roman architecture and gothic architecture are both architecture; I'd claim that there's much more to build something large then being a craftsman; I'd claim that if it was languages, the relation of software development to its state 20 years before should be the relationship of modern greek versus antic greek, rather than italian versus latin.

If I had to define a basic part, I'd rather concentrate on a CPU instruction set, or similar, as zeros and ones rarely occur to be actually defining anything, 0 and 1 weren't the ones where we began.

Let me state something: in outsourcing, where I spent the largest part of my professional carreer, you get what you get. It's not me who produces crap code, it's already there I'm afraid. Most outsourcing projects were about maintenance / further development of existing codebases.

My job for that time was to stop people continuing the tradition of spaghetti code. My internal memos of code conventions are still required course material long since I've left. I was well-known for "maximalism" when it came to so-called internal quality.

Then I switched to in-house enterprise development, and seen where did those spaghetti codes came from.

It came from the very people who defied SE practices. It came from "The Scrum Company", which I don't want to name here, it came from people who defined themselves "agilists". I met also with one of the largest agile coaching companies. I was horrified by their ideas on how software should be developed.

Craftmanship doesn't define zoom-out. Layers are not part of craftmanship knowledge, it's code-focused.

Actually, persistence layer is a rarely changing stuff,which needs pretty good performance, it's a perfectly defendable approach if one of the architects - or even a sole developer - decides that his code would be persistence-engine dependent, just like it's a pretty good guess that the environment is Win32-compatible, or runs on x86, or DirectX is present. In case of applications where 70% of the code is about really complex queries, it may make sense to make it dependant on one of the SQL dialects or platforms.

This decision is enough to be made once over a 10 years lifetime: if they do it for just a month, it'll take at least a month for you to find that spot and fit it to the new platform, especially as there'll be no useable documentation available from 9 years ago (especially not in today's state, remember, working code over docs!)

And again: nearly any kind of reconfiguration is expensive! I use the example of a bug here, a single bug is usually not a big deal in terms of amount of code to be modified.

Even if you exactly know what to do, you have to type it in, test it, etc. A horrible amount of work when it comes to paychecks.

But in case you want to see it in real life, try to make Fitnesse, a creation of the leaders of the craftmanship movement, talk into anything not HTML. Hint: HTML is hardcoded into java strings through the whole project in every layer, or at least so it was in a version I checked out a year ago.

Engineering is about accepted risks and costs. There's no free lunch in software development either.

I can't agree that the notion of "everything before was wrong" applies to Craftsmanship. On the contrary the whole movement is based on individuals with extremely rich history which spans over few decades of experience. At the language arena there aren't many new things - the stuff we use is mostly language constructs which are over 30 years old.

It is evolution of practice, not abandoning the old.

There is no such conflict such as OOP vs. Structured Programming - there's just Data/Object anti-Symmetry

There are point which we can agree on:1. Building large and complex stuff is mainly about architecture.2. Architecture & Design conceptualization happen in a social setting.3. Architecture is emergent discovery of non-obvious synergies where simple parts and materials can be combined and multiplied in a very large scale.

The main difference between our opinions is in the following:1. To me software construction is not quite like building - rarely parts fit in a fixed way, software "parts" are structures of a higher-order abstraction and complexity.2. Building software can not be done by following simple instructions or whatever instructions, even when there's an already prepared architecture and designs.3. Building clean software can happen only by following strict maxims and principles (not to be mistaken with instructions and recipes).4. The effective application of maxims greatly depends on the maturity of the practices of the given programmer.5. There's still not a predictable and repeatable way of growing a professional to the needed level of maturity and skill.

1. To me software construction is not quite like building - rarely parts fit in a fixed way, software "parts" are structures of a higher-order abstraction and complexity.

I think we use a different view on building; I'd like to recommend reading A Pattern Language from Christopher Alexander; it's a required reading in some architecture schools, therefore easy to get from libraries (or from other sources). It's also the source of the pattern languages ("design patterns") movement.

2. Building software can not be done by following simple instructions or whatever instructions, even when there's an already prepared architecture and designs.

The role of the build worker is for the computer, that's something I agree on with the presenter. Therefore the building itself is about following programming language instructions.

3. Building clean software can happen only by following strict maxims and principles (not to be mistaken with instructions and recipes).

So is any livable place, like a park or an office space. No disagreement here, albeit one: a general recipe can and should be followed (such recipes are the development methodologies), albeit never exactly. Such recipes also exist in most of the engineering fields (somewhere more regulated than at other fields).

4. The effective application of maxims greatly depends on the maturity of the practices of the given programmer.

No disagreement.

5. There's still not a predictable and repeatable way of growing a professional to the needed level of maturity and skill.

I also recommend reading the Software Engineering Educational Knowledge I linked above. This is exactly what IEEE wanted to achieve, and although it needs talent from the student's side of course, in the first years it's crucial to demand formal knowledge (like, SOLID principles or (lighthearted!) RUP processes, or MVC) from a junior developer after graduating. In my experience, it worked amazingly well.

If you think about that a bit more you'll notice that these are two different aspects:

1. The first 3 points define the theory behind software - call it Computer Science or Software Engineering. This includes the scientific basis, general purpose knowledge such as mathematics, algorithms, data structures and programming paradigms.... etc, etc, etc.

This first aspect I would agree to call an Engineering.

2. The second aspect (the second 5 points) is related to growing someone's practice and maturity to the level of an expert. Here, I'm almost convinced that even in matured engineering fields the way this is achieved is somewhat more closer to Craftsmanship. The seniors and the experts in the role of the mentors, while the juniors are in the role of the apprentice.

So, do we agree that development could have an engineering-like background?

Because there's nothing wrong with apprenticeship: to the contrary! It's actually part of every engineering studies.

What I'm attacking here is this notion that the participants of the 1968 conference were wrong, and that our education is built on false premises. It's not perfect, of course, as long as a student thinks that waterfall is to be executed to modern sized software or that RUP requires you to write unneeded documentation and find a suitable participant for each of the 92 roles. Yet I believe they got some things right.

What I'm attacking here is that instead of trying to attack based on layman's notions of architecture, we should get close to other engineering / architecture studies and watch them actually working. I'd actually require everyone to spend a week at an architecture / civil engineering / urban architecture company, look at what they do for real, and talk about their work with them.

And lastly, what I'm attacking here is that maybe we should stop reinventing the wheel and trying to find an entirely new direction for software development. I'm perfectly fine with software engineering as a self-definition (which is bit more than a metaphor), just as I'm perfectly fine with the notion of Structural Programming and the problems it tries to solve (namely that the brain capacity of a sole person is way smaller than what a real life project could fit into, and we have to find methods to keep at least some parts under control).

This doesn't mean we shouldn't try out new approaches, it's just it would be the time to recognize values in our past as the situation that development is chaotic by definition gets embarassing, even if statistically, we fail less projects now.

I haven't seen yet the talk. I just read your very negative comments on Craftsmen. I thought it is very distorted. Take the time and read their manifecto

Focus on the first sentence: "...we are raising the bar of professional software development by practicing it and helping others learn the craft" That's the whole essence of this movement - a movement which grew out of the frustration of dealing with incompetent colleagues or with their crappy codemess. This is not a movement which abandons old practices or rejects past achievements. Even the manifesto is some sort of an upgrade to Agile.

As to the word engineering - I don't mind using it when I'm talking to my colleagues. The care and responsibility with which I work can be attributed to an engineer. But I would avoid that word when dealing with my customers. It's just setting wrong expectations. And there's quite a long way before it can be considered an engineering.

I have nothing wrong with most of what Uncle Bob says: Clean Code is a required reading (along with the SOLID excerpt of CCC), the InfoQ presentation about Bad Code, Craftmanship, etc is watched by every peer at my team (and since then a required video material for newbies at my previous workplace).

Actually, I believe SOLID will become core educational material in every single software engineering course on the world in a few years, albeit parts of it were always taught.

But I still have word against the naming and against the actual practice behind it.

(By actual practice I mean the source code of Fitnesse, but more importantly, some people who claim that they're better than engineers, they're craftsmen and provide... well... questionable practices, or to be precise, end-user and customer satisfaction wasn't too high, it wasn't zero either but internal quality can be best described with "Unit Tested Spaghetti".)

Is your profile up-to-date? Please take a moment to review and update.

Email Address

Note: If updating/changing your email, a validation request will be sent

Company name:

Keep current company name

Update Company name to:

Company role:

Keep current company role

Update company role to:

Company size:

Keep current company Size

Update company size to:

Country/Zone:

Keep current country/zone

Update country/zone to:

State/Province/Region:

Keep current state/province/region

Update state/province/region to:

Subscribe to our newsletter?

Subscribe to our architect newsletter?

Subscribe to our industry email notices?

You will be sent an email to validate the new email address. This pop-up will close itself in a few moments.

We notice you're using an ad blocker

We understand why you use ad blockers. However to keep InfoQ free we need your support. InfoQ will not provide your data to third parties without individual opt-in consent. We only work with advertisers relevant to our readers. Please consider whitelisting us.