Featured in AI, ML & Data Engineering

In this article, author shows how to use big data query and processing language U-SQL on Azure Data Lake Analytics platform. U-SQL combines the concepts and constructs both of SQL and C#. It combines the simplicity and declarative nature of SQL with the programmatic power of C# including rich types and expressions.

Featured in Culture & Methods

The book Agile Leadership in Practice - Applying Management 3.0 by Dominik Maximini is an experience report of the agile transformation journey of NovaTec. Maximini shares his experiences from applying principles and practices from Management 3.0, success stories, failure stories, and learnings from experiments.

Featured in DevOps

Yuri Shkuro presents a methodology that uses data mining to learn the typical behavior of the system from massive amounts of distributed traces, compares it with pathological behavior during outages, and uses complexity reduction and intuitive visualizations to guide the user towards actionable insights about the root cause of the outages.

Becoming Software Craftsmen

The software craftsmanship movement talks about practicing as a way to to develop programming skills to become software craftsmen. Technical practices are considered to be important, it takes time to learn them and become better programmers.

As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:

Not only working software, but also well-crafted software

Not only responding to change, but also steadily adding value

Not only individuals and interactions, but also a community of professionals

Not only customer collaboration, but also productive partnerships

That is, in pursuit of the items on the left we have found the items on the right to be indispensable.

In the blog post the codesmith Chris ODell shares his definition of a craftsperson:

in my mind a craftsperson is someone who not only creates something from nothing from materials of their choice, but usually puts a part of themselves into what they make. A good craftsperson takes the time and effort to make something of quality which is worthwhile to have and which will still function way after the crafted item has been made. Good craftsmanship is synonymous with quality products.

Using an analogy with metalsmiths and blacksmiths Chris coined the term codesmiths which for him “implies a certain level of craftsmanship and implies a progression from apprentice to master”. In his follow up blog post anyone can be a codesmith he explained how codesmiths can develop themselves to become codesmith masters:

The barrier to being a codesmith should be low. This is why I think that anyone who writes code in any shape or form should feel free about calling themselves a codesmith. The ideals of craftmanship should be followed, with a healthy dose of pragmatism, but learning the skills to become a good coder takes time and experience.

Matt Badgley: “We need to renew the emphasis on agile engineering good practices and embrace the ideas of craftsmanship — without this, agility does not happen.”

Robert Martin: “if you don't use [the 13 practices of XP], you'd better use some that are as good or better. And the practices you use will define your culture and be an expression of your values.”

Ruud Wijnands: “the companies that I have seen that were successful in their agile adoption all valued and implemented technical practices. The companies less successful did not.”

How can programmers develop their skills to become software craftsmen? John Sonmez wrote the blog post why I don’t do code katas in which he stated that programmers cannot improve their skills by doing the same exercise repeatedly. To learn something, the bar must be raised continuously:

Repeatedly writing the code to solve the same programming problem over and over again is exactly the same thing, except unless you are introducing new challenges by adding constraints and twists, you are only benefiting from the exercise the first, and perhaps, the second time, you undertake the activity.

(…) if you want to get better at something, repeating practice alone is not enough. You must practice with increased difficulty and challenge.

According to John the a good way to develop skills is by doing. Programmers have to program to become software craftsmen:

I get lots of emails asking me about the best way to learn programming or how someone can improve their skills. Recently, I’ve gotten a lot of emails asking about how to learn Android development. Do you know what I tell all these inquisitive minds?

Make Android apps.

But, umm, what book can I read?

No book, just come up with an idea and try to create it. Figure out what you need to do along the way. When you get stuck, go look for the answer or seek help.

Steve Ropa wrote about building software craftsmen. Universities teach the theory of software development. To become craftsmen programmers need to gain “real-world experience and practical applications of knowledge” said Steve. Apprenticeship programs can provide this:

My suggestion is to turn the clock back a few hundred years, and turn your team room into a workshop. Let’s populate that workshop with Craftsmen. We may not have all of the Craftsmen we need to begin with, so we need to build and grow them. This can be done by applying an apprenticeship program and using the craftsmen’s model for further career development.

In his blog post Steve provides five main areas for software development apprentices to become craftsmen:

Crafting Code – The art of using one or more programming languages to create clear, well-factored code. (…)

Applied Principles – (…) An apprentice needs to understand principles like SOLID, and know how to apply them.

Technologies and Tools – While programmers need to be able to practice activities like Refactoring by hand, they also need to know how to use certain tools, as well as which tool to choose for a particular task.

Work Habits – (…) we need to be able to build muscle memory around the activities that make good programmers great, such as TDD, Continuous Integration, etc.

Soft Skills – (…) Software apprentices will learn how to work in a team, how to communicate with others, and other soft skills that tend to be forgotten in the traditional learning environment.

Some Disagreements With This Essay...

Your message is awaiting moderation. Thank you for participating in the discussion.

I tend to agree with most of the ideas presented here in this essay. Where I disagree is when the author promotes certain types of frameworks and\or tools as a basis for "software craftsmanship".

I have been doing professional software development for over 40 years, now specializing in the Microsoft Development Environments. As a result, I received my most critical training during the mainframe era where software disciplines were at an all-time high even though we had our project failures as much then as now.

One of the things that is rarely ever mentioned when discussing software development talent is simple common-sense. It is not a framework, a tool, or even a paradigm followed that makes for a great software developer but instead the ability to craft code that is easily readable by the least experienced person who may have to deal with it while also processing efficiently.

Too many times I have seen the recent promotion of frameworks and paradigms as the answer to software quality when they have little to do with it. In fact many of the current frameworks are mere repetitions of what has already come and disappeared in the past. For example, the new trend in JavaScript is nothing more than a re-hash of “Classic ASP” with a different twist. It is nothing new since it is another version of an interpretive language.

TDD or Test Driven Development has been found to offer nothing over well designed traditional forms of testing.

Agile development is often used to pay lip-service by organizations that want to promote themselves as using modern development techniques but in reality are just using a newer form of “guerrilla programming” or “Just get it done!”.

ASP.NET MVC is just another paradigm designed to pander to the “purist crowd”. Not once has it ever been clinically demonstrated that this paradigm produces better results in terms of actual benefits to a business.

These are just some of the examples of what is occurring now in the mainstream, world of software development but at the same time the rate of project failure still stands at around 70% which has been a constant for over 35 years.

True software craftsmanship can only be attained from a developer truly has a love for the development of fine software no matter the language or tools used. And this can be only accomplished when the least amount of complexity is used to produce the results keeping the overall work simple and easy to understand. This is why I still use the basic tools I have always used to develop my projects. My work is very easy to understand and as easy to maintain once one understands the standard n-tiered pattern that is used across all my work.

On a final note, the suggestion that one should develop Android applications to learn software craftsmanship is somewhat far-fetched. For one there is only so much one can do with a mobile application. Second, there are so many mobile applications now in existence that this genre has quickly become saturated with such software. A better approach is to develop applications across several topologies for different interests that a developer may have. This means developing for the desktop, the web, and the mobile space. However, it is the desktop and the web where a new technician will learn the necessities of enterprise-level systems that in the midst of the current trends in mobile computing is a talent that is fast becoming lost on the newer generations of development professionals.

Re: Some Disagreements With This Essay...

Your message is awaiting moderation. Thank you for participating in the discussion.

"TDD or Test Driven Development has been found to offer nothing over well designed traditional forms of testing."

That is not true. TDD has been found to eliminate more than 60% of bugs otherwise found by testing teams.

Agile development is "not just a new form of guerilla programming". It is a very effective software development methodology which, when applied correctly, lowers development cost over time. However, software craftsmen should be able to develop quality software both using Agile methodologies and otherwise traditional methodologies.

Re: Some Disagreements With This Essay...

Your message is awaiting moderation. Thank you for participating in the discussion.

I disagree with your statement on TDD. I have spoken with a former long-term Silicon Valley Software Engineer who has used both types of testing methods in large projects. He stated that if the quality of both methodologies were equal than their was no advantage to one over the other. In fact, that is what I stated in my post. I did not state that TDD was of no value.

Agile Development has not shown to provide any reduction in the overall project failure rates. This has been demonstrated over the years via the analysis of project failure, which is still at approximately 70%. Second, Agile is just a form of the incremental or evolutionary development, which has been part of the recommended software engineering paradigms for many years. It is nothing new. However, most IT organizations do not practice true Agile techniques as they relate to actual software engineering paradigms. I stated this also in my post. As I have found on many assignments, Agile is just a trend used by many IT organizations to simply mask the useless method of "guerrilla programming".

Overwhelmingly, most corporations in the United States simply cannot develop quality software since they are too high-bound with bureaucracy and politics.

Re: Some Disagreements With This Essay...

Your message is awaiting moderation. Thank you for participating in the discussion.

@Steve Naidamast I feel you may not fully understand the practice and motivation of doing TDD. If you're comparing TDD with 'traditional methods' of testing, then you're not seeing the full picture.

TDD does not usually eliminate other forms of testing. TDD is as much about design as it is about having unit-test coverage, allowing one to refactor with confidence. Some would even argue it is all about design, and the test coverage being a good side-effect. To write testable code, it most likely should have a simple-to-use API, have high-cohesion and low-coupling.

Re: Some Disagreements With This Essay...

Your message is awaiting moderation. Thank you for participating in the discussion.

Michael:

I can understand what you are saying about TDD. However, the reality is that even if TDD does in fact provide a superior framework to developing quality software, few organizations actually implement the paradigm as it should.

Second, though I may be incorrect about this, the way I understand TDD is unit-test development first and then actual implementation. Such a techniques requires additional time that most companies are not willing to afford. However, as there may be design paradigms figured into this paradigm, TDD is generally compared to traditional testing methodologies, which do not require that additional time involved. Maybe I shoudl have been clearer in my statements on this but my intent was to state that between the two testing paradigms no one has been proven superior when both are of equal quality in their use.

I am a senior software engineer trained in the development of low-defect software. I know all of the required paradigms needed to do this and attempt to implement them in my own projects whenever I can. However, even in companies that touted the use of quality software engineering techniques, the best that happened was lip services to the concepts. And even where I have proven substantially the quality of such paradigms via the output I produced, project leaders still had no idea as to why they would even consider them.

I have been in the field over 40 years doing this work in a variety of environments and none of them, with the exception of certain project leaders, ever encouraged quality software development...

Re: Some Disagreements With This Essay...

Your message is awaiting moderation. Thank you for participating in the discussion.

Hi Steve,I agree to some extent with some of what you are saying here. I think every agilist would say that what makes Agile work is that it is a combination of techniques that were successful in the past. In other words, if things like evolutionary design and well factored code are good ideas, lets do them all the time. Agile has been very helpful in getting that word out, which to me is the biggest benefit of all.

While some corporations do indeed use agile as a way to mask guerrilla programming, the same guerrilla programming would exist in a more formalized approach. I contend that it is not up to the corporation to determine whether we as programmers write good code, but it is up to us. In other words, to abuse Shakespeare, "The fault, dear Brutus, lies not in the stars, but in ourselves."

lambda-calculus and software craftsmanship

Your message is awaiting moderation. Thank you for participating in the discussion.

If you write code for long enough and ask yourself what programs are made of, you can see four key concepts emerging [1]:- actions- types- states- relationships

You could argue there are a few more (say collections, events/messages or services) but that would not change my argument by much.

The core reason why we are "craftsmen" (and craftswomen) is simply because programming languages come from a different era, an era driven by lambda-calculus, an era where everything was either an action or a type.

We have to craft our code because we need to constantly (and inconsistently) encode state (an invoice is issued, received, paid, in default, reimbursed...) and relationships into types and actions. The programming languages that we are given are so insane that even SQL reifies a relationship behind the attribute of a type, or a type itself. There is no "relationship" concept in SQL (not to mention state), it's all the head of the developer. OO is by far the worst offender (by amount of code written) where action and types subsume state machines as a bunch of attributes and disjointed actions, and relationships as mere type composition mechanisms.

The sad part of this story is that States and Relationships are the structure of software, no type or action would make sense outside the frame(work) they provide. We are coding a bit like a residential architect would be build a house with beams and studs reified in paint and siding elements. Wouldn't work too well? hey?

That's why we have no choice other than being craftsmen. Perhaps it is time to change course [2], we tried that one for 50 years.

Re: Some Disagreements With This Essay...

Your message is awaiting moderation. Thank you for participating in the discussion.

There is often a confusion in how people understand TDD. And how they see a non-TDD development. It can be same level of unit testing, but tests created immediately after the "main" code. Or at unit tests are added some time later "as time permits". Or there might be no unit test coverage, only component/integration tests. Or just manual testing.