Sunday, February 14, 2016

Why Should Software Architects Write Code?

In the software engineering community there is a divide on who needs architects, what responsibilities should architects have, and whether architects should code. We all have heard these questions in one form or another, all motivated by a wide range of pragmatic opinions. Some practitioners argue that architects responsibility is for the integrity of the entire system, and satisfying business goals while mitigating the risks; Therefore, architects can postpone other lower-level decisions, including the coding decisions, to developers that typically have more limited responsibilities. On the other side of this spectrum, a group of practitioners argue architects should code. Some critics even take the criticism of “architects not practicing coding” further and argue that: “powerpoint architects'' are ineffective while expensive to afford. These architects join the project kickoff meetings, draw all sort of diagrams and leave before implementation starts. They are ineffective because of their absence and lack of ongoing feedback during the development cycle. We should also keep in mind that, such disconnection will create more problems considering that the requirements which the same architecture addressed will more likely change later on.

Other practitioners choose a moderated perspective. In his recent keynote talk, Martin Fowler explores different ways of stimulating collaboration and communication between programmers and architects, and he advocates the idea of architects pair-programming with developers. Ward Cunningham in an anti-pattern called “Architects Don't Code'' addresses the same issue--- that “The Architect responsible for designing your system hasn't written a line of code in two years''. Ward recommends getting architects involved at the implementation level.

In his new book, “Software Architecture for Developers'', Simon Brown advocates a transition from architects as “ivory tower'' to a role which is about coding, coaching and collaboration. All these practitioners agree on the necessity of having some form of architecture design. Simon Brown argues that most software developers are not architects or do not have extensive design skills. Therefore, he advocates increasing the design knowledge of developers as a way to bridge the gap.

These discussions are just a few samples of pragmatic perspectives. Following either of these opinions largely depends on having the necessary trust in the person advocating it.

While there have been numerous studies about the impact of architecture in a software system, there has not been an empirical study to examine the influence that software architects can have during coding activities of a software system. In our research we use mining software repository techniques to answer the question of Why should architects write code?We studied the architecture and implementation of six software systems. In these case studies we looked at the involvement of developers in implementing architectural patterns/tactics (e.g. authentication, audit trail, thread pooling, scheduling).

The developers contributing to these projects were observed to have difficulties implementing architectural tactics. The tactical files for all of these projects underwent more refactoring efforts than non-tactical files and contained more defects. In fact, from November 2008 through November 2011, 2.8 times as many defects were found in tactical files than non-tactical files for the Hadoop project, while from January 2009 through November 2011, 2.0 times as many defects were found in these files for the OFBiz project.Following this observation, we formulated the following hypothesis: When software architects write code, the number of defects in the tactical fragments of the systems will be reduced.

To examine this hypothesis, we conducted a second study to examine the impact that architecture-savvy and non-architecture savvy developers have on defects in design fragments of a system.

We divided the source code commits into architecturally-significant commits and functional commits. Code changes impacting the architectural patterns/tactics were classified as architecturally significant while those commits only impacting the functional features of the system were labeled as functional commits. Then we created a profile for each developer involved in the implementation of tactics. We used a persona-based human modeling approach to extract the architecture and design experience of the developers from the six software projects. For each developer, we created a profile to document his/her design expertise. Next, these profiles were categorized into architecture-savvy and non-architecture-savvy personas. Using these profiles, we created a design contribution matrix, examining the relationship between developers design backgrounds and defects in design fragments. In this post, I report a some of the results of our study and what we observed across these six projects.

Overall in four out of six projects, we have strong statistical results, validating our research hypothesis. This provides empirical evidence on why software architect should write code or, at least, be engaged in the development of tactical fragments in a system. Developers with little or no background in architecture and design struggled to implement architectural patterns/tactics. The majority of the defects in the tactical files were introduced because of a misinterpretation of the design concept, being unaware of the rationale for the tactic, or just plainly implementing it incorrectly.

I am looking forward to read your comments and feedback on this post. Also I would like to encourage researchers and practitioners to conduct similar empirical studies to examine the controversial pragmatic opinions related to Architecture and Coding.Next post is going to summarize a few practices to bridge the gap between design and implementation. Stay tuned!~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Acknowledgement:"This post includes joint work with Inayat Rehman, Matthew Thornton and Azat Aralbay Uulu, graduate students at RIT. I would like to thank Mei Nagappan for crunching the numbers for the experiment reported in this blog."

Don´t mean to sound clerical, but noticed that "threat" should actually be "thread" up there. For the rest, definitely every developer should every now and then rise high above programming language constructs, and see the "grand design" into what he's doing. A great program to enhance those skills could be https://www.tue.nl/en/university/departments/mathematics-and-computer-science/education/graduate-programs/pdeng-programs/software-technology/

May I add that architects should code along _with_ the team implementing the project, not outside of it. More than half the code of our projects nowadays is non-production : unit, functional and stress tests, deployment, management. It's easy to ignore those aspects if you're coding alone towards a solution. But such code doesnt have much value if it doesnt have the support harness to guarantee trouble-free maintenance over its lifetime.

Then I also met QA guys unable to do their job because they were hired as non coders.

I could point that recently to break the resistance of coders to code stuff that don't work, more and more important coding decision with "added value" or "firctions" from the job of coders have been deported to non coders.

Such as project management, business analyst, product managers, Human Resources, even boss. People that lead the work of skilled coders are not themselves coders taking decision without the experimental knowledge of cost and feasibility. Which seems to be the only true knowledge that betters software cost estimation hence pricing and costing. Hence should lead cost efficient business decisions.

Having people steal our jobs without skills sure hurt the whole business.

But in the first place it hurts even more to deny the true nature of creation that is both hard to make work and which benefits of the creation are given as an incentive to fools who can just need to capture more coders (designers, redactors, journalists, event organizer) and squeeze them like lemons to make more money.

Our market is rigged by IP and it makes very counter productive systems. The regulations creates this unstable equilibrium by favouring the capital over the craftsman.

Even amongst coders: they are the ones who get their legitimity by experience others by expensive diploma even though CS diploma are still not correlated to any proof of indeed producing better (or worst) coders.

Regulations and perverted macro economical effects are affecting locally our work organization and it is proving to be having an adverse effect.

It is time we face the truth: this is a clear effect of the intellectual property economy.

Giving the benefits and the authority of the works not to those who know their jobs (praxis) but to those who to know to make benefits out of it (doxein).

Hence the world used to be describing pro-net-arians could be crafted on this solemn occasion to describe this.

And this does not apply to coders only, it apply hence for the heart of the who "service" economy.

And that is the ugly secret behind our modern crisis. Giving the incentive to create working innovative business models to those who have interest in conservatism and lack of disruption.

I am an architect and team lead so my view might not be the same as others but I emphatically agree architects should code. It's absolutely necessary for architects to have practical, continuing experience with the patterns, approaches, library decisions, etc. that they're making. That said, an architect should never be on the critical path for delivery when it comes to code. What I mean is, if an architect is put in the position of having to implement features on a timeline, they won't be available to coach, solve problems, revisit a decision, etc. at the architecture level without risking the timeline.

To me that means pair programming is a non-starter when it comes to architects and developers. An architect needs to be focused on the bigger design and integrity of the software, yes, but they also need to communicate effectively with the development team, ensure the design is understood, coach a bit on the parts that are less common or are not part of the usual way the team is used to working, etc. To have the architect tied to deliverables necessarily means other things will not get the focus they should.

Thanks Erik for the comment. I'm glad to hear from someone who is serving as an architect. You are articulating an interesting point and I certainly will use that in my next blog post. Out of curiosity, how often do you get involved with implementation issues? Either coaching the programmers or going over a coding problem with them that relates to architectural choices?

As I mentioned, I'm both an architect and team lead so my perspective is slightly different than people who are pure architects so keep that in mind.

The frequency of me getting involved is directly related to how often changes to the way the developers approach things change. Developers tend to learn an approach, get familiar and comfortable with it and then that becomes how they handle all new projects. Over time this starts to become stale. The field moves forward to solve common problems, introduce better abstractions, libraries mature or die and are replaced, etc. It's necessary to "refresh" our approach from time to time. That's when I end up getting involved. I'd say a handful of times a year we introduce a change that requires some coaching and monitoring and often that includes writing code to either demonstrate the concept or actually implement one component to use as the guide for developers to implement the rest.

Each new topic has a drop-off where I spend a lot of time up front making sure things are done correctly and that people understand the how and why. As the developers become accustomed to the change, my time required for that change is much less to the point I'm not needed at all by the second or third implementation leveraging that design element. When that happens, I introduce a new change. It's a slow process of constant improvement. I look for a high-value change that can be made to make things easier for the developers or more robust or gives new value in the applications, etc., add it to a design, work through an implementation and then give some time for it to become routine. Then I identify the next high-value change and repeat the process. Changing too many things at once is very risky as it becomes difficult to track down exactly where problems are and as the architect I become the critical resource for many developers to resolve their issues. It causes developers to fall back to "the old way which worked perfectly fine" and makes introducing changes in the future an uphill battle.

I'd say the amount of time I spend on design is eclipsed by time spent coaching, providing reference implementations, answering questions and debugging problems but I think it's important that I do these things to make sure I'm not setting the developers up for failure. I made that mistake by telling the developers to use repository pattern to make DI easier without explaining what I meant or providing a reference and then spent a lot of time pulling things back together. Conversely giving them a reference for DI using a specific library was quite successful. I'm actually currently working on a new reference implementation of some new things we've been asked to start including in projects going forward which includes better error logging and the inclusion of application profiling because whatever our next project is will have to have these features.

As one of the Hadoop committers during the time period in question (stevel@apache), I am very interested about these findings, and would love to be a reviewer.

In particular, I'm curious about

-How did you determine authors of patches? As it is often not the committer. I assume you went through JIRA, right?

-What did you conclude about my role? High-level architecture or low-level things like diagnostics and getting tests to work?

-How to define "strategic" vs "tactical" files? This is an interesting topic: if we can identify them automatically, then that's taking some of our folklore (leave "FileSystem alone") and making it visible.

-Similarly, how good was you analysis tool at inferring "troublesome" files. I guess we have that information (the one with the most diffs), but really the nature of the patch type/JIRA issue needs to be included, to find those files which get the most bug fixes rather than enhancements.

Some bits of code may seem tactical (UserGroupInformation; IPC Client and Server) but they are trouble not so much due to code quality, but they are the interface to stress points in the cluster, Kerberos and the network itself. They fail in ways which could be described as "interesting". We're scared of them. I don't know how they'd fit in to this tactical/strategic role, but certainly in the case of UserGroupInformation, the reason it doesn't get refactored or edited much is not due to contentment with its architecture: it's fear.

I don't disagree with the premise: architects need to code. I also think they need to write tests, to understand how to design for testability, and get involved in the low-level issues of metrics and monitoring, of deployment and fielding support calls: otherwise you don't get systems designed for management and supportability. I'm pleased to say my colleagues and I do get to do all this, including fielding support calls about UGI if we are particularly unfortunate. It only takes a few weekend-long support calls related to a part of the system before you start to see flaws in its current design.

Thanks for the feedback!!!I very much like different pieces of your comment (e.g. fear of UserGroupInformation, Architects write tests, design for testability, etc.) I would love to be in touch with you for a follow-up study. I've been impressed by the amount of design knowledge shared between Hadoop developers. Next step for us is to interview someone from Hadoop project, If you are interested, please drop me an email (mehdi at se.rit.edu)

Thanks for the comments. I didn't describe the details of the methodology used for this study in the blog post, but I'm going to put that online.

The roles were identified through developers LinkedIn profiles plus a complimentary step.

Yes we looked at code commits, and JIRA. We extracted the developers emails account, looked up their profile on social media (e.g. Linkedin) Primarily we used the work history reported on LinkedIn to see if they have previously served as an architect or not. So many of the architecture-savvy developers were identified through self-identification on the LinkedIn profile. We also looked at the developers discussions on JIRA and their comments within the commits. That info gave us complementary data on who is architecture-savvy and who is not architecture-savvy. These data were peer-reviewed to gain higher confidence. We had senior developers who were in the non-architecture savvy group because we couldn't find evidence that they have served as architect role in the past (through LinkedIn) through their comments we didn't find discussions related to architecture. (Note: This dataset has been kept fully private, sometimes we researcher need such datapoints to provide empirical support for what's happening in practice)

Buggy files were identified by matching commits and JIRA reports. This is a common method, and the most reliable technique so far. Yes, we differentiated the bugs from enhancements and other changes. It requires matching GitHub commit histories with issue tracking systems.

The tactical files were detected through our reverse engineering approach; we used higher thresholds to include files which very high confidence in the study.

Thanks Dr.Mehdi, it a very interested study and I can say that I totally agree on the important role of the architect with the team and that this role should include more than just the design and high-level decisions.

I thought these comments were interesting "Simon Brown argues that most software developers are not architects or do not have extensive design skills. " and "Developers with little or no background in architecture and design struggled to implement architectural patterns/tactics. "

Which is to say, the author defines "Developers" as people with very little skill or experience, and notes that when some experienced people got involved, things worked better.

So you could really call this piece "why you should hire a few competent and experienced developers rather than expecting a bunch of unsupervised juniors to do useful work on their own."

Hi Simon, Thanks for the feedback.One interesting point is that actually a good number of the developers in non-architecture savvy group were senior developers with years of experience but not necessarily architecting experience. So we don't make the assumption that developers are people with little skill.

Thanks for the feedback! Question, How can one be a good developer if he/she doesn't know how an MVC pattern, client-server style, fault detection design, or many other design choices work. What about trade-offs between performance and security choices? At the end of the day, our software systems will have an architecture, and developers work within the framework enforced by that architecture, missing the design knowledge can compromise software quality.

The second observation can be explained more general: It is most of the time easier to implement a quick and dirty local solution what will work for the Happy flow simple testing. Code/Design reusing is a case of continuous learning en syncing with each other...

Wow! Very interesting post, I am really happy to see that I am not the only one with these ideas. This post looks really similar to one I wrote last year, based on my own experience: http://missarchitectblog.blogspot.nl/2015/11/do-architects-need-to-write-code.html. I am curious to see the results of your investigation, it can help me to prove my point :)

Thanks for your comments!!! I'd be happy to send you a copy of our results. Please drop me an email!!! In this article we planned to provide empirical evidence for pragmatic views on architecture and coding.

It seems to me that this survey provides some evidence that developers who have some understanding of architectural principles make fewer mistakes when developing and changing architecture-related code than those developers who lack this understanding. Like skumar suggests above 'hire architecture-savvy developers' would seem to me to be the valid conclusion of this survey.

What the survey does not really help us with is the key question - 'To what extent should the architect of a particular software system be involved in the programming of that system'. Like most software engineering questions, there's no easy answer. Sometimes it makes sense for the architect to be involved in the system from its outset to its delivery and maybe even its support. Sometimes, however, say when a system is developed by multiple organizations by developers speaking different languages, the best use of the architect is during the early stage of systems engineering not in the coding process.

So, the question 'Should software architects write code' is not really a very good question because the only universally valid answer is 'It depends'. We can only answer the question more specifically if we define the context of work .

What you have provided evidence for is the need for all developers to understand architectural concepts and principles and it is a pretty sad reflection on computer science education (and other programmer education programs) that so many developers don't seem to have this understanding.