Sunday, May 22, 2011

Unsolicited and uninformed rant about Software Engineering Research

Jorge Aranda interviewed industry professionals regarding their opinions of software engineering research. Needless to say I wasn't asked, and also needless to say I'll shoot my mouth off anyway. As with most of the interviewees, my knowledge of software engineering research goes little beyond Brooks' classic papers. Despite my reduced exposure, the impression I have of S.E. research is less then stellar. I have a feeling that a lot of the research tries to be too helpful, and ends up lacking in enlightenment.

It seems the goal is often to evaluate a certain practice in order to recommend it or not to the industry. I find this goal suspect for a number of reasons: the definition of success is too context bound (e.g. some environments value fast feedback while others value more a well crafted user experience), the measures of project success may mislead, some practices can help at certain parts of a project and hinder at different parts, people respond differently to different incentives, personal preferences play a major factor, etc. Perhaps Cockburn's "Characterizing people as non-linear first-order components in software development" helps to explains my uncertainty.

I'm not writing just to complain that research I'm not familiar with goes about the wrong way. I'm also writing to offer unsolicited advice regarding what the aforementioned research should do. In a nutshell, I would like to learn more about programmers, teams, and culture. Research about programmers could investigate stuff like correlation between psychological attributes and professional attributes (e.g. are people with certain personality profiles more attracted to some kinds of software than others), differences in thinking processes between programmers doing different kinds of work, differences in the work of programmers that have been doing the same thing for an extended amount of time versus programmers with a more diverse career, etc.

Research about teams could look into communication patterns (how and when they tend to take place, not which are more "effective" than others), how people react to homogeneous versus heterogeneous skill-set compositions, how the amount of time spent as a team affects the work (not just wrt productivity), at what team sizes people tend to jell into cliques (and what other factors are in play in the process), how the physical layout affects the communication (again, not trying to find "optimal placements"), ...

The "culture" part may seem a strange focus for software engineering research, but I would enjoy reading anthropological accounts of the differences an similarities between behaviours of the different programming cultures: systems programming, the smalltalk/OOPSLA/XP culture, lispers, modern large-scale web development, game developers, etc. The very question of whether it is meaningful to apply the word "culture" here would be worthy investigating.

Anyway, the particulars aren't important, I just want to emphasize the most interesting research, from my point of view, is the one that tries to understand, not merely to optimze.

Edit: Jorge Aranda commented to point out the interviews were conducted by a team including himelf, Margaret-Anne Storey, Marian Petre, and Daniela Damian.

5 comments:

I think the real problem is the overloaded term, "Software Engineering." You seem to be focusing on a rather small piece of software engineering (process).

The focus of software engineering is the construction, delivery, and deployment of "systems of systems", and the managerial aspects in producing those systems.

Software architecture, requirements elicitation and specification, validation and verification, formal and semi-formal methods, software and system evolution, software project management, and Software development life cycles are all fair game for research.

Much of what is commonplace in software development today came from research in all of these fields.

It appears like you're only looking at "research" being published by industrial labs or reported in Software Developer Times. Rather, you should look at leading journals and conferences for substantial research in the field.

I agree though, despite software engineering being concerned with the human/person piece of software development, one has to hope that researches in cognitive psychology, behavioral science, and communication attributes and theories within certain processes, focus on our industry for relevant and influential information.

"I just want to emphasize the most interesting research, from my point of view, is the one that tries to understand, not merely to optimze."

Great point, Rafael, and great post overall. I should say that a lot of current research tries to do just that (to understand), though this was not the case some ten years ago. I think our community is becoming a bit more humble. Perhaps in its earlier times it did not recognize that its idea of software development was completely misguided, and it has taken lots of time to recognize that we don't really know much that we should know about what goes on in software organizations, and to rectify this.

I also must point out that it was a bunch of us doing the interviews you refer to besides myself: Margaret-Anne Storey, Marian Petre, and Daniela Damian.

@paulIndeed I was applying a narrow interpretation of the term "software engineering" and my concerns don't necessarily apply to the broader view. A view so broad, in fact, that it makes it hard to say anything about it in general.