Let SQA Be Your Guide

When you think about a software quality Assurance (SQA) function in your organization, what's the first thing that comes to mind? Organizational saviors? Bane of our existence? Pain in the patootie? Ignorant nitpickers with no business sense? Source of all knowledge as we know it today? You're not alone. Because SQA performs reviews and audits, it may be seen as a direct threat to Engineering. Rather than operating as partners, SQA and Engineering can become adversaries. SQA should function more like Jiminy Cricket. Remember the story of Pinocchio, the little puppet who wanted to become a boy? He had a creature who sat on his shoulder and reminded him when, in his heart of hearts, he wasn't doing what he knew he should be doing. Jiminy Cricket was the physical embodiment of Pinocchio’s conscience. And that's what SQA should be to an organization—its conscience.

Who Is Responsible for Quality?Who is responsible for quality, for assuring that products meet established standards and that the process is followed to create those products? Is quality everyone's responsibility? This typical assertion actually creates an interesting paradox of human behavior in organizations. When quality is declared to be everyone's responsibility, no one is truly designated to be responsible for it, and quality issues fade into the chaos of the crisis du jour.

Is quality SQA's responsibility? Not necessarily. A conscience is not responsible for proper behavior. It's merely a fair reminder. (Let your conscience be your guide.) Let me illustrate with a personal story.

Years ago, I was brought in as the new SQA "hired gun" to help straighten out a contract that had been operating for several years and was in big trouble. I started by poking around the declared project standards and artifacts. I noticed that this organization had a wonderful standard for doing Software Development Notebooks (SDNs). The SDN was a loose-leaf binder, one for each software unit, created by the original software developer, then transitioned to the software maintainer to serve as an ongoing record of the life of the applicable software unit.

This project had been running for years, and there were hundreds of these SDNs in the project's library, each with the applicable software unit's name recorded on the spine. I decided that a good way for me to get an idea of the software status would be to audit the SDN library. I was shocked to discover that over 90% of the SDNs were empty! I wrote a memo that described the findings of my audit.

10 Ways to Improve Your SQA RoleOkay, so you've been designated as a part of (or all of) the SQA function in your organization. And you don't feel like a conscience. Or perhaps you're experiencing contention or just tension. What now? Let's look at the significant human-interaction element involved in SQA that if not approached with the right attitude (organizationally and personally) can cause an SQA role to be less than fun—this can range from feeling that SQA is a thankless task to actually experiencing out-and-out shunning from coworkers.

1. Be willing to change. Organizational improvement means that humans have to change their behaviors, and behavioral change is not an easy thing for humans to do. SQA helps the organization to be honest with itself, so you have to be honest with yourself as well. Can you change your behaviors? What are your weaknesses (especially in personal interactions)? What can you do about them? Are you willing to learn and grow?

2. Be objective. Remember that most noncompliance issues are the result of miscommunication and significant performance variations (often unintended). You must maintain your objectivity and focus on process as your imperfect human peers learn to do things better. While you can be passionate about quality and your job, you should be dispassionate about how you register noncompliance. They are not violating YOUR rules: these are THEIR rules (or management's rules, or the organization's rules). You are not an enforcer, merely an objective observer.

Provide a process hook. It is easier to check products for adherence to standards than to check human activities for adherence to procedural (process) guidance. But the Software Capability Maturity Model (SW CMM) says that both must be checked for compliance. Why? One reason is that a product could conceivably be created to a standard without necessarily following the prescribed process. For example, let’s say that estimates for the project (size, effort, cost, etc.) must be created and recorded on a form, using a particular estimation technique (algorithmic, Delphi wide-band, etc.). Could someone just fill out the form, providing final numbers, without following the prescribed process? How would you know the difference? Should you care? The SW CMM wants you to care.

Does this mean that someone has to watch each process being performed? Not necessarily. In the case of the estimation example, final estimates are based on assumptions or ranges of guesses or past history. These can be thought of as the inputs to the estimation process. If the inputs are recorded and the final numbers are recorded, then it would seem more trouble that it was worth to not follow the process to get from the inputs to the final numbers. It's also easy to spot-check by ensuring that the estimates can be re-created. So, for this example, recording both the input assumptions and the final numbers gives reasonable confidence that the prescribed process was followed.

Therefore, one of the roles for SQA during process creation is to ensure that the process is documented in such a way that it can be objectively verified, preferably without having to watch someone do it. I call this the role of the "Process Hooker"—ensuring that the verification "hook" is present when processes are documented. If you have done your job as a Process Hooker for internal standards and procedures, the payback in objective observational ability will be priceless when they are implemented.

4. Establish your credibility. Be a constant learner. Know what standards and procedures are applicable internally, as well as any special constraints of a specific operating environment (FDA, NASA, NRC, DOD, MMC, etc.). You will be expected to stay ahead of the standards game, so prepare yourself technically. Becoming a Certified Software Quality Engineer through the American Society for Quality (www.asq.org) is an excellent way to establish credibility in your field.

5. Get involved early. Your opportunity to proactively affect a project’s quality culture decreases with time. The proposal is a great time for providing a reality check on what behaviors are desired versus the way things have happened in the past. It is very important to understand the organization’s estimation and commitment process, as these early behaviors could wreak havoc with the cost/schedule/quality-balancing act later.

6. Become a people-person. Much of your job will be dealing with others as a salesperson, coach, sounding board, arbitrator, and decision maker. Learn how to communicate effectively and comfortably. Remember to separate the person from the process. Maintain a sense of humor and a thick skin. When folks are having trouble adapting to a new procedure or standard, their frustrations with the situation will probably get vented to the nearest, most convenient human being. Recognize that anger about a situation is not really anger at you. It just feels that way.

7. Think before you react. Remember that we all carry the accumulated baggage of previous events in our lives. Someone's seemingly unreasonable reaction to an organizational situation may have actually been influenced by a particularly good or bad past experience. Of course, you are not qualified to practice psychology, but you should be sensitive to the fact that reactions to your role in observing process discipline can be skewed by all kinds of things that may not be apparent on the surface.

Observe your own behaviors and try to be congruent in dealing with those around you. Being congruent means thinking before reacting and keeping yourself, the other person(s), and the context in mental focus as you respond to others. This is not simple or easy. It takes a lot of practice.

8. Learn how to accept criticism. Remember that when someone is criticizing you, they are always telling more about themselves than about you. What an opportunity for learning! (And something to remember when criticizing others!) For

9. Develop negotiating skills. A few things to remember when you are in "convincing mode":

You truly must believe in what you are "selling." If you don't, it will eventually show and you will no longer be convincing.

Don't hog the airtime. Learn to listen (hearing is not the same as listening). Ask questions that imitate childlike (not childish!) curiosity and wonder. Your compatriot has a legitimate issue, from their perception. Do you understand it? Reframe it to them to see if they agree with what you thought you heard. Maybe the standard or process really doesn't apply in their context. (Time for an update or adjustment.) Follow the 80/20 rule—spend at least 80% of the time listening and only 20% (or less) talking.

A bad habit that I have to battle is to stop thinking of "the answer"—solution, comeback, or retort—while I am listening. If you are thinking about how you will respond, then you are not really listening—you're just waiting for your chance to speak.

When behaviors are changing in an organization, you as an SQA practitioner will often find yourself on the vanguard of that change. Remember that people change behaviors when there is something in it for them. Hopefully, from an organizational reward and recognition system, this will be obvious. But when it's not, y ou have your work cut out f or you

Learn to read the organization. There is almost always an informal structure or hierarchy that differs from the official organizational chart with regard to influence, communication, and even power. Most organizations have opinion leaders, folks to whom others gravitate for advice or expertise. These individuals typically have superior credibility that comes from past success and fosters future success. When these opinion leaders are aligned with and champion a cause, the changes associated with that cause become easier to accomplish. For example, when opinion leaders play an expert role in procedural guidance or standards documentation, implementation seems to happen more smoothly.

Being an organizational conscience isn’t easy, but it’s easier than being an adversary by virtue of your occupation. Ultimately, your role and effectiveness in human interaction will depend on management's approach to a quality culture and acceptance of their responsibilities. If you are not being met halfway, and are not allowed to discuss the problem, the lack of fun in the future should cause you to consider other options.

Software Quality.The term "quality" has many definitions. As a matter of fact, a universally accepted definition of quality really does not exist (despite what you read in dictionaries!). The classic Zen and the Art of Motorcycle Maintenance by Robert Pirsig actually is about neither Zen nor motorcycle maintenance, although both are touched on in the book. It is the story of a man who loses his mind trying to define quality.

Philip Crosby defines quality as "conformance to requirements," Joseph Juran as "fitness for use," and Jerry Weinberg as "value to some person(s)." And, of course, one of the most popular definitions of quality is “the absence of defects” (remembering that there are also multiple definitions of "defect"!).

If we examine the function of Software Quality Assurance (SQA) in the Software Capability Maturity Model (SW CMM), we see a unique image of "Software Quality" derived from IEEE-STD-610 that essentially answers two questions:

Do the software products conform to applicable standards?

XAre we following our prescribed processes in developing the software?

Remember that in the context of the SW CMM, "software product" is typically more than just the code. It includes any documentation that may accompany the code, including documentation used to manage the project (e.g., the development plan). So the processes used to develop the software include both engineering and management functions.

This perspective on software quality can be thought of as an extension of the Crosby definition of quality (conformance to requirements), but Crosby’s definition is usually accomplished by testing—testing the software to its customer-recognized requirements. Many software shops equate their SQA function to a test function. This does not satisfy the concept of SQA in the SW CMM, where testing is viewed as an extremely valuable engineering activity, and the SQA role in testing is to examine the products and processes of testing to see if they comply with applicable test standards and procedures.

Remarkably soon, I was called into one of those luxurious mahogany conference rooms with the huge rectangular table and plush chairs. There sat the Software Development Manager, along with most of the pertinent managers on the contract. The Software Development Manager had my memo in front of him. He slid it across the table to me and declared, "You can’t write this!"

I replied, "Well, the information is true, and if the client's representatives should perform the same audit, it could be yet another embarrassment for us."

The manager folded his arms over his chest, rocked back in his chair and said, "We have a deal with the client regarding our library: They can only look at quantity, not at quality."

That was why the empty SDNs were on the shelves. They could only be counted, not opened!

Having an SQA function doesn't assure anything. The workforce will do what management recognizes and rewards. Management didn’t really care whether the SDNs were done, so few programmers actually did them. Having me there didn't really make a difference.

So what is SQA's role? The Software Capability Maturity Model (SW CMM) version 1.1 states: "The purpose of Software Quality Assurance is to provide management with appropriate visibility into the process being used by the software project and of the products being built."

Notice that this purpose definition does not include "test" or "enforce" or "establish quality standards." Yet in many organizations, SQA is viewed as the dreaded "Compliance Police"—those jackbooted thugs that march through the project, put their thumbs on the backs of the developers' necks, and make them do things they hate to do.

When I see this type of adversarial relationship, I usually find that the organization hasn’t internalized the need to do disciplined process stuff (we don't need no stinking process!), and they fool themselves into believing that by implementing an SQA function, somehow the process stuff will happen.

Let's look back at the purpose paragraph quoted from the SW CMM. Notice that it provides "management with ...visibility." So the presumption for an effectively operating SQA function is that management is interested in how the work is being performed. Because management is interested, and they don't have the time to look themselves, they rely on an SQA function to fill them in on how the work is performed. This is much the same way that management relies on an accounting function for cost data.

So who is responsible for quality? Management! And when management truly commits to a quality culture, all of a sudden everyone will, indeed, be responsible for quality. Then SQA can function as it was intended to do.

What Can Noncompliance Teach Us?Even when management does dance to the tune they are whistling, things don't always go as expected. Otherwise, it would make no sense to have an SQA function monitor it. Why might the expected standards, procedures, or processes not be used? The least encountered reason is that some practitioner simply refuses to follow the standard or procedure. If management is truly interested in the attainment of procedural or product standards, it would be career suicide to deliberately conflict with management desires. Most of the time when noncompliance occurs, there is some process reason for it:

There wasn't enough time to do it the expected way

Someone doesn't understand how to do it the expected way (lack of training)

Someone didn't know there was an expected way (lack of communication)

The process documentation didn't make sense (in this context)

The tool or other necessary support was not available

Others

To illustrate, imagine that part of your development process involves unit testing by the programmer—white-box testing before the code is delivered to integration testing. The SQA function, in examining a portion of a project, finds that some folks have skipped unit testing. The noncompliance is documented and the manager or supervisor closest to the problem is apprised of the situation.

There are three ways to resolve (close out) the noncompliance:

COMPLY. Have the developers go back and perform the missing unit tests.

ADJUST. What if the process was initially documented for large projects of 20 or more developers, and it is discovered that on a smaller (1 or 2 developer) project, later stages of testing cover the elements of unit testing? Ah! Learning has occurred! The organization should augment its process guidance to allow for this context.

WAIVE. Sometimes the customer wants the product really, really bad—so they get it really, really bad! What if the developer skipped unit testing because the customer demanded that the product be shipped without (adequate) testing? You are allowed to waive your process, as long as you document the waiver. But if you waive it more than you do it, I would question whether it’s really your process.

At the end of the project, when you perform a post-project review, you should look at the waivers and see if they made any difference. I would hope that if you waived all testing, you (or your customer) would find that it wasn’t such a good idea after all. But you may find that waiving a process, or process step, didn't seem to make much difference in the end. Time to re-examine the process to see if a contextual or universal modification is in order. To quote Frank Zappa, "Without deviation, progress is not possible."

Should SQA Be Independent?A conscience must be free to speak the truth. Does this mean that it has to be independent? There is a subpractice in the SQA Key Process Area of the SW CMM that says that SQA should have "a reporting channel to senior management that is independent of the project manager, the project's software engineering group, and the other software-related groups." Does SQA have to be a completely independent function to satisfy the SW CMM? No. The issue at the SQA goal level is one of objectivity. SQA needs to be able to safely tell the truth—to tell the emperor he has on no clothes!

One of the easiest ways to demonstrate objectivity is to have an SQA function that is not beholden to the project for his/her performance review, i.e., to make the function independent. But this creates what some organizations feel is unnecessary overhead. SQA is one of the most flexibly interpreted concepts in the SW CMM. I have seen it implemented successfully a number of different ways that don't require a substantial investment in a functionally independent segment of the organization.

For instance, one organization I know of implements the "buddy system" for all tasks. Each assigned task has at least two individuals involved—one who has the primary responsibility for completing the task, and someone else who has the responsibility for ensuring that the applicable standards and procedures are followed. A part-time management function spot-checks to make sure that collusion ("Esther, I know you think this standard is a bunch of crap and I agree. So you tell them I did it and I'll tell them you did it. Okay?") doesn't occur. Actually, subversive collusion is quite rare, but time pressures can create collusive shortcuts.

every project manager’s time (4 hours per week) to serve as the SQA function on another project. In this case, a middle manager spot-checks to ensure that no collusion is occurring. (Remember, management is supposed to be interested in how the work is being done.)

Several organizations use Peer Reviews to implement almost all of their SQA function. A staff position at or above the project management level serves as the subject-matter expert for Peer Reviews and monitors them for fidelity.

I have seen a couple of presentations declaring a "person-less" SQA function in which the process itself has built-in checks and balances to ensure that processes and standards are followed as the product moves through the development life cycle. While those who utilize this technique vociferously defend it, I would caution time/schedule-stressed organizations against trying this as a first implementation of SQA. I have visited organizations that try to perform a selfchecking process in "pressure cooker" environments, and have found that the time pressures were inevitably forcing the smart practitioners to work together in implementing shortcuts in order to meet aggressive deadlines. Many of the shortcuts involved skipping steps that later increased project and product risk. The end result is that the process the organization says it is following is not actually what is happening. But without a human-based conscience, it is so easy to either fool oneself into believing or lull oneself into complacency.

What’s the Bottom Line?Procedures and standards cannot always be religiously followed. Life is just not that perfect. Watts Humphrey is one of the prime movers behind establishing process rigor. But he reminds us that "rigor" is in the dictionary between "rigmarole" and "rigor mortis." Rigor, despite its connotations, walks the boundary between these states. However, you should be consciously aware of how the work is being performed—when it's done to expectations and when it's not. An improving organization is a learning organization. SQA can serve as a vehicle for that learning, but only if it is free to operate as the organizational conscience.

User Comments

1 comment

Robbie Bridgewater

Browserstack should suffice for almost any purpose you need it for, though it might be frustrating to use. If you do not mind the lag, or do not need to constantly use it, it may be the perfect solution.

Thanks for the comment!

June 28, 2013 - 11:14pm

About the author

With one of the oldest degrees in Computer Science (1971), W. Mark Manduke has spent over 30 years as a software developer, manager, SQA specialist, consultant, trainer, and assessor. He is one of the founders of Process Enhancement Partners, Inc., an SEI Transition Partner. Reach him at manduke@pep-inc.com.