At my firm, we don't really have a QA department. If you've followed some of my otherquestions you know essentially, we have no automated tests and our 'testers' are the analysts who design the projects.

When I recently brought up the idea of a more formal (I actually used the term 'industry standard') QA department, I was told "The decision was made to go to a lean development process." Turns out that decision was made about 7 or 8 years ago. Only in the last 5 months have I convinced them to move to a 2 week deploy cycle (it was 3-4 months before) and only in the last month are they cutting a 'build' that goes for the 'testers' to test. Also, any changes necessary are made to that 'build' (the code is able to be compiled or interpreted, so modification is easy.) The reason 'build' is in quotes is because it's really just a copy of trunk, with the most recent changes put higher up in the path so they get found first, and it all gets interpreted on the fly.

So, it seems there's a fundamental misunderstanding of what "lean development" is. They had a QA department in the past, but were bothered by their lack of productivity. Of course, we know that a QA department isn't 'productive'. If you look at it from a manufacturing point of view (which they do,) QA by definition hinders productivity by increasing the chance of failed and extended-time projects. (We've tried using the argument that QA for software is like QA for the products they manufacture. They don't see it the same way because there's government mandates for their QA, but not for the software.)

Maybe I missed this because I haven't seen your other posts but why do you feel you need a formal QA or more accurately a Testing Department?
–
Chris KenstOct 20 '11 at 21:28

I hope you don't mind that I converted your answer to a comment (since it's a request for clarification). To answer your question, because right now we have no one doing a real QA. We have analysts who go down the list of projects to make sure they still work, but no one tests the other aspects of the system. There's also no standards for testing, it's just whether or not they feel it "should be approved" or not. For a team with 12 developers and 8 analysts, we should have at least 2 or 3 dedicated QA personnel.
–
corsiKa♦Oct 20 '11 at 21:37

6 Answers
6

This has got to be one of the most common situations a developer faces in a company without QA. Since I'm a QA professional, and not a developer, let me try to explain it from the QA standpoint.

First off, QA productivity is very hard to measure. There isn't really an industry standard for how to do it, and it is very likely impossible. The return-on-investment, however, is a bit easier. QA are only human, so we'll miss 20% of bugs that exist on average. Now think of the issues you've had when the software went to production. Each software failure costs money, in time spent fixing it, in production halted, in lost revenue or somewhere else. Saving 80% of that money is always useful.

Second, bugs are always easier (read cheaper) to fix the sooner they are found in the cycle. A bug in design is 10 times easier to fix than a bug in code, which is much easier than a bug in production code.

Third, the metaphor I find that works for old-fashioned "real-industry" guys is that of HR or accounting. They're there to facilitate the core work of the business, even if they don't run the machines that make the products.

Aside from that, I like to ask someone if they'd drive a car with untested motor-control software. The software is improtant, otherwise you wouldn't be doing it.

+1 for fixing/finding the issues earlier in the lifecycle. Finding a bug at the developer level cost pennies to fix when compared to one found in production. In addition, once the bug comes up in prod, the cost is not always monetary; loss of company reputation, sales, trust - if the problem is notable enough, these make the dollar cost of QA look low...just ask Toyota.
–
StevenMay 11 '11 at 17:13

+1 for everything that I would have said but in less space and just as clear. Wish I could add another one for pointing out that we are only human.
–
Lyndon VroomanMay 11 '11 at 18:02

@Lyndon Vrooman: I have discovered that the next challenge after the one in the question is how to explain to management that QA occasionally miss bugs, in the same way that developers sometimes write bugs. Humanity is an imperfection we're all blessed with.
–
CarmiMay 11 '11 at 18:15

Maybe I'm too cocky around management, but that's typically when I ask management to explain how a bad business decision was made in a circumstance where they were not aware of an important business unit that it would hinder.
–
Lyndon VroomanMay 11 '11 at 18:38

How much of the developer time is currently being spent on maintenance activities? Skilled QA doesn't just find bugs, but also helps increase other aspects of code and development quality, so that developers are often freed from mundane, low-level activities that eat away at productivity. The result can be a huge increase in actual production (the word "skilled" is key here, though). Furthermore, having a QA department should make it easier for your company to hire better developers and retain current developers, since the best developers will be used to working with QA. The absence of QA could give possible hires the impression that your company is not a technical company, and doesn't take software development seriously. You could use the argument that all the big technical companies invest huge amounts of effort into QA (Microsoft, Google, Amazon), and why would they if there weren't an advantage?

But honestly, I get the impression that management might just not be listening to you.

What about your immediate manager? Is he technical enough to be persuaded? Could your team get away with "work-around" QA, where one of your devs takes a quarter or so to be an SDET, and the team rotates through this responsibility? This isn't as good as a dedicated tester who is focused on nothing but quality in their career, but might be better than the current situation. You will then have a person focused on finding ways the product can fail for 3 months or so at a time, and that is a huge improvement over only having people find ways to succeed and then investing a bit of effort into also making sure it doesn't fail. If one person on dev is test-oriented, maybe you could convince management that a shift from SDE to SDE-T permanently would be warranted after a few months of trying the arrangement.

The best argument you might be able to make is finding a better job, and then jumping ship unless your current company gets a real QA department - or just leaving, and telling them why. "Voting with your feet" is a valid option for employees. Of course, that depends on how strongly you feel about this issue and what other pros and cons there are to working at your company. I do get the impression that your company doesn't understand technology, but that doesn't mean that they aren't successful or that you are poorly compensated. Just make sure that you stay current enough to transition to a more technical company in the future, and don't let your skills stagnate! "Career death by poor management" is not unheard of in tech.

TL;DR: If management isn't listening, you can try working around them by making devs temporary SDETs or trying to move a dev to an SDET position permanently. Otherwise, you might want to consider leaving for a more technically-savvy company and letting them know why.

Use facts and data to backup your claim as to why you need a QA department. One thing on your side is the fact that you release every 2 weeks. One thing to begin tracking is the number of defects that occur per release, and also how many other releases/fixes do you have to deploy if you do find a bug.

When you can factually explain faster development cycles and higher quality your case becomes a little easier, but it's still a tough road to hoe.

+1 because sticking to facts goes a long way with me. There is not much else I can add glowcoder to all this great advice you are getting...except possibly to find out what the competition is doing and if they have a QA team. (Sometimes I think we work at the same place.)
–
Laura HensleyMay 11 '11 at 19:04

It seems that management just does not see the point of dedicated QA since they don't see lot of serious bugs in the final release. Exposing these bugs, showing basic bug statistics, conducting end-user satisfaction survey have a chance to convince them.

Unfortunately, we don't have access to the end users. 7200 users, 10 devs, and all communication to them is funneled through BAs, and then plant managers. They actually see many bugs: their solution is to report it, open a new project, and fix it. About 80% of the projects, which take about 50% of the time, are 'bugs'. Of course, a 'bug' is defined as 'when it doesn't do what we want.' So, if they requirements change, it's now a bug. You could say it's an uphill battle but surely I'm not the first person to go down this path before. :)
–
corsiKa♦May 11 '11 at 17:09

Have you thought about expressing your desire/request in terms of quality and costs to the business, i.e. how much it costs (time+money) to set up the large number of projects versus the costs for having a dedicated Q.A. team?

Also is it possible you can collaborate with the B.A. team to publish the costs (time+effort) needed to rework the requirements once the product has gone into production?

What I'm thinking of is can you "sell" the idea of doing preventative measures rather than curative actions?

Development has been trying to 'sell' it to BA for about two years now. It only recently has started to gain any traction. Judging by past experience, it would actually hurt relations to suggest that. For example, the BA-lead used to share my office when he came by for the bi-weekly meeting. I asked him his thoughts on the BAs doing the testing as opposed to a QA team, and he said there's nothing wrong with BAs doing testing. He also hasn't come to my office since. shrug
–
corsiKa♦Oct 19 '11 at 15:49

Is there any value in speaking directly with the plant managers and selling the idea to them? Also is it possible you could speak with the actual budget holders in your company and explain the benefits to them? Looking back at your original post could you pull some metrics out of the 2 week drop cycle you are working within and then massage those metrics to show how a dedicated QA team/department could be used to make the development even leaner, i.e. leave the developers and BA's enough time to concentrate on what they are supposed to be doing?
–
devonpsOct 20 '11 at 7:38

I have worked in product development and quality assurance in three different industries.

Justifying QA is an education and organizational maturity issue. The value of dedicated QA varies depending on the size of the team (specialization of roles) and the risk associated with problems in the software. Some teams cannot justify it. Every project is unique.

In my experience, the need for dedicated QA comes down to two things:
1) Perspective and Risk.
It is easy to say everyone should check their own work, however this is a utopian construct similar to saying your kid should grade their own math homework. Developers are by nature of their work are almost always removed from the user and product perspective that is critical to good QA. The nature of their job requires them to focus on drilling down into the detail of complex logic problems. If they are providing good code and good throughput, most developers will not be able to also do good QA. Most people can indentify with writing a research paper or thesis, and how when you come back to it after doing something else for several days you to see significant issues in your writing. Developers do not have the luxury of letting code sit around for a few days while they shoot hoops and then look at it with "fresh eyes", and they rarely have the perspective of the end user or client. It isn't even fair to ask them to do their own QA. They are super smart people, but they are not magicians. If the risk of problems in the software is important to your company/product/clients, then how QA is executed needs to be an ongoing discussion during development.
2) Turnaround time and build cycles.
If you want rapid iterations of releases and builds, you will need simultaneous development and testing. It is not possible to code, QA, fix, and QA fixes with fast turnaround without some specialization of roles.

Who should do it then?
A good transition step to a full staff allocation for dedicated software QA is to combine it with a business analyst or product role. Business analysis and testing are two sides of the same coin. Verification of can be done by most anyone with some distance from the code and familiarity with the SDLC of the team, but true validation requires someone on the product side that has the perspective of the end user at the forefront of their mind. Bringing someone from the product side into close collaboration with a software team is a beautiful thing. This person really needs to decide to become an extension of the development team, and learn the way things work on the development side. A good QA person winds up doing a lot of BA in reverse anyway, just to model information to make things testable and consumable by developers.

A last thing to consider is "testing" vs. "QA". Testing is a task, is more easily outsourced, and depending on the work can be completed by developers with some success. On the other hand, QA is a profession/career, and incorporates a lot of testing but takes on a lot of holistic perspective. If you have ever worked in a regulatory driven, manufacturing, healthcare, or financial organization you know QA is a whole profession even without a line of code. In quality professions this is often articulated as quality control vs. quality assurance, and is worth a 10-minute Google tour of the subject matter. Calling QA "testing" really takes it down a bunch of notches, but it might be the best way get your foot in the door with your management team. The value of testing is more tangible to them.

Steps (may take multiple iterations, perhaps over months or a couple years:
1) Define the requirements or user stories or their equivalent for each software iteration. Leverage your business analyst if you have one. This is actually far more valuable than dedicated QA.
2) Define QA deliverables to ensure these are met and divvy them up across the team so the deliverables themselves become justified, regardless of who does them.
3) Engage management in discussions about intelligent use of resources based on skill set. Many managers figure out they need to keep their developers coding and it is easier to have someone else do the QA. After a while, in many environments it becomes clear you need someone who is dedicated to the profession to do a really good job. Your project and company will have unique needs, and these will become clear over time.
4) Sometimes the company culture or the education barrier just cannot be overcome. If you are passionate about good product, and your management team is unwilling work toward that end you might be in the wrong company. Software has an intangible quality, and many companies just can't wrap their heads around the unique challenges in the industry. If you find you are speaking a different language and people are not understanding you, you may have outgrown your environment and need to move on.