I'm permanently stuck with the following trouble: being Senior QA I need to quickly check and hire people for short-time testing projects. I specialize mostly in manual testing, while the range of applications is practically unlimited: web portals, standalone apps, educational resources, etc.

So, the problem is: there's a lack of quick (not more than 8-10 hrs) and easily available tasks for staff evaluation. It would be nice to have 5-10 typical testing apps for checking various aspects of QA specialist skills:

Are you talking about using these in the hiring process before extending an offer?
–
Sam WoodsJan 28 '13 at 17:44

@SamWoods well, not exactly. Most often I have a list of currently available QAs from my company, which may be added to starting project. However, I have a list of requirements / skills for the particular project, and I'd like to choose people based on the results of such test tasks.
–
Peter L.Jan 28 '13 at 18:19

Don't quite understand the problem, those seem like general testing skills, don't your pool of testers have these ?
–
Phil KirkhamJan 28 '13 at 18:52

@PhilKirkham well, not all, and not all QAs have enough skills that I especially need for one particular project. Preparing nice test cases for web apps not the same that for standalone ones - as an example.
–
Peter L.Jan 28 '13 at 18:56

6 Answers
6

You said "here's a lack of quick (not more than 8-10 hrs) and easily available tasks for staff evaluation. It would be nice to have 5-10 typical testing apps for checking various aspects of QA specialist skills".

Other than "Create them", I'm not sure what kind of advice we can offer?

I've created "buggy" applications in the past, more as "fun" exercises for my team rather than some sort of skills evaluation. You could do the same (if you are a capable developer with some time on your hands), or you could have someone else do it for you.

Far better to have someone build them for you, or find a tool you could use to build your own. That way, you can use apps that are actually "typical" of your shop, and your staff can't search the vast interwebs for solutions.
–
Joe StrazzereJan 28 '13 at 21:10

It seems to me that if they are currently available QA's at your company, you should have some understanding of their skills, or the ability to find out. Would it not be easier to discuss what you are looking for with their previous leads, or directly with them and see if their skills match up with what you are looking for? Perhaps you are over-complicating it? A day's work seems like a lot just to try to figure out something your company should already know about its employees.

Are you running into a specific issue, where QA members come to work with you on a project and are unable to fill the role requirements? What kind of screening process do you currently have in place?

The problem is that we have many newcomers, that are not very skilled / experienced yet. Perhaps that's my personal approach, but I don't trust words much - better try it out on my own) Maybe because of that "feature" colleagues say I'm a brilliant QA)))
–
Peter L.Jan 28 '13 at 19:02

3

So could you spend your time training them to be better rather than testing them to see if they match your standards ?
–
Phil KirkhamJan 28 '13 at 20:18

@PhilKirkham sure thing we do invest in our staff, but even for that we need something to check level and reveal weak points. Moreover, sometimes TIME really matters, and I need to allocate proper resources in a matter of days - if not hours.
–
Peter L.Jan 29 '13 at 11:49

If you build on-premise software that includes an installer, it should be possible to identify past builds with defects, install the build on a machine somewhere, and then get the tester to re-execute a set of regression scripts, or do some exploratory testing, with the aim of then a) seeing if the find the known bugs (or bonus points for other bugs) and b) interviewing them afterwards to determine the process they went through in finding those bugs, and then what information they captured about them to relay to the rest of the team.

Alternatively if build web applications and doing continuous deployment, adjusting your development process to make it easy to deploy old/different builds also works really well for this.

In the .Net community the use of octopus deploy (http://octopusdeploy.com/) is gaining traction - this means you could retrieve old builds and then deploy them to a specific QC environment, where you could then "test" or train the tester on finding those kinds of issues + what diagnostic information to capture about them, so that wider team can reproduce/resolve the issue.

Working against real issues (preferably those which have since been resolved) means you can review the issue history/comments on the issue as well with them, to show how the issue was then progressed, resolved and re-tested.

As a developer I find this approach works well with up-skilling new developers on a team into how bugs are discovered, selected for fixing and re-tested too e.g. checkout an earlier branch, have a go at fixing the issue, then review what the actual commit for fixing it looked like, and how the fix was then retested etc. (which is normally all in the comment history of the issue in your defect tracker, hopefully!)

This approach is more suitable for staff training, but not for quick skills evaluation. We use almost the same techniques as a long-time staff improving strategy, but I need here smth different. And, training QAs is not just the same as training DEVs)
–
Peter L.Jan 30 '13 at 23:20