Artificial Intelligence (AI) and Machine Learning (ML) can perhaps solve some testing challenges, but not all testing. The testing vs. checking debateand all the shift-left of checking, have revealed that some of testing is about critical thinking and some…

The hyped mnemonic “DevOps” is equally true the other way around: OpsDev – that is, more and more work in the operations and infrastructure departments happens as development activities with scripts, code repositories and build managers. OpsDev is as tool-heavy as DevOps, and test involvement similarly pipeline focussed.

]]>https://jlottosen.wordpress.com/2017/11/04/opsdev-more-dev-work-by-ops/feed/0jlottosenTeaser for Experiences in Testing Infrastructure Projectshttps://jlottosen.wordpress.com/2017/10/23/ukstar2018teaser/
https://jlottosen.wordpress.com/2017/10/23/ukstar2018teaser/#respondMon, 23 Oct 2017 18:07:16 +0000http://jlottosen.wordpress.com/?p=2562Continue reading Teaser for Experiences in Testing Infrastructure Projects]]>I’m presenting at UKSTAR 2018 on the Topic: Experiences in Testing Infrastructure Projects. The content is a continuation of my materials and talks about testing outside the SDLC, in this context operations and infrastructure. It’s additional problems and examples compared to “How to test in IT Operations” at Nordic Testing Days 2016.UKSTAR 10% OFF ANY TICKET WITH CODE

The talk is in the same field as this talk – by Mike Talks @testsheepnz although with no applications on top… and other stories

Advertisements

]]>https://jlottosen.wordpress.com/2017/10/23/ukstar2018teaser/feed/0jlottosenUKSTAR 10% OFF ANY TICKET WITH CODEA 30 Days Agile Experiencehttps://jlottosen.wordpress.com/2017/10/04/30-days-agile/
https://jlottosen.wordpress.com/2017/10/04/30-days-agile/#respondWed, 04 Oct 2017 11:28:39 +0000http://jlottosen.wordpress.com/?p=2488Continue reading A 30 Days Agile Experience]]>In September 2017 the Ministry of Testing had a crowd-based knowledge sharing event called “30 Days of Agile Testing” with a small learning activity for each day of the month. As with the similar security event I set up a weekly schedule at work to meet for an time-boxed hour and discuss 3-5 selected topics each time.

Our score was 17 topics discussed – some more discussed than actually tried out. Hence the half marks on the poster on the window below. Me and my coworkers work on many different teams – so to dig into specific team tools and processes was out of scope.

Here is a few of our findings:

Day 3 – YouTube is full of videos on agile, but our Chinese colleague cannot view them. They can though view videos locally on the Dojo and TestHuddle.

Day 12/26 – We need the test cases in many contexts, but do we need the Test Plan specifically. Perhaps we can reduce the test plan to “Requirements/scope”, “test cases” and “staffing” as in Plutora Test.

]]>https://jlottosen.wordpress.com/2017/10/04/30-days-agile/feed/0jlottosenIMG_0007Less Test Managers, More Test Coacheshttps://jlottosen.wordpress.com/2017/08/31/less-test-managers-more-test-coaches/
https://jlottosen.wordpress.com/2017/08/31/less-test-managers-more-test-coaches/#commentsThu, 31 Aug 2017 11:24:37 +0000http://jlottosen.wordpress.com/?p=2351Continue reading Less Test Managers, More Test Coaches]]>One of the trends/shifts I experience in testing & test management in particular is the Test Coach as discussed initially here: The Shift-Coach Testing Trend (Oct, 2016). Recently (Aug 2017) it came up again in a Twitter thread, where Stephen Janaway stated the inspiration to the title of this blog post.

Less Test Managers and more coaches. That’s how I see it going.

Fittingly as he inspired the first post with his talk “How I Lost My Job As a Test Manager” presented at Test Bash 2015. This post is a further elaboration of the Shift-Coach test management trend. Here are some of my experiences:

I have been assigned to an agile development team to introduce them to 3 Amigos, Test data driven test automation and such things. The purpose of my involvement was to enable the team to continue the practices without me, and without testers besides the business analyst / product owner (See The domain expert is the tester) as they are doing Shift-left. Similar to an agile or scrum coach, my approach was to look at it as a change in the way of working.

Another project is an infrastructure project, there are no testers only technicians configuring Cisco routers that by software can replace firewalls, iron ports, VM servers and other network equipment. The project has to implement 80+ of these, so I setup both a test process and an ITIL change request process acting as a test and release manager – another quite frequent trend. I could continue in the project for the duration, but instead I setup guidance and leave when it’s sufficiently in place.

This might be similar to a test architect, a (internal) test consultant activity. It has nothing to do with diminishing testing. Rather I see it as more testing happening, something that would not have been done without the coaching from a test manager. It’s all about finding a test approach that is fit for the context.

The competence of the test coach is to have enough change management expertise (people skills) and test management expertise (domain skills) to know how to coach and facilitate the change. Should test coaches test too, perhaps when required, but not necessarily. The activity is primarily to up-skill the team to continue on their own.

The “Test Coach” is a trend similar to “shift-left” and all the other shifts in testing and test management. I see it as a pattern, and what I read from the threads and discussions is that many test managers gradually shift towards test coaches.

Advertisements

]]>https://jlottosen.wordpress.com/2017/08/31/less-test-managers-more-test-coaches/feed/52017-07-03 13.57.42jlottosen2017-07-03 13.57.42Don’t request the kitchen sinkhttps://jlottosen.wordpress.com/2017/08/23/andthekitchensinktoo/
https://jlottosen.wordpress.com/2017/08/23/andthekitchensinktoo/#commentsWed, 23 Aug 2017 09:10:19 +0000http://jlottosen.wordpress.com/?p=2325More and more often I see outsourcing contracts that requests 10-15 test phases. It looks like someone has simply thrown the book at it, and not considered if it is an infrastructure project, a software development project or COTS implementation or – what on earth, they actually want to learn from the testing.

These are the phases of a recent project:

Environment Acceptance testing

Hardware and integration testing

Component testing

Component- integration testing

Installation test

System testing

Functional testing

Regression testing

Security testing

Performance testing

Operational acceptance testing

Service Level testing

It’s a challenge in the vendor reply. The vendor will want to reply to all test phases in order to be compliant with the tender, and not loose points. There is no room for elaboration or discussion if you want in on the bid process.

Quite often the requester comes back and say “we didn’t want all those weird testing things, we just want something that works for us”. But when the contract is signed and the work set in motion the project team have challenge to make the testing practical within the framework of the contract. This goes from both sides. Many good hours can be wasted with unwinding cumbersome contractual terms.

What I usually do in such a situation is to bundle the contract’s testing scope into fewer activities, and setup a mapping so that everything is covered. That is if the client allows me to make the activities practical and context-driven. If not – my hands are tied, and we deliver according to spec – even if the chapters of the test plans are set in stone.

Let’sworktowardsbetterdealsfortestingactivities. If you are looking to prepare a BID include a test manager – and have a discussion of the value-add and learning of testing up front. There is no one book of how to do testing. Instead spend the time and money figuring out your context. Figure out what phases are on the client side, and what is on the vendor side. Have a test management consultant on retainer for before and after the bid process. Do something to discuss your test strategy and put the guidelines in the contracts, so that the vendors can propose a solution.

Don’t request everything and the kitchen sink too

Everything and the kitchen too

Advertisements

]]>https://jlottosen.wordpress.com/2017/08/23/andthekitchensinktoo/feed/1Everything and the kitchen sinkjlottosenEverything and the kitchen sinkTest ALL the thingshttps://jlottosen.wordpress.com/2017/07/27/test-all-the-things/
https://jlottosen.wordpress.com/2017/07/27/test-all-the-things/#commentsThu, 27 Jul 2017 14:32:06 +0000http://jlottosen.wordpress.com/?p=2292Continue reading Test ALL the things]]>TL;DR: We can add testing to all requirements and all business risks. Testing to document requirements and to debunk risks provides valuable information for the business. Let us not limit testing to things that can be coded. The intellectual activity of trial and learning is happening anyways, we might as well pitch in with ways to find important evidence for the decision makers.

Test all the requirements

Traditionally testing was all about testing the functional requirements that could be coded. Non-functional requirements was left for the specialists, or plainly disregarded. I know I have done my share of test planning, with a range of requirements left “N/A” with regards to testing. Especially performance scope, batch jobs, hardware specs, data base table expansions and virus scanning have been left out of my functional test plans…

When I look at a list of requirements now – I see that we can indeed test all the things, or we can at least work on how to document that the requirement is fulfilled. Some of the requirements are actually quite easy to document. If it’s on a screen somewhere, take a screen shot and attach it to a simple test case. Done deal really. Additionally with a testing mind-set I can think of ways to challenge the details. But do we really, really need to fill up a disk to establish if it’s exactly a 1 Gb allocation – probably not. Do we really really need to document all requirements – yes in some contracts/contexts it’s important for the customer to know that everything has indeed been established. Sometimes the customer doesn’t trust you otherwise, sometimes the tests are more about your ability to deliver and provide evidence that matters.

Test all the business risks

Look into the business case of your project and find the business risks. Sometimes they are explicitly stated and prioritized. A a recent Ministry of Testing Meetup we looked into a case for a large national health system. We looked at the tangible benefits, intangible benefits and on the scored business risks. What worried the business and management most was budget, time and whether the new system would be used in a standardized way. There is an opportunity for testing here to help address, document and challenge the most important business risks. Traditional testing would usually look at functional requirements that can be coded or configured, and miss totally to address what matters most to the business.

OK, how do we test the project costs? How about frequent checkpoints of expected spending – would that be similar to tracking test progress. Perhaps – let’s find out. Testing is all about asking questions for the stakeholders and solving the most important problems first. Can we help to analyse risks and setup mitigation activities – sure we can. We just have to step out of our traditional “software only bubble”.

]]>https://jlottosen.wordpress.com/2017/07/27/test-all-the-things/feed/6MEME - Test ALL the thingsjlottosenMEME - Test ALL the thingsThe domain expert is the testerhttps://jlottosen.wordpress.com/2017/05/30/the-domain-expert-is-the-tester/
https://jlottosen.wordpress.com/2017/05/30/the-domain-expert-is-the-tester/#commentsTue, 30 May 2017 14:03:52 +0000http://jlottosen.wordpress.com/?p=2183Sometimes the best tester is the domain expert, the person that knows all the in’s and out’s and corners of the system. I have worked with testers that have had hands-on on a system since the late 1970’es, but I also know testers of mobile app’s that marvel in being the subject matter expert of the domain. Sometimes the professional tester doubles as agile Product Owner(1) too given her vast knowledge. The tester becomes the the SME …

The UAT is not dead, but the classic role of the tester testing on behalf of the business is declining. The business would rather test their own, with in-house subject matter experts. The field is active, as there is tool support for this activity. Panaya(2) is a tool that specializes in managing the UAT of a corporate system like SAP, and one of the key elements is that test cases can be broken up in steps and handed over between persons. Not even classic HP ALM’s handle hand-over between testers well. While ALM’s support that the tester does the testing, Panaya supports that tests are distributed across many people. People that have other (“real business”) tasks during the work day.

Testing can also be pushed even further out to the users with crowd-based testing, beta releases etc. In both crowd-based and UAT-based testing, the role of the pro’ tester is missing but the testing is still happening. IT’s being done by the most skilled – most valuable for the task.

So what can we as testers do when our tasks are gone – skill up, go with the change and become the expert – or move out to other skills: Coaches, delivery leads etc.

The expert says 5 pieces should be in the build – though the customer is OK.

this post is not sponsored. I’m just making observations – not recommendations.

Advertisements

]]>https://jlottosen.wordpress.com/2017/05/30/the-domain-expert-is-the-tester/feed/32015-12-18 13.49.28jlottosen2015-12-18 13.49.28Testing across the IT landscapehttps://jlottosen.wordpress.com/2017/05/24/testing-across-the-it-landscape/
https://jlottosen.wordpress.com/2017/05/24/testing-across-the-it-landscape/#commentsWed, 24 May 2017 14:43:21 +0000http://jlottosen.wordpress.com/?p=483Good testing is much more than confirmation of explicit requirements. It’s figuring out the implicit requirements, what blocks the business and what drives the business. Businesses are not driven by SDLC’s but by decisions and strategies. IT road maps are just a representation of the business strategy, an engine to build business solutions on. The is much more to the business than the software solutions and related risk mitigation.

Very often the biggest business risks are outside the project scope. When we look at it this way we see that testers and testing activities has an opportunity outside the classic project life cycle. Testing is about experimenting with a IT solution, to evaluate if it fits the business requests. IT solutions that supports the business exists in many forms, I am certain that explicit testing (*) can add value in other parts of the IT landscape.

Here is a model of an enterprise IT landscape consisting of business ideas, solution development, operations and end user devices + support. Solution delivery is boring in the sense of well-known software creation and maintenance. What if the item under test and the requirements are around network, servers, end-user devices and IT support tickets. I am certain that it’s valuable to test business cases before the project is even formally assembled.

*: implicit testing in the form of critical evaluation always happen. .. similarly does checking.

Advertisements

]]>https://jlottosen.wordpress.com/2017/05/24/testing-across-the-it-landscape/feed/1modeljlottosenCreate, Maintain, Move and Closehttps://jlottosen.wordpress.com/2017/05/19/cmmc-application-cycle/
https://jlottosen.wordpress.com/2017/05/19/cmmc-application-cycle/#commentsFri, 19 May 2017 10:41:53 +0000http://jlottosen.wordpress.com/?p=2132Usually when we talk testing it’s about the road to production. It’s about getting requests from the customer/Product owner and shipping it. We tend to forget that there is more to the life cycle of application than adding to the pile. Inspired by the old CRUD I identify the following stages in the application life. Create, Maintain, Move and Close. Testing can add value all of the four modes, with twists for each one.

Create: “oh shiny” – Creating an application is usually novel, but the more times you have build similar applications it becomes routine. Some applications have to be build from scratch, others merely configured. It matters a lot if you are building a unique app – or if it yet another roll-out of a COTS application. The testing in “create” usually focuses on bugs to be fixed before go-live, and very little on what happens afterwards. Building a new application is usually a strategic decision to the business that solves a problem or builds on a potential. Requests are numerous for new things.

Maintain: “ship, but don’t destroy production” – At some point the customer sends you more requests to the stuff already build than new features. Application maintenance is all about balancing new features and updates to existing features. Existing features are being used by the end users, and they will eventually request updates and bug fixes. Fragmentation, merging and branching becomes and issue – especially if you maintain the application as a solution for a range of customers. Customers might want to differentiate between their requests – as they won’t want to pay for bugs in previous releases, but rather want to pay for new additions.

Move: “It has to work as before, just with a new team“. To many businesses maintaining an application is not their key area; They might be a public organisation with no need to have their own staff of developers. So “Application Outsourcing” becomes a thing for many applications, and with deals being won and lost – it will happen that the development tasks moves from one supplier to another. Testing can make sure that processes are in place in the new location and that the state of the application is known in the new location. The testing during “move” doesn’t involve the functionality of the application, but rather the ability of the new team. Sometimes the hosting of the application stays they same, in other cases it is the hosting of the application that changes and nothing else.

Close: “test that it’s gone” Sometimes IT strategies and businesses decide to close down an application. Perhaps it’s being replaced, perhaps it’s redundant after a long time. Examples could be hospitals moving from one patient journal to another or whole systems being sunsetted. It could also be the closing of end-of-life applications (Windows servers, HPQC etc). The value to the business is knowing the application is gone, and the information in the old systems not trusted anymore.

It is very much possible to have testing in all modes of the application life cycle. Similarly it is very much possible for testing to add value in all stages of the software development life cycle. It’s a matter of perspective.