Question: When Is It Acceptable To Patch QA Environment Ahead Of The Production Environment?

A buddy of mine is head of information security at a large insurance company, and we were talking about a common area of passion for us: implementing controls in pre-production.

He told me about an argument that came up between him and his QA manager. This QA manager was already getting harassed by the rest of the business for delaying needed software releases and needing more QA engineers each year to keep up.

To make matters worse, they found out that one of their pre-production environments was accidentally patched ahead of the production environment. Now they had to figure out what to do about it.

My buddy had made the following argument:

“My job is to protect the shareholders. A bunch of us, including the QA manager, are always complaining about how the Development and QA environments were always too lax and open, and that we needed to lock them down. That’s why we need to patch them — it just doesn’t feel right to rollback the patches. We should keep the QA systems as is.”

After my buddy told me this story, I shared my opinion, but wasn’t able to verbalize why it was true. But over the weekend, I couldn’t help thinking about this problem. And then I remembered about a problem that I’m working on in two of the largest Internet companies, and suddenly the correct answer was obvious.

On Twitter, I posted this question, with a Visible Ops book to the best answer.

“When is it acceptable to patch the QA environment ahead of the production environment?”