How to Stop Putting Out Fires and Solve IT Problems Fast

Problem-solving in many university IT departments can feel like a game of Whack-a-Mole--rushing to address random incidents with no time to investigate the bigger picture.

Kriss Dinkins knows exactly what that's like: As director of knowledge and service support in Information Systems at Wake Forest University (NC), he is responsible for customer support through an onsite service desk, phone, web, and self-service. He describes an example of how IT problems were dealt with a few years ago: When the school distributed laptop computers to freshmen as part of a ubiquitous laptop program, IT staff in residence halls started seeing profile errors. Eventually they realized that the problem was more widespread than a few machines and they created emergency work areas to address the profile errors. "We scrambled and got through it, but because no one team owned this problem, it took longer to coordinate our efforts and get a handle on it," he recalls.

After years of studying its IT processes, the university decided to make a concerted effort to streamline the way it handled tech-related issues. "We realized that when we had a problem related to technology, we weren’t necessarily efficient about tracking the number of incidents and who was in charge," explains Dinkins.

Wake Forest has gradually adopted the ITIL (Information Technology Infrastructure Library) problem-management approach, which is different from incident management in that it seeks to identify trends across incidents and create a knowledgebase for corrective or preventive actions. Service desk pilots in early 2012 led to a yearlong problem-management implementation, the establishment of a problem advisory board, and the funding of a permanent problem manager position.

To help guide its change management process, Wake Forest turned to a former employee who now works as a consultant. John Borwick, the founder and manager of Higher Education IT Management, worked in several positions at Wake Forest from 2003 to 2012, including director of service management and director of the portfolio management office.

"Part of our focus with ITIL problem management is to help find root causes for tickets and particularly to try to encourage people to identify problems," says Borwick, who led a series of "Building a Problem-Solving Culture" workshops to help the university's IT leaders learn how to identify, understand, and solve problems. He stresses that technical issues are easier to think about because they are more black-and-white. "Process and political issues are tougher because they don’t have black-and-white answers, but we want a culture where identifying them is encouraged," says Borwick.

Now that the new problem-management approach is taking hold, incidents can be resolved more quickly. For example, last summer, teams distributing laptops to faculty and staff started seeing random "Blue Screen of Death" errors. These groups reported to the problem manager, who discerned there was a bigger issue. "Problem management raised the visibility of this issue with the leadership team," Dinkins points out. An investigation led to an escalation with the vendor, which identified a hardware issue that required two weeks of on-site repair. A laptop refresh for juniors was delayed until the vendor repaired computers.

"We were able to come up with a better solution," Dinkins notes. "Although only 3 percent of users reported the problem, the new process helped us measure the impact, identify the root cause, and start looking at the vendor," he says. "It created a venue to escalate it to management and to the CIO."

Borwick and Dinkins say the clear support of Wake Forest CIO Rick Matthews and Deputy CIO Nancy Crouch has been crucial. Matthews has regularly stressed one of the values underscored in problem management: "Don’t look for blame," Borwick recounts. "It is one thing to believe these things and another to translate them to action."

After the pilot period ended in September 2012, Wake Forest made the decision to move forward and invest in the full-time problem manager position. The person the university hired, John Vientos, has plenty of service desk experience, Dinkins says. "We needed a systemic thinker who understands the importance of process and how to get people working together, how to prioritize and not fix obscure problems. He has to identify priority problems and figure out which resources to apply."

Dinkins has 30 employees in his group and the centralized IT operation employs 108 people. Besides the problem manager, several other roles contribute to the problem-management process: problem champions, who are advocates within an IS team; problem specialists, who are subject-matter experts; and problem coordinators, who are point people for the problem.

Borwick says not every organization is ready to move directly to problem management. Some universities focus just on processes and haven’t developed any way to document their issues. "They need ownership of the process to be effective, so they can engage all the people on their team. Wake Forest has done a good job fitting processes to people and was in the right place with its leadership team."

According to Dinkins, the general response to the approach from staff has been a widespread "wow." People are excited to work on a team, and appreciate having someone to shepherd the process and help with communications, he says. "But I think we will have to keep it high profile and have it recognized at staff meetings to keep the ball bouncing," Dinkins says. "It is still new."

About the Author

David Raths is a Philadelphia-based freelance writer focused on information technology. He writes regularly for several IT publications, including Healthcare Informatics and Government Technology.