Objectives: To demonstrate the feasibility of using process mining concepts, techniques, and tools to examine and improve the systematic review process. Study Design and Setting: We conducted a simulation study evaluating a process used by one research team over one year. The process was characterized using an event log of review activities, start/end dates for review tasks, reviewers, and person-hours spent on tasks. We obtained process models from mining event logs for visual display/animation/replay of review activities. We analyzed the social networks of reviewer interactions to discern how reviewers worked together. Key outcomes included review timelines and person-time. Results: The 12 reviews included in the study included an average of 3831 titles and abstracts (range:1565-6368) and 20 studies (6-42). The average time was 463 days (range: 289-629) [881 person-hours (range: 243-1752)] per review. The average person-hours on each step were: study selection 26%, data abstraction 24%, report preparation 23%, and meta-analysis 17%. Social network analyses showed that the team handled tasks according to their expected roles (e.g., methodologists developed review questions, librarians conducted searches, and review coordinators coordinated tasks). Conclusion: Process mining is valuable for review teams interested in improving and modernizing the conduct of systematic reviews.

Bibtex Entry:

@article{jce2018,
author = {Ba' Pham and Ebrahim Bagheri and Patricia Rios and Asef Pourmasoumi and Reid C. Robson and Jeremiah Hwee and Wanrudee Isaranuwatchai and Nazia Darvesh, Matthew Page and Andrea Tricco},
title = {Improving the Conduct of Systematic Reviews: A Process Mining Perspective},
journal = {Journal of Clinical Epidemiology},
url = {https://www.journals.elsevier.com/journal-of-clinical-epidemiology},
year = {2018},
abstract = {Objectives: To demonstrate the feasibility of using process mining concepts, techniques, and tools to examine and improve the systematic review process. Study Design and Setting: We conducted a simulation study evaluating a process used by one research team over one year. The process was characterized using an event log of review activities, start/end dates for review tasks, reviewers, and person-hours spent on tasks. We obtained process models from mining event logs for visual display/animation/replay of review activities. We analyzed the social networks of reviewer interactions to discern how reviewers worked together. Key outcomes included review timelines and person-time. Results: The 12 reviews included in the study included an average of 3831 titles and abstracts (range:1565-6368) and 20 studies (6-42). The average time was 463 days (range: 289-629) [881 person-hours (range: 243-1752)] per review. The average person-hours on each step were: study selection 26%, data abstraction 24%, report preparation 23%, and meta-analysis 17%. Social network analyses showed that the team handled tasks according to their expected roles (e.g., methodologists developed review questions, librarians conducted searches, and review coordinators coordinated tasks). Conclusion: Process mining is valuable for review teams interested in improving and modernizing the conduct of systematic reviews.},
webpdf = {http://ls3.rnet.ryerson.ca/wiki/images/f/f1/Jce2018.pdf}
}