Discovering Processes In Unusual Places

Since 2008, I’ve spent many September days kicking off the fall conference season by attending the International Conference on Business Process Management: the BPM research conference hosted by a different university each year, and featuring academic papers and tutorials presented by professors, graduate students and researchers. This year, the conference was held virtually (we all missed that trip to Sevilla) and I attended many of the sessions even though they started at 4am in my time zone. Although it’s not quite the same as being there in person, the advantage is that all presentations were recorded and are available for replay.

I like hearing the new research ideas, and although some of them will probably never make it beyond the walls of academia, I always see a few interesting presentations that I know will impact commercial solutions in years to come. This year was no exception: I saw three presentations that brought new insights to process mining, and some ideas about how it might evolve in the future. In my August post, I provided an overview of process mining and why I think that this type of data-driven analysis is a critical skill for business process analysts.

Process mining traditionally works on structured log files from transactional systems, such as ERP systems, where activities within a process are well-defined but the order of the activities is unknown. What if, however, the activities are not well-defined, and are embedded in the content of email messages rather than transactional systems? That’s exactly the problem that Marwa Elleuch of Orange Labs (Paris) is tackling, along with several colleagues, and she presented their paper on “Discovering Activities from Emails Based on Pattern Discovery Approach” at the conference. In short, they are using pattern-discovery techniques to find activities and related business data within the text of email messages, and using these to create a structured event log that can then be analyzed using traditional process mining tools. Any one email (or chain of emails) can be related to multiple activities and multiple instances/cases; in fact, a single sentence in an email could relate to multiple activities and instances.

She showed their research approach (shown in the diagram above) and their paper in the conference proceedings covered the mathematical details of their work. To test their approach, they used a large public domain set of email messages: a half-million messages collected from Enron during the US Federal Energy Regulatory Commission’s investigation. The results look promising, although this is still early days and more research needs to follow: compared to manual review of the emails, their algorithms were able to discover a good percentage of the activities and related business/instance data. The same methods could be applied to other unstructured communications such as text or chat messages. It’s not perfect, but consider that it’s doing unsupervised activity discovery without prior knowledge, this has incredible potential.

Most of the companies that I work with have so many of their internal processes buried in email, performed in an ad hoc manner; in fact, when I visit a large operations area for the first time, I tend to look for “email and spreadsheets” as the location of many uncontrolled processes. Imagine if you could have an algorithm track through your company email messages and suggest processes to be standardized and automated? That would bring the power of process mining to a previously manual, time-consuming analysis task. It would be interesting to see what the addition of AI techniques to these researchers’ pattern discovery techniques could do to gather even more information from the unstructured text of email messages.

Another process discovery-related presentation I found interesting was “Analyzing a Helpdesk Process through the Lens of Actor Handoff Patterns”, based on research by Akhil Kumar of Pennsylvania State University and a colleague. In this case, they’re using the structured logs of known activities and handoffs between people within an IT help desk operation, but doing classical process modelling on these activities doesn’t provide a lot of answers: the process model is simple, but potentially contains a number of loops as a help ticket is directed back to someone who touched it earlier in the process. They looked at the patterns of handoffs in order to better understand the type of collaboration that was occurring, and use machine learning to predict incident resolutions times. This is an interesting type of analysis for case management scenarios such as help desks, which may be able to predict time-to-resolution for cases very soon after they are initiated, just based on the pattern of handoffs between case participants. I see future potential to combine this with analysis of the issue complexity based on other instance data to provide a much more accurate resolution time; this, in turn, leads to greater customer satisfaction since a customer that understands the timeline to expect in their case is less likely to complain about the resolution time, even if the time is long.

I finished up my look at process mining and discovery methods at the conference by attending a tutorial “Queue Mining: Process Mining Meets Queueing Theory” presented by the three researchers involved: Avigdor Gal of Technion (Israel Institute of Technology), Arik Senderovich of University of Toronto, and Matthias Weidlich of Humboldt-Universität zu Berlin. This was a really interesting look at how to expand process mining – which typically looks at what happens in a single instance/case of a process – to consider how the queue of instances impacts the outcomes. They looked at situations such as emergency medical waiting rooms, where queue congestion (too many people waiting) could cause some people to abandon the queue altogether: this provides insights into why some instances end early, or take another path that can’t be explained by the individual instance data alone.

All of this fascinating research is just that: research. There are no commercial solutions that include these techniques and algorithms (yet), but I am predicting that we will see the outcome of some of this research in future commercial products. If you’re just starting to use process mining in your organization – possibly based on recommendations from my previous post – or are experienced at process mining techniques, rest assured that there is a lot of interesting research going on in this field, and you can expect many new capabilities in years to come.