BPM+ : The Enduring Value of Business Automation Standards

While it no longer has the marketing cachet it once enjoyed, the fact that the foundations of the Trisotech platform are the three business automation standards from the Object Management Group – BPMN, CMMN, and DMN – remains a key advantage, with enduring benefits to users. In this post we’ll discuss what these standards do, how we got them, and their enduring business benefits.

In the Beginning

In the beginning, lets say the year 2000, there were no standards, not for business automation, not for almost anything in tech, really. The web changed all that. I first heard about BPMN in 2003. I had been working in something called workflow, automating and tracking the flow of human tasks, for a dozen years already, first as an engineering manager and later as an industry analyst. There were many products, all completely proprietary. But suddenly there was this new thing called BPM that promised to change all that.

Previously, in addition to workflow, there had been a completely separate technology of enterprise application integration or EAI. It was also a process automation technology but based on a message bus, system-to-system only, no human tasks, and again all proprietary technology. It was a mess.

Now a new technology called web services offered the possibility to unify them based on web standards. In the earlier dotcom days, the web just provided a way to advertise your business, maybe with some rudimentary e-commerce. Now people realized that the standard protocols and XML data of the web could be used to standardize API calls in the new Service-Oriented Architecture (SOA). A Silicon Valley startup called Intalio proposed using web services to define a new process automation language called BPML that unified workflow and application integration in this architecture. But they went further: Surprisingly, solutions using this language would be defined graphically, using a diagram notation called BPMN. The goal was engaging business users in the transition to SOA, and BPMN offered the hope of bridging the business-IT divide that had made automation projects fail in the past.

Enterprise application software vendors liked it too. Functionality that previously was embedded in their monolithic applications could be split off into microservices that could be invoked as needed and sequenced using the protocols of the world wide web. This was a game-changer. Maybe you could make human workflow tasks into services as well.

As it evolved, BPMN 1.0 took on the look of swimlane flowcharts, which were already used widely by business users, although without standard semantics for the shapes and symbols. Now with each shape bound to the semantics of the execution language BPML, BPMN promised “What You Draw Is What You Execute.” Intalio was able to organize a consortium of over 200 software companies called BPMI.org in support of this new business automation standard.

But in fact, they were too early, since the core web services standard WSDL was not yet final, and in the end BPML did not work with it. So BPML was replaced by another language BPEL that did. While the graphical notation BPMN remained popular with business for process documentation and improvement, it was not a perfect fit with BPEL for automation. There were patterns you could draw in BPMN that did not map to BPEL. So no longer could it claim “what you draw is what you execute.” BPMI.org withered and ultimately was acquired by OMG, a true standards organization.

Even so, BPMN remained extremely popular with business users. They were not, in fact, looking to automate their processes, merely to document them for analysis and potential improvement. Still the application software vendors and SOA middleware vendors pushed for a new version of BPMN that would make the diagrams executable. But it was not until 2010 that BPMN 2.0 finally gave BPMN diagrams a proper execution language. I had been doing BPMN training on version 1.x since 2007, and somehow I wound up on the BPMN 2.0 task force, representing the interests of those business users doing descriptive modeling.

BPMN 2.0 was widely adopted by almost all process automation vendors. Later OMG developed two other business automation standards, CMMN for case modeling and DMN for decision modeling, all based on a similar set of principles. There are three standards because they cover different things. They were developed at different times for different reasons. So let me briefly describe what they do, their common principles, and why they remain important today.

BPMN

BPMN was the first. BPMN describes a process as a flowchart of actions leading from some defined triggering event to one of several possible end states. Each action typically represents either a human task or an automated service, for the first time unifying human workflow and EAI. The diagrams include both the normal “happy path” and branching paths followed by exceptions. It is easy to see the sequence of steps the process may follow, but every instance of the process must follow some path in the diagram. That is both its strength and its basic limitation.

BPMN remains very popular because the diagrams are intuitive. Users understand them in general terms without training, and what you draw is what you execute. BPMN can describe many real-world processes but not all of them.

There are actually two distinct communities of BPMN users. It is interesting that they interact hardly at all. One uses BPMN purely for process description. They have no interest in automation, just documentation and analysis. Those BPMN users simply want to document their current business processes with hope of improving them, since the diagrams, if made properly with something like Method and Style, more clearly express the process logic than a long text document. For them, the value of BPMN is it generally follows the look of traditional swimlane flowcharts.

The other community is interested in process automation, the actual implementation of new and improved processes. For them, the value of BPMN is what you draw is what you execute. It is what allows business teams to better specify process requirements and generally collaborate more closely with development teams. This reduces time to value and generally increases user acceptance.

As a provider of BPMN training, my interactions have been mostly with the first community. In a prior life as an industry analyst, they were exclusively with the second community. So I have a good sense of the expectations of both groups. And while I believe the descriptive modeling community is probably still larger, it is now declining, and heres why.

In a descriptive modeling team, there are typically meetings and workshops where the facilitator asks how does the process start? What happens next? And so forth. Thats good for capturing the happy path, the normal activity flow to a successful end state, which is the main interest. But BPMN diagrams also contain gateways and events that define exception paths, since, remember, every instance of the process must follow some path drawn in the diagram. In reality, however, you are never going to capture all the exceptions that way. 

So over the past decade, a lot of process improvement efforts have shifted away from descriptive process modeling in this way to something called process mining, instrumenting key steps of the process as it is running however they are achieved and correlating the timestamps to create a picture of all the paths from start to end including, importantly, that small fraction of instances that take much too long, cost way too much, and are responsible for complaints to customer service. Dealing with those cases leads to significant process improvement, and automating them requires accommodating them somehow.

Some process mining tools map their findings back to BPMN, but BPMN demands a reason some gateway condition or event to explain each divergence from the happy path. This points out the key weakness of BPMN as an automation language, the difficulty in accommodating all the weird exceptions that occur in real world scenarios. Something more flexible is needed.

CMMN

CMMN was created to handle those processes that BPMN cannot. It focuses on a key difference from BPMN, work taking an unpredictable path of over its lifetime, emphasizing event-triggered behavior and ad-hoc decisions by knowledge workers.

A case in CMMN is not described by a flowchart, where you can clearly see the order of actions, but rather a diagram of states, called stages, each specifying actions that MAY be performed or in some cases MUST be performed, in order to complete some milestone or transition to some other state. That allows great flexibility, a better way than BPMN, in fact, to handle real-world scenarios in which it is impossible to enumerate the all the possible sequences of actions needed.

This comes at a cost, however. While what you draw is still what you execute, what you draw is not easy to understand. CMMN diagrams are not flowcharts but state diagrams. They are not intuitive, even if you understand the shapes and symbols well. Each diagram shape goes through various lifecycle states Available, Enabled, Active, Completed or Terminated – and each change of state is represented by a standard event that triggers state transitions in other shapes. So in the end the diagram, when fully expanded down to individual tasks, shows actions that MAY be performed or MUST be performed, and the events that trigger other shapes.

In practice, CMMN is almost always used in conjunction with BPMN, as the specific actions described, typically a mix of automated services and human interaction, are better modeled as BPMN processes. So typically in a business automation solution, CMMN is at the top level, where the choice of what actions to perform is flexible and event-driven. Many of the CMMN actions are modeled as process tasks, meaning enactment of a BPMN process. Organizing the solution in this way has the benefit of CMMNs flexibility to handle the events and exceptions of real-world end-to-end scenarios, and BPMNs business-understandable automation of the details of the required actions.

Above is an example from my book CMMN Method and Style, based on social care services in a European country. The stage Active Case contains 4 sub-stages without any entry conditions. That means they can be activated at any time. The triangle marker at the bottom means manually activated; without that they would activate automatically. And notice within each stage are a mix of human tasks, with the head and shoulders icon, and process tasks, with the chevron icon. At the bottom are 4 more tasks not in a substage that deal with administrative functions of the case. They potentially can occur at any time when Active Case is in the active state, but they have entry conditions the white diamonds meaning they must be triggered by events, either a user interaction or a timer. This diagram does not include any file item events receipt or update of some document or data which also triggers tasks and stage entry or exit. As you can see, the case logic is very flexible almost anything could be started at any time but at this level not too revealing as to the details of the actions. Those are detailed within the BPMN processes and tasks.

Here is another CMMN example that illustrates the flexibility needed for what seems like a simple thing like creating a user account in a financial institution. Yes, that is a straightforward process BPMN but it lives inside a wider context, which is managing the users status as a client. A new client has to be initialized, set up thats another BPMN process and then you can add one or more accounts as long as the client remains an Active User. You can deactivate the user thats another process which automatically terminates the Manage Active User stage, thats the black diamond, and triggers the Manage Inactive User stage thats the link to the white diamond labeled exit, exit being the lifecycle event associated with the Terminated state of Manage Active User. 

The only thing that happens in the Manage Inactive User stage is you can reactivate the user, another process. And when that is complete, since there is nothing else in that stage, the stage is complete, meaning the link to the white diamond in Manage Active User labeled complete triggers activation of that stage. The hashtag marker on both stages means after completion or termination they can be retriggered; without that marker they could not. And finally, the process task Delete user has no entry conditions, so it can be activated manually at any time, and when complete terminates the case, the black diamond. So what seemed initially like a simple process, Add User Account, in a real world application expands to a case handling the whole lifecycle of that client.

DMN

DMN describes not actions but business logic, understandable by non-programmers but deployable nevertheless as an executable service. DMN is a successor to business rule engine technology from the 1990s, better aligned with today’s microservice-oriented architecture. In DMN, a decision service takes data in and returns data out. It takes no other actions. Like BPMN and CMMN, design is largely graphical.

The diagram you see here, called a Decision Requirements Diagram, shows the dependencies of each rectangle, representing a decision, on supporting decisions and input data connected to it via incoming arrows called information requirements. The value expression of each decision can reference only those direct information requirements. Even if they are unable to model the value expressions, business users are able create these diagrams, which serve as verifiable business requirements for the executable solution.

But DMN was designed to let non-programmers go further, all the way in fact to executable decision services. Instead of program code, DMN provides standard tabular formats called boxed expressions. These are what define the expressions that compute the value of each decision.

Decision tables, like the one shown above, are the most familiar boxed expression type, but there are many more, such as Contexts, like the one shown below, where rows of the table define local variables called context entries, used to simplify the expressions in subsequent rows. Note here that the value expression of a context entry, such as Loan amount here, can itself be a context or some other boxed expression.

The expressions of each context entry, the cells in gray here, use a low-code expression language called FEEL. Although FEEL is defined in the DMN spec, it has the potential to be used in BPMN and CMMN as well. More about that later.

The combination of DRDs, boxed expressions, and FEEL means business users can create directly executable decision logic themselves without programming. In BPMN and CMMN, graphical design is limited to an outline of the process or case logic, the part shown in the diagram. To make the model fully executable, those standards normally still require programming.

On the Trisotech platform, however, that’s no longer the case. When BPMN and CMMN were developed, they did not define an expression language, because they assumed, correctly at the time, that the details of system integration and user experience would require programming in a language like Java. But DMN was published 6 years later, when software tools were more advanced. DMN was intended to allow non-programmers to create executable decision services themselves using a combination of diagrams, tables, and FEEL, a standard low-code expression language. An expression language is not a full programming language; it does not define and update variables, but just computes values through formulas. For example, the Formulas ribbon in Excel represents an expression language.

While adoption of FEEL as a standard has been slow, even within the decision management community, many business automation vendors have implemented their own low-code expression languages with the objective of further empowering business teams to collaborate with developers in solutions.. Low-code expression languages should be seen today as the important other half of model-based business automation, along with diagrams composed of standardized shapes, semantics, and operational behavior. The goal is faster time-to-value, easier solution maintenance, and improved user acceptance through close collaboration of business and development teams.

Shared Principles

Three key principles underlie these OMG standards, which play a key role in the Trisotech platform today, and I want to say more about why they are important.

The first is that they are all model-based – meaning diagrams and tables and those models are used both for outlining the desired solution, what have been called business requirements, and creating the executable solution itself, going back to the original promise of What You Draw Is What You Execute.

The traditional method of text-based business requirements is a well-known cause of project failure. Instead, models allow subject matter experts in the business to create an outline of the solution in the form of diagrams and tables that can be verified for completeness and self-consistency. Moreover, the shapes and symbols used in the diagrams are defined in a spec, so the models look the same, and work the same, in any tool. And even better, the execution semantics of the shapes is also spelled out in a spec, so what you draw is actually what you execute. Engaging business users together with IT from the start of the project speeds time to value and increases user acceptance.

Second, the models have a standard xml file format, so they can be interchanged between tools without loss of fidelity. In practice, for example, that means business users and developers can use different tools and interchange models between them. Ive been involved in engagements where the business wants to use modeling tool X, and IT, in a separate organization, insists on using tool Y for the runtime. Thats not great, but doable, since you can interchange models between the tools using the standard file format.

Third, OMG procedures ensure vendor-neutrality. Proprietary features cannot be part of the standard, and the IP is free to use. In practice this tends to result in open-source tools and lower software cost. It may not be widely appreciated but many, possibly the majority, of BPMN diagramming tools today stem from an open source project, and another open source project created a popular BPMN runtime. DMN is a similar story. Tool vendors were stumped by the challenge of parsing FEEL, where variable and function names may contain spaces, until Red Hat made their FEEL parser and DMN runtime open source. Implementing these standards is difficult, and open source software has been a big help in encouraging adoption.

Enduring Business Value

So where is the business value of these standards? What are the benefits to customers? In my view, these fall into 3 categories:

First, common terminology and understanding. I remember the days of workflow automation before BPMN. There was no common terminology. You learned a particular tool, and tools were for programmers. Now BPMN, CMMN, and DMN provide tool-independent understanding of processes, cases, and decision logic. There is much wider availability of information about these technologies, through books, web posts, and training, down to the details of modeling and automation. This in turn makes it easier to hire employees and engage service providers already conversant and experienced in the technology. It also lowers the re-learning cost if it becomes necessary to switch platform vendors.

Benefit #2 is faster time-to-value. Today this might be the most critical one for customers. Model-based solutions are simply faster to specify, faster to implement, and faster to adapt to changing requirements than traditional paradigms. Faster to specify because you can more easily engage the subject matter experts up front in requirements development and more quickly verify the requirements for completeness and self-consistency. Faster to implement because you are building on top of standard automation engines. The custom code is just for specific system integration and user experience details. And faster to adapt to changing requirements because often this involves only changing the model not the custom code.

So, for example, a healthcare provider can go live with a Customer Onboarding portal in only 3 months instead of a year, leveraging involvement from business users across multiple departments and close collaboration between business and IT. Everyone is using the same terminology, the same diagrams, and what you draw is what you execute.

A key feature of the 3 OMG standards is they were designed to work together. CMMN, for example, has a standard task type that calls a BPMN process and another that calls a DMN decision service. BPMN can call a DMN decision service and, on the Trisotech platform, a CMMN case. And so you have platforms like Trisotech that handle the full complement of business automation requirements event-driven case management, straight through process automation, long-running process automation, and decision automation. The alternative is a lot of custom code expensive and hard to adapt to changing requirements or separate platforms for process, decision, and case management, again expensive and requiring a lot of system integration.

A single platform that handles all three especially one based on models, business engagement, and vendor-independent standards lowers costs the cost of software, of implementation, and of maintenance of the solution over the long term.

SDMN, a Common Data Language

While BPMN, CMMN, and DMN work together well, they still lack a common format for data. DMN uses FEEL, but in most tools, executable models BPMN and CMMN use a programming language like Java to define data. It would be better if they shared a way to define data. Trisotech uses FEEL for all three model types, which is ideal. But now there is an effort to make that common data language a standard.

BPM+ Health is an HL7 “community of practice” in the healthcare space seeking industry convergence around such a common data language, SDMN. That language is essentially FEEL. Technically, it is defined by the OMG Shared Data Model and Notation (SDMN) standard, a key BPM+ Health initiative seeking to unify data modeling across BPMN, CMMN, and DMN. 

Data modeling is primarily used in executable models and omitted from purely descriptive models. The challenge for SDMN is to unify DMN variables, BPMN data objects, and CMMN case file items, and describe them as what DMN calls item definitions.

SDMN defines a standards-based logical data model for data used across BPMN, CMMN, and DMN. Data linked to SDMN definitions is not specific to a particular model but usable consistently across models of different types. In this sense it is the logical data model equivalent to the Trisotech Knowledge Entity Modeler. But where KEM defines a conceptual data model, with focus of KEM is on term semantics, SDMN is a logical data model, with focus on data structure, what DMN calls item definitions.

At this time, SDMN 1.0 is still in beta, but two artifacts illustrate what is planned. The DataItem diagram, shown below, describes the names and types of data items, and structural relationships between them.

In addition, an ItemDefinition diagram, again shown below, details the structure of each data item.

Trisotech users will recognize this as an alternative way to create item definitions, easier for tool vendors without Trisotech’s graphics capability. I still prefer Trisotech’s collapsible implementation:

The Path Forward

Going forward, look for more solutions developed using BPMN, CMMN, and DMN in combination. Typically CMMN is used at the top level, where it can handle all possibilities in a real-world process application. Specific actions are best implemented as BPMN processes, invoked from CMMN as a process task. Decision logic can be called from either CMMN or BPMN as a decision task. So you see why having common data definitions for all three modeling languages is valuable in such integrated solutions.

On the Trisotech platform, SDMN, in the form of FEEL, is already supported in all three languages, providing the additional benefit of Low-Code business automation. Starting in healthcare, SDMN will make this integration enabler available on other platforms as well.