Stateful Decision Models

Recently I viewed a DecisionCamp presentation on an application of DMN to a public-facing portal in the Netherlands for what we call in the USA building permits.

Here is the link. The scope of the project is impressive, combining regulations defined at the national, provincial, and municipal level, plus those of special districts. The ultimate decision on what kind of permit is required depends on not only the location of the property but the nature of the property improvement. Dealing with the latter dependency, however, caused the implementation, in my view, to break fundamental principles of DMN. The public-facing portal implementation also illustrates how deeply legacy concepts of business rules stubbornly persist in nominally “DMN” solutions. The issue in question here is how to add orchestration, that is decision sequencing, to DMN. It’s a real issue, but fortunately there is a better way to do it, one that remains true to the requirements and spirit of the standard.

The variety of projects that may or may not require a building permit is very broad, and thus the information required to make the decision covers a vast range. Think about the different information required to erect a fence around the property, install rooftop solar, or add an entire auxiliary dwelling unit. In the portal app, you cannot ask the user to provide input data for all possible improvements up front. You need to start by asking about the nature of the proposed project, as this narrows the required information considerably. To do this within a single decision model requires statefulness, that is, sequencing of the decision logic. But the fundamental unit of execution in DMN is a decision service. Its definition is declarative and execution is always stateless, meaning you cannot specify the order of evaluating various parts of the service.

Stateless vs Stateful

What’s the Difference?

Statelessness is a design principle of web services that improves scalability by not requiring storage of data on the server. A stateless service depends only on the data provided in its request; it cannot have any dependence on prior requests. A stateless decision service is executed all at once using the input data provided in the invocation. In contrast, a stateful service may be executed in discrete steps, in which each step may depend on the result of prior steps. This is often valuable, but it means the server must store and retrieve state data, typically in a database. A DMN decision service is always stateless. In contrast, a process service such as defined by BPMN is normally stateful.

Legacy of Business Rule Thinking

Why would one even try to implement the portal as a single decision service? One reason is that legacy business rule management systems include stateful decision logic in the form of a ruleflow, a kind of decision tree where selected fragments of the decision called rulesets are executed in some defined order. This ruleflow mindset is leading some practitioners to assume that DMN includes this stateful notion, or that it should. And the DMN spec itself is hardly blameless, because the DMN example in the spec, ever since DMN 1.0, has been a stateful DRD! This example, which predates DMN’s formal definition of decision services, has misled many, myself included: The first edition of my book DMN Method and Style tied itself into a knot trying to interpret stateless execution of the example. Later versions of the spec added a bit of clarity by defining various fragments of the DRD as stateless decision services, orchestrated by some kind of process. But the damage was already done. While the spec is intended for implementers, i.e. software developers, not end users, in practice the spec example defines what most users know about DMN.

A second bit of “business rule thinking” in the DecisionCamp portal implementation is the use of backward chaining. Depending on the values of certain inputs, certain “rules” could become irrelevant to the final result. In consequence, users should not be asked for their input data. The public-facing portal implementation resorted to developing some custom Java code to allow rules to make certain other rules irrelevant. So we have statefulness plus custom code that turns off certain rules based on other rules. These are legacy business rule management notions with a DRD pasted on top. They are inconsistent with the stateless intent of DMN services. While the public-facing portal implementers took care to use the extensions mechanism from standard, the logic is no longer transparent, a key guiding principle of DMN.

A Better way

There is a better way to do it, one that remains true to DMN, both technically and in spirit. The solution is defined as a collection of stateless decision services, orchestrated by BPMN. BPMN is the standard for orchestration, and it even has a task type specifically meant for invoking decision services. A Decision task – in the spec called a Business Rule task – invokes a DMN service. The DMN service inputs and outputs are, by definition, the Decision task inputs and outputs. In the orchestration model, process variables (data objects) are mapped to the decision service inputs, and the decision service outputs are mapped to other process variables. Gateways branch the flow to the next decision service by testing the value of the process variables. You can publish this “ruleflow” process as a REST service, one that follows the rules of both BPMN and DMN. In the end, you have a stateful decision service, but it’s defined by BPMN not DMN.

You might complain about having to learn another language, BPMN, but what if this BPMN “ruleflow” used the same language, DMN? On the Trisotech platform, that’s exactly what happens. The mapping from process variables to decision service inputs looks and acts just like a DMN boxed invocation. And the mapping from decision service outputs to other process variables looks and acts just like a DMN boxed context. The gateway logic uses FEEL boolean expressions. If you know DMN, including boxed expressions and FEEL, you already know all you need in BPMN.

The diagram above illustrates a stateful decision service modeled in BPMN. The service inputs are the data inputs Input A, Input B, and Input C. The service output is the data output Output B. Each of the Decision tasks invokes a distinct stateless decision service. In this example, it is likely that Input C will be null if it is known that in this instance Decision C will not be needed. The decision service inputs and outputs are not visible in this diagram, but the dotted arrows, called data associations, indicate mappings between process variables and those inputs and outputs. On the Trisotech platform, these mappings are modeled using FEEL and boxed expressions.

An Unexpected Bonus

The public-facing portal example requires statefulness in the decision logic itself, but I maintain that in real-world implementations, even nominally stateless decision logic should be embedded in a process. You need a process to ask for user input, to retrieve other input data, to validate the input data, and to take action based on the decision result. In a nutshell, DMN in practice always requires BPMN, and – on the Trisotech platform, at least – they were designed to work together!

And it’s even better than that, because there is little difference between a BPMN “ruleflow” and BPMN used for any kind of Business Automation. There is nothing in DMN that restricts it to business decisions based on rules. It’s actually a Low-Code language for any kind of business logic. Moreover, Service tasks in Trisotech BPMN use FEEL and boxed expressions to invoke any REST API. Effectively then, the combination of BPMN and DMN provides a Low-Code language for any kind of Business Automation. Learning to add statefulness to DMN thus comes with a huge bonus: You’ve just learned Low-Code Business Automation.