"Business Process Task Library", that's quite a mouthful, and what does it actually mean? For that, we need to lay out some context first.
The Common Ground movement is an initiative to improve the Dutch municipal data landscape. One of the core values is to get a grip on the data being exchanged, with a focus on eliminating duplication, facilitate innovation and make it possible to be GDPR compliant. It embraces privacy-by-design.
To achieve that, an architecture with 5 distinct 'layers' is to be implemented, breaking up the traditional, opaque silos that applications/processes currently are. The layers are defined as:
Up until now there's been a substantial effort to standardize on some APIs - short for Application Programming Interfaces. This is a broad term, but in the current context, we're mostly talking about web-based APIs to exchange data between applications. This plays out on the second layer.
VNG Realisatie has developed some API standards supporting "Zaakgericht Werken" (Zaken API, Documenten API, Catalogi API, Besluiten API, Notificaties API and Autorisaties API), while Haal Centraal has been working on the BRP, BRK, BAG, BRT, BGT...
Probably a number of projects haven't been mentioned here, but that's mostly because we haven't been involved with them yet, and the relevance for this blog post is limited to the common concepts anyway.
While efforts so far have mostly been focused (and advertised) on layers 2 and 3 in the form of API standards and projects such as NLX or the App store for integration, the interaction and process layers are still pretty vague.
Enter the Utrecht Proeftuin ("playground"), where we can explore these new ideas, implement and test the theory and promises of Common Ground. All of this in a greenfield approach without affecting the real business or daily operations. Maykin Media has been involved in this for some time now, together with other market suppliers, each playing to their strengths.
For municipalities, low- or zero-code environments are desirable, as it allows them to design, build, implement and tweak real business processes without being dependent on developers or software suppliers too much. This should in theory also prevent unexpected costs because of development spikes. Another advantage is that it reduces vendor lock-in.
BPMN (Business Process Model and Notation) is an international standard to graphically model and represent internal business procedures. It's also machine-readable by existing software to execute these processes. This is a pretty sweet deal - having a process engine powered by BPMN allows you to follow the process without deviating (as a staff member), and it provides room to automate (parts of) the execution while being self-documenting. Ideally, a minimal amount of code would have to be written, and what needs to be developed can be done so once and re-used easily.
This fits the Common Ground model on layer 4. Process definitions can be shared between organizations because of the BPMN standard, encouraging re-use.
However, there is one major disconnect. BPMN is a well-known global standard. Standard tasks such as assigning to a particular user, sending e-mails (automatically) are readily available in the major process engines. On the other hand, Dutch-government-specific APIs (such as the BRP or Zaken API) require custom development to actually be consumed. While the APIs may be standardized in the Netherlands, on a global scale they are not relevant enough for pre-made solutions to exist (yet).
The Business Process Task Library (BPTL) bridges this gap. It's a project compromised of what we call "work units". These work units are tied to the specific API standards. Such a work unit can be added as a task in your business process, fitting in the BPMN language. It's where the custom development meets the BPMN standard.
The following sections zoom in on the specifics and technical details.
As the project name implies, the project is a library of 'tasks'. These are the work units, they represent a logical set of things to do - mostly making API calls. Some examples:
The work units in this case are simple Python functions or classes, and agnostic of the actual process engine being used. They receive input sourced from process variables, and return some result. That result can be set as new or updated process variables.
As new services are introduced, or new patterns emerge, they can be added as (new) work units to the BPTL. We aim to be batteries-included - that is, a standard installation ships with a ready-to-use set of work units.
One last important thing we have to mention is that work units are connected to topics. A topic name is the link between some task in the process definition and the actual work to be performed. The details vary between process engines, as you'll read next.
There are a number of process engines supporting BPMN. Each engine has its own extensions, and these are usually not compatible with other engines. There are also functionality differences between community and enterprise editions, usually in the features available in the user-interface of the engine.
BPTL aims to be easily extended into supporting the popular process engines. Currently, we have support for Camunda and Alfresco Process Services (based on Activiti). Other engines may follow later, please contact us if you're interested ;)
For reference, a very simple process model would look like this:
startEvent ---> workUnit ---> endEvent
where workUnit is the task to be performed by BPTL and is an engine-specific extension.
Camunda has an extension in the form of External Tasks that's not seen in other process engines. It's a pretty clever way to off-load work in a decoupled way to specialized handlers. In the above example, workUnit is a task of the type External Task.
Whenever Camunda execution arrives at an External Task, it will put it on a queue to be picked up by a worker. The process effectively halts at this stage. An External Task is characterized by one attribute - the topicName, which you use to classify the type of work to perform. In our example, we could use initialize-zaak as topic name.
Execution is by definition asynchronous, since some worker needs to pick up the work load from Camunda, do the needful, and then mark the task as completed, after which the process execution continues with any subsequent tasks.
BPTL supports Camunda, of course, else we wouldn't be writing this blog post. When you install BPTL in your environment, you can configure where the Camunda process engine lives and set up the credentials.
Next, BPTL will periodically poll Camunda for external tasks to process, and it does this only for the topics you have configured BPTL to handle. This means that other applications can still handle different subsets of specialized work if needed. This is good, as we still don't like vendor lock-in ;)
Whenever a work unit is picked up from Camunda's queue, BPTL will schedule it to one of the async workers. That worker executes it and sets the resulting process variables while completing the External Task. Camunda then automatically proceeds to the next step in the process definition upon receiving the completion.
The interesting part about this model is that you can tackle scaling challenges by scaling more BPTL workers, as opposed to having to scale the process engine itself. With BPTL, we pay attention to performance of course and aim for low CPU and memory footprints.
Activiti (or the enterprise version "APS") has a different model. There is no External Task type, but there is a REST-call task type. An administrator can define endpoints for tenants. You would typically define an endpoint to the BPTL API here. The pre-defined endpoints can then be integrated into the process definitions by regular process designers.
Effectively, this means that for APS, the workUnit task is a task of the type REST call. Within the process definition, you specify the request body - which can contain process variables to pass to BPTL. You can also map the response back into process variables.
So, to serve Activiti (and really, anything that can make REST calls), BPTL has its own RESTful API. You trigger a work unit execution by making a REST call to the appropriate endpoint, specifying the topic name and variables. BPTL then executes the connected work unit, and the resulting variables are returned in the response.
NLX is one of the core integration services in Common Ground. In a nutshell, it facilitates logging why and when data was requested, while providing connectivity between (different) organizations.
We're providing first-class support for NLX. Work units can optionally support NLX-variables to be set as HTTP headers in API calls. Effectively, this means your NLX properties (such as "doelbinding") can be embedded in the process rather than imposing extra development effort on the end-user application to support NLX.
Integration is currently as simple as configuring your services to make use of the NLX Outway instead of their public/canonical URLs and providing the relevant process variables.
You may notice that there's been a particular focus on API calls so far, but work units aren't necessarily limited to that. There could be work units for document generation, sending e-mails, calculating things...
We even feel that BPTL may be able to bridge the old-standard StUF communication to the new RESTful API approach! Messages could be buffered into a process instance, and when all the relevant data is available in a given process step, translation to and from StUF would be possible. This can ease municipalities into gradually migrating to the Common Ground vision.
We're designing BPTL to be cloud- and Kubernetes native. The Docker images are published publicly on Docker Hub, and the source code is public on Github.
Open Source does not mean that you cannot use closed-source parts! We've thought about a plugin system to include additional (closed source) work units in a BPTL installation. BPTL is batteries included, but you can definitely add extra batteries ;)
First, a thank you to the Common Ground organization, for encouraging innovation. It has inspired a lot of suppliers and municipalities to do better.
Second, the municipality of Utrecht. In the Proeftuin, there is a lot of room and freedom to explore ideas, develop them into an MVP, and do all of this Open Source!
We're Maykin Media, a development agency. We have a passion for open source software and contributing to solid and secure applications.
The BPTL project is developed with the Django framework as part of Utrecht's Proeftuin.
Git Rebasing - make your commit history pretty Maykin & Common Ground - vijf jaar later
Maykin Media focust op de technologie achter uw webapplicatie. Door te specialiseren op Python en Django bieden we efficiënte en professionele webdevelopment aan.
Copyright © 2024 Maykin Media. All rights reserved.
