jBPM Workitems Spotlight - GitHub and Slack

So far we have posted a number of articles describing the jBPM Workitem Repository and its features but there was little mention about its actual content and the powerful integration capabilities it brings to your business processes.

Currently the workitem repository includes 25 different workitem groups and a total of 57 individual workitem handlers. These include various integration services with Google,  Dropbox, IBM, Twitter and many more. New workitems can easily be added as was described in this previous article.  Given that the number of workitems in growing at a steady pace we thought to start showcasing them with simple working examples that you can take and use to start implementing your custom integration solutions.

In this article we will focus on two workitem groups, namely GitHub and Slack:

For our simple example to showcase these workitems we will create a business process which fetches all open issues from one of your GitHub repositories, accumulate them into a message and then send this message to a specified channel on Slack notifying you of these open issues and give you simple links to each so you can start addressing them.

First let's look at the pre-requisites for each of the workitems. To start using GitHub workitems the required information is your login data, namely your username and password. For this example we will use the FetchIssuesWorkitemHandler where you will also need to know the name of the repository where you want to fetch the issues from.

To start using the Slack workitem handlers you need to first create a Slack App. Follow this link to get started with that. Once you create your app make sure to enable OAuth so Slack will create an access token for it. This access token is then passed to the workitem handler as it needs it for authentication with the Slack service. The workitem handler we will use here is PostMessageToChannelWorkitemHandler which also needs the name of one of the channels your slack app has access posting messages to.

To follow this example we also assume that you have the KIE workbench+server set up and ready but if you don't you can follow this article to do that literally in minutes. We also assume that you have an existing GitHub repository set up with some real or test issues defined. If you don't you could use this test one. 

Test GitHub repo with some test issues

With our KIE worbench running log in and create a new project. Within this project create a new business process (with legacy editor as it's currently only one capable of installing workitems from a repository).  Click on the repository icon in the process editor menu bar and connect to your running service repository

Installing workitems from workbench legacy bpmn2 editor

Find the two workitem handlers we mentioned earlier, namely FetchIssuesWorkitemHandler and PostMessageToChannelWorkitemHandler and click on the little wrench icon next to each to install them into your workbench. Note that the install also updates your project pom with all necessary maven dependencies to execute these workitems as well as your projects deployment descriptor xml. 

To finish the setup we do need to edit the projects deployment descriptor slightly, passing in our login information to the github workitem as well as the Slack access token created to the slack workitem so they can authenticate with their respective services correctly:

For this in the project explorer navigate to your projects /src/main/resources/META-INF directory and edit the kie-deployment-descriptor.xml file. Note that under the work-item-handlers element in this xml file the workitem installer has already added the definitions of two workitem handlers we want to use, however it needs to be slightly changed to add the sensitive info we talked about:

So for the GitHub handler we need to add:

<identifier>new org.jbpm.process.workitem.github.FetchIssuesWorkitemHandler("YOUR_GITHUB_LOGIN_NAME", "YOUR_GITHUB_LOGIN_PASSWORD")</identifier>

and for the Slack handler:

<identifier>new org.jbpm.process.workitem.slack.PostMessageToChannelWorkitemHandler("YOUR_SLACK_APP_ACCESS_TOKEN")</identifier>

Note that in the image below I have cut off the sensitive info as I used my personal login information when making the demo

Modifying the kie deployment descriptor

            

Now we should be all set up to create our business process. You can download or import the one used in this example from this Gist.

Our test business process

The example process first fetches all open issues from our testrepo GitHub repository. This list is then saved in a process variable which the script task iterates to create a message. The created message is then used by the Slack workitem to connect to your defined channel and create a new post to it.

You can now build and deploy your worbench process and start and instance of our example process

Starting new instance of the example process

Process instance completed info panel

Once the process has completed execution you should see in your Slack channel the result message:

And that's it :) Let us know what other workitems you would like to see featured next. 

Easy Workitem installation (jBPM WorkItem Repository)

We added the ability to install workitems hosted by the  jBPM Workitem Repository on any running KIE Worbench instance directly from the repository. itself This lifts some limitations of installing workitems which was so far only possible from within process editors inside each running workbench. It also allows for future integrations with other runtime systems that can take advantage of the hosted workitems.

When browsing workitems inside the repository you will now find the "Install" column:

Install column - Repository homepage

Selecting "Kie Workbench" will trigger the installation modal:

Workitem installation modal

Here you can enter specific information about the running KIE Workbench instance such as URL, Username, Password, Space name, and Project info where this workitem should be installed in.

Clicking the Install button will install all workitem handlers from the selected workitem group to your workbench instance. Upon successful installation you should be able to see the installed workitem handlers inside the selected workbench project.

Install success message

Archive workitem installed in "myproject"

Once installed you can start using your workitems when modeling your business processes:

Start using the installed workitem!

Note that this feature is available only if you are running the workitem repository in "hosted" mode, so by hosting it via the repository-springboot, or the repository-wildfly modules. When serving the repository content from the file system the Install column will not be displayed. 

Another thing to note is that since in almost all cases your hosted repository will have a different origin(domain) than the running workbench instances you are installing workitems to we need to enable CORS on the servers the workbench is running on. If you are running on Wildfly, this can simply be done inside your standalone.xml config:

Wildfly CORS config

Don't forget to change the value of the "Access-Control-Allow-Origin" header to the domain where you are hosting your workitem repository. In this case we use the default http://localhost:8090 which is used when running spring-boot repository setup. 

Last thing to mention is that currently the KIE Workbench servlet filters do not handle CORS preflight requests unfortunately (hopefully this will be changed in the future). Because of this by default the workitem repository is not able to issue CORS requests to the workbench intances. To lift this limitation add this filter definition to your workbench web.xml and restart the server:

Filter definition within KIE workbench

As always feel free to leave any comments and suggestions. Enjoy!

Performance baseline for jBPM 7 (7.8.0)

The aim of this article is to show a base information about performance of the jBPM to set a baseline and to answer basic question how good jBPM performs when it comes to execution. This is not to be seen as competitive information or show jBPM is faster or slower than other engines but more for setting a stage and open the door for more performance tests that can be performed in different types of environments.

Overview

The performance test is executed on KIE Server so it actually measures performance of the jBPM as a running service instead of focusing on raw execution of the APIs. So anyone can perform this tests by following the instructions at the end of this article.

Environment

The test has been executed on:

• community 7.8.0 single zip distribution that you can download on jbpm.org
• WildFly 11
• Postgres data base 
• hardware
• macOS 10.13.4
• Processor Intel Core i7 2,3 GHz
• Memory 16GB
• JMeter as the test client

All components (client, application server and database) are on the same hardware, meaning they share the resources.

Scenarios

There are three scenarios selected for this test that are executed with various concurrency settings.

Script task

Most basic process definition that runs directly from the beginning till the end without persisting any state in between.

This test consists of just single call to KIE Server.

User task

User task based process that will persist its state when reaching user task activity. Completion of the task is done in another call. 

This test consists of three calls to KIE Server

• start process
• get tasks for given process instance
• complete first task

Parallel script and user tasks

More advanced process definition that combines both user and script tasks with parallel gateways.

This test consists of five calls to KIE Server

• start process
• get tasks for given process instance
• complete first task
• get tasks for given process instance
• complete second task

Performance test

Tests are separated per scenario and then number of concurrent threads. The test is designed to run fixed number of process instances (1000) in the shortest possible time.

Script task execution results

Actual figures

• 1 thread - 11 421 ms
• 4 threads - 4 428 ms
• 8 threads - 3 124 ms

Throughput: 

• 1 thread - 91 instances/s
• 4 threads - 240 instances/s
• 8 threads - 361 instances/s

User task execution results

Actual figures

• 1 thread - 64 439 ms
• 4 threads - 18 397 ms
• 8 threads - 13 927 ms

Throughput: 

• 1 thread - 16 instances/s
• 4 threads - 52 instances/s
• 8 threads - 72 instances/s

NOTE: throughput is for complete process instance execution including completion of user task

Parallel script and user tasks execution results

Actual figures

• 1 thread - 153 543 ms
• 4 threads - 34 769 ms
• 8 threads - 20 426 ms

Throughput: 

• 1 thread - 11 instances/s
• 4 threads - 45 instances/s
• 8 threads - 70 instances/s

NOTE: throughput is for complete process instance execution including completion of user tasks

Conclusion

These performance results show the base performance of the jBPM execution through KIE Server - meaning it adds network and marshalling overhead. The application server or hardware has not been tuned in anyway and the sample processes are simple as well. So with that said, it's not meant to provide complete performance report but rather a base line. More advanced performance tests can be performed on dedicated hardware and with tuned application server and database for optimal performance.

Instruction for execution

In case someone would like to try these tests themselves, here are few steps one how to do it.

1. Download and install jBPM 7.8.0 (or newer)
1. download 
2. getting started 
2. Change data base to PostgreSQL or MySQL - see getting started (bottom of the page)
3. Import this project into workbench on your running jBPM server
4. Download and start JMeter
5. Open this script in JMeter
6. Run the selected scenario.

jBPM WorkItem Repository updates

Wanted to share some recent updates to the jBPM WorkItem Repository

1. Visual updates: the repository home page includes alot more information about workitems in the repository

Updated repo index page

Each row now displays the number of workitem handlers it includes. Also each row can be selected to display the handlers information and link to the handlers documentation page:

Workitem Handler information

The text download links have been replaced with a dropdown so if you extend the repository for your own needs you easily add more items to the dropdown:

Downloads dropdown

2. Functionality Search: Often users won't know what all they can do with workitems in the repository by just looking at their names alone. We have included a functionality search where you can search for actual functionalities you wish to include in your business processes, for example "upload", "detect", "send message" etc. The search then filters only the workitems in the repository which can perform this functionality:

3.  WildFly repository module: Similar to the repository-springboot module we have now added a repository-wildfly module as well.  This module builds a wildfly war as well as an out-of-the-box zip distribution that includes WildFly 11 and the repository already installed. For more info on how to set this up and use it see here. 

4. Repository rest api: If you are running your workitem repository with Spring Boot or WildFly the repository exposes a rest api for you. The currently implemented endpoints include:

/rest/services

/rest/services/{name}
/rest/services/{name}/parameters
/rest/services/{name}/results
/rest/services/{name}/mavendepends
/rest/services/category/{category}
/rest/servicetriggers
/rest/servicetriggers/count
/rest/serviceactions
/rest/serviceactions/count

and more can be easily added. Currently the response type of all these endpoints is application/json.

Sample results of repository rest api

The hope is that community can easily build client-side apps like installers etc that can consume and utilize the repository info.

That's it for now. We are looking to further improve the workitem repository and add a lot more to it. As always any help from our great jBPM community is more than welcome.

jBPM 7.8 native execution of BPMN2, DMN 1.1 and CMMN 1.1

with upcoming 7.8 release of jBPM there is quite nice thing to announce - native execution of:

• BPMN2 - was there already for many years
• DMN 1.1 - from the early days of version 7
• CMMN 1.1 - comes with version 7.8

The biggest thing coming with 7.8 is actually CMMN execution. It is mainly added for completeness of the execution so people who would like to model case with CMMN can actually execute that directly on jBPM (via KIE Server or embedded).

Although jBPM supports now CMMN, it is still recommended to use BPMN2 and case management features of jBPM for advanced cases to benefit from features that both specification brings rather to be limited to particular approach. Nevertheless CMMN can be a good visualisation for less complex cases where data and loosely coupled activities can build a good business view.

Disclaimer: jBPM currently does not provide nor plans to provide any modelling capabilities for CMMN.

With that said let's take a quick look at what is supported from the CMMN specification as obviously it's not covering 100% of the spec.

• tasks (human task, process task, decision task, case task)
• discretionary tasks (same as above)
• stages
• milestones
• case file items
• sentries (both entry and exit)

Not all attributes of tasks are supported - required, repeat and manual activation are currently not supported. Although most of the behaviour can still be achieved using different constructs.

Sentries for individual tasks are limited to entry criteria while entry and exit are supported for stages and milestones.

Decision task by default maps to DMN decision although ruleflow group based is also possible with simplified syntax - decisionRef should be set to ruleflow-group attribute.

Event listeners are not supported as they do not bring much value for execution and instead CaseEventListener support in jBPM should be used as substitute.

Let's have a quick look at how the sample Order IT case would look like designed in CMMN

some might say it's less or more readable and frankly speaking it's just a matter of preferences.

Here is a screencast showing this CMMN model being executed 

Next I'd like to show the true power of jBPM - execution of all three types of models:

• CMMN for top level case definition
• DMN for decision service
• BPMN2 for process execution

you can add all of them into kjar (via import asset in workbench) build, deploy from workbench directly to KIE Server and execute. So here are our assets

A case definition that has:

• decision task that invokes DMN decision that calculates vacation days (Total Vacation Days)
• two human tasks that are triggered based on the data (entry criterion)
• process task that invokes BPMN2 process if the entry condition is met

Here is our DMN model

and last but not least is the BPMN2 process (actually the most simple one but still a valid process)

Another thing to mention is that, all the models where done with Tristotech Editors to illustrate that they can be simply created with another tool and imported into kjar for execution.

Here is another screencast showing this all step by step, from exporting from Tristotech, importing into workbench, building and deploying kjar and lastly execute on KIE Server.

That's all to share for now, 7.8 is just around the corner so keep your eyes open and visit jbpm.org to learn more.

And at the end here are the links to the projects (kjars) in github

• CMMN Order IT case sample
• CMMN, DMN, BPMN sample

Enjoy!

single zip distribution for jBPM

To simplify getting started experience for users I'd like to showcase a single zip distribution that includes:

• WildFly server (at the moment version 11.0.0.Final)
• workbench (aka jbpm console)
• kie server with all capabilities enabled
• jBPM case management show case application

All of them are perfectly configured and ready to run with just single and short command:

./stanalone.sh

or on windows

standalone.bat

The only thing user needs to do is download, unzip and run!

But that's not all that comes with this single zip distribution - it comes with very handy scripts that allow to switch to different databases as easy as just one click.

There are three databases supported out of the box:

• H2 - default with file based data base stored under WILDFLY_HOME/standalone/data
• MySQL
• PostgreSQL

MySQL and PostgreSQL must be installed before use. Moreover the scripts assume following values:

• host -> localhost
• port -> 3306 for MySQL and 5432 for PostgreSQL
• database name -> jbpm
• user name -> jbpm
• password -> jbpm

in case the values are not correct, edit them in the script files 

• jbpm-mysql-config.cli for MySQL 
• jbpm-postgres-config.cli for PostgreSQL 

in both scripts values to be updated are on line 17 and these are located under WILDFLY_HOME/bin.

To switch to MySQL stop the server and use following command when server is stopped

<WILDFLY_HOME>/bin/jboss-cli.sh --file=jbpm-mysql-config.cli           (Unix / Linux)

<WILDFLY_HOME>\bin\jboss-cli.bat --file=jbpm-mysql-config.cli     (Windows)

To switch to PostgreSQL stop the server and  use following command when server is stopped

<WILDFLY_HOME>/bin/jboss-cli.sh --file=jbpm-postgres-config.cli      (Unix / Linux)

<WILDFLY_HOME>\bin\jboss-cli.bat --file=jbpm-postgres-config.cli     (Windows)

next, start the server again and all your data will be stored in external database.

All this in action can be seen in this "not so short" screencast

As usual feedback welcome and please share your opinion if you'd like to see this in the official distribution of jBPM.

For those that would like to give it a go directly here is the project - just clone it and build locally - in case you want to use another version of jBPM change property named kie.version to the version number you want to use.

Contract Net Protocol with jBPM

jBPM provides lots of capabilities that could be used out of the box to build rather sophisticated solutions. In this article I'd like to show one of them - Contract Net Protocol.

Contract Net Protocol (CNP) is a task-sharing protocol in multi-agent systems, consisting of a collection of nodes or software agents that form the 'contract net'. Each node on the network can, at different times or for different tasks, be a manager or a contractor. [1]

This concepts nicely fits into case management capabilities of jBPM 7. It allows to easily model interaction between Initiator and Participant(s).

source http://www.fipa.org/specs/fipa00029/SC00029H.html#_ftnref1

Contract can be announced to many participants (aka bidders) that can either be interested in the contract and then bid or simply reject it and leave the contract net completely.

with jBPM 7, contract net can be modelled as a case definition where individual phases of the protocol can be externalised to processes to carry on with additional work. This improves readability and at the same time promotes reusability of the implementation.

Announce contract and Offer contract are separate processes that can be implemented separately according to needs. For this basic showcase they are based on human decision and look as follows

Each of the participants of the contract net will have dedicated instance of the announce contract process. Each should make the decision if they will place a bid or not. In case they won't do it at all, main contract net case definition keeps a timer event on them to remove given bidder if the deadline was reached.

As soon as all bidders replied (or time for reply elapsed) there are set of business rules that will evaluate all provided bids and select only one. Once it is selected an Offer contract subprocess will be initiated - after milestone of selecting a bid is completed.

So the bidder who placed the selected bid will get the "Work on contract" task assigned to actually perform the work. Once done the worker indicates if the work was done or she failed at doing the work. In case of successful completion of the work additional business rules are invoked to verify it.

Completion of the work (assuming it was done) will show the results of the work to the initiator for final verification. Once the results are reviewed the contract is ended - by that case instance is ready to be closed.

All this in action can be seen in the following screencast

Again, this is just basic implementation but shows the potential that can be unleashed to build advanced Contract Net Protocol solutions.

Complete project that can be easily imported into workbench and executed in KIE server can be found here.

To start the case instance you can use following payload that includes data (both contract and bidders) and case role assignments

{
    "case-data": 
    {
        "contract": 
        {
            "Contract": 
            {
                "name" : "jBPM contract",
                "description" : "provide development expertise for jBPM project",
                "price" : 1234.40
            }
        },
        "bidders": [
        {
            "Bidder": 
            {
                "id" : "maciek",
                "name" : "Maciej Swiderski",
                "email" : "maciek@email.com"   
            }
        },
        {
            "Bidder": 
            {
                "id" : "john",
                "name" : "John doe",
                "email" : "john@email.com"   
            }
        }
        ]
    },

    "case-user-assignments": 
    {
        "Initiator": "mary",
        "Participant": "john"
    },

    "case-group-assignments": 
    {
        
    }
}

If you need more bidders just add them by copying the single bidder in the payload

[1] Source - https://en.wikipedia.org/wiki/Contract_Net_Protocol