Keep your jBPM environment healthy

Once jBPM is deployed to given environment and is up and running most of actual maintenance requirements come into the picture. Running BPM deployment will have different maintenance life cycle depending on the personas involved:

• business users would need to make sure latest versions of processes are in use
• administrators would need to make sure that entire infrastructure is healthy 
• developers would need to make sure all projects are available to their systems

In this article I'd like to focus on administrators to give them a bit of power to maintain jBPM environments in a easier way. So let's first look at what sort of thing they can be interested with....

jBPM when configured to use persistence will store its state into data base over JPA. That is regardless if jbpm-console/kie-wb is used or jBPM runs in embedded mode. Persistence can be divided into two sections:

• runtime data - current state of active instances (processes, tasks, jobs)
• audit data - complete view of all states of instances (processes, tasks, events, variables)

Above diagram presents only subset of data model for jBPM and aims at illustrating important parts from maintenance point of view.

Important information here is that "runtime data" is cleaned up automatically on life cycle events:

• process instance information will be removed upon process instance completion
• work item information will be removed upon work item completion
• task instance information (including content) will be removed upon completion of a process instance that given tasks belongs to
• session information clean up depends on the runtime strategy selected
• singleton - won't be removed at all
• per request will be removed as soon as given request ends
• per process instance - will be removed when process instance mapped to given session completes (or aborts)
• executor's request and error information is not removed

So far so good, we have cleanup procedure in place but at the same time we loose all trace of process instances being executed at all. In most of the case this is not an acceptable solution...

And because of that there are audit data tables available (and used by default) to keep trace of what has been done, moreover it does keep track of what is happening in the environment as well. So it is actually great source of information in any given point in time. Thus name audit data might be slightly misleading ... but don't worry it is the first class citizen and is actually used by jbpm services  to provide you with all the details about current view on past and present.

So that puts us in tight spot - that data is gathered in audit tables but we do not have control over how long would that be stored in these tables. In environments that do operate on large number of process instances and task instances this might be seen as a problem. To help with this maintenance burden a clean up procedure has been provided (from version 6.2) that will allow two approaches to the topic:

• automatic clean up as scheduled job running in background on defined intervals
• manual clean up by taking advantage of the audit API

LogCleanupCommand

LogCleanupCommand is jbpm executor command that consists of logic to clean up all (or selected) audit data automatically. That logic is simply taking advantage of audit API to clean it up but provides one significant benefit - it can be scheduled and executed repeatedly by using reoccurring jobs feature of jbpm executor. Essentially this means that once job completes it provides information to the jbpm executor if and when next instance of this job should be executed. By default LogCleanupCommand is executed one a day from the time it was scheduled for the first time. It of course can be configured to run on different intervals.

NOTE: LogCleanupCommand is not registered to be executed out of the box to do not remove data without explicit request so it needs to be started as new job, see short screen cast on how to do it.

LogCleanupCommand comes with several configuration options that can be used to tune the clean up procedure.

Name	Description	Is exclusive
SkipProcessLog	Indicates if clean up of process instance, node instance and variables log cleanup should be omitted (default false)	No, can be used with other parameters
SkipTaskLog	Indicates if task audit and task event logs cleanup should be omitted (default false)	No, can be used with other parameters
SkipExecutorLog	Indicates if jbpm executor entries cleanup should be omitted (default false)	No, can be used with other parameters
SingleRun	Indicates if the job should run only once default false	No, can be used with other parameters
NextRun	Date for next run in time expression e.g. 12h for jobs to be executed every 12 hours, if not given next job will run in 24 hours from time current job completes	No, can be used with other parameters
OlderThan	Date that logs older than should be removed - date format YYYY-MM-DD, usually used for single run jobs	Yes, cannot be used when OlderThanPeriod is used
OlderThanPeriod	Timer expression that logs older than should be removed - e.g. 30d to remove logs older than 30 day from current time	Yes, cannot be used when OlderThan is used
ForProcess	Process definition id that logs should be removed for	No, can be used with other parameters
ForDeployment	Deployment id that logs should be removed for	No, can be used with other parameters
EmfName	Persistence unit name that shall be used to perform delete operations	N/A

Another important aspect of the LogCleanupCommand is that it protects the data it removes by making sure it won't delete active instances such as still running process instances, task instance or executor jobs.

NOTE: Even though there are several options to use to control what data shall be removed, recommended is to always use date as all audit data tables do have timestamp while some do not have other parameters (process id or deployment id).



A short screencast shows how LogCleanupCommand can be used in practice. It shows to scenarios (two execution of a command) where both are just single run:

• first that attempts to remove everything that is older than 1 day
• second that removes everything that is older than current time - not parameter for date is given

For the first run we only see that one job has been removed as only that met the criteria to be older than 1 day and all other was started same day. Then the second run that removes everything that was completed did actually removed them as expected.

Manual cleanup via audit API

Instead of having automatic cleanup of jobs, administrators can make use of audit API to do the clean up manually with more control over parameters to control what is to be removed. Audit API is divided in three areas (same as shown on the diagram) that covers different parts of the environment:

• process audit to clean up process, node and variables logs via jbpm-audit module
• task audit to clean up tasks and task events via jbpm-human-task-audit module
• executor jobs to cleanup jbpm executor jobs and errors via jbpm-executor module

API cleanup support is done in hierarchical way so in case all needs to be cleaned up it's enough to take the last audit service from hierarchy and all operations will be available.

• org.jbpm.process.audit.JPAAuditLogService
• org.jbpm.services.task.audit.service.TaskJPAAuditService
• org.jbpm.executor.impl.jpa.ExecutorJPAAuditService

Example 1 - remove completed process instance logs

JPAAuditLogService auditService = new JPAAuditLogService(emf);
ProcessInstanceLogDeleteBuilder updateBuilder = auditService.processInstanceLogDelete().status(ProcessInstance.STATE_COMPLETED);
int result = updateBuilder.build().execute();

Example 2 - remove task audit logs for deployment org.jbpm:HR:1.0

TaskJPAAuditService auditService = new TaskJPAAuditService(emf);
AuditTaskInstanceLogDeleteBuilder updateBuilder = auditService.auditTaskInstanceLogDelete().deploymentId("org.jbpm:HR:1.0");
int result = updateBuilder.build().execute();

Example 3 - remove executor error and requests

ExecutorJPAAuditService auditService = new ExecutorJPAAuditService(emf);
ErrorInfoLogDeleteBuilder updateBuilder = auditService.errorInfoLogDeleteBuilder().dateRangeEnd(new Date());
int result = updateBuilder.build().execute();

RequestInfoLogDeleteBuilder updateBuilder = auditService.requestInfoLogDeleteBuilder().dateRangeEnd(new Date());
result = updateBuilder.build().execute();

NOTE: when removing jbpm executor entries make sure first error info is removed before request info can be removed due to constraints setup on data base.

See API for various options on how to configure cleanup operations:

• AuditTaskInstanceLogDeleteBuilder
• ErrorInfoLogDeleteBuilder
• NodeInstanceLogDeleteBuilder
• ProcessInstanceLogDeleteBuilder
• RequestInfoLogDeleteBuilder
• TaskEventInstanceLogDeleteBuilder
• VariableInstanceLogDeleteBuilder

Equipped with these features jBPM environment can be kept clean and healthy for long long time without much effort.

Ending always same way - feedback is more than welcome.

Process instance migration made easy

jBPM 6 comes with an excellent deployment model based on knowledge archives which allows different versions of given project to run in parallel in single execution environment. That is very powerful but at the same time brings some concerns about how to deal with them, to name few:

• shall users run both old and new version of the processes?
• what shall happen to already active process instances started with previous version?
• can active versions be migrated to newer version and vice versa?

While there might be other concerns, one of the most frequently asked question is such situation is: can I migrate active process instance?

So can we do process migration with jBPM?

The straight answer is - YES

... but it was not easily available to be performed via jbpm console (aka kie-workbench). This article is about to introduce solution to this limitation by providing ... knowledge archive that can be deployed to your installation and simply used to migrate any process instance. I explicitly use term "migrate" instead of upgrade because it can actually be used in both directions (from lower to higher version or from higher to lower version).

So there are quite few things that might happen when such operation is performed. All depends on the changes between versions of the process definition that are part of the migration. So what does this process migration comes with:

• it can migrate from one process to another within same kjar
• it can migrate from on process to another across kjars
• it can migrate with node mapping between process versions 

While the first two options are simple, the third one might require some explanation. What is node mapping? While doing changes in process versions we might end up in situation that nodes/activities are replaced with another node/activity and by that when migrating between these versions mapping needs to take place. Another aspect is when you would like to skip some nodes in current version (see second example).

NOTE: mapping will happen only for active nodes of the process instance that is being migrated.

Be careful what you migrate...

Migration does not affect any data so please take that into account when performing migration as in case there were changes on data level process instance migration will not be able to resolve potential conflicts and that might lead to problems after migration.

To give you some heads up about how does it work, here comes two screencasts that showcase its capabilities in actions. For this purpose we use our standard Evaluation process that is upgraded with new version and active process instance is migrated to next version.

Simple migration of process instance

This case is about showing how simple it can be to migrate active process instance from one version to another:

• default org.jbpm:Evaluation:1.0 project is used which consists of single process definition - evaluation with version 1
• single process instance is started with this version
• after it has been started, new version of evaluation process is created
• upgraded version is then released as part of org.jbpm:Evaluation:2.0 project with process version 2
• then migration of the active process instance is performed
• results of the process instance migration is the illustrated on process model of active instance and as outcome of the process instance migration

Process instance migration with node mapping

In this case, we go one step further and add another node to Evaluation process (version 2) to skip one of the nodes from original version. So to do that, we need to map nodes to be migrated. Steps here are almost the same as in first case with the difference that we need to go over additional steps to collect node information and then take manual decision (over user task) what nodes are mapped to new version. Same feedback is given about results.

Ready to give it a go?

To play with it, make sure you have jBPM version 6.2 (currently available at CR level from maven repository but soon final release will be available) and then grab this repository into your jbpm console (kie-wb) workspace - just clone it directly in kie-wb. Once it's cloned simply build and deploy and you're ready to migrate any process instance :).

Feedback, issues, ideas for improvements and last but not least contribution is more than welcome.

Cross framework services in jBPM 6.2

jBPM version 6 comes with quite a few improvements that allow developers build their own systems with BPM and BRM capabilities just to name few:

• jar based deployment units - kjar
• deployment descriptors for kjars
• runtime manager with predefined runtime strategies
• runtime engine with configured components
• KieSession
• TaskService
• AuditService (whenever persistence is used)

While these are certainly bringing lots of improvements in embedability of jBPM into custom application they do come with some challenges in terms of how they can be consumed. Several pieces need to come along to have it properly supported and reliably running:

• favor use of RuntimeManager and RuntimeEngine whenever performing work instead of using cached ksession and task service instance
• Cache only RuntimeManager not RuntimeEngine or runtime engine's components
• Creating runtime manger requires configuration of various components via runtime environment - way more simplified compared to version 5 but still...
• On request basis always get new RuntimeEngine with valid context, work with ksession, task service and then dispose runtime engine

All these (and more) were sometimes forgotten or assumed will be done automatically while they weren't. And even more issues could arise when working with different frameworks - CDI, ejb, spring, etc.

Rise of jBPM services (redesigned in version 6.2)

Those who are familiar with jBPM console (aka kie workbench) code base might already be aware of some services that were present from version 6.0 and through 6.1. Module that encapsulated these services is jbpm-kie-services. This module was purely written with CDI in mind and all services within it were CDI based. There was additional code to ease consumption of them without CDI but that did not work well - mainly because as soon as the code was running in CDI container (JEE6 application servers) CDI got into the way and usually caused issues due to unsatisfied dependencies.

So that (obviously not only that :)) brought us to a highly motivated decision - to revisit the design of these services to allow more developer friendly implementation that can be consumed regardless of what framework one is using.

So with the design we came up with following structure:

• jbpm-services-api - contains only api classes and interfaces
• jbpm-kie-services - rewritten code implementation of services api - pure java, no framework dependencies
• jbpm-services-cdi - CDI wrapper on top of core services implementation
• jbpm-services-ejb-api - extension to services api for ejb needs
• jbpm-services-ejb-impl - EJB wrappers on top of core services implementation
• jbpm-services-ejb-client - EJB remote client implementation - currently only for JBoss

Service modules are grouped with its framework dependencies, so developers are free to choose which one is suitable for them and use only that. No more issues with CDI if I don't want to use CDI :)

Let's now move into the services world and see what we have there and how they can be used. First of all they are grouped by their capabilities:

DeploymentService

As the name suggest, its primary responsibility is to deploy (and undeploy) units. Deployment unit is kjar that brings in business assets (like processes, rules, forms, data model) for execution. Deployment services allow to query it to get hold of available deployment units and even their RuntimeManager instances.

NOTE: there are some restrictions on EJB remote client to do not expose RuntimeManager as it won't make any sense on client side (after it was serialized).

So typical use case for this service is to provide dynamic behavior into your system so multiple kjars can be active at the same time and be executed simultaneously.

// create deployment unit by giving GAV
DeploymentUnit deploymentUnit = new KModuleDeploymentUnit(GROUP_ID, ARTIFACT_ID, VERSION);
// deploy        
deploymentService.deploy(deploymentUnit);
// retrieve deployed unit        
DeployedUnit deployed = deploymentService.getDeployedUnit(deploymentUnit.getIdentifier());
// get runtime manager
RuntimeManager manager = deployed.getRuntimeManager();

Deployment service interface and its methods can be found here.

DefinitionService

Upon deployment, every process definition is scanned using definition service that parses the process and extracts valuable information out of it. These information can provide valuable input to the system to inform users about what is expected. Definition service provides information about:

• process definition - id, name, description
• process variables - name and type
• reusable subprocesses used in the process (if any)
• service tasks (domain specific activities)
• user tasks including assignment information
• task data input and output information

So definition service can be seen as sort of supporting service that provides quite a few information about process definition that are extracted directly from BPMN2.

String processId = "org.jbpm.writedocument";
        
Collection<UserTaskDefinition> processTasks = 
bpmn2Service.getTasksDefinitions(deploymentUnit.getIdentifier(), processId);
        
Map<String, String> processData = 
bpmn2Service.getProcessVariables(deploymentUnit.getIdentifier(), processId);
        
Map<String, String> taskInputMappings = 
bpmn2Service.getTaskInputMappings(deploymentUnit.getIdentifier(), processId, "Write a Document" );

While it usually is used with combination of other services (like deployment service) it can be used standalone as well to get details about process definition that do not come from kjar. This can be achieved by using  buildProcessDefinition method of definition service.

Definition service interface can be found here.

ProcessService

Process service is the one that usually is of the most interest. Once the deployment and definition service was already used to feed the system with something that can be executed. Process service provides access to execution environment that allows:

• start new process instance
• work with existing one - signal, get details of it, get variables, etc
• work with work items

At the same time process service is a command executor so it allows to execute commands (essentially on ksession) to extend its capabilities.

Important to note is that process service is focused on runtime operations so use it whenever there is a need to alter (signal, change variables, etc) process instance and not for read operations like show available process instances by looping though given list and invoking getProcessInstance method. For that there is dedicated runtime data service that is described below.

An example on how to deploy and run process can be done as follows:

KModuleDeploymentUnit deploymentUnit = new KModuleDeploymentUnit(GROUP_ID, ARTIFACT_ID, VERSION);
        
deploymentService.deploy(deploymentUnit);

long processInstanceId = processService.startProcess(deploymentUnit.getIdentifier(), "customtask");
     
ProcessInstance pi = processService.getProcessInstance(processInstanceId);     

As you can see start process expects deploymentId as first argument. This is extremely powerful to enable service to easily work with various deployments, even with same processes but coming from different versions - kjar versions.

Process service interface can be found here.

RuntimeDataService

Runtime data service as name suggests, deals with all that refers to runtime information:

• started process instances
• executed node instances
• available user tasks 
• and more

Use this service as main source of information whenever building list based UI - to show process definitions, process instances, tasks for given user, etc. This service was designed to be as efficient as possible and still provide all required information.

Some examples:
1. get all process definitions
Collection definitions = runtimeDataService.getProcesses(new QueryContext());

2. get active process instances

Collection instances = runtimeDataService.getProcessInstances(new QueryContext());

3. get active nodes for given process instance

Collection instances = runtimeDataService.getProcessInstanceHistoryActive(processInstanceId, new QueryContext());

4. get tasks assigned to john

List taskSummaries = runtimeDataService.getTasksAssignedAsPotentialOwner("john", new QueryFilter(0, 10));

There are two important arguments that the runtime data service operations supports:

• QueryContext
• QueryFilter - extension of QueryContext

These provide capabilities for efficient management result set like pagination, sorting and ordering (QueryContext). Moreover additional filtering can be applied to task queries to provide more advanced capabilities when searching for user tasks.

Runtime data service interface can be found here.

UserTaskService

User task service covers complete life cycle of individual task so it can be managed from start to end. It explicitly eliminates queries from it to provide scoped execution and moves all query operations into runtime data service.
Besides lifecycle operations user task service allows:

• modification of selected properties
• access to task variables
• access to task attachments
• access to task comments

On top of that user task service is a command executor as well that allows to execute custom task commands.

Complete example with start process and complete user task done by services:

  
long processInstanceId = 
processService.startProcess(deployUnit.getIdentifier(), "org.jbpm.writedocument");

List<Long> taskIds = 
runtimeDataService.getTasksByProcessInstanceId(processInstanceId);

Long taskId = taskIds.get(0);
     
userTaskService.start(taskId, "john");
UserTaskInstanceDesc task = runtimeDataService.getTaskById(taskId);
     
Map<String, Object> results = new HashMap<String, Object>();
results.put("Result", "some document data");
userTaskService.complete(taskId, "john", results);

That concludes quick run through services that jBPM 6.2 will provide. Although there is one important information left to be mentioned. Article name says it's cross framework services ... so let's see various in action:

• CDI - services with CDI wrappers are heavily used (and by that tested) in jbpm console - kie-wb. Entire execution server that comes in jbpm console is utilizing jbpm services over its CDI wrapper.
• Ejb - jBPM provides a sample ejb based execution server (currently without UI) that can be downloaded and deployed to JBoss - it was tested with JBoss but might work on other containers too - it's built with jbpm-services-ejb-impl module
• Spring - a sample application has been developed to illustrate how to use jbpm services in spring based application

The most important thing when working with services is that there is no more need to create your own implementations of Process service that simply wraps runtime manager, runtime engine, ksession usage. That is already there. It can be nicely seen in the sample spring application that can be found here. And actually you can try to use that as well on OpenShift Online instance here.

Go to application on Openshift Online

 Just logon with:

• john/john1
• mary/mary1

If there is no deployments available deploy one by specifying following string:

org.jbpm:HR:1.0

this is the only jar available on that instance.

And you'll be able to see it running. If you would like to play around with it on you own, just clone the github repo build it and deploy it. It runs out of the box on JBoss EAP 6. For tomcat and wildfly deployments see readme file in github.

That concludes this article and as usual comments and ideas for improvements are more than welcome.

As a side note, all these various framework applications built on top of jBPM services can simply work together without additional configuration just by configuring to be backed by the same data base. That means:

• deployments performed on any of the application will be available to all applications automatically
• all process instances and tasks can be seen and worked on via every application

that provides us with truly cross framework integration with guarantee that they all work in the same way - consistency is the most important when dealing with BPM :)

Let's WildFly with jBPM6

Let's go into the Wild....

It's been a while since JBoss AS7 as community was released and that's the default server used by jBPM in community distribution - when using jbpm installer but not only limited to that as it's frequently first choice to give it a try with jBPM 6.
No wonder as it's state of the art application server leading in the industry and very developer friendly. Some time ago JBoss AS has moved to its next version called WildFly and now is available as final release - there's even already second version released - 8.1.0.Final. You can download it from here.

Source - wildfly.org

All details about WildFly can be found at its web site but just to highlight the most important:

• Java EE 7
• extremely lightweight
• low memory footprint
• comes with latest version of best open source projects
• hibenrate
• infinispan
• resteasy
• weld
• IronJacamar
• HornetQ
• and more...

With that short introduction, it's high time to give jBPM 6 a WildFly as well. By that I mean have kie workbench (aka jbpm console) running on full speed on WildFly application server. The version chosen is the latest one - 8.1.0.Final due to some issues found in 8.0.0.Final that limited remote capabilities - REST interface did not work properly - though this is not an issue with 8.1.0.Final.

So jBPM comes with additional distribution of:

• kie-wb - fully featured workbench that includes both jbpm (process) and drools (rules) authoring and runtime capabilities
• kie-drools-wb - similar to what kie-wb is but it's stripped out from jbpm (process) capabilities and focusing mainly on rules authoring (including projects and repositories)

there are two dedicated web application archives tailored for WildFly application server that are named as follows:

• kie-wb-${version}-wildfly.war
• kie-drools-wb-${version}-wildfly.war

the assemblies are being built while this article is written so be bit patient and let jenkins build properly the artifacts so you'll be able to download it from JBoss Nexus repository. Keep in mind that these are still snapshots and might have some issues so for those that are willing to wait a bit in like a week from now 6.1.0.CR1 should be out in the wild as well.

Installation

Just download WildFly 8.1.0.Final (useful getting started guide), extract it and drop the dedicated wildfly distribution war file into standalone/deployments directory (you can also rename the war file to kie-wb.war to make it bound to kie-wb context path). Next same as for JBoss AS7 add users to the security domain so you can logon to the system once it's up and running:

• use add-user.sh/add-user.bat script in JBOSS_HOME/bin
• follow the wizard where important are the following
• Application Realm
• Roles: admin to have access to all capabilities of the system

Next is to ensure that server is started with standalone-full profile:

./standalone.sh --server-config=standalone-full.xml

then wait for application server to complete startup and visit http://localhost:8080/kie-wb

And that's it - jBPM 6 is running on latest JBoss application server - WildFly 8.1.0.Final.

Currently known issues

As usual with first major releases of core components there are some issues, fortunately not blockers. Obviously there might be still some other so feel free to report anything that you find not working on WildFly with jBPM6 either on user forum or as jira issues.

• class cast exceptions thrown by Errai only clean of CDI caches - see ERRAI-750 issue for details and to keep an eye on it
• on logout there are errors written to server log about broken pipe or closed stream caused by some Errai interaction - see ERRAI-754 issue for details and to keep track of the progress

While this issues might be bit annoying they do not cause any overall issues for the application (at least none were identified so far). The first one can he mitigated by hiding these warning via logging configuration. Add following to standalone-full.xml to hide these warnings:

       <logger category="EventDispatcher">

           <level name="ERROR"/>

       </logger>

As always, all comments and issues are more than welcome.

jBPM 6 on WebSphere - installation notes...

Brand new tooling for jBPM 6 is out for a while now and it was mainly targeting the open source world so by default it was deployable to JBoss AS 7.x / JBoss EAP 6.x and Tomcat 7. Now it's time to expand more into other containers so let's start with WebSphere (version 8.5.x).

NOTE: This article covers deployment of kie workbench (aka business central). Although this is just one option to make use of jBPM.

So first of all, we need to get WebSphere Application Server 8.5, if you don't have one already you can download the developer edition from here, which is free to be used for development work and not for production.
Once we have binaries downloaded, it's time to install it, I will not cover installation steps here as it's well documented on IBM documentation and there is no special requirements for WebSphere installation. Make sure that after installation you create a server profile, this article covers application server profile.

Tip: when running on Linux you can encounter problem on deployment, actually at upload time that manifest with very weird exception mostly referring to internal classes of WebSphere. To resolve that increase number of open files for example by issuing following command before starting the server:
ulimit -n 300000

Once WebSphere is properly installed and verification script confirm it is up and running we can move on to configuring server instance. Let's start with process definition configuration where we can specify JVM parameters such as heap size and system properties (aka custom properties):

Logon to WebSphere Administrative Console

Java Virtual Machine configuration

Go to Servers > Server Types > WAS app servers
Go to MyServer > Server Infrastructure > Process Definition > Java Virtual Machine

• Increase heap size 
• Initial heap size - 1024
• Max heap size - 2048

NOTE: that heap settings will depend on your environment so please consider these as starting point that might require some adjustments.

Go to Additional properties > Custom properties

• Set JVM system properties
• jbpm.ut.jndi.lookup set to jta/usertransaction
• kie.services.jms.queues.response set to jms/KIE.RESPONSE.ALL 
• kie.services.rest.deploy.async set to false

• org.apache.sshd.registerBouncyCastle set to true

• org.uberfire.domain set to WSLogin

This is the mandatory set of system properties to be set but more can be specified, check jbpm documentation for available system properties.

Security configuration

Go to Security > Global security

Ensure the option Enable Application security is checked. 

Go to Users and groups > Manage groups

Create groups: 

• Application groups :
• admin, analyst, developer, manager, user
• Task service groups
• Accounting, HR, IT, PM

Go to Users and groups > Manage users

Create a single user and add to selected groups above.

Register the SSL certificate from Github.com

This is needed in order to enable repository cloning from Github. This is the case of the kie-wb repository examples which are fetched from Github. 

Go to Security > SSL Certificate and Key Management > Manage endpoint security configurations

Go to Outbound section. Go to your server node within the tree. Select the HTTP subnode.

Go to Related Items > Key Stores and certificates

Select the row in the table named NodeDefaultTrustStore

Go to Additional properties > Signer certificates

Click button Retrieve from port

Fill out the form with these values: Host=github.com, Port=443, Alias=github.com

Click on Retrieve signer information button, then Ok, and finally, Save to master configuration.

Data source configuration

Create the JDBC provider

Left side panel, click on Resources > JDBC > JDBC Providers

Select the appropriate scope and click on the New button.

Fill out the form. For non-listed database types (i.e: H2, Postgres & Mysql) you need to provide the path to the JDBC driver jar plus the following class name:

•   H2            org.h2.jdbcx.JdbcDataSource                                 
•   Postgres   org.postgresql.xa.PGXADataSource                            
•   Mysql      com.mysql.jdbc.jdbc2.optional.MysqlConnectionPoolDataSource 

When you finish, click Ok. If there are no data entry errors, you should be back at the list of JDBC Providers, where you should now see your new provider displayed.

Create the data source

Left side panel, click on Resources > JDBC > Data sources

Select the appropriate scope and click on the New button.

Fill out the creation form. Set the following JNDI name jdbc/jbpm (must match the data source defined in the persistence.xml file contained in the kie-wb.war)

Select the existing JDBC provider you created. Click Next.

Keep clicking Next until Finish.

Save to master configuration.

Edit the datasource you just created and click on the Custom properties link.

Edit and fill the appropriate values required to set-up the connection. This depends on the database type.

•    H2            URL, user, password                                  
•    Postgres   serverName, databaseName, portNumber, user, password 
•    Mysql      serverName, databaseName, port, user, password       

   

JMS Configuration

Create Service Integration Bus that will host all the queues required by jbpm.

Go to Service integration > Buses

Next step is to assign bus members that will host message engine which is essentially application server instance.

Next let's create destinations for our queues, all of type Queue

Go to Service Integration > Busses > {bus name}

Destination resources > Destinations

and create following queues:

• KIE.AUDIT
• KIE.RESPONSE.ALL
• KIE.SESSION
• KIE.TASK

Then let's create the actual JMS resources such as Connection factories, Queues, Activation specifications

Connection factories

Create following connection factories to allow integration over JMS

• KIE.AUDIT - used for audit logging over JMS to make them asynchronous 
• KIE.RESPONSE.ALL - used for returning replies after processing incoming messages
• KIE.SESSION - used for incoming message for process operations e.g. start process
• KIE.TASK - used for incoming messages for task operations e.g. complete task

Queues

Create following queues

• KIE.AUDIT
• KIE.RESPONSE.ALL
• KIE.SESSION
• KIE.TASK

Activation specification

Create following activation specifications

• KIE.AUDIT
• KIE.RESPONSE.ALL
• KIE.SESSION
• KIE.TASK

Worth mentioning is that KIE.AUDIT activation specification should be additionally configured to prevent from processing message concurrently to avoid out of sync messages. Set "Maximum concurrent MDB invocations per endpoint" to 1.

Deploy the application

Upload the WAR file

Go to Applications > Application types > Websphere enterprise applications

Click on Install, select the kie-wb-was8.war file from your local filesystem. Click Next

From here, you will be asked several deployments settings.

You'll need to select the datasource created above as the datasource to be used by the application.

Screen Bind listeners for message-driven beans - select for every bean Activation Specification and fill the corresponding activation specification JNDI name into Target Resource JNDI Name (e.g. jms/activation/KIE.SESSION). You may also specify Destination JNDI name using JNDI name of the appropriate JMS queue (e.g. jms/queue/KIE.SESSION).

We also recommend to set is the context path of the webapp to kie-wb.

Screen Map resource references to resources - for both beans provide JNDI name of KIE.RESPONSE.ALL connection factory (e.g. jms/conn/KIE.RESPONSE.ALL).

Application settings

Go to Applications > Application types > Websphere enterprise applications > kie-wb app > Security role to user/group mapping

Select the five BPMS roles: admin, analyst, developer, manager, user.

Click on Map Special Subjects and select the All Authenticated in Application's Realm option.

Go to Applications > Application types > Websphere enterprise applications > kie-wb app > Class loading and update detection

Ensure the following radio buttons are checked:

• Classes loaded with local class loader first (parent last)
• Single class loader for application

Configure SSH GIT server with proper security libraries

Create shared library that will bring in Bouncy Castle library for enhanced security that is required for GIT SSH server to operate properly. Make sure the it will have proper version of bouncy castle defined (tested with bcprov-jdk16-1.46.jar).
Next make a reference to the shared library for the jbpm console application.

Save the configurations to the master and restart the server.

Dashbuilder (BAM) configuration

Follow instructions in this article for installing the dash builder application on WebSphere Application Server.

In addition to this you might want to reduce logging level for class 

com.ibm.websphere.rsadapter.WSCallHelper

 - by reduce, set it to war level to avoid spamming your server log whenever this class is used.

This can be done in Troubleshooting -> Logs and trace -> Change log details levels

Very important to note when using both kie-wb and dash builder is that both must use same data base (in come data bases even same data base user) as dash builder depends on tables created and populated by kie-wb so dash builder to work proper (and to actually start the application correctly).

Session management settings

in case of running in combination with dash builder (BAM component) it's is recommended to set following session management property to avoid issues with SSO between kie-wb and dash builder.

Go to:
Application Servers -> {servername} -> Session management -> Custom properties

and add custom property:
name: InvalidateOnUnauthorizedSessionRequestException
value: true

Once restarted you should be able to access the kie-wb application by typing the following URL: http://localhost:9080/kie-wb (unless you used another context root at deploy time).

Have fun and as usual all comments are more than welcome.

Reuse your business assets with jBPM 6

As described in the article about deployment model in jBPM 6, business assets are included in so called knowledge archives - kjars. Kjar is nothing more than regular jar file with knowledge descriptor - kmodule.xml that is under control of Apache Maven. But there is more in this...

Since kjars are managed by maven one kjar can declare other kjar as its dependency. Which means that assets included in the dependent one will be available for execution as well. That is all available when working with jbpm console (aka kie workbench). So to provide more information on this let's look at an example.

Use case definition

There is a need to prepare a simple registration process that will ask user who starts the process for basic personal information like name, age. Then there will be some business rules that will evaluate if that person is adult or a teenager. Once completed it will be presented to reviewer to see the details of the evaluation. Last but not least is to proceed with actual registration in the system. So we can see that there is part of this logic that might be a very well considered a reusable - part that is responsible for gathering information about a person.

So let's design it this way:

• first project - reusable project - will actually deal only with gathering personal information and presenting that to verifying personnel after business rules were evaluated.

           As you can see, besides business assets data model is included in reusable-project so it can

           be used by projects that declare it as dependency, same as with any other Maven based project.

• second project - top project - will provide additional process logic on top of the common collect info procedure and do registration stuff.

So, this is the structure of the projects we are going to use to support the case described.

What must be actually done to make this work? First of all the reusable project needs to be created as it will be a dependency of the top project so it must exists. In the reusable project we need to define knowledge base and knowledge session to disable auto deploy, as we don't want to have it on runtime as a standalone project but included in top project.  With that said we create:

• one knowledge base (kbase) - ensure it's not marked as default
• one stateful knowledge session (ksession) - ensure it's not marked as default
• include all packages - use * wildcard for it

Note: we do this to illustrate what configuration options we have here and to ensure that auto deployment to runtime environment will not be possible - no default knowledge base and knowledge session.

Let's create this collect info process that could look like this:

a simple process, two user tasks and business rule task. So what will it do:

• Enter person details will collect personal information from a user - name and age
• Evaluate will execute business rules that are responsible for marking given person as adult if (s)he is older than 21
• Verify simply presents the results of the process

Both rule and user tasks operate on data model, to be more specific org.jbpm.test.Person class. It was created using Data Modeler and places inside the kjar.

Next process and tasks forms are generated and altered to ask for the right information only. Person class includes three properties:

• name - string
• age - integrer
• adult - boolean

Since we have business rules for evaluating if user is adult or teenager we don't want to ask for it via forms. So these are removed from the "Enter person details" task.

With all that we are ready to build the project so it can be used as dependency. So, just hit the build and deploy button in Project Editor and observe Problems Panel. If everything went ok, it will display single error saying that deployment failed because it cannot find defaultKBase. That is expected as we defined knowledge base and knowledge session that is not marked as default and thus auto deploy fails. But the kjar is available in maven repository so it can be used as dependency.

Next we create top project and add single dependency to it of reusable-project. This is done in Project Editor in Dependencies list section. You can add it from repository as it's already built. Next we need to define knowledge base and knowledge session:

• one knowledge base (kbase-top) - ensure it's marked as default
• one stateful knowledge session (ksession-top) - ensure it's marked as default
• include all packages - use * wildcard for it
• include kbase defined in reusable project - kbase

Note: make sure that names do no collide between kjars as that will result in failing compilation of knowledge base.

Now we are ready to create our top level process that could look like this:

Again simple process, that will:

• log incoming request for registration using script task - Log registration
• invoke common part to collect info - Collect Info - by invoking the reusable project process, rules, forms etc
• and lastly will show the outcome of the collection process for approval

The most important part here is that Collect Info activity will use process (and other assets) from another kjar thanks to usage of maven dependencies and kbase inclusion.

To examine this example in details you can clone the repository in your jbpm console (kie workbench) and build both projects - first reusable project and then top project. 

This illustrates only the top of the mountain that is provided by maven dependencies and knowledge inclusion in jBPM 6. I would like to encourage you to explore this possibilities and look at options to reuse your knowledge in structured and controlled way - remember this is all standardized by maven so things like versioning are supported as well.

This is a feature that will be available in 6.1.0 so feel free to jump into the wild right away by making use of it in nightly builds. Comments and feedback are welcome.

jBPM 6 - store your process variables anywhere

Most of jBPM users is aware of how jBPM stores process variable but let's recap it here again just for completeness.

NOTE: this article covers jBPM that uses persistence as without persistence process variables are kept in memory only.

 jBPM puts single requirement on the objects that are used as process variables:

• object must be serializable (simply must implement java.io.Serializable interface)

with that jBPM engine is capable to store all process variables as part of process instance using marshaling mechanism that is backed by Google Protocol Buffers. That means actual instances are marshaled into bytes and stored in data base. This is not always desired especially in case of objects that are actually not owned by the process instance. For example:

• JPA entities of another system
• documents stored in document/content management system 
• etc

Luckily, jBPM has a solution to that as well called pluggable Variable Persistence Strategy. Out of the box jBPM provides two strategies:

• serialization based, mentioned above that actually works on all object types as long as they are serializable (org.drools.core.marshalling.impl.SerializablePlaceholderResolverStrategy)
• JPA based that works on objects that are entities (org.drools.persistence.jpa.marshaller.JPAPlaceholderResolverStrategy)

Let's spend some time on the JPA based strategy as it might become rather useful in many cases where jBPM is used in embedded mode. Consider following scenario where our business process uses entities as process variables. The same entities might be altered from outside of the process and we would like to keep them up to date within the process as well. To do so, we need to use JPA based strategy for variable persistence that is capable of storing entities in data base and then retrieving them back.

To configure variable persistence strategy you need to place it into the environment that is the used when creating knowledge sessions. Note that the order of the strategies is important as they will be evaluated which one will be used in the order they are given. best practice is to always set the serialization based strategy to be the last one. 

An example how you can use it with RuntimeManager:

// create entity manager factory
EntityManagerFactory emf = Persistence.createEntityManagerFactory("org.jbpm.sample");

RuntimeEnvironment environment = 

     RuntimeEnvironmentBuilder.Factory.get().newDefaultBuilder()
     .entityManagerFactory(emf) 
     .addEnvironmentEntry(EnvironmentName.OBJECT_MARSHALLING_STRATEGIES, 

          new ObjectMarshallingStrategy[]{
// set the entity manager factory for jpa strategy so it 

// know how to store and read entities     
               new JPAPlaceholderResolverStrategy(emf),
// set the serialization based strategy as last one to

// deal with non entities classes
               new SerializablePlaceholderResolverStrategy( 

                          ClassObjectMarshallingStrategyAcceptor.DEFAULT  )
         })  
     .addAsset(ResourceFactory.newClassPathResource("cmis-store.bpmn"), 

               ResourceType.BPMN2)

     .get();
// create the runtime manager and start using entities as part of your process  RuntimeManager manager = 

     RuntimeManagerFactory.Factory.get().newSingletonRuntimeManager(environment);

Once we know how to configure it, let's take some time to understand how it actually works. First of all, every process variable on the time when it's going to be persisted will be evaluated on the strategy and it's up to the strategy to accept or reject given variable, if accepted only that strategy will be used to persist the variable, if rejected other strategies will be consulted.

Note: make sure that you add your entity classes into persistence.xml that will be used by the jpa strategy

JPA will accept only classes that declares a field with @Id annotation (javax.persistence.Id) that allows us to ensure we will have an unique id to be used when retrieving the variable.
Serialization based one simply accepts all variables by default and thus it should be the last strategy inline. Although this default behavior can be altered by providing other acceptor implementation.

Once the strategy accepts the variable it performs marshaling operation to store the variable and unmarshaling to retrieve the variable from the back end store (of the type it supports).

In case of JPA, marshaling will check if entity is already stored entity - has id set - and:

• if not, it will persist the entity using entity manager factory that was assigned to it
• if yes, it will merge it with the persistence context to make sure up to date information is stored

when unmarshaling it will use the unique id of the entity to load it from the database and provide as process variable. It's that simple :)

With that, we quickly covered the default (serialization based) strategy and JPA based strategy. But the title of this article says we can store variables anywhere, so how's that possible?
It's possible because of the nature of variable persistence strategies - they are pluggable. We can create our own and simply add it to the environment and process variables that meets the acceptance criteria of the strategy will be persisted by that given strategy. To not leave you with empty hands let's look at another implementation I created for purpose of this article (although when working on it I believe it will become more than just example for this article).

Implementing variable persistence strategy is actually very simple, it's a matter of implementing single interface: org.kie.api.marshalling.ObjectMarshallingStrategy

public interface ObjectMarshallingStrategy {
    
    public boolean accept(Object object);

    public void write(ObjectOutputStream os,
                      Object object) throws IOException;
    
    public Object read(ObjectInputStream os) throws IOException, ClassNotFoundException;
    

    public byte[] marshal( Context context,
                           ObjectOutputStream os,
                           Object object ) throws IOException;
    
    public Object unmarshal( Context context,
                             ObjectInputStream is,
                             byte[] object,
                             ClassLoader classloader ) throws IOException, ClassNotFoundException;

    public Context createContext();
}

the most important methods for us are:

• accept - decides if this strategy will be responsible for persistence of given object
• marshal - performs operation to store process variable
• unmarshal - performs operation to retrieve process variable

the other remaining are for backward compatibility reasons with old marshaling framework prior to protobuf, so it's not mandatory to be implemented but it's worth to put the logic there too as most likely it will be same as for marshal (write) and unmarshal (read).

So the mentioned example implementation is for storing and retrieving process variables as document from Content/Document management systems that support access to the repository using CMIS. I used Apache Chemistry as the integration component that can easily talk to CMIS enabled systems like for example Alfresco.

So first bit of requirements:

• process variables must be of certain type to be stored in the content repository
• documents (process variables stored in cms) can be:
• created
• updated (with versioning)
• read
• process variables must be kept up to date

so all these sounds simple and of course that's the point to keep it simple at this point. CMS can be used for much more but we wanted to get started and then enhance it if needed. So the implementation of strategy org.jbpm.integration.cmis.impl.OpenCMISPlaceholderResolverStrategy supports following:

• when marshaling
• create new documents if it does not have object id assigned yet
• update document if it has already object id assigned
• by overriding existing content
• by creating new major version of the document 
• by creating new minor version of the document
• when unmarshaling
• load the content of the document based on given object id

So you can actually use this strategy for:

• creating new documents from the process based on custom content
• update existing documents with custom content
• load existing documents into process variable based on object id only

These are very high level details but let's look at the actual code that does that "magic", let's start with marshal logic - note that is bit simplified for readability here and complete code can be found in github.

public byte[] marshal(Context context, ObjectOutputStream os, Object object) throws IOException {
 Document document = (Document) object;
 // connect to repository
 Session session = getRepositorySession(user, password, url, repository);
 try {
  if (document.getDocumentContent() != null) {
   // no object id yet, let's create the document
   if (document.getObjectId() == null) {
    Folder parent = ... // find folder by path
    if (parent == null) {
     parent = .. // create folder
    }
    // now we are ready to create the document in CMS
   } else {
      // object id exists so time to update     
   }
  }
 // now nee need to store some info as part of the process instance
 // so we can later on look up, in this case is the object id and class
 // that we use as process variable so we can recreate the instance on read
     ByteArrayOutputStream buff = new ByteArrayOutputStream();
     ObjectOutputStream oos = new ObjectOutputStream( buff );
     oos.writeUTF(document.getObjectId());
     oos.writeUTF(object.getClass().getCanonicalName());
     oos.close();
     return buff.toByteArray();
 } finally {
  // let's clear the session in the end
  session.clear();
 }
}

so as you can see, it first deals with the actual storage (in this case CMIS based repository) and then save some small details to be able to recreate the actual object instance on reading. It stores objectId and fully qualified class name of the process variable. And that's it. Process variable of type Document will be stored inside content repository.

Then let's look at the unmarshal method:

public Object unmarshal(Context context, ObjectInputStream ois, byte[] object, ClassLoader classloader) throws IOException, ClassNotFoundException {
 DroolsObjectInputStream is = new DroolsObjectInputStream( new ByteArrayInputStream( object ), classloader );
 // first we read out the object id and class name we stored during marshaling
 String objectId = is.readUTF();
 String canonicalName = is.readUTF();
 // connect to repository
 Session session = getRepositorySession(user, password, url, repository);
 try {
  // get the document from repository and create new instance ot the variable class
  CmisObject doc = .....
  Document document = (Document) Class.forName(canonicalName).newInstance();
  // populate process variable with meta data and content
  document.setObjectId(objectId);
  document.setDocumentName(doc.getName());   
  document.setFolderName(getFolderName(doc.getParents()));
  document.setFolderPath(getPathAsString(doc.getPaths()));
  if (doc.getContentStream() != null) {
   ContentStream stream = doc.getContentStream();
   document.setDocumentContent(IOUtils.toByteArray(stream.getStream()));
   document.setUpdated(false);
   document.setDocumentType(stream.getMimeType());
  }
  return document;
 } catch(Exception e) {
  throw new RuntimeException("Cannot read document from CMIS", e);
 } finally {
  // do some clean up...
  is.close();
  session.clear();
 }
}

nothing more that the logic to get ids and class name so the instance can be recreated and load the document from cms repository and we're done :)

Last but not least, the accept method.

public boolean accept(Object object) {
    if (object instanceof Document) {
 return true;
    }
    return false;
}

and that is all that is needed to actually implement your own variable persistence strategy. The only thing left is to register the strategy on the environment so it will be evaluated when storing/retrieving variables. It's done the same way as described for JPA based on.

Complete source code with some tests showing complete usage case from process can be found here. Enjoy and feel free to provide feedback, maybe it's worth to start producing repository of such strategies so we can have rather rich set of strategies to be used...