KIE Server clustering and scalability

This article is another in KIE Server series and this time we'll focus on clustering and scalability.

KIE Server by its nature is lightweight and easily scalable component. Comparing to execution environment included in KIE workbench it can be summarized with following:

• allows to partition based on deployed containers (kjars)
• in workbech all containers are deployed to same runtime
• allows to scale individual instances independently from each other
• in workbench scaling workbench means scaling all kjars
• can be easily distributed across network and be managed by controller (workbench by default)
• workbench is both management and execution which makes it a single point of failure
• clustering of KIE Server does not include any additional components in the infrastructure 
• workbench requires Zookeeper and Helix for clustered git repository

So what does it mean to scale KIE Server?

First of all it allows administrators to partition knowledge between different KIE Server instances. With that said, HR department related processes and rules can run on one set of KIE Server instances, while Finance department will have its own set of KIE Server instances. By that each department's administrator can easily scale based on the needs without affecting each other. That gives us unique opportunity to really focus on the components that do require additional processing power and simply add more instances - either on the same server or on different distributed across your network.

Let's look at most common runtime architecture for scalable KIE Server environment

As described above the basic runtime architecture will consists of multiple independent sets of KIE Servers where number of actual server instances can vary. In the above diagram all of them have three instances but in reality they can have as many (or as little) as needed.

Controller in turn will have three server templates - HR, Finance and IT. Each server template is then defined with identifier which is used by KIE Server instances via system property called org.kie.server.id.

In above screenshot, server templates are defined in the controller (workbench) which becomes single point of configuration and management of our KIE Servers. So administrators can add or remove, start or stop different containers and controller is responsible for notifying all KIE Server instances (that belongs to given server template) with performed operations. Moreover when new KIE server instances are added to the set they will directly receive all containers that should be started and by that increase processing power.

As mentioned, this is the basic setup, meaning actual usage of the server instances is by calling them directly - each individual KIE Server instance. This is a bit troublesome as users/callers will have to deal with instances that are down etc. So to solve this we can put load balancer in front of the kie servers and then utilize that load balancer to the heavy lifting for us. So users will simply call single URL which is then configured to work with all instances in the back end. One of the choices of a load balancer is Apache HTTP with ModCluster plugin for efficient and highly configurable load balancer.

In version 7, KIE Server client will come with pluggable load balancer implementation so whenever using KIE Server client users could simply skip additional load balancer as infrastructure component. Though it will provide load balancing and failure discovery support it's client side load balancer which has no knowledge of underlying backend servers and thus won't be as efficient as mod cluster can be.

So this covers, scalability of KIE Server instances as they can be easily multiplied to provide more power for execution and at the same time distribution both on the network and knowledge (containers) level. But looking at the diagram, a single point of failure is the controller. Remember that in managed mode (where KIE Server instances depend on controller) they are limited in case controller is down. Let's recap on how KIE Server interacts with controller:

• when KIE Server starts it attempts to connect to any of the defined controllers (if any)
• it will only connect to one once connection is successful
• controller will then provide list of containers to deploy and configuration
• based on this information KIE Server deploys and starts to serve requests

But what happens when none of the controllers can be reached when KIE Server starts? KIE Server will be pretty much useless as it does not know what container it should deploy. And will keep checking (with predefined intervals) if controller is available. So until controller is not available KIE Server has not containers deployed and thus won't process any requests - most likely response you'll get from KIE Server when trying to use it will be - no container found.

Note: This affects only KIE Servers that starts after controller went down, those that are currently running are not affected at all.

So to solve this problem workbench (and by that controller) should be scaled. Here default configuration of KIE workbench cluster applies, meaning with Apache Zookeeper and Apache Helix as part of the infrastructure.

In this diagram, we scale workbench by using Apache Zookeeper and Helix for cluster of GIT repository. This gives us replication between server instances (that runs workbench) and by that provides several controller endpoints (which are synchronized) to secure KIE Server instances can reach the controller and collect configuration and containers to be deployed.

Similar as it was for KIE Servers, controller can be either reached directly by independent endpoints or again fronted with load balancer. KIE Server allows to be given list of controllers so load balancer is not strictly required though recommended as workbench is also (or even primarily) used by end users who would be interested in load balanced environment as well.

So that would conclude the description of clustering and scalability of KIE server to gain most of it, let's now take a quick look what's important to know when configuring such setup.

Configuration

Workbench

We start with configuration of workbench - controller. The most important for controller is the authentication so connecting KIE Server instances will be authorized. By default KIE Server upon start will send request with Basic authentication corresponding to following credentials:

• username: kieserver
• password: kieserver1!

so to allow KIE Server to connect, make sure such user exists in application realm of your application server.

NOTE: the username and password can be changed on KIE Server side by setting following system properties:

• org.kie.server.controller.user
• org.kie.server.controller.pwd

This is the only thing needed on application server that hosts KIE workbench.

KIE Server

On KIE Server side, there are several properties that must be set on each KIE Server instance. Some of these properties must be same for all instances representing same server template defined in the controller.

• org.kie.server.id - identifier of the kie server that corresponds to server template id. this must be exactly the same for all KIE Server instances that represent given server template
• org.kie.server.controller - comma separated list of absolute URL to the controller(s). this must be the same for all KIE Server instances that represents given server template
• org.kie.server.location - absolute URL where this KIE Server instance can be reached. This must be unique for each KIE Server instances as it's going to be used by controller to notify requested changes (e.g start/stop container). 

Similar to how workbench authenticates request KIE Server does the same, so to allow controller to connect to KIE Server instance (based on given URL as org.kie.server.location) application realm of the server where KIE Server instances are running must have configured user. By default workbench (controller) will use following credentials:

• username: kieserver
• password: kieserver1!

so it must exist in application realm. In addition it must be member of kie-server role so KIE Server will authorize it to its REST api.

NOTE: the username and password can be changed on KIE Workbench side by setting following system properties:

• org.kie.server.user
• org.kie.server.pwd

There are other system properties that can be set (and most likely will be needed depending on what configuration of KIE Server is needed). For those look at the documentation.

This configuration applies to any way you run KIE Server - on standalone Wildfly, domain mode of Wildfly, Tomcat, WAS or WebLogic. It does not really matter as long as you follow the set of properties you'll be ready to go with clustered and scalable KIE Server instances that are tailored to your domain.

That would be all for today, as usual comments are more than welcome.

Community extension to KIE Server - welcome Apache Thrift

In previous articles about KIE Server I described how it can be extended to bring in more features to it, starting with enhanced REST endpoints, through building addition transport layers, and finishing at building custom kie server client implementations.

It didn't take long and we got official confirmation that it works!!!

Maurice Betzel, has done excellent job and implemented KIE Server extensions that brings in Apache Thrift into the picture. That allowed him to bridge the gap between Java and PHP to make use of rule evaluation using KIE Server.

KIE Server with Apache Thrift

I'd like to encourage every one to look at detailed description about Maurice's work and take it for a spin to see how powerful it is.

All the credit goes to Maurice and I'd like to thank you as well for keeping me in the loop and verifying extensions mechanism of KIE Server in real life.

jBPM UI extension on KIE Server

KIE Server that was first released in 6.3.0.Final with jBPM capabilities (among others) was purely focused on execution. Though it was lacking part of functionality BPM users expects:

• process diagram visualization 
• process instance diagram visualization
• process and task forms information 

Since KIE Server is execution server thus it does not come with any UI and to be able to interact with it a custom UI needs to be built. Technology used to build such UI does not really matter and is left to developers to choose. Though certain parts should be possible to get out from KIE Server to improve the UI capabilities.

One of the most desired use case is to be able to visualize state of given process instance - including graphical annotations about which nodes are active and which are already completed, showing complete flow of the process instance.

This has been added to KIE Server as part of jBPM UI extensions and provides following capabilities:

• display process definition diagram as SVG
• display annotated process instance diagram as SVG
• greyed out are completed nodes
• marked as red are active nodes
• display structure of process forms
• display structure of task forms

While displaying process diagrams is self-explanatory, then operation around forms might be bit confusing. So let's go over them first to understand their usage. 

Primary authoring environment is KIE workbench where users can build various assets such as processes, rules, decision tables, data model and forms. Forms in workbench are built with Form Modeler that allows good integration with process and task variables providing binding between inputs and outputs - how data are taken out from process/task variable and displayed in the form and vice-versa how form data is put back to process variables.

Since KIE Server does not provide any UI it does not allow to render task nor process forms. It simply expect the data to be given that will be mapped (by name) to their process or task variables. While this is completely ok from execution point of view, it's not so great from UI and data collection stand point. So to ease a bit in this area, KIE Server is now capable to return form structure that can be used later on to render the form with whatever UI technology/framework you like.

Let's take a test drive of it. We will use our well known HR example to guide you through the usage of this UI support jBPM extension to KIE Server.

Form operations

First endpoint we are going to discuss is to get process form for given process definition - similar to what you get when you start a process instance in workbench.

Endpoint:

• http://localhost:8230/kie-server/services/rest/server/containers/hr/forms/processes/hiring

Method:

• GET

where:

• hr - is container id
• hiring - is process id

When you issue this request you'll get following response:

You can notice few important properties there:

• form/name - hiring-taskform - is the name of the form built in form modeler - you'll find it in workbench under "Form definitions" section in Project explorer
• form/field/name is the name of the first field on that form
• under field properties you can find lots of details and depending on your form design you'll see more or less data, though still important
• fieldName
• fieldRequired
• readonly
• inputBinding
• outputBinding

This form structure directly translates to what KIE workbench will render when you start hiring process

Similar thing can be done for task forms with slightly different endpoint url as it refers to tasks (already active tasks)

Endpoint:

• http://localhost:8230/kie-server/services/rest/server/containers/hr/forms/tasks/123

Method:

• GET

where:

• hr - is container id
• 123 - is task id

same content as in case of process forms are returned for tasks. You can notice that there are different data filled for different fields. Like some have inputBinding set some have outputBinding set. 

So this structure represents this form rendered by workbench:

So with this you can build a custom renderer that is based on same form structure that was designed in Form Modeler that comes with KIE workbench.

Note: In the above example the content is XML but by changing the Accept header to be application/json you'll get JSON content instead.

Image operations

There are two operations available - get "pure" process definition diagram or get annotated process instance diagram.

To get process diagram use following endpoint 

Endpoint:

• http://localhost:8230/kie-server/services/rest/server/containers/hr/images/processes/hiring

Method:

• GET

where:

• hr - is container id
• hiring - is process id

and this is what you'll get in your browser

To get annotated process instance, first of all you have to have process instance active and one you have its process instance id you can issue following 

Endpoint:

• http://localhost:8230/kie-server/services/rest/server/containers/hr/images/processes/instances/123

Method:

• GET

where:

• hr - is container id
• 123 - is process instance id

and you'll get this

Here you can see that start event is greyed out and thus means it was already completed and currently process instance is in HR Interview task.

Returned content for image operations are SVG - where its MIME type is: application/svg+xml

so make sure you have your client capable of displaying SVG content to properly display the diagrams. Note that all major browsers do support SVG and if you can display process diagram in KIE workbench with given browser you'll be fine.

Now, the most important configuration parameter to enable image operations. KIE workbench by default does not store SVG version of the process so that means such SVG will not be included in kjar and thus won't be available to KIE Server. To be able to take advantage of this feature you need to enable it in workbench configuration files.

Enable SVG on store in workbench

Edit file jbpm.xml that is stored in (depending on what installation you have):

• jbpm installer: 
• jbpm-console.war/org.kie.workbench.KIEWebapp/profiles/jbpm.xml
• manual installation 
• kie-wb{version-container}.war/org.kie.workbench.KIEWebapp/profiles/jbpm.xml
• Red Hat JBoss BPMS: 
• business-central.war/org.kie.workbench.KIEWebapp/profiles/jbpm.xml

in this file you need to find 

        <storesvgonsave enabled="false"/>

and set it to true

        <storesvgonsave enabled="true"/>

Once this enabled (re)start workbench and go to your process definition to save it again (any modification will be required) and that will trigger SVG file for that process to be generated and stored in kjar.

Then deploy that kjar to KIE Server and you can enjoy KIE Server shipping process images for your custom UI.

That's it for the jBPM UI extensions that is coming with 6.4.0.Final very soon so stay tuned. 

Are you ready to dive into (wildfly) swarm?

KIE Server is a lightweight execution server that comes with various capabilities where out of the box are following:

• BRM - rules execution (Drools)
• BPM - business process execution, task management, background jobs (jBPM)
• BPM-UI - visualize your bpm components on runtime such as process definition and instance (since 6.4)
• BRP - business resource planning (Optaplanner) (since 6.4)

It's by default packaged as JEE application (web archive) and deployed to various containers, such as:

• JBoss EAP
• Wildfly
• Tomcat
• WebLogic
• WebSphere

While all this is already quite nice coverage, we don't stay idle and do work on bringing more to you. Let's see what's coming next...

All the hype about micro service is bringing in tons of new stuff that allows alternative approach for packaging and deployment of our systems or services if you like. Taking into consideration what capabilities KIE Server comes with it would be a crime to not take advantage of it to start building micro services with it. Instead of rewriting all the stuff in different way.

It's time to introduce Wildfly Swarm (to those that haven't heard about it yet) ...

Swarm offers an innovative approach to packaging and running JavaEE applications by packaging them with just enough of the platform to "java -jar" your application

So what Wildfly Swarm means in context of KIE Server?

Actually it means a lot:

• first of all it allows us to build executable jars that will bring in KIE Server capabilities to simple java -jar way of working with all it's power!
• next you can have a "executable kjar" just by starting it with argument that identifies kjar to be available for execution (Group Artifact Version)
• you can still run in managed mode - connected to controller and managed from within controller but without a need to provision your application server

With this in mind let's take a look at how to use it with Wildfly Swarm.

• Clone this repository kie-server-swarm into your local environment.
• Build the project with maven (mvn clean package)
• Make sure you run it with latest Maven otherwise you might run into build errors - I tested it with 3.3.9 so it works certainly with it
• Once it's successfully build you'll find following file inside the target folder
• kie-server-swarm-1.0-swarm.jar
• Now you're ready to rock with KIE Server on Wildfly Swarm

but before we start our KIE Server on Swarm, let's look at what options we have for the project we just built. This project, same as KIE Server, is modularized and allows us to pick only the things we are interested with. While KIE Server allows to disable extensions on runtime (via system properties) sometimes it does not make sense to bring in lots of dependencies if they are not going to be used.

So you can build the project with following profiles:

• BRM - includes BRM capability of the KIE Server that allows rules execution only
• no server components besides REST is configured
• build it with - mvn clean package -PBRM
• BPM - includes both BRM and BPM capabilities of the KIE Server - this is the default profile
• configures Swarm to have transactions and data sources enabled
• build it with - mvn clean package -PBPM or mvn clean package

So why it's important to have it done as profiles? It's because the size of resulting file (executable jar) will be smaller. Moreover it reduces number of things Swarm is going to configure and boot when we start our system. So keep this in mind as it might become handy one day or another :)

Let's get our hands dirty with running KIE Server on Wildfly Swarm

First thing, let's just start empty server that will let us manage it manually - creating containers, running rules and processes via REST api

Make sure you're in the project folder (where you executed maven build) and then simply run this command:

java -Dorg.kie.server.id=swarm-kie-server -Dorg.kie.server.location=http://localhost:8380/server -Dswarm.port.offset=300 -jar target/kie-server-swarm-1.0-swarm.jar

Wait for a while to boot Wildfly Swarm and KIE Server on it. Once it's completed you should be able to access it at http://localhost:8380/server

NOTE: since KIE Server requires authentication, whenever you attempt to access its REST endpoints you need to logon - by default you should be able to logon with kieserver/kieserver1!
you can customize users and roles by editing following files:

kie-server-swarm/src/main/config/security/application-users.properties

kie-server-swarm/src/main/config/security/application-roles.properties

Now let's examine a bit what all these parameters mean:

•  -Dorg.kie.server.id=swarm-kie-server - specifies the unique identifier of the kie server - it is important when running in managed mode but good to use it always to make it a habit
• -Dorg.kie.server.location=http://localhost:8380/server - specifies the actual location where our KIE Server is going to be available - this must be a direct URL to actual instance even it if's behind load balancer - again important when running in managed mode
• -Dswarm.port.offset=300  - sets global port offset to avoid port conflicts when running many instances of wildfly on same machine

Next, let's run our first executable KJAR... to do so we just extend the command from first run and add arguments to the execution

java -Dorg.kie.server.id=swarm-kie-server -Dorg.kie.server.location=http://localhost:8380/server -jar target/kie-server-swarm-1.0-swarm.jar org.jbpm:HR:1.0

as you can see the only difference is:
org.jbpm:HR:1.0

which is GAV of a KJAR that is going to be deployed upon start of KIE Server on Swarm. So with just single command line we have fully functional server with BPM capabilities and HR project deployed to it.

Last but not least, let's run it in fully managed way - with controller.
Before you start wildfly Swarm with KIE Server, make sure you start controller (KIE workbench) so you'll see how nicely it registers automatically upon start.

Once controller (workbench) is running issue following command:

java -Dorg.kie.server.id=swarm-kie-server -Dorg.kie.server.location=http://localhost:8380/server -Dorg.kie.server.controller=http://localhost:8080/kie-wb/rest/controller -jar target/kie-server-swarm-1.0-swarm.jar

Again, a singe parameter difference between the first command we used to start empty KIE Server on Swarm - in this case it's controller url:

• -Dorg.kie.server.controller=http://localhost:8080/kie-wb/rest/controller

Make sure that this URL matches your controller being deployed - it can differer in terms of

• host (localhost in this case)
• port (8080 in this case)
• context root (kie-wb in this case)

Now you're ready to rock with Wildfly Swarm and KIE Server to build your own micro services backed by business knowledge.

Enjoy your dive into Swarm and as usual comments are more than welcome.

Advanced queries in KIE Server

As a follow up of Advanced queries in jBPM 6.4 article let's take a look at queries in KIE Server - BPM capability.
Since KIE Server's BPM capability is based on jbpm services api, it does provide access to QueryService and its advanced (DashBuilder DataSets based) operations.

We are going to use the same use case, product sale with 10 000 loaded process and task instances. Next we show how you can query data both via KIE Server client and directly via raw REST api.

KIE Server capabilities when it comes to advanced queries mirrors what's available in services api, so users can:

• register query definitions
• replace  query definitions
• unregister query definitions
• get list of queries or individual query definition
• execute queries on top of query definitions with 
• paging and sorting
• filter parameters
• query with custom param builder and mappers

So let's start simple and build our KIE Server client to use query services:

KieServicesConfiguration configuration = KieServicesFactory.newRestConfiguration(serverUrl, user, password);

Set<Class<?>> extraClasses = new HashSet<Class<?>>();
extraClasses.add(Date.class); // for JSON only to properly map dates
        
configuration.setMarshallingFormat(MarshallingFormat.JSON);
configuration.addJaxbClasses(extraClasses);
        
KieServicesClient kieServicesClient =  KieServicesFactory.newKieServicesClient(configuration);
        
QueryServicesClient queryClient = kieServicesClient.getServicesClient(QueryServicesClient.class);

now we are ready to make use of the query service via QueryServicesClient

List available query definitions available in the system

List<QueryDefinition> queryDefs = queryClient.getQueries(0, 10);
System.out.println(queryDefs);

Next let's register new query definition that we can use for advanced queries

QueryDefinition query = new QueryDefinition();
query.setName("getAllTaskInstancesWithCustomVariables");
query.setSource("java:jboss/datasources/ExampleDS");
query.setExpression("select ti.*,  c.country, c.productCode, c.quantity, c.price, c.saleDate " +
                       "from AuditTaskImpl ti " +
                       "    inner join (select mv.map_var_id, mv.taskid from MappedVariable mv) mv " +
                       "      on (mv.taskid = ti.taskId) " +
                       "    inner join ProductSale c " +
                       "      on (c.id = mv.map_var_id)");
        
queryClient.registerQuery(query);

Once the query is registered with can make use of it and start fetching data. At first very basic query:

List<TaskInstance> tasks = queryClient.query("getAllTaskInstancesWithCustomVariables", "UserTasks", 0, 10, TaskInstance.class);
System.out.println(tasks);

this will return task instances directly from the data set without any filtering and use UserTasks mapper to build up object representation and apply paging - first page and 10 results at most.

Now it's time to use more advanced queries capabilities and start filtering by process variables. As described in the Advanced queries in jBPM 6.4 article to be able to map custom variables we need to provide their column mapping - name and type. Following is an example that searches for tasks that:

• processInstanceId is between 1000 and 2000 - number range condition
• price is over 800 - number comparison condition
• sale date is between 01.02.2016 and 01.03.2016 - date range condition
• product in sale are EAP or Wildfly - logical and group condition
• order descending by saleDate and country

SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
              
Date from = sdf.parse("2016-02-01");                        
Date to = sdf.parse("2016-03-01");
        
QueryFilterSpec spec = new QueryFilterSpecBuilder()
   .between("processInstanceId", 1000, 2000)
   .greaterThan("price", 800)
   .between("saleDate", from, to)
   .in("productCode", Arrays.asList("EAP", "WILDFLY"))
   .oderBy("saleDate, country", false)
 .addColumnMapping("COUNTRY", "string")
 .addColumnMapping("PRODUCTCODE", "string")
 .addColumnMapping("QUANTITY", "integer")
 .addColumnMapping("PRICE", "double")
 .addColumnMapping("SALEDATE", "date")
.get();        
        
List<TaskInstance> tasks = queryClient.query("getAllTaskInstancesWithCustomVariables", "UserTasksWithCustomVariables", spec, 0, 10, TaskInstance.class);
System.out.println(tasks);

The query in above example uses QueryFilterSpec (and its builder) that allows to specify query parameters and sorting options. In addition it allows to specify column mapping for custom elements to be set as variables next to default column for task details. These column mappings are then delivered to mapper for transforming results - in this case we used built in mapper UserTasksWithCustomVariables that will collect all data details and given column mappings as custom variables data.

QueryFilterSpec maps to use of QueryParams in services api so in inherits the same limitation - all conditions are AND based and thus means all must match to get a hit.

To overcome the problem, services api introduced QueryParamBuilder so users can build advanced filters. Similar is on KIE Server, though they need to be built and included in one of following:

• KIE Server itself (like in WEB-INF/lib)
• Inside a project - kjar
• Inside a project's dependency

Implementing QueryParamBuilder to be used in KIE Server requires a factory so it can be discovered and created on query time - every time query is issues new instance of QueryParamBuilder will be requested with given parameters.

Using QueryParamBuilder in KIE Server

To be able to use QueryParamBuilder user needs to:

• Implement QueryParamBuilder that will produce new instance every time is requested and given a map of parameters

public class TestQueryParamBuilder implements QueryParamBuilder<ColumnFilter> {

    private Map<String, Object> parameters;
    private boolean built = false;
    public TestQueryParamBuilder(Map<String, Object> parameters) {
        this.parameters = parameters;
    }
    
    @Override
    public ColumnFilter build() {
        // return null if it was already invoked
        if (built) {
            return null;
        }
        
        String columnName = "processInstanceId";
        
        ColumnFilter filter = FilterFactory.OR(
                FilterFactory.greaterOrEqualsTo(((Number)parameters.get("min")).longValue()),
                FilterFactory.lowerOrEqualsTo(((Number)parameters.get("max")).longValue()));
        filter.setColumnId(columnName);
       
        built = true;
        return filter;
    }

}

Above builder will produce filter that will accept processInstanceId that are grater that min or lower that max. Where min and max are given on each query issued as part of the request.

• Implement QueryParamBuilderFactory 

public class TestQueryParamBuilderFactory implements QueryParamBuilderFactory {

    @Override
    public boolean accept(String identifier) {
        if ("test".equalsIgnoreCase(identifier)) {
            return true;
        }
        return false;
    }

    @Override
    public QueryParamBuilder newInstance(Map<String, Object> parameters) {
        return new TestQueryParamBuilder(parameters);
    }

}

Factory is responsible for returning new instances of the query param builder only if the given identifier is accepted by the factory. Identifier is given as part of query request and there can be only one query builder factory selected based on the identifier. In this case "test" identifier needs to be given to use this factory, and in turn query param builder.

There is last tiny bit required to make this to work - we need to make it discoverable so let's add service file into META-INF folder of the jar that will package these implementation.

META-INF/services/org.jbpm.services.api.query.QueryParamBuilderFactory

where the content of this file is fully qualified class name of the factory.

with this we can issue a request that will make use of newly created query builder for advanced filters:

Map<String, Object> params = new HashMaplt;String, Object>();
params.put("min", 10);
params.put("max", 20);

Listlt;TaskInstance> instances = queryClient.query("getAllTaskInstancesWithCustomVariables", "UserTasksWithCustomVariables", "test", params, 0, 10, TaskInstance.class);

So what we have done here:

• reference registered query by name - getAllTaskInstancesWithCustomVariables
• reference mapper by name - UserTasksWithCustomVariables
• reference query param builder identifier - test
• sent params (min and max) that will be used by new instance of query builder before query is executed

Similar to this you can register and use custom mappers and it is even simpler than query param builders as there is no need for factory as services api comes with registry that KIE Server uses to register found mappers by ServiceLoader based discovery.

Implement mapper so it can be used in KIE Server:

public class ProductSaleQueryMapper extends UserTaskInstanceWithCustomVarsQueryMapper {

    private static final long serialVersionUID = 3299692663640707607L;

    public ProductSaleQueryMapper() {
        super(getVariableMapping());
    }

    protected static Map<String, String> getVariableMapping() {
        Map<String, String> variablesMap = new HashMap<String, String>();
        
        variablesMap.put("COUNTRY", "string");
        variablesMap.put("PRODUCTCODE", "string");
        variablesMap.put("QUANTITY", "integer");
        variablesMap.put("PRICE", "double");
        variablesMap.put("SALEDATE", "date");
        
        return variablesMap;
    }

    @Override
    public String getName() {
        return "ProductSale";
    }
}

Here we simply extend the UserTaskInstanceWithCustomVarsQueryMapper and provide directly column mapping so it can be used without column mapping on request level. To be able to use it, mapper needs to be made discoverable so we need to create service file within META-INF folder of the jar that will package this implementation.

META-INF/services/org.jbpm.services.api.query.QueryResultMapper

where the content of this file is fully qualified class name of the mapper.

Now we can directly use it by referencing it by name:

List<TaskInstance> tasks = queryClient.query("getAllTaskInstancesWithCustomVariables", "ProductSale", 0, 10, TaskInstance.class);
System.out.println(tasks);

Raw REST API use of described examples

Get query definitions

Endpoint:

• http://localhost:8230/kie-server/services/rest/server/queries/definitions?page=0&pageSize=10

Method:

• GET

Register query definition

Endpoint:

• http://localhost:8230/kie-server/services/rest/server/queries/definitions/getAllTaskInstancesWithCustomVariables

Method:

• POST

Request body:

{

  "query-name" : "getAllTaskInstancesWithCustomVariables1",

  "query-source" : "java:jboss/datasources/ExampleDS",

  "query-expression" : "select ti.*,  c.country, c.productCode, c.quantity, c.price, c.saleDate from AuditTaskImpl ti     inner join (select mv.map_var_id, mv.taskid from MappedVariable mv) mv       on (mv.taskid = ti.taskId)     inner join ProductSale c       on (c.id = mv.map_var_id)",

  "query-target" : "CUSTOM"

}

Query for tasks - no filtering

Endpoint:

• http://localhost:8230/kie-server/services/rest/server/queries/definitions/getAllTaskInstancesWithCustomVariables/data?mapper=UserTasks&orderBy=&page=0&pageSize=10

Method:

• GET

Query with filter spec

Endpoint:

• http://localhost:8230/kie-server/services/rest/server/queries/definitions/getAllTaskInstancesWithCustomVariables/filtered-data?mapper=UserTasksWithCustomVariables&page=0&pageSize=10

Method:

• POST

Request body:

{

  "order-by" : "saleDate, country",

  "order-asc" : false,

  "query-params" : [ {

    "cond-column" : "processInstanceId",

    "cond-operator" : "BETWEEN",

    "cond-values" : [ 1000, 2000 ]

  }, {

    "cond-column" : "price",

    "cond-operator" : "GREATER_THAN",

    "cond-values" : [ 800 ]

  }, {

    "cond-column" : "saleDate",

    "cond-operator" : "BETWEEN",

    "cond-values" : [ {"java.util.Date":1454281200000}, {"java.util.Date":1456786800000} ]

  }, {

    "cond-column" : "productCode",

    "cond-operator" : "IN",

    "cond-values" : [ "EAP", "WILDFLY" ]

  } ],

  "result-column-mapping" : {

    "PRICE" : "double",

    "PRODUCTCODE" : "string",

    "COUNTRY" : "string",

    "SALEDATE" : "date",

    "QUANTITY" : "integer"

  }

}

Query with custom query param builder

Endpoint:

• http://localhost:8230/kie-server/services/rest/server/queries/definitions/getAllTaskInstancesWithCustomVariables/filtered-data?mapper=UserTasksWithCustomVariables&builder=test&page=0&pageSize=10

Method:

• POST

Request body:

{

  "min" : 10,

  "max" : 20

}

Query for tasks - custom mapper

Endpoint:

• http://localhost:8230/kie-server/services/rest/server/queries/definitions/getAllTaskInstancesWithCustomVariables/data?mapper=ProductSale&orderBy=&page=0&pageSize=10

Method:

• GET

With this, we have went over support for advanced queries in KIE Server for BPM capability.

As usual, feedback is welcome :)

Advanced queries in jBPM 6.4

While working with BPM, access to data that are being processed by the engine are very important. In many cases users would like to have options to easily and efficiently search for different data:

• process instances started by...
• process instances not completed until...
• tasks assigned to ... for a given project
• tasks not started for a given amount of time
• process instances with given process variable(s)
• tasks with given task variable(s)

These are just few examples of advanced queries that are useful but might be tricky to provide out of the box because:

• different data bases have different capabilities when it comes to efficient searches
• ORM in between adds layer of complexity while it helps to mitigate db differences
• out of the box solution relies on compile time data - that can be used in queries - like jpa entities
• not possible to build data structure that will fit all cases and that will be efficient to query on

Again, just few items that makes the query out of the box limited in terms of functionality. jBPM in version 6.3 comes with efficient query builders based on JPA Criteria API that aims at solving many issues that are listed above but is blocked by compile time dependency as this is JPA based solution so the entity manager must be aware of all possible types used in queries.

What's new in 6.4?

jBPM 6.4 comes with solution to address these problems. And this solution is based on DashBuilder DataSets. DataSets are like data base views - users can define them to pre filter and aggregate data before they will be queried or filtered if you like.

QueryService is part of jbpm services api - a cross framework api build to simplify usage of jBPM in embedded use case. At the same time jbpm services api is backbone of both KIE workbench and KIE Server (with its BPM capabilities).

QueryService exposes simple yet powerful set of operations:

• Management operations
• register query definition
• replace query definition
• unregister query definition
• get query definition
• get queries
• Runtime operations
• query - with two flavors:
• simple based on QueryParam as filter provider
• advanced based on QueryParamBuilder as filter provider 

DashBuilder DataSets provide support for multiple data sources (CSV, SQL, elastic search, etc) while jBPM - since its backend is RDBMS based - focuses on SQL based data sets. So jBPM QueryService is a subset of DashBuilder DataSets capabilities to allow efficient queries with simple API.

How to use it?

Let's define use case that we can use throughout this article...

We are about to sale software and for doing that we define very simple process that deal with the sale operation. For that we have data model defined that represents our produce sale:

ProductSale:

   String productCode
    String country
    Double price
    Integer quantity
    Date saleDate

As you can see the process is very simple but aims at doing few important things:

• make use of both processes and user tasks
• deals with custom data model as process and user task
• allows to store externally process and task variables (here as JPA entity)

To be able to take advantage of the advanced queries we need to make sure we have various data being processed by jBPM so we can actually measure properly how easy we can find the relevant data. For that we create 10 000 process instances (and by that 10 000 user tasks) that we can then try to search for using different criteria.

Define query definitions

First thing user needs to do is to define data set - view of the data you want to work with - so called QueryDefinition in services api. 

 
SqlQueryDefinition query = new SqlQueryDefinition("getAllProcessInstances", "java:jboss/datasources/ExampleDS");
query.setExpression("select * from processinstancelog");
        
queryService.registerQuery(query);

This is the simplest possible query definition as it can be:

• constructor takes 
• a unique name that identifies it on runtime
• data source JNDI name used when performing queries on this definition - in other words source of data
• expression - the most important part - is the sql statement that builds up the view to be filtered when performing queries

Once we have the sql query definition we can register it so it can be used later for actual queries.

Perform basic queries

Next make use of it by using queryService.query methods:

Collection<ProcessInstanceDesc> instances = queryService.query("getAllProcessInstances", ProcessInstanceQueryMapper.get(), new QueryContext());

What happened here...

• we referenced the registered query by name - getAllProcessInstances
• we provided ProcessInstanceQueryMapper that will be responsible for mapping our data to object instances
• we provided default query context that enables paging and sorting

Let's see it with query context configuration...

QueryContext ctx = new QueryContext(0, 100, "start_date", true);
        
Collection<ProcessInstanceDesc> instances = queryService.query("getAllProcessInstances", ProcessInstanceQueryMapper.get(), ctx);

here we search the same query definition (data set) but we want to get 100 results starting at 0 and we want to have it with ascending order by start date.

But that's not advanced at all... it just doing paging and sorting on single table... so let's add filtering to the mix.

// single filter param
Collection<ProcessInstanceDesc> instances = queryService.query("getAllProcessInstances", ProcessInstanceQueryMapper.get(), new QueryContext(), QueryParam.likeTo(COLUMN_PROCESSID, true, "org.jbpm%"));

// multiple filter params (AND)
Collection<ProcessInstanceDesc> instances = queryService.query("getAllProcessInstances", ProcessInstanceQueryMapper.get(), new QueryContext(),
 QueryParam.likeTo(COLUMN_PROCESSID, true, "org.jbpm%"),
 QueryParam.equalsTo(COLUMN_STATUS, 1, 3));

here we have filtered our data set:

• first query - by process id that matches "org.jbpm%"
• second query - by process id that matches "org.jbpm%" and status is in active or aborted

but that's still not very advanced, isn't it?? Let's look at how to work with variables.

Perform queries with process and task variables

Common use case is to find process instances or tasks that have given variable or have given variable with particular value.

jBPM from version 6.4 indexes task variables (and in previous versions it already did that for process instance variables) in data base. The indexation mechanism is configurable but default is to simple toString on the variable and keep it in single table:

• Process instance variables - VariableInstanceLog table
• Task variables - TaskVariableImpl table

equipped with this information we can define data sets that will allow us to query for task and process variables.

// process instances with variables
SqlQueryDefinition query = new SqlQueryDefinition("getAllProcessInstancesWithVariables", "java:jboss/datasources/ExampleDS");
query.setExpression("select pil.*, v.variableId, v.value " +
                       "from ProcessInstanceLog pil " +
                            "INNER JOIN (select vil.processInstanceId ,vil.variableId, MAX(vil.ID) maxvilid  FROM VariableInstanceLog vil " +
                                "GROUP BY vil.processInstanceId, vil.variableId ORDER BY vil.processInstanceId)  x " +
                                "ON (v.variableId = x.variableId  AND v.id = x.maxvilid )" +
                            "INNER JOIN VariableInstanceLog v " +        
                                "ON (v.processInstanceId = pil.processInstanceId)");
        
queryService.registerQuery(query);

// tasks with variables
query = new SqlQueryDefinition("getAllTaskInputInstancesWithVariables", "java:jboss/datasources/ExampleDS");
query.setExpression("select ti.*, tv.name tvname, tv.value tvvalue "+
                        "from AuditTaskImpl ti " +
                             "inner join (select tv.taskId, tv.name, tv.value from TaskVariableImpl tv where tv.type = 0 ) tv "+
                                "on (tv.taskId = ti.taskId)");
        
queryService.registerQuery(query);

now we have registered new query definitions that will allow us to search for process and task and return variables as part of the query.

NOTE: usually when defining query definitions we don't want to have always data set to be same as the source tables so it's good practice to initially narrow down the amount of data for example by defining it for given project (deploymentId) or process id etc. Keep in mind that you can have query definitions as many as you like.

Now it's time to make use of these queries to fetch some results

Get process instances with variables:

List<ProcessInstanceWithVarsDesc> processInstanceLogs = queryService.query("getAllProcessInstancesWithVariables", ProcessInstanceWithVarsQueryMapper.get(), new QueryContext(), QueryParam.equalsTo(COLUMN_VAR_NAME, "approval_document"));

So we are able to find process instances that have variable called 'approval_document'...

Get tasks with variables:

List<UserTaskInstanceWithVarsDesc> taskInstanceLogs = queryService.query("getAllTaskInputInstancesWithVariables", UserTaskInstanceWithVarsQueryMapper.get(), new QueryContext(), 
                                                            QueryParam.equalsTo(COLUMN_TASK_VAR_NAME, "Comment"), 
                                                            QueryParam.equalsTo(COLUMN_TASK_VAR_VALUE, "Write a Document"));

... and here we can find tasks that have task variable 'Comment' and with value 'Write a Document'.

So a bit of a progress with more advanced queries but still nothing that couldn't be done with out of the box queries. Main limitation with out of the box variables indexes is that they are always stored as string and thus cannot be efficiently compared on db side like using operators >, < between, etc

... but wait with query definitions you can take advantage of the SQL being used to create your data view and by that use data base specific functions that can cast or convert string into different types of data. With this you can tune the query definition to provide you with subset of data with converted types. But of course that comes with performance penalty depending on the conversion type and amount of data.

So another level of making this use case covered is to externalize process and task variables (at least some of them that shall be queryable) and keep them in separate table(s). jBPM comes with so called pluggable variable persistence strategies and ships out of the box JPA based one. So you can create your process variable as entity and thus it will be stored in separate table. You can then take advantage of mapping support (org.drools.persistence.jpa.marshaller.VariableEntity) that ensures that mapping between your entity and process instance/task will be maintained.

Here is sample ProductSale object that is defined as JPA entity and will be stored in separate table

@javax.persistence.Entity
public class ProductSale extends org.drools.persistence.jpa.marshaller.VariableEntity implements java.io.Serializable
{

   static final long serialVersionUID = 1L;

   @javax.persistence.GeneratedValue(strategy = javax.persistence.GenerationType.AUTO, generator = "PRODUCTSALE_ID_GENERATOR")
   @javax.persistence.Id
   @javax.persistence.SequenceGenerator(name = "PRODUCTSALE_ID_GENERATOR", sequenceName = "PRODUCTSALE_ID_SEQ")
   private java.lang.Long id;

   private java.lang.String productCode;

   private java.lang.String country;

   private java.lang.Double price;

   private java.lang.Integer quantity;

   private java.util.Date saleDate;

   public ProductSale()
   {
   }

   public java.lang.Long getId()
   {
      return this.id;
   }

   public void setId(java.lang.Long id)
   {
      this.id = id;
   }

   public java.lang.String getProductCode()
   {
      return this.productCode;
   }

   public void setProductCode(java.lang.String productCode)
   {
      this.productCode = productCode;
   }

   public java.lang.String getCountry()
   {
      return this.country;
   }

   public void setCountry(java.lang.String country)
   {
      this.country = country;
   }

   public java.lang.Double getPrice()
   {
      return this.price;
   }

   public void setPrice(java.lang.Double price)
   {
      this.price = price;
   }

   public java.lang.Integer getQuantity()
   {
      return this.quantity;
   }

   public void setQuantity(java.lang.Integer quantity)
   {
      this.quantity = quantity;
   }

   public java.util.Date getSaleDate()
   {
      return this.saleDate;
   }

   public void setSaleDate(java.util.Date saleDate)
   {
      this.saleDate = saleDate;
   }
}

When such entity is then used as process or task variable it will be stored in productsale table and referenced as mapping in mappedvariable table so it can be joined to find process or task instances holding that variable.

Here we can make use of different types of data in that entity - string, integer, double, date, long and by that make use of various type aware operators to filter efficiently data. So let's define another data set that will provide use with tasks that can be filtered by product sale details.

// tasks with custom variable information
SqlQueryDefinition query = new SqlQueryDefinition("getAllTaskInstancesWithCustomVariables", "java:jboss/datasources/ExampleDS");
query.setExpression("select ti.*,  c.country, c.productCode, c.quantity, c.price, c.saleDate " +
                       "from AuditTaskImpl ti " +
                       "    inner join (select mv.map_var_id, mv.taskid from MappedVariable mv) mv " +
                       "      on (mv.taskid = ti.taskId) " +
                       "    inner join ProductSale c " +
                       "      on (c.id = mv.map_var_id)");
        
queryService.registerQuery(query);

// tasks with custom variable information with assignment filter
SqlQueryDefinition queryTPO = new SqlQueryDefinition("getMyTaskInstancesWithCustomVariables", "java:jboss/datasources/ExampleDS", Target.PO_TASK);
queryTPO.setExpression("select ti.*,  c.country, c.productCode, c.quantity, c.price, c.saleDate, oe.id oeid " +
                            "from AuditTaskImpl ti " +
                            "    inner join (select mv.map_var_id, mv.taskid from MappedVariable mv) mv " +
                            "      on (mv.taskid = ti.taskId) " +
                            "    inner join ProductSale c " +
                            "      on (c.id = mv.map_var_id), " +
                            "  PeopleAssignments_PotOwners po, OrganizationalEntity oe " +
                            "    where ti.taskId = po.task_id and po.entity_id = oe.id");
        
queryService.registerQuery(queryTPO);

here we registered two additional query definitions:

• first to load into data set both task info and product sale info
• second same as first but joined with potential owner information to get tasks only for authorized users

In second query you can notice third parameter in the constructor which defines the target - this is mainly to instruct QueryService to apply default filters like user or group filter for potential. Same filter parameters can be set manually so it's just short cut given by the API.

Marked in blue are variables from custom table and in orange task details

Now we can perform queries that will benefit from externally stored variable information to be able to find tasks by various properties (of different types) using various operators

Map<String, String> variableMap = new HashMap<String, String>();
variableMap.put("COUNTRY", "string");
variableMap.put("PRODUCTCODE", "string");
variableMap.put("QUANTITY", "integer");
variableMap.put("PRICE", "double");
variableMap.put("SALEDATE", "date");

//let's find tasks for product EAP and country Brazil and tasks with status Ready and Reserved");
List<UserTaskInstanceWithVarsDesc> taskInstanceLogs = queryService.query(query.getName(), 
                UserTaskInstanceWithCustomVarsQueryMapper.get(variableMap), new QueryContext(), 
                QueryParam.equalsTo("productCode", "EAP"), 
                QueryParam.equalsTo("country", "Brazil"), 
                QueryParam.in("status", Arrays.asList(Status.Ready.toString(), Status.Reserved.toString())));


// now let's search for tasks that are for EAP and sales data between beginning and end of February
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
              
Date from = sdf.parse("2016-02-01");                        
Date to = sdf.parse("2016-03-01");
taskInstanceLogs = queryService.query(query.getName(), 
                UserTaskInstanceWithCustomVarsQueryMapper.get(variableMap), new QueryContext(), 
                QueryParam.equalsTo("productCode", "EAP"), 
                QueryParam.between("saleDate", from, to),
                QueryParam.in("status", Arrays.asList(Status.Ready.toString(), Status.Reserved.toString())));

Here you can see how easy and efficient queries can be using variables stored externally. You can take advantage of type based operators to effectively narrow down the results.

As you might have noticed, this time we use another type of mapper - UserTaskInstanceWithCustomVarsQueryMapper - that is responsible for mapping both task information and custom variable. Thus we need to provide column mapping - name and type - so mapper know how to read data from data base to preserve the actual type.

Mappers are rather powerful and thus are pluggable, you can implement your own mappers that will transform the result into whatever type you like. jBPM comes with following mappers out of the box:

• org.jbpm.kie.services.impl.query.mapper.ProcessInstanceQueryMapper
• registered with name - ProcessInstances
• org.jbpm.kie.services.impl.query.mapper.ProcessInstanceWithVarsQueryMapper
• registered with name - ProcessInstancesWithVariables
• org.jbpm.kie.services.impl.query.mapper.ProcessInstanceWithCustomVarsQueryMapper
• registered with name - ProcessInstancesWithCustomVariables
• org.jbpm.kie.services.impl.query.mapper.UserTaskInstanceQueryMapper
• registered with name - UserTasks
• org.jbpm.kie.services.impl.query.mapper.UserTaskInstanceWithVarsQueryMapper
• registered with name - UserTasksWithVariables
• org.jbpm.kie.services.impl.query.mapper.UserTaskInstanceWithCustomVarsQueryMapper
• registered with name - UserTasksWithCustomVariables
• org.jbpm.kie.services.impl.query.mapper.TaskSummaryQueryMapper
• registered with name - TaskSummaries

Mappers are registered by name to simplify lookup of them and to avoid compile time dependency to actual mapper implementation. Instead you can use:

org.jbpm.services.api.query.NamedQueryMapper

that simple expects the name of the actual mapper that will be resolved on time when the query is performed

Here you can find complete product-sale project that can be imported into KIE Workbench for inspection and customization.

QueryParamBuilder

Last but not least is the QueryParamBuilder that provides more advanced way of building filters for our data sets. By default when using query method of QueryService that accepts zero or more QueryParam instances (as we have seen in above examples) all of these params will be joined with AND operator meaning all of them must match. But that's not always the case so that's why QueryParamBuilder has been introduced so users can build up their on builders can provide them at the time the query is issued.
QueryParamBuilder is simple interface that is invoked as long as its build method returns non null value before query is performed. So you can build up an complex filter options that could not be simply expressed by list of QueryParams.

Here is basic implementation of QueryParamBuilder to give you a bit of jump start to implement your own - note that it relies on DashBuilder Dataset API.

public class TestQueryParamBuilder implements QueryParamBuilder<ColumnFilter> {

    private Map<String, Object> parameters;
    private boolean built = false;
    public TestQueryParamBuilder(Map<String, Object> parameters) {
        this.parameters = parameters;
    }
    
    @Override
    public ColumnFilter build() {
        // return null if it was already invoked
        if (built) {
            return null;
        }
        
        String columnName = "processInstanceId";
        
        ColumnFilter filter = FilterFactory.OR(
                FilterFactory.greaterOrEqualsTo((Long)parameters.get("min")),
                FilterFactory.lowerOrEqualsTo((Long)parameters.get("max")));
        filter.setColumnId(columnName);
       
        built = true;
        return filter;
    }

}

This concludes introduction to new QueryService based on Dashbuilder Dataset API to allow tailored queries against all possible data including (but not being limited to) jBPM data.

This article focused on jbpm services api but this functionality is also available in KIE Server for remote use cases. Stay tuned for another article describing remote capabilities.

KIE Server: Extend KIE Server client with new capabilities

Last but not least part of KIE Server extensions is about extending KIE Server Client with additional capabilities.

Use case

On top of what was built in second article (adding Mina transport to KIE Server), we need to add KIE Server Client extension that allow to use Mina transport with unified KIE Server Client API.

Before you start create empty maven project (packaging jar) with following dependencies:

<properties>
    <version.org.kie>6.4.0-SNAPSHOT</version.org.kie>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.kie.server</groupId>
      <artifactId>kie-server-api</artifactId>
      <version>${version.org.kie}</version>
    </dependency>

    <dependency>
      <groupId>org.kie.server</groupId>
      <artifactId>kie-server-client</artifactId>
      <version>${version.org.kie}</version>
    </dependency>
    
    <dependency>
      <groupId>org.drools</groupId>
      <artifactId>drools-compiler</artifactId>
      <version>${version.org.kie}</version>
    </dependency>
  </dependencies>

Design ServicesClient API interface

First thing we need to do is to decide what API we should have to be exposed to the callers of our Client API. Since the Mina extension is an extension on top of Drools one so let's provide same capabilities as RulesServicesClient:

public interface RulesMinaServicesClient extends RuleServicesClient {

}

As you can notice it simply extends the default RulesServiceClient interface and thus provide same capabilities. 

Why we need to have additional interface for it? It's because we are going to register client implementations based on their interface and there can be only one implementation for given interface.

Implement RulesMinaServicesClient

Next step is to actually implement the client and here we are going to prepare a socket based communication for simplicity sake. We could use Apache Mina client API though this would introduce additional dependency which we don't need for sample implementation.

Note that this client implementation is very simple and in many cases can be improved, but the point here is to show how it can be implemented rather than provide bullet proof code.

So few aspects to remember when reviewing the implementation:

• it relies on default configuration from KIE Server client and thus uses serverUrl as place where to provide host and port of Mina server
• hardcodes JSON as marshaling format
• decision if the response is success or failure is based on checking if the received message is a JSON object (start with {) - very simple though works for simple cases
• uses direct socket communication with blocking api while waiting for first line of the response and then reads up all lines that are available
• does not use "stream mode" meaning it disconnects from the server after invoking command

Here is the implementation

public class RulesMinaServicesClientImpl implements RulesMinaServicesClient {
    
    private String host;
    private Integer port;
    
    private Marshaller marshaller;
    
    public RulesMinaServicesClientImpl(KieServicesConfiguration configuration, ClassLoader classloader) {
        String[] serverDetails = configuration.getServerUrl().split(":");
        
        this.host = serverDetails[0];
        this.port = Integer.parseInt(serverDetails[1]);
        
        this.marshaller = MarshallerFactory.getMarshaller(configuration.getExtraJaxbClasses(), MarshallingFormat.JSON, classloader);
    }

    public ServiceResponse<String> executeCommands(String id, String payload) {
        
        try {
            String response = sendReceive(id, payload);
            if (response.startsWith("{")) {
                return new ServiceResponse<String>(ResponseType.SUCCESS, null, response);
            } else {
                return new ServiceResponse<String>(ResponseType.FAILURE, response);
            }
        } catch (Exception e) {
            throw new KieServicesException("Unable to send request to KIE Server", e);
        }
    }

    public ServiceResponse<String> executeCommands(String id, Command<?> cmd) {
        try {
            String response = sendReceive(id, marshaller.marshall(cmd));
            if (response.startsWith("{")) {
                return new ServiceResponse<String>(ResponseType.SUCCESS, null, response);
            } else {
                return new ServiceResponse<String>(ResponseType.FAILURE, response);
            }
        } catch (Exception e) {
            throw new KieServicesException("Unable to send request to KIE Server", e);
        }
    }

    protected String sendReceive(String containerId, String content) throws Exception {
        
        // content - flatten the content to be single line
        content = content.replaceAll("\\n", "");
        
        Socket minaSocket = null;
        PrintWriter out = null;
        BufferedReader in = null;

        StringBuffer data = new StringBuffer();
        try {
            minaSocket = new Socket(host, port);
            out = new PrintWriter(minaSocket.getOutputStream(), true);
            in = new BufferedReader(new InputStreamReader(minaSocket.getInputStream()));
        
            // prepare and send data
            out.println(containerId + "|" + content);
            // wait for the first line
            data.append(in.readLine());
            // and then continue as long as it's available
            while (in.ready()) {
                data.append(in.readLine());
            }
            
            return data.toString();
        } finally {
            out.close();
            in.close();
            minaSocket.close();
        }
    }
}

Once we have the client interface and client implementation we need to make it available for KIE Service client to find it.

Implement KieServicesClientBuilder

org.kie.server.client.helper.KieServicesClientBuilder is the glue interface that allows to provide additional client apis to generic KIE Server Client infrastructure. This interface have two methods:

• getImplementedCapability - which must much the server capability (extension) is going to use
• build - which is responsible for providing map of client implementations where key is the interface and value fully initialized implementation

Here is a simple implementation of the client builder for this use case

public class MinaClientBuilderImpl implements KieServicesClientBuilder {

    public String getImplementedCapability() {
        return "BRM-Mina";
    }

    public Map<Class<?>, Object> build(KieServicesConfiguration configuration, ClassLoader classLoader) {
        Map<Class<?>, Object> services = new HashMap<Class<?>, Object>();

        services.put(RulesMinaServicesClient.class, new RulesMinaServicesClientImpl(configuration, classLoader));

        return services;
    }

}

Make it discoverable

Same story as for other extensions ... once we have all that needs to be implemented, it's time to make it discoverable so KIE Server Client can find and register this extension on runtime. Since KIE Server  Client is based on Java SE ServiceLoader mechanism we need to add one file into our extension jar file:

META-INF/services/org.kie.server.client.helper.KieServicesClientBuilder

And the content of this file is a single line that represents fully qualified class name of our custom implementation of  KieServicesClientBuilder.

How to use it

The usage scenario does not much differ from regular KIE Server Client use case:

• create client configuration
• create client instance
• get service client by type
• invoke client methods

Here is implementation that create KIE Server Client for RulesMinaServiceClient

protected RulesMinaServicesClient buildClient() {
    KieServicesConfiguration configuration = KieServicesFactory.newRestConfiguration("localhost:9123", null, null);
    List<String> capabilities = new ArrayList<String>();
    // we need to add explicitly capabilities as the mina client does not respond to get server info requests.
    capabilities.add("BRM-Mina");
    
    configuration.setCapabilities(capabilities);
    configuration.setMarshallingFormat(MarshallingFormat.JSON);
    
    configuration.addJaxbClasses(extraClasses);
    
    KieServicesClient kieServicesClient =  KieServicesFactory.newKieServicesClient(configuration);
    
    RulesMinaServicesClient rulesClient = kieServicesClient.getServicesClient(RulesMinaServicesClient.class);
    
    return rulesClient;
}

And here is how it is used to invoke operations on KIE Server via Mina transport

RulesMinaServicesClient rulesClient = buildClient();

List<Command<?>> commands = new ArrayList<Command<?>>();
BatchExecutionCommand executionCommand = commandsFactory.newBatchExecution(commands, "defaultKieSession");

Person person = new Person();
person.setName("mary");
commands.add(commandsFactory.newInsert(person, "person"));
commands.add(commandsFactory.newFireAllRules("fired"));

ServiceResponse<String> response = rulesClient.executeCommands(containerId, executionCommand);
Assert.assertNotNull(response);

Assert.assertEquals(ResponseType.SUCCESS, response.getType());

String data = response.getResult();

Marshaller marshaller = MarshallerFactory.getMarshaller(extraClasses, MarshallingFormat.JSON, this.getClass().getClassLoader());

ExecutionResultImpl results = marshaller.unmarshall(data, ExecutionResultImpl.class);
Assert.assertNotNull(results);

Object personResult = results.getValue("person");
Assert.assertTrue(personResult instanceof Person);

Assert.assertEquals("mary", ((Person) personResult).getName());
Assert.assertEquals("JBoss Community", ((Person) personResult).getAddress());
Assert.assertEquals(true, ((Person) personResult).isRegistered());

Complete code of this client extension can be found here.

And that's the last extension method to provide more features in KIE Server then given out of the box.

Thanks for reading the entire series of KIE Server extensions and any and all feedback welcome :)