czwartek, 14 lipca 2016

jBPM v7 - workbench and kie server integration

As part of ongoing development of jBPM version 7 I'd like to give a short preview of one of the changes that are coming. This in particular relates to changes how workbench and kie server are integrated.  In version 6 (when kie server was introduced with BPM capability) we have had two independent execution servers:

  • one embedded in workbench 
  • another in in kie server
in many cases it caused bit of confusion as users were expecting to see processes (and tasks, jobs etc) created in kie server vie workbench UI. To achieve that users where pointing workbench and kie server to same data base and that lead to number of unexpected issues as these two were designed in different way and were not intended to work in parallel.

jBPM version 7 is addressing this problem in two ways:
  • removes duplication when it comes to execution servers - only kie server will be available - no execution in workbench
  • integrates workbench with kie server(s) so its runtime views (e.g. process instances, definitions, tasks) can be used with kie server as backend

While the first point is rather clear and obvious, the second takes a bit to see its full power. It's not only about letting users use workbench UI to start processes or interact with user tasks, it actually makes the flexible architecture of kie server to be fully utilized (more on kie server can be found in previous blog series).
In version 6.4 there was new Server Management UI introduced to allow easy and efficient management of kie servers. This came with concept of server templates - that is a definition of the runtime environment regardless of how many physical instances are going to run with this definition. That in turn allow administrators to define partitioned environment where different server templates are representing different part of the organization or domain coverage.

Server template consists of:
  • name
  • list of assigned containers (kjars)
  • list of available capabilities
Once any kie server starts and connects to workbench (workbench acts as controller) then it will be presented in server management under remote servers. Remote servers reflects the current state of controller knowledge - meaning it only changes the list upon two events triggered by kie servers:
  • start of the kie server in managed mode - which connects to controller and registers itself as remote server
  • shutdown of the kie server in managed mode - which notifies controller to unregister itself from remote servers
With this setup users can create as many server templates as they need. Moreover at the same time each server template can be backed by as many kie server instances as it needs. That gives the complete control of how individual server templates (and buy that part of your business domain) can scale individually.

So enough of introduction, let's see how it got integrated for execution. Since there is no more execution server embedded in workbench, all that needs it will be carried on by kie server(s). To accomplish this workbench internally relies on two parts:
  • server management (that maintains server templates) to know what is available and where
  • kie server client to interact with remote servers
Server management is used to collect information about:
  • server templates whenever project is to be deployed - regardless if there are any remote servers or not - again this is just update to the definition of the server
  • remote servers whenever kie server interaction is required - start process, get process instances, etc
NOTE: in case of multiple server templates are available there will be selection box shown on the screen so users can decide which server template they are going to interact with. Again, users do not care about individual remote servers as they represent same setup so it's not important which server instance will be used for given request, as long as one of them is available.
Server templates that do not have any remote servers available won't be visible on the list of server templates.
And when there is only one server template selection is not required and that one becomes the default - for both deployment and runtime operations.


On right top corner users can find server template selection button in case there are more than one server template available. Once selected it will be preserved across screen navigation so it should be selected only once.

Build & Deploy has been updated to take advantage of the new server management as well, whenever users decide to do build and deploy:
  • if there is only one server template:
    • it gets selected as default
    • artifact name is used as container id
    • by default container is started
  • if there are more than one server templates available user is presented with additional popup window to select:
    • container id
    • server tempalte
    • if container should be started or not

That concludes the introduction and basic integration between kie server and workbench. Let's now look what's included and what's excluded from workbench point of view (or the differences that users might notice when switching from 6 to 7). 

First of all, majority of runtime operations are suppose to work exactly the same what, that includes:
  • Process definition view
    • process definition list
    • process definition details
    • Operations
      • start process instance (including forms)
      • visualize process definition diagram


  • Process instance view
    • process instance list (both predefined and custom filters)
    • process instance details
    • process instance variables
    • process instance documents
    • process instance log
    • operations
      • start process instance (including forms)
      • signal process instance (including bulk)
      • abort process instance (including bulk)
      • visualize process instance progress via diagram

  • Tasks instance view
    • task list (both predefined and custom filters)
    • task instance details (including forms)
    • life cycle operations of a task (e.g. claim, start, complete)
    • task assignment
    • task comments
    • task log

  • Jobs view
    • jobs list (both predefined and custom filters)
    • job details
    • create new job
    • depending on status cancel or requeue jobs


  • Dashboards view
    • out of the box dashboards for processes and tasks


All of these views retrieve data from remote kie server so that means there is no need for workbench to have any data sources defined. Even the default that comes with wildfly is not needed - not even for dashboards :) With that we have very lightweight workbench that comes with excellent authoring and management capabilities.

That leads us to a last section in this article that explains that changed and what was removed. 

Let's start with changes that are worth noting:
  • asynchronous processing of authoring REST operations has been moved from jbpm executor to UberFire async service - that makes the biggest change in cluttered setup where only cluster member that accepted request will know its status
  • build & deploy from project explorer is unified - regardless if the project is managed or unmanaged - there are only two options
    • compile - which does only in memory project build
    • build and deploy - which includes build, deploy to maven and provision to server template
Now moving to what was removed

  • since there is no jbpm runtime embedded in workbench there is no REST or JMS interfaces for jBPM, REST interfaces for the authoring part is unchanged (create org unit, repository, compile project etc)
  • jobs settings is no longer available as it does not make much sense in new (distributed) setup as the configuration of kie servers is currently done on server instance level
  • ad hoc tasks are temporally removed and will be introduced as part of case management support where it actually belongs
  • asset management is removed in the form it was known in v6 - the part that is kept is
    • managed repository that allows single or multi module projects
    • automatic branch creation - dev and release
    • repository structure management where users can manage their modules and create additional branches
    • project explorer still supports switch between branches as it used to
  • asset management won't have support for asset promotion, build of projects or release of projects
  • send task reminders - it was sort of hidden feature and more of admin so it's going to be added as part of admin interface for workbench/kie server


Enough talking (... writing/reading depends on point of view) it's time to see it in action. Following are two screen casts showing different use cases covered.

  • Case 1 - from zero to full speed execution
    • Create new repository and project
    • Create data object
    • Create process definition with user task and forms that uses created data object
    • Build and deploy the project
    • Start process instance(s)
    • Work on tasks
    • Visualize the progress of process instance
    • Monitor via dashboards



  • Case 2 - from existing project to document capable processes
    • Server template 1
    • Deploy already build project (translations)
    • Create process instance that includes document upload
    • Work on tasks
    • Visualize process instance details (including documents)

    • Server template 2
    • Deploy already build project (async-example)
    • Create process instance to check weather in US based on zip code
    • Work on tasks 
    • Visualize process instance progress - this project does not have image attached so it comes blank
    • Monitor via dashboards


Before we end a short note for these that want to try it out. Since we have integration with kie server and as you noticed it does not require any additional login to kie server and workbench uses logged in user, there is small need for configuration of Wildfly:
workbench comes with additional login module as part of kie-security.jar so to enable smooth integration when it comes to authentication, please declare in standalone.xml of your wildfly following login module:

so the default other security domain should look like this:
  <security-domain name="other" cache-type="default">
    <authentication>
       <login-module code="Remoting" flag="optional">
          <module-option name="password-stacking" value="useFirstPass"/>
        </login-module>
        <login-module code="RealmDirect" flag="required">
            <module-option name="password-stacking" value="useFirstPass"/>
        </login-module>
        <login-module code="org.kie.security.jaas.KieLoginModule" flag="optional"
                                module="deployment.kie-wb.war"/>
     </authentication>
  </security-domain>

important element is marked with red as it might differ between environments as it relies on the actual file name of the kie-wb.war. Replace it to match the name of your environment.

NOTE: this is only required for kie wb and not for kie drools wb running on wildfly. Current state is that this works on Wildfly/EAP7 and Tomcat, WebSphere and WebLogic might come later...

That's all for know, comments and ideas more than welcome

14 komentarzy:

  1. Hi Maciej,
    Any chance that the code you demoed at https://www.youtube.com/watch?v=GDl0zKpXXMU can be found on github?

    OdpowiedzUsuń
    Odpowiedzi
    1. I do plan to write an article next week with code pushed to github

      Usuń
  2. Hey Maciej!

    Thanks for another great blog post. The New and Noteworthy section for version 6.5 KIE server says that it is possible for the workbench to manage processes/tasks for execution servers by sharing the database. However, I am having trouble doing this with the Docker distributions. Are the jbpm-installer and Docker distributions up to date with this change?

    OdpowiedzUsuń
    Odpowiedzi
    1. Chris,
      they should be. the main requirement is to use the same container id as deployment id (GAV). Can you share what kind of issues you run into?

      Usuń
    2. Thanks for the reply Maciej! I am using the 6.5.0.Final jBPM workbench Docker container. If I look inside with "docker exec -it jbpm /bin/bash", I only see dash builder and jbpm-console wars in the JBoss deployment folder. I get 404 or page not found if I hit 192.168.1.100:8080/kie-server/services/rest/server or a few variations on it. Perhaps it is not shipped in the Docker container, just the jbpm-installer or perhaps I need to enable something else. I also do not see the server show up in the Server Templates either. I could configure separate Docker containers for execution server and workbench and share a database connection. I have tried on 6.4 but it does not work completely (dash builder does not load, tasks do not load in workbench). Any advice is welcome.

      Usuń
    3. kie server is on another docker container, it's not bundled with the same docker image as workbench, so most likely that's the issue. See details here http://blog.athico.com/2015/06/drools-jbpm-get-dockerized.html

      Usuń
    4. OK, I think I am pretty close. I set up both Docker showcase images and a shared MySQL image. They both run correctly on their own and the KIE server registers with the Workbench. I can manage kjars on the KIE server with the workbench. The logs are clear, no errors. However, it seems like all the process instances are separated in the two servers. In the Workbench dashboards and process instance lists, I only see process instances started from Workbench and vice versa with the KIE server REST API. From the documentation on 6.5, it seems like they should now see both. Perhaps it is because I am using admin user in Workbench and kieserver user in KIE server? Or is it still not possible for these servers to share process instances and tasks? Thanks again!

      Usuń
    5. I suspect that kie server is running with in memory db. To verify it it's as easy as just stopping it and starting again and process instances should be gone. Keep in mind that kie server persistence is controlled via system properties - data source jndi name and hibernate dialect

      Usuń
    6. I think they're both correctly using the MySQL database, I can restart them without losing any data. Can you confirm that it should be possible to see KIE server process instances/tasks in the Workbench with the correct database configuration in 6.5? Assuming everything was connected correctly, would the user, group, or role have anything to do with it?

      Usuń
    7. I just checked, it seems that if I go into Dashbuilder and choose the jBPM Dashboard, it shows me all the processes. Some are owned by admin and some are owned by KIE server. It's just in the Workbench where I am logged in as Admin where I cannot see all the processes and when I call the KIE server, it's authenticated as the kieserver user. I think I need to figure out how to share/merge their user management before I take up more of your time. Do you think that might be the issue?

      Usuń
  3. Ok, just double checked the database, externalID in ProcessInstanceLog is different. Deploying through Workbench generates a different container name than the one I manually specified in the Server Template for KIE server. Once I got them to be the same, I can start and query processes from both Workbench and KIE Server. Thanks for your help! Looking forward to version 7!

    OdpowiedzUsuń
    Odpowiedzi
    1. For anyone interested, this can be done easily through Docker https://github.com/Shumakriss/workbench-execution-integration

      Just make sure to have your kjar/container names match up in the Workbench and KIE Server!

      Usuń
  4. I want to Integrate Kie Workbench UI with my own application. Is there any way to do this?

    OdpowiedzUsuń