Integrate jBPM with Apache Kafka Server – Complete Setup Guide

 💡 Introduction

In modern event-driven architectures, integrating ⚙️ jBPM (Business Process Management) with 📡 Apache Kafka unlocks real-time visibility and automation across systems.

jBPM can publish its process events, task updates, and logs directly to a Kafka server (broker) — enabling analytics, auditing, and microservice event chaining.

This guide shows you how to connect jBPM to a running Kafka Server without writing any Java code, using only configuration changes in WildFly / KIE Server.

⚙️ 1️⃣ Prerequisites

Before starting, make sure you have:

✅ Kafka Server (Broker) up and running
(e.g., localhost:9092 or your remote broker IP/port)
✅ jBPM / KIE Server deployed on WildFly
✅ Access to modify standalone.xml or standalone.conf

If you’re using a cloud Kafka (like Azure Event Hubs or Confluent Cloud), note the connection parameters (host, port, protocol, username, password or connection string).

🧠 2️⃣ Enable jBPM Kafka Event Emitter

jBPM includes a ready-to-use module:
org.jbpm:jbpm-event-emitters-kafka

It automatically pushes process and task events to Kafka when enabled.

🔹 Add this dependency to your KIE Server (if not already bundled):

<dependency>
    <groupId>org.jbpm</groupId>
    <artifactId>jbpm-event-emitters-kafka</artifactId>
    <version>7.74.1.Final</version>
</dependency>

🧩 3️⃣ Configure Kafka Connection in WildFly

Open your WildFly standalone.xml (or standalone-full.xml)
and add this inside the <system-properties> section:

🧾 Example: Local Kafka Broker (No Security)

<system-properties>
    <property name="org.jbpm.event.emitters.kafka.bootstrap.servers" value="localhost:9092"/>
    <property name="org.jbpm.event.emitters.kafka.topic" value="jbpm-logs"/>
    <property name="org.jbpm.event.emitters.kafka.client.id" value="jbpm-emitter"/>
</system-properties>

This tells jBPM to send all business events to Kafka topic jbpm-logs.

🔐 4️⃣ If Kafka Uses SSL or Authentication

If your Kafka cluster requires a password, SSL, or a connection string (like Azure Event Hubs), use these properties instead:

🔹 For SSL Authentication

<system-properties>
    <property name="org.jbpm.event.emitters.kafka.bootstrap.servers" value="10.10.1.25:9093"/>
    <property name="org.jbpm.event.emitters.kafka.topic" value="jbpm-logs"/>
    <property name="org.jbpm.event.emitters.kafka.client.id" value="jbpm-emitter"/>
    <property name="org.jbpm.event.emitters.kafka.security.protocol" value="SSL"/>
    <property name="org.jbpm.event.emitters.kafka.ssl.truststore.location" value="/opt/wildfly/config/truststore.jks"/>
    <property name="org.jbpm.event.emitters.kafka.ssl.truststore.password" value="changeit"/>
</system-properties>

🔹 For Cloud Kafka (Azure Event Hubs, SASL_SSL)

<system-properties>
    <property name="org.jbpm.event.emitters.kafka.bootstrap.servers" value="YOUR_NAMESPACE.servicebus.windows.net:9093"/>
    <property name="org.jbpm.event.emitters.kafka.security.protocol" value="SASL_SSL"/>
    <property name="org.jbpm.event.emitters.kafka.sasl.mechanism" value="PLAIN"/>
    <property name="org.jbpm.event.emitters.kafka.sasl.jaas.config"
              value="org.apache.kafka.common.security.plain.PlainLoginModule required username='$ConnectionString' password='Endpoint=sb://YOUR_NAMESPACE.servicebus.windows.net/;SharedAccessKeyName=YOUR_SAS_KEY_NAME;SharedAccessKey=YOUR_SAS_KEY;EntityPath=YOUR_EVENT_HUB';"/>
    <property name="org.jbpm.event.emitters.kafka.topic" value="YOUR_EVENT_HUB"/>
    <property name="org.jbpm.event.emitters.kafka.client.id" value="jbpm-emitter"/>
    <property name="org.jbpm.event.emitters.kafka.client.dns.lookup" value="use_all_dns_ips"/>
</system-properties>

💡 use_all_dns_ips ensures that the Kafka client connects to all IPs returned by DNS — improving reliability with cloud load balancers.

📈 5️⃣ Restart and Verify the Integration

Restart WildFly or KIE Server:

./bin/standalone.sh -c standalone-full.xml

Check logs:

INFO  [org.jbpm.event.emitters.kafka.KafkaEventEmitter] Connecting to Kafka broker at localhost:9092
INFO  [org.jbpm.event.emitters.kafka.KafkaEventEmitter] Successfully sent event to topic jbpm-logs

✅ If you see these messages, the integration is working!

🔍 6️⃣ Validate from Kafka Side

Open your Kafka console consumer:

bin/kafka-console-consumer.sh --topic jbpm-logs --bootstrap-server localhost:9092 --from-beginning

You should see JSON messages like:

{
  "processId": "OrderProcess",
  "instanceId": 101,
  "eventType": "PROCESS_STARTED",
  "timestamp": "2025-10-07T14:21:05Z"
}

Each line represents a jBPM event — process started, task completed, variable changed, etc.

📊 7️⃣ Common Configuration Tips

GoalPropertyExample
Change topic nameorg.jbpm.event.emitters.kafka.topicprocess-events
Adjust retriesorg.jbpm.event.emitters.kafka.retries5
Set timeoutorg.jbpm.event.emitters.kafka.request.timeout.ms60000
Acknowledgment modeorg.jbpm.event.emitters.kafka.acksall

🔄 8️⃣ Optional – Use REST Task Instead of Event Emitter

If you don’t want to use the event emitter, you can add a REST Task in your BPMN process to call Kafka REST Proxy directly — no code needed.

Example:

  • URL → http://<kafka-rest>:8082/topics/jbpm-logs

  • Method → POST

  • Body:

{
  "records": [
    { "key": "jbpm", "value": { "processId": "#{processInstanceId}", "event": "#{eventType}" } }
  ]
}

🧠 9️⃣ Benefits of jBPM + Kafka Integration

✅ Real-time process monitoring
✅ Streamlined log collection and analytics
✅ Easy integration with ELK, OpenTelemetry, or Flink
✅ Scalable event-driven orchestration across microservices

🏁 Conclusion

🎯 Integrating jBPM with Kafka Server makes your process automation event-driven, scalable, and transparent.

Once connected, every jBPM process or task event becomes a Kafka message — allowing you to feed dashboards, alerting systems, and analytics tools in real time.

This architecture is the foundation for intelligent BPM, data-driven workflows, and real-time decision systems.


💼 Professional Support Available

 If you are facing issues in real projects related to enterprise backend development or workflow automation, I provide paid consulting, production debugging, project support, and focused trainings. 

Technologies covered include Java, Spring Boot, PL/SQL, Azure, and workflow automation (jBPM, Camunda BPM, RHPAM).

📧 Contact: ishikhanirankari@gmail.com | info@realtechnologiesindia.com

Comments

Post a Comment

Popular posts from this blog

jBPM Installation Guide: Step by Step Setup

  jBPM Installation Guide – Step by Step for Beginners If you are starting with jBPM (Java Business Process Management) , the first step is to set up your environment. In this guide, we’ll walk through how to install jBPM, configure it, and run your first business process. 🔎 What is jBPM? Before we dive into installation, a quick recap: jBPM is an open-source Business Process Management (BPM) suite , written in Java, that allows you to model, execute, and monitor workflows using BPMN 2.0 standards. It can run standalone , embedded in applications , or deployed on a Java EE server . 🖥️ Prerequisites Before installing jBPM, make sure you have: Java JDK 11 or later (Java 17 is commonly used) Maven 3.x (for building and managing dependencies) Git (optional, if you want to clone sample projects) A Database (H2, MySQL, PostgreSQL, or Oracle) – jBPM ships with H2 for testing At least 4GB RAM (8GB recommended for Workbench & KIE Server) 📦 Download ...
Read more

Scopes of Signal in jBPM

Image
💡 Introduction Signals in jBPM 7 are a core part of event-driven workflow design. They allow asynchronous communication between process instances, making it possible to start , interrupt , or coordinate processes without direct linking. However, signals can have different scopes , which define how far the signal travels and who receives it . Let’s explore the four available scopes in detail: 🔹 Process Scope 🔹 Default Scope 🔹 External Scope 🔹 Project Instance Scope 🧩 1️⃣ Process Scope 📘 Definition Process scope signals are local to a single process instance or its child subprocesses . They are not visible outside that instance hierarchy. 📋 Use Case Used for communication inside the same process , like triggering subprocesses or event sub-processes. 🖼️ Process Scope Example Below is an example of a simple process-scoped signal. Only this process (or its subprocess) will catch this signal. ⚙️ 2️⃣ Default Scope 📘 Definition The default scope ...
Read more

OOPs Concepts in Java | English | Object Oriented Programming Explained

Image
🔹OOPS (Object Oriented Programming System) in Java Object-Oriented Programming (OOPS) is a programming paradigm that uses classes and objects to design software. It simplifies development and maintenance by applying key concepts: 📌 Class: Class   is a user-defined data type which defines its properties and its functions. Class is the only logical representation of the data. For example, Human beings in an organisation is a class like Employee. The age, name, department, salary etc. of a human being/employee are its properties, and the jobs performed by the employees in an organisation are known as functions.  The class does not occupy any memory space till the time an object is instantiated. A class is a user-defined data type that defines properties and functions. For example: Class: Employee Properties: name, age, salary Functions: work(), attendMeeting() A class does not occupy memory until an object is created. ...
Read more