Integrate jBPM with Apache Kafka Server – Complete Setup Guide

 💡 Introduction

In modern event-driven architectures, integrating ⚙️ jBPM (Business Process Management) with 📡 Apache Kafka unlocks real-time visibility and automation across systems.

jBPM can publish its process events, task updates, and logs directly to a Kafka server (broker) — enabling analytics, auditing, and microservice event chaining.

This guide shows you how to connect jBPM to a running Kafka Server without writing any Java code, using only configuration changes in WildFly / KIE Server.

⚙️ 1️⃣ Prerequisites

Before starting, make sure you have:

✅ Kafka Server (Broker) up and running
(e.g., localhost:9092 or your remote broker IP/port)
✅ jBPM / KIE Server deployed on WildFly
✅ Access to modify standalone.xml or standalone.conf

If you’re using a cloud Kafka (like Azure Event Hubs or Confluent Cloud), note the connection parameters (host, port, protocol, username, password or connection string).

🧠 2️⃣ Enable jBPM Kafka Event Emitter

jBPM includes a ready-to-use module:
org.jbpm:jbpm-event-emitters-kafka

It automatically pushes process and task events to Kafka when enabled.

🔹 Add this dependency to your KIE Server (if not already bundled):

<dependency>
    <groupId>org.jbpm</groupId>
    <artifactId>jbpm-event-emitters-kafka</artifactId>
    <version>7.74.1.Final</version>
</dependency>

🧩 3️⃣ Configure Kafka Connection in WildFly

Open your WildFly standalone.xml (or standalone-full.xml)
and add this inside the <system-properties> section:

🧾 Example: Local Kafka Broker (No Security)

<system-properties>
    <property name="org.jbpm.event.emitters.kafka.bootstrap.servers" value="localhost:9092"/>
    <property name="org.jbpm.event.emitters.kafka.topic" value="jbpm-logs"/>
    <property name="org.jbpm.event.emitters.kafka.client.id" value="jbpm-emitter"/>
</system-properties>

This tells jBPM to send all business events to Kafka topic jbpm-logs.

🔐 4️⃣ If Kafka Uses SSL or Authentication

If your Kafka cluster requires a password, SSL, or a connection string (like Azure Event Hubs), use these properties instead:

🔹 For SSL Authentication

<system-properties>
    <property name="org.jbpm.event.emitters.kafka.bootstrap.servers" value="10.10.1.25:9093"/>
    <property name="org.jbpm.event.emitters.kafka.topic" value="jbpm-logs"/>
    <property name="org.jbpm.event.emitters.kafka.client.id" value="jbpm-emitter"/>
    <property name="org.jbpm.event.emitters.kafka.security.protocol" value="SSL"/>
    <property name="org.jbpm.event.emitters.kafka.ssl.truststore.location" value="/opt/wildfly/config/truststore.jks"/>
    <property name="org.jbpm.event.emitters.kafka.ssl.truststore.password" value="changeit"/>
</system-properties>

🔹 For Cloud Kafka (Azure Event Hubs, SASL_SSL)

<system-properties>
    <property name="org.jbpm.event.emitters.kafka.bootstrap.servers" value="YOUR_NAMESPACE.servicebus.windows.net:9093"/>
    <property name="org.jbpm.event.emitters.kafka.security.protocol" value="SASL_SSL"/>
    <property name="org.jbpm.event.emitters.kafka.sasl.mechanism" value="PLAIN"/>
    <property name="org.jbpm.event.emitters.kafka.sasl.jaas.config"
              value="org.apache.kafka.common.security.plain.PlainLoginModule required username='$ConnectionString' password='Endpoint=sb://YOUR_NAMESPACE.servicebus.windows.net/;SharedAccessKeyName=YOUR_SAS_KEY_NAME;SharedAccessKey=YOUR_SAS_KEY;EntityPath=YOUR_EVENT_HUB';"/>
    <property name="org.jbpm.event.emitters.kafka.topic" value="YOUR_EVENT_HUB"/>
    <property name="org.jbpm.event.emitters.kafka.client.id" value="jbpm-emitter"/>
    <property name="org.jbpm.event.emitters.kafka.client.dns.lookup" value="use_all_dns_ips"/>
</system-properties>

💡 use_all_dns_ips ensures that the Kafka client connects to all IPs returned by DNS — improving reliability with cloud load balancers.

📈 5️⃣ Restart and Verify the Integration

Restart WildFly or KIE Server:

./bin/standalone.sh -c standalone-full.xml

Check logs:

INFO  [org.jbpm.event.emitters.kafka.KafkaEventEmitter] Connecting to Kafka broker at localhost:9092
INFO  [org.jbpm.event.emitters.kafka.KafkaEventEmitter] Successfully sent event to topic jbpm-logs

✅ If you see these messages, the integration is working!

🔍 6️⃣ Validate from Kafka Side

Open your Kafka console consumer:

bin/kafka-console-consumer.sh --topic jbpm-logs --bootstrap-server localhost:9092 --from-beginning

You should see JSON messages like:

{
  "processId": "OrderProcess",
  "instanceId": 101,
  "eventType": "PROCESS_STARTED",
  "timestamp": "2025-10-07T14:21:05Z"
}

Each line represents a jBPM event — process started, task completed, variable changed, etc.

📊 7️⃣ Common Configuration Tips

GoalPropertyExample
Change topic nameorg.jbpm.event.emitters.kafka.topicprocess-events
Adjust retriesorg.jbpm.event.emitters.kafka.retries5
Set timeoutorg.jbpm.event.emitters.kafka.request.timeout.ms60000
Acknowledgment modeorg.jbpm.event.emitters.kafka.acksall

🔄 8️⃣ Optional – Use REST Task Instead of Event Emitter

If you don’t want to use the event emitter, you can add a REST Task in your BPMN process to call Kafka REST Proxy directly — no code needed.

Example:

  • URL → http://<kafka-rest>:8082/topics/jbpm-logs

  • Method → POST

  • Body:

{
  "records": [
    { "key": "jbpm", "value": { "processId": "#{processInstanceId}", "event": "#{eventType}" } }
  ]
}

🧠 9️⃣ Benefits of jBPM + Kafka Integration

✅ Real-time process monitoring
✅ Streamlined log collection and analytics
✅ Easy integration with ELK, OpenTelemetry, or Flink
✅ Scalable event-driven orchestration across microservices

🏁 Conclusion

🎯 Integrating jBPM with Kafka Server makes your process automation event-driven, scalable, and transparent.

Once connected, every jBPM process or task event becomes a Kafka message — allowing you to feed dashboards, alerting systems, and analytics tools in real time.

This architecture is the foundation for intelligent BPM, data-driven workflows, and real-time decision systems.

🔹 📚 Recommended Articles

👉 Continue learning:

Getting started with Kafka?

https://shikhanirankari.blogspot.com/2025/10/install-apache-kafka-step-by-step-guide.html

💼 Professional Support Available

 If you are facing issues in real projects related to enterprise backend development or workflow automation, I provide paid consulting, production debugging, project support, and focused trainings. 

Technologies covered include Java, Spring Boot, PL/SQL, Azure, and workflow automation (jBPM, Camunda BPM, RHPAM).

📧 Contact: ishikhanirankari@gmail.com | info@realtechnologiesindia.com

🎥 Learn IT with Shikha on YouTube

Prefer learning through videos? Watch practical tutorials on Kafka, Camunda, Alfresco, Java, Spring Boot, Microservices and Enterprise Architecture.

▶ Subscribe to Learn IT with Shikha on YouTube

Comments

Post a Comment

Popular posts from this blog

Top 50 Camunda BPM Interview Questions and Answers for Developers (2026 Guide)

Image
  Top 50 Camunda BPM Interview Questions and Answers for Developers (2026 Guide) Introduction Camunda BPM is one of the most widely used workflow automation platforms for building process-driven applications using BPMN, DMN, and microservices architecture . Many companies use Camunda for: Business process automation Microservices orchestration Workflow management Enterprise integration Because of its growing popularity, Camunda BPM interview questions are frequently asked in Java developer interviews . In this guide, we cover 50 important Camunda BPM interview questions and answers ranging from beginner to advanced levels .
Read more

10 BPMN Best Practices Every Camunda Developer Should Know

Image
  10 BPMN Best Practices Every Camunda Developer Should Know Designing BPMN processes in Camunda is easy—but designing them correctly, scalable, and maintainable is what separates beginners from experts. In this guide, you’ll learn 10 essential BPMN best practices that every Camunda developer should follow. 🔹 1. Keep Your Process Simple Avoid overcomplicating diagrams. 👉 A simple process is: Easier to debug Easier to maintain Easier for business users to understand 🔹 2. Use Meaningful Names Bad: Task1 ProcessA Good: Validate Customer Data Approve Loan Request 👉 Always use verb + business context 🔹 3. Use Async Continuations Wisely Use: asyncBefore asyncAfter 👉 Helps in: Retry handling Transaction boundaries Avoiding long DB locks 🔹 4. Avoid Long-Running Service Tasks Problem: Blocks Job Executor Solution: 👉 Use External Tasks for: API calls Heavy processing 🔹 5. Use Exclusive Gateways Properly 👉 Always define conditions clearly Avoid: Multiple true conditions Missing d...
Read more

OOPs Concepts in Java | English | Object Oriented Programming Explained

Image
🔹OOPS (Object Oriented Programming System) in Java Object-Oriented Programming (OOPS) is a programming paradigm that uses classes and objects to design software. It simplifies development and maintenance by applying key concepts: 📌 Class: Class   is a user-defined data type which defines its properties and its functions. Class is the only logical representation of the data. For example, Human beings in an organisation is a class like Employee. The age, name, department, salary etc. of a human being/employee are its properties, and the jobs performed by the employees in an organisation are known as functions.  The class does not occupy any memory space till the time an object is instantiated. A class is a user-defined data type that defines properties and functions. For example: Class: Employee Properties: name, age, salary Functions: work(), attendMeeting() A class does not occupy memory until an object is created. ...
Read more