This blog is part of a series where my goal is to offer you a practical overview of SAP Integration Suite, Advanced Event Mesh. As I continue to write more blogs, I will provide summaries on the page below.
Advanced Event Mesh: Create your first event broker
In the first blog, we created our broker instance, marking the initial step in the Mission Control component. First, let's clarify some concepts before examining the broker.
The broker works as a middleware, facilitating the transmission of messages between producers and consumers. It possesses the capability to receive a message from a publisher and route it to one or more consumers that are listening to the event, allowing for a "publish once and receive in many" scenario. Consumers attract published events by subscribing to a topic.
Producers or publisher represents who is publishing an event.
Consumers or subscribers represents who us receiving the event.
Events
An event is a changed state in a system. To simplify, a sales order being created is an event, and so is an update to a sales order. Other events include the order being blocked, released, and so on.
Messages
Messages are a consequence of an event, and exchanged messages between producers and consumers are structured in three components: header, properties, and body. While the header contains information about the destination (topic), correlation ID, message ID, and delivery mode; properties can include custom elements such as an ID to track your message across systems, and the body serves as the payload.
Imagine a message as an email, where the destination (topic) is like the email subject that indicates the main topic, while the payload is equivalent to the email body, containing the actual information or message being communicated.
Topic
We use topics to classify the information/message that are sent.
When producers send a message, in addition to the message body, they also include the topic, which serves as the subject or classification of that message. Consumers utilize this topic, or subject, to filter messages they are interested in. This topic information is crucial for the broker to route the message to the correct consumer.
In the email analogy, it's as if a producer is sending me various emails, but my interest lies in emails with a specific subject. So, I create a rule in my email service to filter by subject. It's essentially the same concept.
Due to this, designing your topic structure well is very important because it will facilitate your understanding of the arriving messages, and consumers can filter the messages they are interested in more effectively.
Spend some time to understand how to design your topic well.
Topic Architecture Best Practices (cloud.sap)
Message Protocols
Advanced Event Mesh supports various message protocols such as SMF, AMQP, MQTT, and REST. You can publish an event using the SMF protocol and consume the event using the AMQP protocol effortlessly, the broker handles all necessary conversions internally. The same applies to coding languages, you can create a microservice to publish events using Python and consume it using Node.js.
It provides APIs that you can use to start developing your application.
Regardless of your chosen protocol, we publish events to the broker following two important concepts of message delivery:
Direct Messages
Direct Messages provide reliable but not guaranteed delivery of messages to a consumer. This implies that when you send an event to your broker and there's a consumer listening to that event, the message will be delivered. However, if the consumer disconnects, and the publisher continues sending events to the broker, the broker will accept and receive the events, but they won't be delivered to anyone, resulting in message loss. You don't need to set up anything on your broker to start sending direct messages; just send them.
You can use it when your scenario can tolerate message loss.
Guaranteed Messages
Guaranteed Messages are used when you want a guarantee that all received messages will be delivered to a consumer. Upon receipt, messages are first persisted on the broker and delivered to the consumer in the order they were published.
With guaranteed messages, messages can be persisted on a topic endpoint or a queue. Even if there's no consumer connected to the queue, the messages are stored. When a consumer connects to the queue, the broker operates on a first-in-first-out basis, pushing messages in sequence. As the consumer receives and starts processing the first message, the broker waits for the consumer's acknowledgment (ACK) indicating successful processing before removing the message from the queue. If the consumer is unable to process the message and returns an unacknowledged (NACK) to the broker, the broker redirects the message for processing again, either by the same or another consumer.
You can use guaranteed messages when you cannot afford to lose any events.
Message Delivery Modes (cloud.sap)
There are two types of guaranteed messages:
Queues
There are two access types to set up a queue, and these determine how messages will be delivered to consumers:
In exclusive mode, the first consumer connected to your queue will receive the messages, and a second one acts as a standby consumer, waiting for its turn. It will only receive messages when the first connected consumer disconnects.
In the non-exclusive mode, messages are delivered to consumers in a round-robin fashion, like a load balancer.
In non-exclusive mode, there's no guarantee that messages will be processed in the correct order. Although messages are delivered in the correct order, one consumer may process a message faster than another. If maintaining the correct order of processing is essential, it's recommended to use the exclusive mode.
Filtering messages
As mentioned earlier, consumers attract messages by subscribing to topics. This allows us to use topics to filter only the messages that a specific consumer is interested in, even when the broker is receiving hundreds or thousands of different events.
Let's check some examples:
Publisher sends events of a sales order on topics:
This topic is composed of the application name that produces the event, domain, action verb, topic versioning, and some properties representing Sales Organization, Distribution Channel, and Division.
A first consumer subscribed to topic BusinessSuite/Sales/SalesOrder/Created/v1/BR01/10/01 will receive only the sales order created events related to sales organization BR01, distribution channel 10, and division 01.
A second consumer aims to receive messages related to sales order within the same sales organization, dist. channel and division but, they are interested in events for sales order created, updated, released and blocked.In this scenario, the use of wildcards enables us to efficiently receive all distinct events related to the sales order without the need to subscribe to numerous specific topics.
The second consumer subscribes to BusinessSuite/Sales/SalesOrder/*/v1/BR01/10/01 instead of individual subscriptions:
BusinessSuite/Sales/SalesOrder/Created/v1/BR01/10/01 BusinessSuite/Sales/SalesOrder/Updated/v1/BR01/10/01 BusinessSuite/Sales/SalesOrder/Blocked/v1/BR01/10/01 BusinessSuite/Sales/SalesOrder/Released/v1/BR01/10/01
A third consumer aims to receive messages related to sales orders with various actions, irrespective of the sales organization, distribution channel, and division. Leveraging wildcards, the third consumer subscribes to BusinessSuite/Sales/SalesOrder/*/v1/> to capture events across different contexts.
In summary, SAP Integration Suite, Advanced Event Mesh allows us to use wildcards (*) and (>) for effectively filtering events.
On the next blog, we'll explore some of the broker features
Advanced Event Mesh: Create your first event broker
Basic Concepts
In the first blog, we created our broker instance, marking the initial step in the Mission Control component. First, let's clarify some concepts before examining the broker.
The broker works as a middleware, facilitating the transmission of messages between producers and consumers. It possesses the capability to receive a message from a publisher and route it to one or more consumers that are listening to the event, allowing for a "publish once and receive in many" scenario. Consumers attract published events by subscribing to a topic.
Producers or publisher represents who is publishing an event.
Consumers or subscribers represents who us receiving the event.
Events
An event is a changed state in a system. To simplify, a sales order being created is an event, and so is an update to a sales order. Other events include the order being blocked, released, and so on.
Messages
Messages are a consequence of an event, and exchanged messages between producers and consumers are structured in three components: header, properties, and body. While the header contains information about the destination (topic), correlation ID, message ID, and delivery mode; properties can include custom elements such as an ID to track your message across systems, and the body serves as the payload.
Imagine a message as an email, where the destination (topic) is like the email subject that indicates the main topic, while the payload is equivalent to the email body, containing the actual information or message being communicated.
Topic
We use topics to classify the information/message that are sent.
When producers send a message, in addition to the message body, they also include the topic, which serves as the subject or classification of that message. Consumers utilize this topic, or subject, to filter messages they are interested in. This topic information is crucial for the broker to route the message to the correct consumer.
In the email analogy, it's as if a producer is sending me various emails, but my interest lies in emails with a specific subject. So, I create a rule in my email service to filter by subject. It's essentially the same concept.
Due to this, designing your topic structure well is very important because it will facilitate your understanding of the arriving messages, and consumers can filter the messages they are interested in more effectively.
Spend some time to understand how to design your topic well.
Topic Architecture Best Practices (cloud.sap)
Message Protocols
Advanced Event Mesh supports various message protocols such as SMF, AMQP, MQTT, and REST. You can publish an event using the SMF protocol and consume the event using the AMQP protocol effortlessly, the broker handles all necessary conversions internally. The same applies to coding languages, you can create a microservice to publish events using Python and consume it using Node.js.
It provides APIs that you can use to start developing your application.
Regardless of your chosen protocol, we publish events to the broker following two important concepts of message delivery:
- Direct Messages
- Guaranteed Messages
Direct Messages
Direct Messages provide reliable but not guaranteed delivery of messages to a consumer. This implies that when you send an event to your broker and there's a consumer listening to that event, the message will be delivered. However, if the consumer disconnects, and the publisher continues sending events to the broker, the broker will accept and receive the events, but they won't be delivered to anyone, resulting in message loss. You don't need to set up anything on your broker to start sending direct messages; just send them.
You can use it when your scenario can tolerate message loss.
Guaranteed Messages
Guaranteed Messages are used when you want a guarantee that all received messages will be delivered to a consumer. Upon receipt, messages are first persisted on the broker and delivered to the consumer in the order they were published.
With guaranteed messages, messages can be persisted on a topic endpoint or a queue. Even if there's no consumer connected to the queue, the messages are stored. When a consumer connects to the queue, the broker operates on a first-in-first-out basis, pushing messages in sequence. As the consumer receives and starts processing the first message, the broker waits for the consumer's acknowledgment (ACK) indicating successful processing before removing the message from the queue. If the consumer is unable to process the message and returns an unacknowledged (NACK) to the broker, the broker redirects the message for processing again, either by the same or another consumer.
You can use guaranteed messages when you cannot afford to lose any events.
Message Delivery Modes (cloud.sap)
There are two types of guaranteed messages:
- Topic Endpoints: This is more focused on JMS applications.
- Queues: This is usually the more commonly used case.
Queues
There are two access types to set up a queue, and these determine how messages will be delivered to consumers:
- Exclusive
- Non-Exclusive
In exclusive mode, the first consumer connected to your queue will receive the messages, and a second one acts as a standby consumer, waiting for its turn. It will only receive messages when the first connected consumer disconnects.
In the non-exclusive mode, messages are delivered to consumers in a round-robin fashion, like a load balancer.
In non-exclusive mode, there's no guarantee that messages will be processed in the correct order. Although messages are delivered in the correct order, one consumer may process a message faster than another. If maintaining the correct order of processing is essential, it's recommended to use the exclusive mode.
Filtering messages
As mentioned earlier, consumers attract messages by subscribing to topics. This allows us to use topics to filter only the messages that a specific consumer is interested in, even when the broker is receiving hundreds or thousands of different events.
Let's check some examples:
Publisher sends events of a sales order on topics:
- BusinessSuite/Sales/SalesOrder/Created/v1/BR01/10/01
- BusinessSuite/Sales/SalesOrder/Updated/v1/BR01/10/01
- BusinessSuite/Sales/SalesOrder/Blocked/v1/BR01/10/01
- BusinessSuite/Sales/SalesOrder/Released/v1/BR01/10/01
- BusinessSuite/Sales/SalesOrder/Created/v1/US01/99/01
- BusinessSuite/Sales/SalesOrder/Updated/v1/US01/99/01
This topic is composed of the application name that produces the event, domain, action verb, topic versioning, and some properties representing Sales Organization, Distribution Channel, and Division.
A first consumer subscribed to topic BusinessSuite/Sales/SalesOrder/Created/v1/BR01/10/01 will receive only the sales order created events related to sales organization BR01, distribution channel 10, and division 01.
A second consumer aims to receive messages related to sales order within the same sales organization, dist. channel and division but, they are interested in events for sales order created, updated, released and blocked.In this scenario, the use of wildcards enables us to efficiently receive all distinct events related to the sales order without the need to subscribe to numerous specific topics.
The second consumer subscribes to BusinessSuite/Sales/SalesOrder/*/v1/BR01/10/01 instead of individual subscriptions:
BusinessSuite/Sales/SalesOrder/Created/v1/BR01/10/01 BusinessSuite/Sales/SalesOrder/Updated/v1/BR01/10/01 BusinessSuite/Sales/SalesOrder/Blocked/v1/BR01/10/01 BusinessSuite/Sales/SalesOrder/Released/v1/BR01/10/01
A third consumer aims to receive messages related to sales orders with various actions, irrespective of the sales organization, distribution channel, and division. Leveraging wildcards, the third consumer subscribes to BusinessSuite/Sales/SalesOrder/*/v1/> to capture events across different contexts.
In summary, SAP Integration Suite, Advanced Event Mesh allows us to use wildcards (*) and (>) for effectively filtering events.
On the next blog, we'll explore some of the broker features
- SAP Managed Tags:
4 Comments
