SAP has recently published integration flow design guidelines and patterns for SAP Cloud Integration that should help you to design the most common Enterprise Integration Patterns and to build enterprise-grade integration flows. The information is available on the SAP Help Portal. Furthermore, we provide sample integration content packages on the SAP API Business Hub. For more details, check out the announcement blog from a colleague of mine.
Regarding the patterns, while writing this blog I just noticed that bhalchandra.wadekar had the same idea and actually has been started working on this great blog series Enterprise Integration Patterns in Cloud Platform Integration with the promise to blog about a new pattern every Sunday. Now, that we obviously worked on this in parallel, some of the patterns that are on his ToDo list have been already covered by us here. So, @Bhalchandra, you may tick off those patterns from your list, this will save you some Sundays. On the other side, might be interesting how you would model the patterns since there might be different ways to do it. I am also grateful for all your effort, a lot of patterns that were on our list have now been covered by you already, so this saves a lot of time on our end. And it is definitely an added value for the community.
Anyway, with this blog I would like to start a blog series covering some of the patterns that we have published so far. Our intention however is not only to show how those patterns can be implemented on SAP Cloud Integration but also to compare with how we modeled this on SAP Process Orchestration for the different use cases as listed below.
Remember, we did a similar exercise for SAP Process Orchestration, see the following blog. At this point of time, we looked into the implementation of typical integration patterns using either the Process Integration (PI) runtime for stateless message processing or Business Process Management (BPM) for stateful message orchestration, so called integration-centric business processes.
For each pattern, I intend to recap how we have modeled the patterns on SAP Process Orchestration and will explain how those patterns can be implemented on SAP Cloud Integration addressing the following use cases and situations:
In the current blog, I will cover the Scatter-Gather pattern. This is actually a stateful pattern that describes how to broadcast a message to multiple receivers and to collect the individual responses in an aggregated message. For a definition of this and other patterns, see also Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf.
In the following, I will briefly explain the process model of a typical bidding process that we have implemented in SAP Process Orchestration. The process itself is modeled using BPM, the message exchange with the various systems is done using PI as part of the SAP Process Orchestration solution bundle. Here, I stick to the BPM process only. For a detailed description of the overall implementation, please refer to this blog.
As you may know, BPM supports the BPMN (Business Process Model & Notation) specification. The process model starts with a message start event. For a new loan request, a new process instance is created. After having carried out a solvency check, the request is broadcast to multiple banks. Here, this is a service call, the distribution of the request to the banks is done within PI (not shown here). The bidding process should be open for a limited time, say 2 days for instance. Within this time frame, we collect all bids from the banks. Once the time has been exceeded, we close the bidding, so no further bids are accepted. This is implemented via an event-based choice gateway with two branches. One branch contains an intermediate message event with a correlation condition so that the bids are assigned to the right process instance. The other branch contains an intermediate timer event which is triggered once the time is up. In this case, the best quote is calculated and a response is sent to the original requester. If a bid arrives after the time has been exceeded, the message is simply discarded because there is no active intermediate message event that meets the correlation condition.
For a detailed description of the Scatter-Gather pattern on SAP Cloud Integration, please check out the SAP Help Portal. In the following, I will only outline the rough model.
Since we do not support an intermediate message event on SAP Cloud Integration, we model the Scatter and the Gather parts as separate integration processes. This is not really a disadvantage, the distribution and the collection does not necessarily have to be modeled in the same process. If you somehow need information from one part in the other, you may exchange this via a data store. Furthermore, to be able to correlate both parts in the message monitor, you may define a common Application ID based on the request ID for instance.
The Scatter part takes the request and broadcasts it to multiple banks. Before, the timer is triggered via a separate message sent to the Gather part.
The Gather part collects the bids using an aggregator flow step, calculates the best quote, and sends the response to the original requester. The best quote calculation is done via a message mapping which we actually re-use from the Enterprise Service Repository (ESR) of SAP Process Orchestration. See how message mappings can be imported from an ESR within this blog.
By default, if a bid arrives after the time has been expired, a new process instance would be created. To avoid this, we have modeled a third integration process which actually checks if the bidding process is still active and eventually forwards the bids to the Gather part. The check is based on a respective entry in the data store. Once the request has been received, the request is stored in the data store with the request ID as key. This is done in the Scatter part, see above. Once the time has been exceeded, the data store entry is deleted. This is done in the Gather part, see above. In the model below, we check if the data store entry still exists. If not, the exception subprocess is triggered, and the message processing is stopped hence discarding the message. Otherwise, the message is sent to the Gather part.
So, get the Integration Flow Design Guidelines - Enterprise Integration Patterns package from the SAP API Business Hub, and start right away with testing out the scenario on your SAP Cloud Integration tenant. Or optionally, if you run an SAP Process Orchestration 7.5 system, you can deploy the very same integration flow there.
Stay tuned, more patterns to come:
Regarding the patterns, while writing this blog I just noticed that bhalchandra.wadekar had the same idea and actually has been started working on this great blog series Enterprise Integration Patterns in Cloud Platform Integration with the promise to blog about a new pattern every Sunday. Now, that we obviously worked on this in parallel, some of the patterns that are on his ToDo list have been already covered by us here. So, @Bhalchandra, you may tick off those patterns from your list, this will save you some Sundays. On the other side, might be interesting how you would model the patterns since there might be different ways to do it. I am also grateful for all your effort, a lot of patterns that were on our list have now been covered by you already, so this saves a lot of time on our end. And it is definitely an added value for the community.
Anyway, with this blog I would like to start a blog series covering some of the patterns that we have published so far. Our intention however is not only to show how those patterns can be implemented on SAP Cloud Integration but also to compare with how we modeled this on SAP Process Orchestration for the different use cases as listed below.
Remember, we did a similar exercise for SAP Process Orchestration, see the following blog. At this point of time, we looked into the implementation of typical integration patterns using either the Process Integration (PI) runtime for stateless message processing or Business Process Management (BPM) for stateful message orchestration, so called integration-centric business processes.
Use cases
For each pattern, I intend to recap how we have modeled the patterns on SAP Process Orchestration and will explain how those patterns can be implemented on SAP Cloud Integration addressing the following use cases and situations:
- You run SAP Process Integration / SAP Process Orchestration and are used to its tools and the way how integration scenarios are implemented. So, those blogs should help you to jump start getting familiar to the modeling environment and the capabilities of SAP Cloud Integration.
- You plan to move selected integration scenarios from SAP Process Integration / SAP Process Orchestration to SAP Cloud Integration such as scenarios bridging ground to cloud or where you connect to systems and applications outside your domain. So, we show you how to model an existing integration scenario on SAP Cloud Integration.
- You would like to implement an integration scenario on SAP Process Orchestration where you need to keep a state, however you would like to avoid using BPM because of its higher footprint compared to the PI runtime. As an alternative, you may use the cloud integration runtime inside SAP Process Orchestration. In this case, modeling is done in the cloud (requiring a SAP Cloud Integration tenant) whereas the scenario can run on-premise. For more details about the cloud integration runtime inside SAP Process Orchestration, check out my blog.
- You plan to migrate your SAP Process Integration Dual Stack system to SAP Process Orchestration. For some scenarios, you use cross-component BPM (ccBPM), e.g., for collecting messages or for complex message orchestration. In this case, you may consider to use the cloud integration runtime inside SAP Process Orchestration, see above. In this context, don't miss the blog about the various installation types and upgrade/migration paths.
In the current blog, I will cover the Scatter-Gather pattern. This is actually a stateful pattern that describes how to broadcast a message to multiple receivers and to collect the individual responses in an aggregated message. For a definition of this and other patterns, see also Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf.
Scatter-Gather on SAP Process Orchestration
In the following, I will briefly explain the process model of a typical bidding process that we have implemented in SAP Process Orchestration. The process itself is modeled using BPM, the message exchange with the various systems is done using PI as part of the SAP Process Orchestration solution bundle. Here, I stick to the BPM process only. For a detailed description of the overall implementation, please refer to this blog.
As you may know, BPM supports the BPMN (Business Process Model & Notation) specification. The process model starts with a message start event. For a new loan request, a new process instance is created. After having carried out a solvency check, the request is broadcast to multiple banks. Here, this is a service call, the distribution of the request to the banks is done within PI (not shown here). The bidding process should be open for a limited time, say 2 days for instance. Within this time frame, we collect all bids from the banks. Once the time has been exceeded, we close the bidding, so no further bids are accepted. This is implemented via an event-based choice gateway with two branches. One branch contains an intermediate message event with a correlation condition so that the bids are assigned to the right process instance. The other branch contains an intermediate timer event which is triggered once the time is up. In this case, the best quote is calculated and a response is sent to the original requester. If a bid arrives after the time has been exceeded, the message is simply discarded because there is no active intermediate message event that meets the correlation condition.
Scatter-Gather on SAP Cloud Integration
For a detailed description of the Scatter-Gather pattern on SAP Cloud Integration, please check out the SAP Help Portal. In the following, I will only outline the rough model.
Since we do not support an intermediate message event on SAP Cloud Integration, we model the Scatter and the Gather parts as separate integration processes. This is not really a disadvantage, the distribution and the collection does not necessarily have to be modeled in the same process. If you somehow need information from one part in the other, you may exchange this via a data store. Furthermore, to be able to correlate both parts in the message monitor, you may define a common Application ID based on the request ID for instance.
The Scatter part takes the request and broadcasts it to multiple banks. Before, the timer is triggered via a separate message sent to the Gather part.
The Gather part collects the bids using an aggregator flow step, calculates the best quote, and sends the response to the original requester. The best quote calculation is done via a message mapping which we actually re-use from the Enterprise Service Repository (ESR) of SAP Process Orchestration. See how message mappings can be imported from an ESR within this blog.
By default, if a bid arrives after the time has been expired, a new process instance would be created. To avoid this, we have modeled a third integration process which actually checks if the bidding process is still active and eventually forwards the bids to the Gather part. The check is based on a respective entry in the data store. Once the request has been received, the request is stored in the data store with the request ID as key. This is done in the Scatter part, see above. Once the time has been exceeded, the data store entry is deleted. This is done in the Gather part, see above. In the model below, we check if the data store entry still exists. If not, the exception subprocess is triggered, and the message processing is stopped hence discarding the message. Otherwise, the message is sent to the Gather part.
So, get the Integration Flow Design Guidelines - Enterprise Integration Patterns package from the SAP API Business Hub, and start right away with testing out the scenario on your SAP Cloud Integration tenant. Or optionally, if you run an SAP Process Orchestration 7.5 system, you can deploy the very same integration flow there.
Stay tuned, more patterns to come:
- Scatter-Gather (current blog)
- Composed Message Processor
- Aggregator
- Content-Based Router
- Recipient List
- Message Filter
- Content Filter
- Content Enricher
- Splitter
- SAP Managed Tags:
