This blog describes how to configure the new Poll Enrich pattern in combination with the sftp adapter, elaborating on use cases and different configuration options.
In general, there are three main use cases for the Poll Enrich pattern:
In this blog, I will focus on the second and third use case. Yet, I will shortly also mention the configuration option for the first use case when discussing the integration flow configuration.
This is the simple integration flow we are going to build:
The flow starts with a HTTP inbound adapter that triggers the message processing. In a subsequent Content Modifier, some properties are defined that are used dynamically in the sftp adapter when polling the file from sftp server. At the very end, the message is sent out to the receiver system.
Let's start.
In the integration flow, draw a connection from the Sender participant to the Start Message event and select the HTTP adapter; then add a Content Modifier next to it. Configure the HTTP adapter as you like.
In the Content Modifier, you define the properties that are to be used dynamically in the SFTP Adapter. For dynamically configurable settings, refer to the blog Cloud Integration – Dynamically Configure the SFTP Receiver Adapter. The same settings are allowed dynamically in the sftp adapter in the Poll-Enrich pattern. Note that with the first increment (version 1.0 of the Poll Enrich sftp adapter), Proxy Type and Authentication can't yet be set dynamically.
For my simple scenario, I define the following properties to set them dynamically via XPath from the incoming http request:
Next to the Content Modifier, configure the Poll Enrich step. For this, in the Call section of the modeler, select External Call and then Poll Enrich.
In the Poll Enrich step, we have to configure how the payloads need to be aggregated: the one before the Poll Enrich step and the payload fetched from sftp server need. In addition you configure what to do if no message is found on the sftp server.
The aggregation is configured with the help of the Aggregation Algorithm in the Poll Enrich step:
For our use case (to simply read files via external trigger or via dynamic configuration in the sftp adapter), the Replace option is the correct choice. This algorithm simply replaces the original message (payload and headers) before the Poll Enrich step with the new one fetched from the sftp server.
If you want to enrich your original payload (use case 1), you have to select one of the other two algorithms:
If you want to get an exception in case no message is found on the sftp server you can select the checkbox Throw Exception if no Message Found. If not selected, the header SAP_PollEnrichMessageFound is set to false if no message can be found during processing. Based on the header you can configure alternative processing.
Having defined the Poll Enrich step, we now need to configure the SFTP adapter. To do so, add another Sender participant and draw a connection from this new Sender participant to the Poll Enrich step. Select the SFTP adapter.
The configuration of the SFTP adapter will look familiar to you because it contains almost the same configuration options as the normal SFTP sender adapter. Some configurations, like the Scheduler tab, are missing because they are not relevant in this pattern, but most settings are equal.
In the configuration we specify the properties defined in the Content Modifier:
As a final step, configure a connection to the Receiver participant to send the message out.
Deploy the flow and trigger it with a payload that provides the necessary data; in my example, I use the following simple payload:
Note that in case no file can be found on the sftp server, the processing continues with the payload before the Poll Enrich step. It is planned to offer specific handling in such a case.
I hope this simple scenario helps you to understand the Poll Enrich pattern.
Use Poll Enrich with SFTP Adapter
In general, there are three main use cases for the Poll Enrich pattern:
- You want to enrich the message payload with additional data retrieved from a file on the sftp server.
- You want to poll from an sftp server triggered by an external trigger, for example triggered via HTTP call.
- You need to poll from an sftp server but want to define the configuration in the sftp adapter dynamically, for example from a partner directory.
In this blog, I will focus on the second and third use case. Yet, I will shortly also mention the configuration option for the first use case when discussing the integration flow configuration.
This is the simple integration flow we are going to build:
The flow starts with a HTTP inbound adapter that triggers the message processing. In a subsequent Content Modifier, some properties are defined that are used dynamically in the sftp adapter when polling the file from sftp server. At the very end, the message is sent out to the receiver system.
Let's start.
In the integration flow, draw a connection from the Sender participant to the Start Message event and select the HTTP adapter; then add a Content Modifier next to it. Configure the HTTP adapter as you like.
In the Content Modifier, you define the properties that are to be used dynamically in the SFTP Adapter. For dynamically configurable settings, refer to the blog Cloud Integration – Dynamically Configure the SFTP Receiver Adapter. The same settings are allowed dynamically in the sftp adapter in the Poll-Enrich pattern. Note that with the first increment (version 1.0 of the Poll Enrich sftp adapter), Proxy Type and Authentication can't yet be set dynamically.
For my simple scenario, I define the following properties to set them dynamically via XPath from the incoming http request:
Next to the Content Modifier, configure the Poll Enrich step. For this, in the Call section of the modeler, select External Call and then Poll Enrich.
In the Poll Enrich step, we have to configure how the payloads need to be aggregated: the one before the Poll Enrich step and the payload fetched from sftp server need. In addition you configure what to do if no message is found on the sftp server.
The aggregation is configured with the help of the Aggregation Algorithm in the Poll Enrich step:
For our use case (to simply read files via external trigger or via dynamic configuration in the sftp adapter), the Replace option is the correct choice. This algorithm simply replaces the original message (payload and headers) before the Poll Enrich step with the new one fetched from the sftp server.
If you want to enrich your original payload (use case 1), you have to select one of the other two algorithms:
- Combine XML: This option combines the message polled from the sftp server with the original message in multimapping format. This option makes most sense if both, the original and the polled message, are XML messages.
- Concatenate: The message polled from the sftp server is added to the existing message. This option is useful for strings.
If you want to get an exception in case no message is found on the sftp server you can select the checkbox Throw Exception if no Message Found. If not selected, the header SAP_PollEnrichMessageFound is set to false if no message can be found during processing. Based on the header you can configure alternative processing.
Having defined the Poll Enrich step, we now need to configure the SFTP adapter. To do so, add another Sender participant and draw a connection from this new Sender participant to the Poll Enrich step. Select the SFTP adapter.
The configuration of the SFTP adapter will look familiar to you because it contains almost the same configuration options as the normal SFTP sender adapter. Some configurations, like the Scheduler tab, are missing because they are not relevant in this pattern, but most settings are equal.
In the configuration we specify the properties defined in the Content Modifier:
As a final step, configure a connection to the Receiver participant to send the message out.
Deploy the flow and trigger it with a payload that provides the necessary data; in my example, I use the following simple payload:
<?xml version='1.0' encoding='UTF-8' ?>
<request>
<sftp>
<Directory>test/test1</Directory>
<File>TestFile*</File>
</sftp>
</request>Note that in case no file can be found on the sftp server, the processing continues with the payload before the Poll Enrich step. It is planned to offer specific handling in such a case.
I hope this simple scenario helps you to understand the Poll Enrich pattern.
Planned Extensions:
- Dynamic Configuration of Proxy Type and Authentication (available after the June 2021 update)
- Keep File and Process Again option not available (available with July 2021 update)
- Configure Handling if File is not found on the sftp server (available with August 2021 update)
- Support FTP adapter in Poll Enrich step
- SAP Managed Tags:
76 Comments
