** Disclaimer:While all of you can use these best practices on the projects after evaluating it is relevant for your customer or usecase, you shouldn't cut and paste this best practice blog as the best practices developed by your organization **
*These best practices are not "One Size that fits all customers" and hence you need to evaluate whether it works for your customer and your use case"
1 Introduction
1.1 Objective
The scope of this blog is to set the development guidelines for Integration developers who will use SAP Cloud Platform Integration Service to develop integrations for Client consistently across the projects in Client.
This blog is an end to end comprehensive CPI standards & best practices that is just an extension of SAP guidelines and insights shared by amazing CPI developers across the world coupled with our real world experience of many projects and recent experience of distributing millions of API messages via CPI.This is a living document which will evolve over the period with our learnings.
1.2 Intended Audience
Integration architect designers and developers who are already little familiar with SAP CPI as an Integration tool can easily infer and implement the guidelines in this book.
2 Architecture
2.1 SAP Integration Strategy for Cloud Transformation
SAP provides many apps for integrating on premise and cloud applications but we should be following SAP strategic direction when advising clients on which integration app we need to procure based on Integration Domain and Use Case Pattern. Please don't miss my blog on Do's and Dont's on SAP Cloud projects.
Useful Links:
Please refer to SAP CIO Guide below for understanding SAP Strategic Direction.
Do’s and Don’ts of SAP CLOUD PROJECTS – Moving from ASAP Methodology to Agile SAP Activate
2.2 Licensing Model
The following table can be used as a guideline for choosing the right licensing model and you can use scp license estimator to determine the approximate costs for your requirements.
Licensing Models
SAP Provides 2 licensing models for SAP Cloud Platform Components.
CPEA Consumption-based pricing
Configure and scale cloud services with business agility by accessing eligible services and simplified engagement instantly.
What’s included:
- Use eligible services without a new contract
- Prepay consumption of services with credits
- Renew your subscription at the end of the period
- Pay in advance when cloud credits are used up
- Add credits to your cloud account multiple times during a single consumption period
https://discovery-center.cloud.sap/serviceCatalog provides you the ability to caclulate approximate licensing costs based on services you want to consume.
Subscription-based pricing
Receive a fixed price and length for access to subscribed services to consume and configure them as your business requires. This option is available for select services.
What’s included:
- Modify your contract to access more services
- Pay a fixed cost, regardless of consumption
- Pay in advance when the contract period starts
- Renew the subscription at the end of the period
SAP Store provides you the ability to calculate the price based on the service you want to subscribe and buy on SAP Cloud Platform.
Pricing Details For SAP CPI - CONSUMPTION MODEL
SAP CPI Process Integration Style
Please check SAP Cloud Discovery Centre for pricing of CPI process integration suite.
SAP CPI Integration Suite (New pricing model)
Please check SAP Cloud Discovery Centre for pricing of SAP API MANAGEMENT, CPI, ENTERPRISE MESSAGING BUNDLE.
Please check below SAP Note on how cloud credits and message metrics are calculated for SAP CPI Suite based on type of messages.
SAP CPI Data Integration Style
Please check pricing of ETL, Data Integration of CPI DATA Services here ( Check with SAP on roadmap as they said SAP DATA Intelligence is their way forward)
Please check below for the pricing of strategic and the new Data Intelligence service from SAP for Structured, Un-Structured Data Integration that allows data scientists to design, deploy, and manage machine-learning models with built-in tools for data governance, management, and transparency.
Pricing Details For SAP CPI - SUBSCRIPTION MODEL
SAP CPI Integration Suite
Please check pricing of CPI Suite for subscription model.
2.3 SAP CPI Design Guidelines - API Business Hub Package
It is recommended that every developer should go through the CPI Cloud Exemplar package and SAP CPI Integration Design Guidelines and SAP CPI Troubleshooting Tips templates published by SAP to constantly and share the best practices on Cloud Integration in the form of FAQs, do’s and don’ts, code snippets, integration flow templates, how-to guides, troubleshooting tips, etc.
The packages CPI Cloud Exemplar package and SAP CPI Integration Design Guidelines and SAP CPI Troubleshooting Tips includes not only detailed documentation or FAQs, but also working samples and templates that help you:
- Understand how you can perform basic tasks
- Identify the common pitfalls while designing your flow
- Discover optimal ways of modelling an integration flow
- Determine techniques to achieve better memory footprint
- Define what to keep in mind in order to create performant integration flows
- Solve commonly known errors with ready solution
3 Development Guidelines
3.1 Development Tool Strategy
SAP CPI offers development in two different environments namely eclipse and Web IDE.
3.1.1 Eclipse
Eclipse is java based IDE for software development which needs to install in developer machine. CPI extensions are required to be added in eclipse software.
3.1.2 CPI Web UI
Web UI is SAP’s own browser based platform for the CPI development.
3.1.3 Recommended Development Tool
As per the SAP roadmap, eclipse based development tool will be obsolete soon and hence all the CPI development should be carried out in CPI Web UI where ever possible and integration flows should be imported from Eclipse to CPI Web UI if developer used Eclipse due to any current limitations of Web UI.
3.2 Naming Conventions
The table below summarizes the naming convention to be adopted in Client for SAP CPI development.
| Object Type | Definition | Convention Syntax | Example |
| Custom Package Name | A collection of related APIs or services, belonging to one product or product area, packaged and delivered together. | Name: The name of the package should refer to the two products plus product lines between which the integration needs to take place if it is point to point. If you are developing generic integration packages or country specific packages then refer to generic and country specific sections in the example. | Point to Point: Name: SAP Hybris Cloud for Customer Integration with SAP Hybris Marketing. If you are developing generic interfaces like EDI or API(S) and you don't want to tie IFLOWS to a specific system then use naming conventions like below. Generic: <Business Object/Domain Description>for<Sender or Reciever System> Ex:EDI Integration Templates for SAP Cloud Platform Integration Advisor Ex: EDI Integration Templates for Successfactors Ex: S4 HANA API(S) Country Specific If you are developing package specific to country like tax interfaces then I would follow: <Business Object/Domain Description> for <Country><Sender or Reciever System> Ex: Payroll e-Filing of Employees' Payments and Deductions for UK HMRC |
Point to Point: Technical Name: Z_<Sending System Name>_Integration_ With_<Receiving System Name> Generic: Z_<Business Object/Domain Description><Sender or Reciever System> Country Specific: Z_<Business Object/Domain Description><Country><Sender> OR/AND <Reciever System> | Point to Point : Technical Name: Z_Salesforce_Integration_With_SAPC4HANA Generic: Z_SupplyChain_S4HANAAPI Country Specific Z_PAYROLL_UK_HMRC | ||
| Vendor : <Organisation Name> | Vendor Name: Wales & East Utilities | ||
| Version: The version number should always be 1.0.0 when the package is productionized and should be incremented by +1.0.0 or 1.1.0 or 1.0.1 for every transport based on whether change is major or minor or micro. | Version : 1.0.0(when productionised) ,1.0.1(After micro change of first transport of artefact) on whether change is major or minor or micro. | ||
| Overview: The full long description of the package describing the usage, functionality and goal of the package. | Overview: This integration package enables you to integrate business processes between SAP Hybris Marketing or S/4HANA Cloud Marketing Edition and SAP Hybris Cloud for Customer. The integration scope includes a call center scenario, the creation of leads in SAP Cloud for Customer from a campaign in SAP Hybris Marketing and the replication of accounts, contacts, individual customers, leads and opportunities from SAP Hybris Cloud for Customer to SAP Hybris Marketing. | ||
| TAGS: The package should be tagged with country and line of business and industry using relevant dropdowns and a common keyword search should be based on most commonly used in package short description or Interface ID(S) of package. | Country:UK Line of Business: Metering Industry : Oil and Gas Keyword : Customer, NI000001 Product : SAP Marketing CLOUD, SAP C/4 HANA Project Name : "Wales and West Metering DTE | ||
| Documents: The API / Interface Documentation should be linked for each interface/artefact in the package and name should match exactly with IFLOW Name. | Create Planning Orders | ||
| iFlow | Integration flow is a BPMN-based model that is executable by orchestration middleware | <Interface Action i.eSend/Recieve/Create/Publish/Update/Delete/Replicate/Insert/Maintain><Interface Business Object ShortDescription> | Create Planning Orders |
| Content Modifier | This Step is used to modify content | Modify_<Details of modification> | Modify_logHTTPHeader |
| Script | This step is used to create groovy script to handle complex flows | SC_<Objective of script> | SC_formatMessage |
| Message Mapping | Message Mapping enables to map source message to target message | MM_<SourceMessage>_to_<TargetMessage> | MM_MATMAS_to_ProductCategoryReplication |
| Sender | Sender Endpoint | S_<Sender Partner> | S_C4HANA |
| Receiver | Receiver Endpoint | R_<Receiver Partner> | SIEBEL |
| Value Mapping | Value Mapping is used to map source system values to target system values. | VM_<Source Agency>_to_<Target Agency>_<FieldName> | VM_C4HANA_to_ HYBRISMARKETING_SALUTATION |
| Communication Channel | Communication Channel is used to convert Source message protocols into target message protocols | < Adapter Type>_<Sender/Receiver>_<BusinessSystem>_<Interface Short Description> | ODATA_Marketing Cloud_C4HANA_BusinessPartnerReplication |
| Local Process Flow | Local Process flows are used to modularise the complex integration flow | <Description of the objective that local process flow is designed to achieve> | Successfactors_OData_Paging |
3.3 IFLOW Description Guidelines
3.3.1 Package Short Description Guidelines
This is the text that is displayed on your package tile in the hub and is very important because it is the first description that support team may use to identify and discover existing interfaces to promote reusability.
3.3.2 Package Overview Description Guidelines
This is a longer description, which can be viewed on the Overview tab of the package. You need to provide high level overview detail about the package and its functionality to make it friendlier for support teams.
3.3.3 General Step Description Guidelines
The step names inside the Integration flow and descriptions in the Integration Flow should be meaningful in the given context. It should follow the below guidelines in addition to the English grammar rules:
- Start description sentences with a capital, end with a period.
- Avoid describing low level implementation details and dependencies unless they are important for usage.
- Avoid repetitions and misplacements of information: for example, don't write about parameters in an operation description
- Describe the objective of a step or the task that is executed by a step in the integration flow in plain english.
3.4 IFLOW Look and Feel Guidelines
The following guidelines should be used to design integration flow layout for simplifying maintenance.
- Try to avoid overlapping sequence flows
- Avoid kinks (or confusing twist/turns) in the sequence and message flow connectors – try to keep them as straight as possible.
- Avoid overlapping process steps – in case you need many process steps, try expanding the canvas and arrange the process steps neatly.
- Modularize wherever possible – in case of complex logics, try to break it down into small, easy to understand modules. Move the logic of each module into a sub-process. Name the sub-process appropriately to describe the module’s operation.
- Do not mix multiple transformations in a single script or sub-process – one sub-process should only contain the logic for one function.
- Do not assign the whole XML message to a header or a property unless necessary. Clear it once done.
- Always keep the flow direction from left to right. The sender always comes on the left and the receiver on right.
- It is recommended to limit the total number of steps in integration flow to 10 and use the steps local integration process to modularize complex integration flows for reducing TCO and ease of maintenance.
- Limit Step types to 10 for better readability
Useful Links:
https://api.sap.com/package/DesignGuidelinesKeepReadabilityinMind?section=Artifacts
3.5 IFLOW Modularisation
Complex IFLOWS can be modularized using the following mechanisms in SAP CPI:
- SAP CPI Process Direct Adapter - Modularize multiple IFLOWS
- SAP CPI Local Process Step - Modularize single IFLOW
Useful Links:
https://blogs.sap.com/2018/02/14/processdirect-adapter/
3.6 Concurrent Development
The following practice should be followed when multiple developers across different SAP Cloud Integration development projects to ensure that other developers don’t modify packages and artefacts of other developers.
- For Version, enter “WIP” to indicate that the package is work in progress. Enter your name/team name in the Owned By text box to indicate who should the other team/developer contact if they wish to change this package’s artefact.
- It will help developers to coordinate and contact on how to edit the artefacts in the package. Once the edit is done, then save the package as a version. This will change the version of the package from WIP to the next version number.
4 Performance Guidelines for Large Volumes
The Bulk Extracts should be avoided wherever possible. Process design must leverage change pointers and deltas rather than repeated bulk transfer. We must not be transmitting data that is not needed. The transmission of large volumes of data will have a significant performance impact on Client and External Partner computing systems, networks and thus the end users.
Hence interface design has to optimize the data transfer , we should also look at alternative tools like sap data services or cpi data services or smart data integration if you have to extract data from multiple source systems and transform and load data into target systems.
4.1 File Adapters Performance Guidelines
In case of huge files, below strategy should be used
- Check with participating partner to accommodate splitting message into smaller chunks before hitting CPI or use the approaches listed in section 4.3 and 4.4 for enabling multiple node and parallel processing, or the use of delta messages.
- We need to ensure that the locking mechanisms are built-in the target applications when we are processing large volumes of data.
- You can ask SAP to increase the nodes and memory to pick and process files in parallel by raising SAP incident on SAP CPI system.
Useful Links:
CPI: Tips/tricks for memory consumption on CPI tenants
4.1.1 Processing High Volume Concurrent Files
You should change Max. Messages per Poll from value 20 to 1 or 2 when you want the files to be fetched by several nodes.
4.2 REST API(S) Performance Guidelines
API(S) generally are designed for low volumes. We should use other adapters like JMS, Files, JDBC etc for large volumes or CPI Data Services/Smart Data Integration if we have to extract, transform and distribute large amount of data between many systems.
The following are the performance guidelines to optimize IFLOW when you are integrating systems using API endpoints.
Session Reuse
Creating login session is a resource demanding task. Instead of creating a session for each HTTP transaction or each page of paginated data, reuse login sessions.
In reality, a user session is created for each API call. Creating user session is a resource-intensive process. Large number of API calls will increase the stress on the server and drastically slow down response time. To address this, you can enable session on the integration flow to reuse the session. It is described in the below blog.
Useful Links:
https://blogs.sap.com/2017/07/17/cloud-integration-how-to-configure-session-handling-in-integration-...
Pagination:
Pagination is used when we have to fetch large volumes of data from the backend systems i,e it is a feature implemented in SAP Cloud Platform Integration to be able to process large requests in pages. We need to optimize the size of pagination i.e number of records to be fetched in a single call based on API but in general it is 1000 to 5000 records per call.
Useful Links:
https://blogs.sap.com/2017/04/14/cloud-integration-processing-successfactor-records-in-batches/
https://blogs.sap.com/2019/01/16/sap-cloud-platform-integration-enhanced-pagination-in-successfactor...
How to enable Server Snapshot Based in SuccessFactors OData API using SAP Cloud Platform Integration...
How to avoid missing/duplicated data enabling server based pagination in Boomi, CPI / HCI and Integr...
https://blogs.sap.com/2020/07/27/handle-dynamic-paging-for-odata-services-using-looping-process-call...
API Timeout:
Tune your client wait time to match the system wait time, your client should be set to wait a reasonable amount of time before timing out. Complex operations can take as long as 10 minutes and our network and servers will continue to process a transaction for that long.
Useful Links:
https://apps.support.sap.com/sap/support/knowledge/preview/en/2499167
API Retry Mechanism:
Implement your retry logic properly, Retry logic can help to recover transactions that failed due to internet connectivity or backend server issues, but retry must be done with care based on the type of HTTP error. Only return a maximum number 5 of times before abandoning your task.
OData API Performance Optimization Recommendations:
- Tune your batch requests into proper sizes, The OData API can return a maximum number of 1000 records in a single page. You should tune your batch sizes to be as large as possible. For more complex transactions, you may need to decrease the size to avoid HTTP timeouts.
- Use batch or $filter to get multiple records instead of pulling many records one at a time
- Avoid large $expand statements, the $expand statement can be used to request master-detail data. However, using $expand to join a large number of tables can lead to poor performance. SAP recommends that you first fetch master by batch and get detail via content enricher or expand.
- Do not query properties and expand entities you do not need or use.
Useful Links:
https://blogs.sap.com/2018/08/13/sap-cloud-platform-integration-odata-v2-function-import/
https://blogs.sap.com/2018/04/10/sap-cloud-platform-integration-odata-v2-query-wizard
4.3 Parallel Processing - Splitter
We should look at options of parallelizing the process within a HCI tenant for optimizing interface performance. At the step “Splitter”, you can activate parallel processing. At this step, the content is split in packages with 1000 entries per package. This process can be parallelized by activating the checkbox Parallel Processing. The splitter step also has concurrency which can limit the number of concurrent parallel processes that CPI can trigger in the SAP destination systems.
4.4 Performance Optimization via Several Nodes
We should request SAP to provide additional nodes via support ticket process well in advance to ensure that interface non-functional requirements of performance critical interfaces are met. Please provide the Interface non-functional requirements in the ticket for SAP to allocate the resources appropriately.
4.5 Performance Guidelines for File to OData Loads
POST: It is recommended to split the file at 100k records each and split 100k file recording into 2000 to 5002 packets with parallel processing and streaming enabled in the splitter step before calling OData Endpoint to optimise performance.
| File Size | Object Name | Splitter Grouping | Performance Throughput |
| 100K | Customer | 1000 | 500k records per hour(Post) |
| 100k | Interactions | 5002 | 1 million records per hour(Post) |
Get:
It is recommended to read data in 5k packets and write into file without append mode i.e split page data in multiple files.
OData Page Limit is 5000 which means you will get bad request if the page size is more than 4999.
| OData Page Size | Customer Read Performance | File Configuration |
| 1000 | 1 Million Per hour | Append where all records are appended to same file |
| 4900 | 3 Million Per hour | 5002 records per file |
5 Externalize Configuration Parameters
Environment specific parameters for communication channel should always be externalised instead of using static URL(S)/Authentication details for easier and central management and maintenance. Externalizing parameters is useful when the integration content should be used across multiple landscapes, where the endpoints of the integration flow can vary in each landscape. This method helps you to declare a parameter as a variable entity to allow you to customize the parameters on the sender and receiver side with a single change in landscape. Partial Parameterization enables to change part of a field rather than the entire field. This variable entity of the field is entered within curly braces.
Useful Links:
https://blogs.sap.com/2018/08/01/sap-cpi-externalizing-a-parameter-in-content-modifier-from-web-gui/
6 Modularising IFLOWS
Modularising IFLOWS
We can follow same guidelines for modularizing Complex IFLOWS :
- Create separate IFLOWS for Sender Business Logic, Call Mapping IFLOW via Process Direct
- Create separate IFLOW for processing and mapping Logic, Call Receiver IFLOW via Process Direct
- Create separate IFLOW for Receiver Business Logic, Call Receiver System via actual receiver adapter
Modularising Scripts
It is recommended to use Groovy scripts rather than java scripts unless there is a good reason on why we can’t use groovy script to fulfil a functionality.
The scripts should be created centrally in a central reusable shared library repository for easier maintenance and handler methods for each integration flow to invoke the script inside the integration flow to minimize the maintenance costs. Scripts should be commented for each logical processing block.
While it is very easy to create Groovy scripts in CPI, when the integration flow becomes more complex, inevitably there may be occasions where the same logic is repeated in different scripts. This presents an opportunity to modularise and simplify the development. One approach to perform such modularisation is by using Plain Old Groovy Objects (POGO), which are the Groovy equivalent of POJO. In short, they are just Groovy classes as opposed to the Groovy scripts that are typically generated by the CPI development tools.
Useful Links:
https://blogs.sap.com/2018/03/15/modularising-cpi-groovy-scripts-using-pogo/
Reusable Scripts across Packages
Code reusability is one of the strongest feature of any tool. Earlier SAP Cloud Integration was lacking a prime feature of script reusability like we used to reuse the objects of different software component versions in SAP PI/PO. But now this issue has been addressed in SAP CPI and script reusability feature is available in Integration Suite to avoid redundant script writing. We can create multiple scripts under one artefact ( script Collection) and that can be called in multiple packages/integration flows.
Useful Links:
https://blogs.sap.com/2021/06/07/using-script-collection-across-various-packages-in-sap-cpi/
7 Restarting Messages
In Cloud Platform Integration, when message processing fails, there is no means to retry the message processing automatically by the system out-of-the-box for most of the adapters. It must be resent from the source system. SAP Cloud Platform Integration does not support Quality of Service (QoS) Exactly Once (EO) as a standard feature, however it is on the roadmap. The following sections provide alternative approaches.
7.1 AS2, JMS, SF Adapters
For the AS2, JMS sender channel, we have Retry Handling, and the following parameters can be set in the channel configuration:
For the SuccessFactors adapter, it has a parameter called “Retry on Failure" which enables the adapter to retry connecting to SuccessFactors system in case of any network issues. The adapter tries to re-establish connection every 3 minutes, for a maximum of 5 times by default.
7.2 Other Adapters
We can overcome the standard limitation by designing the integration process to retry only failed messages using CPI JMS Adapter or Data Store and deliver them only to the desired receivers.
JMS Approach
In many cases integration scenarios have to be decoupled asynchronously between sender and receiver message processing to ensure that a retry is done from the integration system rather than the sender system. This can be achieved, for example, by using JMS queues to temporarily store the messages in the cloud integration system if the receiver system cannot be reached, and retry them from there.
Useful Links:
https://api.sap.com/package/DesignGuidelinesRelaxDependenciestoExternalComponents?section=Artifacts
Data Store Approach
The messages are persisted in data store for many days (as configured in the process step – default being 90 days); or a variable which stays in the database for 400 days after the last access. In this approach, messages are persisted for pre-defined period in CPI Data Store and automatically restarted after failures. You can also configure number of retries in a global variable.
Useful Links:
https://blogs.sap.com/2016/05/11/exactly-once-in-sap-hana-cloud-integration/
https://blogs.sap.com/2018/01/16/sap-cpi-exactly-once-with-sequencing/
8 Optimizing IFlow Memory Footprint
- When your flow has multiple branches, through a multicast or router, ensure that before ending the branch, you empty or reset all the data that is not required beyond that step. Unless explicitly reset, the data will be kept in memory until the process ends. The property and headers automatically reset when the context switches to the next branch; however, the body and variables continue to hold the data. Release all unwanted data before exiting the branch.
- Empty the header and property maps after you are done with retrieving all the required information in the script. To access a property or header in a script, you retrieve the entire list into a variable and then get the required property/header value. There are cases when there are many properties/headers or there is large amount of data assigned to any property/header.
- Avoid multicasting wherever possible – it multiplies the data and stores that in the memory.
- Use of Byte instead of String reduces the memory consumption, unless you want to process the message as a string in the next step.
Make sure that the step after the mapping, especially a script can take in a byte data. - When using XPATHs, try to use absolute path as much as possible; relative XPATH expressions are very expensive.
- SAX/STAX parsers are very helpful when working with huge datasets as they stream the XML and do not load the entire XML in memory. Moving the XML back and forth may be expensive with these parsers.
- Keep the tracing turned off unless it is required for troubleshooting. Writing trace adds a lot of overhead on performance as every stage of message processing is persisted along with the message at every step.
9 Guidelines for Transaction Handling
There are three important guidelines to follow:
1. Configure the transaction as short as possible!
Transactions always need resources on the used persistency, because the transaction needs to be kept open during the whole processing it is configured for. When configured in main process, the transaction will already be opened at the begin of the overall process, and is kept open until the whole processing ends. In case of complex scenarios and/or large messages, this may cause transaction log issues on the database or exceeds the number of available connections.
To avoid this, configure the transactions a short as possible!
2. Configure the transaction as long as needed for a consistent runtime execution!
As already explained, for end-to-end transactional behavior you need to make sure all steps belonging together are executed in one transaction, so that data is either persisted or completely rolled back in all transactional resources.
3. Configure only one transaction if multiple JMS components are used!
As already explained, If a JMS, XI or AS2 Sender Channel and one or more JMS Receiver Adapters are used in one integration flow you can optimizes the numbers of used transactions in the JMS instance using a JMS transaction handler because then only one transaction is opened for the whole processing.
4. Avoid mixing JDBC and JMS transactions!
Cloud integration does not provide distributed transactions, so it is not possible to execute JMS and JDBC transactions together in one transaction. In error cases the JDBC transaction may already be committed and if the JMS transaction cannot be committed afterwards, the message will still stay in the inbound queue or will not be committed into the outbound queue. In such cases the message is normally retried from inbound queue, sender system or sender adapter and could cause duplicate messages.
Either the backend can handle duplicates or you must not mix JMS and JDBC resources.
Check out the sample configurations below for more information.
Useful Links:
Cloud Integration – How to configure Transaction Handling in Integration Flow
10 Centralized Reusable Value Mapping API
Value mapping needs to be maintained and created in a central repository i.e in one value mapping package for promoting reusability and should be accessed using a script for the source values in the integration flow to map it to the target value. Value maps can be accessed programmatically from a script with the help of the getMappedValue api of the ValueMappingApi class.
Useful Links:
Autogenerate Value Mapping XML
11 Operational Guidelines
11.1 Tenant Landscape
SAP provides 2 tenants by default for the test and production landscape.It will work for small to medium size clients who doesn’t do heavy customization of SAP Cloud product and who have a small set of interfaces.
I recommend a 3 tier (Development where you test bespoke development, Test & Production Client) architecture for large clients who has more than 40 complex interfaces that integrate into more than 10 systems where the SAP Cloud Business Suites is implemented with high degree of customization.
11.2 Transport Mechanism & Naming Conventions
SAP provides below mechanisms to transport CPI objects:
- Manual Export and Import
- CTS+ (Content Transport Service)
- MTAR Download
- Transport Management Service
It is recommended to use CTS+(If customer has CTS+ System)/TMS transports for transporting objects from one environment into another environment. CTS+/TMS Transport should contain package name and version number and change description for each transport for customers with complex integration landscape and who has solution manager in the to-be landscape. Customers can evaluate SCP TMS and FIGAF for small to medium complexity integration landscape or if customers don't have solution manager on roadmap.
CPI Transport Naming Conventions : <CR Number> <CR Description><Package Name/Artefact Name> <Version Number>
Useful Links:
https://apps.support.sap.com/sap/support/knowledge/en/2651907
https://blogs.sap.com/2018/04/10/content-transport-using-cts-cloud-integration-part-1/
https://blogs.sap.com/2018/04/10/content-transport-using-cts-cloud-integration-part-2/
https://blogs.sap.com/2020/09/21/content-transport-using-sap-cloud-platform-transport-management-ser...
11.3 User Roles
The users are given access to SAP Cloud Platform Integration only after obtaining S user from Client Basis Team. It is recommended to either assign a custom or standard authorisation groups of roles (also referred to as authorization groups) to the users.
There is a set of predefined authorization groups (beginning with AuthGroup) that cover the different tasks associated with an integration project. For service users, you need to assign to the associated technical user the specific role ESBmessaging.send. In that case, as Application choose the one which ends with iflmap (corresponding to a runtime node of the cluster which processes the message).
Useful Links:
Standard Integration Role Persona Templates
11.4 Security
Interface should follow the security standards specified by the organization. SAP API management or API management should be used when integrating user facing web or mobile applications with the on premise or cloud systems and sharing data to multiple systems or users via API(S) unless there is a good reason on why we can’t use API management for the project and it should be agreed prior in solution architecture or discovery project phase.
11.4.1 Transport Level Security
Security artefacts like user credentials, SSH known host (for SFTP connections) can be deployed in CPI dashboard. Only S user with admin access can deploy the artefacts. SAP CPI supports both basic (user/password) and certificate,OAuth/Public Key based authentication. To enable maximum security, it is advised to use certificate/OAuth based authentication in productive environment.
Useful Links:
https://api.sap.com/package/DesignGuidelinesApplySecurity?section=Artifacts
https://blogs.sap.com/2018/03/12/part-1-secure-connectivity-oauth-to-sap-cloud-platform-integration/
https://blogs.sap.com/2018/03/12/part-2-secure-connectivity-oauth-to-sap-cloud-platform-integration/
https://blogs.sap.com/2018/09/06/hci-client-certificate-authorization/
11.4.2 Message Level Security
SAP CPI provides out of the box options for message level security and it has to be used if the client requires to encrypt or sign the data payload. Please check message security table for further details on pro's and con's of message security policies.
Useful Links:
https://api.sap.com/package/DesignGuidelinesApplySecurity?section=Artifacts
https://blogs.sap.com/2018/10/11/hci-encrypt-with-pgp/
https://blogs.sap.com/2015/12/16/hci-using-pgp-message-level-security-in-hci/
11.4.3 Security Algorithms
Some algorithms (like MD2, MD5, DES or RC4) are still supported for legacy reasons, but they are not considered secure any more. We recommend that you check the official recommendations from National Institute of Standards and Technology (NIST) or European Union Agency for Network and Infor...
Useful Links:
https://blogs.sap.com/2015/12/22/sap-hci-security-artifact-checklist/
11.4.4 Securing Design Artefacts
With the 5.22.x/6.14.x release, SAP Cloud Integration provides Access policies in the designer for integration flow to apply more granular access control in addition to the existing role-based access control.
In SAP Cloud Integration, user permissions are granted in a way that all tasks can be performed on all artefacts and data. Access policies provide a way to restrict access to selected artifacts and their data.
Useful Links:
12 Monitoring & Tracing the Messages
SAP CPI supports out of box monitoring capabilities and provides comprehensive auditing of the processed message at each level of its life cycle for support teams to resolve issues quickly.
It is recommended to log the payload tracing only in test systems and payload tracing should be activated in production system based on logging configuration of the IFLOW for optimizing system performance unless it is required form audit perspective.
Useful Links:
https://blogs.sap.com/2018/03/13/troubleshooting-message-processing-in-the-cpi-web-application/
https://blogs.sap.com/2016/04/29/monitoring-your-integration-flows/
How to trace message contents in CPI Web Tooling
Message Processing Log: Adapter Tracing
13 Error Handling
There are integration scenarios where the number of error cases exceed the success cases. It isn’t required to generate alerts for all error cases require alerts but we generally do require that the right level of error information is captured and logged to aid in troubleshooting.
13.1 Types of Errors
We should design integrations to handle errors gracefully and provide mechanisms to handle below errors for every interface:
Errors are broadly classified into two types:
- Recoverable Errors - Recoverable errors are the errors that Client programs can recover from to take appropriate alternate execution paths. Such errors are the result of failure to meet a business rule.
- Non-Recoverable errors - These are the errors that Client programs cannot recover from. This kind of errors are result of some unexpected errors during runtime such as programming errors such null pointers, resources not available etc. This should be minimized by testing NFR(s) and boundary conditions during testing phases and by identifying it from continuous service improvement analysis.
Useful Links:
https://api.sap.com/package/DesignGuidelinesHandleErrors?section=Overview
13.2 Invoke Business Friendly Exception API
SAP CPI provides exception sub flow to raise errors during iflow runtime. It is recommended that we should raise errors using the Generic Exception API Format across all CPI interfaces into SFTP server or HANA tables for helping support teams to quick assess the impact of the interface failure and resolve incident quickly.
| Field | Description | Example |
| Message ID | This field contains the ID of the message | 5576767677SDFDFDDF |
| Interface ID | This field contains the interface ID of the message which will reference to interface metadata table which has details like description, criticality, contact details etc | BS100134 |
| Exception Point | Step in the iFlow where the error occurred | Mapping, Adpater call etc |
| Exception Source | System where the error occured | CPI, C4HANA,S4HANA etc |
| Exception Category | This field will define the category of exception. Please see Exception Category section below for definitions of each category. | Business, System,Technical etc |
| Error Code | This field will contain the appropriate HTTP error codes. Please see the definitions of each code in the error code section. | 500,200 etc |
| Error Description | This field will contain the detailed description of the error message. | Exception.Stacktrace i.e Message Log |
| Date/Time Stamp | Date and Time of the Error | YYYYMMDDThh:mm:ss |
13.3 Customizing Message Log with Business Friendly Info
Integration flows should record business friendly information into standard log entries by using script to provide more contextual information to assess business impact.
The following script is an example on how you can write integration flow specific keys in the message log: Ex: Purchase Order Number, Customer Number.
Custom headers are added to the log using the function addCustomHeaderProperty on the messageLog object:
Signature: void addCustomHeaderProperty(String Name, String Value);
Sample: messageLog.addCustomHeaderProperty("MyCustomeHeader","MyValue");
Useful Links:
https://blogs.sap.com/2015/01/12/blog-4-modelling-exceptions-in-integration-flows-hci-pi/
https://blogs.sap.com/2018/06/08/how-to-tackle-disguised-errors-in-your-integration-flows/
13.4 Archiving Error Files in CPI SFTP
SAP CPI doesn't provide out of the box capability to move the error files automatically into an exception folder which will cause issues as the next polling interval will pick the error file and process it again indefinitely which is not ideal for every business scenario.
The following process should be followed to avoid automatic processing of failed files.
Useful Links:
14 Alerts
Alerts should be generated only for business critical interfaces.
For all the custom iflows, it is recommended to capture runtime errors and consume in exception sub-flow. Exception sub-flow will then either send email alerts or log them in CPI monitoring tool or central monitoring tool using an CPI OData Monitoring API based on criticality and severity of the error.
1.Ensure alerts are tied to a schedule
2.Ensure alerts are actionable by providing insightful information
3.Set Priorities! Classify Your Alerts Based on Importance
4.Route alerts to right teams
5.Create a Wiki Process for How Alerts are Resolved
Useful Links:
https://blogs.sap.com/2017/06/15/qforit-error-alerting-for-sap-cloud-platform-integration/
https://apps.support.sap.com/sap/support/knowledge/preview/en/2472429
15 External Monitoring Tool Integration
Cloud Platform Integration Service also provides out of the box monitoring API CPI Monitoring OData API and SAP Open Connectors to integrate with central monitoring tool chosen by Client Ex: Solution Manager, Service Now, Jira, Slack, Elastic Stack, Dyntrace.
Useful Links:
- CPI OData Monitoring API
- Open Connector Service Now
- Open Connector Jira
- Realcore CPI Dashboard
- Dyntrace Integration
- Elastic Stack Integration
RealCore CPI Dashboard
The RealCore CPI Dashboard is a lightweight free IFlow-based tool that you can install which allows you to monitor your CPI instance (including system parameters like CPU-, RAM- and disk usage), view passwords and log files as also setup an mail-based alerting.
16 Enhancing SAP Cloud Standard Content
Before starting any development, you must search the integration content catalogue on SAP API Business Hub to check if there is any pre-packaged content or the content you can reuse and enhance to fulfil the customer integration requirement.
- If standard content exists then it should be copied to the Client work space.
- It is also advised to use “configure-only” approach for standard content but edited only in unavoidable circumstances as there are no auto-updates for modified content.
- If you wish to add custom code to the pre-delivered standard content without falling out of the content update contract – request the content developer or SAP to include custom flows/exits in the integration flows.
- If the content developer or SAP do not agree to change the content, copy the content package twice to your design view. Ensure that you keep one copy untouched.
- Add all your customized changes to the other copy. Whenever a standard update is released by the content developer, update the untouched copy with the latest changes. Then manually merge the custom update to the updated content.
- Every change from the content developer is backed by a release note – this gives an idea about what has changed in the content with each release. The customer can then decide if he wants to merge the custom changes manually or standard changes based on whichever is less.
Useful Links:
CPI Standard IFLOW Extension Techniques
CPI Standard IFLOW Extension Tutorial - Part 1
CPI Standard IFLOW Extension Tutorial - Part 2
CPI Standard IFLOW Extension Tutorial - Part 3
Pre-Exit Post-Exit Standard IFLOW Example
16.1 SAP Cloud API Extension Best Practices
SAP provides two mechanisms i.e side by side or in-app to extend SAP Cloud Business Suites like C4HANA/S4HANA/Successfactors. You can use below table to carefully evaluate which is right mechanism for fulfilling your integration requirement.
| Business Scenario | Solution Guideline |
| Extend API Fields to match custom legacy systems for the business process to work | Create custom CDS view with additional fields and generate OData API if the fields can't be derived from other CDS views or OData API(S). |
| Extend S/4 or C/4 UI with read only Custom Mashup Screen | Orchestrate API(S) using SCP Integration and Open Connectors to read data and call it from Launchpad if we need mashup screens to just display information from other systems. |
| Extend S/4 or C/4 UI with write/read complex Custom Mashup Screen | If data doesn't needs to be stored in S/4 or C/4 for operational purposes then follow above solution. If data needs to be stored in S/4 or C/4 for operational purposes then create a custom BO, CDS view and enable OData API(S). |
| Orchestrate and Join results from different SAP S/4 or C/4 API(S). Ex: Join Shipment and PO/Delivery API(S) together | In this case, you can search if there are existing CDS views on SAP API Business Hub and create custom CDS views on top of existing CDS views and expose OData API that CPI can consume. |
Useful Links
17 Versioning Integration Flow
The guiding principle to ensure that we release features without breaking existing running scenarios in client systems or in other words we need to make every effort to ensure that the features are always backward compatible especially when we use CPI in multi-country roll out projects. Based on this, new features to components (like flow steps, adapters or pools) are always released through a new version.
The CPI IFLOW will follow the following version management strategy. Each component in SAP Cloud Platform Integration has a version and this version is defined using the paradigm <major>.<minor>.<micro> as depicted below:
FIGAF tool by daniel.graversen can be used along for CPI version management.
Useful Links:
https://blogs.sap.com/2016/03/24/standard-hci-version-management-effective-usage-and-review/
18 Deprecating Integration Flow
You can deprecate the package or artefact using below mechanisms:
- Delete the package or artefact if no system is using and update the "Change Log of the Package"
- Add [Deprecated] as prefix in the short description and in the long description add the link to next version and explain the reason.Additionally,update the "Change Log of the Package"
19 CPI Automated Testing Tools
The biggest challenge in any integration project is not building but test preparation and execution. If client want to optimize the costs of integration projects then we can suggest them to evaluate the below tools to speed up and cut down the costs of integration projects in future.
I am not elaborating details of the tools as it is already done by r_herrmann in his excellent round up blog.
20 CPI QA Tool
CPILint is a linter for SAP Cloud Platform Integration. More specifically, it’s a tool that automatically checks your integration flows against a number of different rules. The rules cover best practices, connectivity, security and more. From the built-in rules, you choose the ones you want to enforce, and CPILint does the work of checking for integration flows that don’t follow those rules.
You might think of the CPILint rules as executable development guidelines. A development guidelines document might state that at company X, we only transform messages with message mapping. That’s all well and good, but up until now, there’s been no way of knowing whether that guideline is actually being followed, short of manually checking every integration flow. With CPILint, however, you can set up a rule specifying allowed mapping types, and completely automate the process.
21 Reading E-mails from multiple inboxes
As the CPI sender mail adapter can only be used as first step in the CPI IFLOW, the following template is the best practice to read multiple e-mail inboxes.
- SAP Managed Tags:
37 Comments
