Open in all directions: SAP CDP and the integratio...

peter_gergen · ‎05-25-2021

By Peter Gergen, Solution Architect CX

For a CDP to deliver its full value, it is essential that as many customer data containing or generating platforms as possible are connected. Only then can a comprehensive profile of the multiple facets of customer profiles - from identity information to activity and event data - be captured and processed in the CDP. However, the integration aspect poses a great challenge to companies for several reasons:

Heterogeneity: too many systems from too many different vendors with too many different integration technologies, making each integration a separate project.

Closed systems: systems with no interfaces, incapable of integrating with a centralized customer data platform, and only using export-import.

Proprietary interfaces: There may be access points to the data silos, but they are not standardized.

This means integration projects can go from a seemingly simple project to a lengthy and costly challenge.

To simplify these challenges, SAP CDP facilitates integration by offering a wealth of out-of-the-box integrations for many SAP and non-SAP systems, offering a vast range of integration options for both SAP and non-SAP systems. But what does that really mean, and what technologies can be used to integrate IT systems with customer data into SAP’s CDP? That is the question I will explore in this blog post.

Put simply, the range of integration possibilities extends from pre-built connectors to API-driven controls. These two antipodes can also be used to approximate the integration effort: Connectors are usually configurable with relatively little effort, while API-driven integrations involve extensive implementation efforts. Let’s shed a bit more light on what this means by taking a closer look at the spectrum of SAP CDP integration options.

How do the data get into the CDP? And how do they get out again?

The SAP CDP concept for the integration of source and target systems is the "Connector Library", a library of application connectors within the CDP with various sources and targets. Here, all connectors that enable the connection to target systems are provided via a guided integration wizard. Put simply, the Connector Library is the first point of contact for checking whether a source or target system can be configured and integrated.

The “Connector Studio”, another part of the SAP CDP, allows you to build further connectors, which then become part of the connector library for your tenant. This is a template driven, wizard-based User Interface. It allows you to configure a REST Template that will represent how to operate with a particular REST-based provider.

Both the Connector Library as well as the Connector Studio are part of SAP CDP. If the required connector is available in the CDP's Connector Library, great! If it is not, a connector can be created in the SAP CDP Connector Studio under wizard control.

The communication to and from the SAP CDP can be configured inbound or outbound:

Events specify inbound data streams into the CDP. For example, if a new user is created in the CIAM system, the CDP Connector registers this as an "event". This information is then evaluated in the CDP and, if threshold values are exceeded, transfer the customer to a specific segment. This then leads to…

Actions. These are data streams from the CDP. Before an action, the CDP calculates the resulting changes in customer data after an event has been received. Does an Activity Indicator change or is the customer reassigned to a segment? Does the preceding event trigger a customer-specific customer journey? All these calculations are performed in the CDP and the resulting data change is transmitted to the target systems as an “action”.

A connector is therefore a connection element that waits for incoming data streams (events) or sends information about customers out of the CDP to the relevant target systems (actions).

Integrating customer data from all connected systems is crucial for the success of a customer experience concept. By seamlessly combining information from various sources, organizations can gain a holistic view of their customers and deliver personalized and relevant experiences.

Real-time or not?

For many customers, "real time" is an important factor in CDP data processing. But what does that mean? Well, within the CDP integrated architecture an IT system can process data in near real time, unless the incoming activities of customers are passed on directly by the source systems. Example: A customer orders goods from an online store. The commerce system in question executes the order process but does not pass this information directly to the CDP. In this case, the only way for the CDP to process the order is to query the commerce system for new orders at regular intervals. Put simply, the information about the orders is not "pushed" by the commerce system but is "pulled" by the CDP. This shifts the real-time effect to the process interval of the CDP queries.

Real-time is therefore only possible if the data is delivered to the CDP in real time, meaning that a source system sends customer activity when it happens. The CDP can then process this information. But for that to happen, the source system must deliver the customer data to the SAP CDP API, which then takes the data and processes it further. Alternatively, it is possible for the source system to send a webhook to the CDP notifying it that some piece of customer information has changed. The CDP can then query this data from the source system in near real time.

Confused? I hope not! We’ve got a lot more to cover.

Conversely, the SAP CDP can pass the data on to the consuming system in real time. However, if this data is not directly processed there (especially if the consuming system is only provided with the data via an upload because it otherwise has no interface available), the real-time aspect is also invalid. In other words, CDPs can process the data fast, up to real time, but the nature of the source and target systems determines how fast they deliver or consume the data.

SAP CDP is designed for real-time. So long as the source and target systems provide the necessary connectivity, activity analysis to segmentation of customer data is processed in the range of milliseconds to seconds (depending on latency in networks and at interfaces). The SAP CDP REST API collection is available – beyond the Connector Studio – and often used for many of our customers’ Real Time use cases.

Now let’s take a closer look at the connectors and the options you have.

Connector Library - The preconfigured connectors

The preconfigured connectors are listed in the SAP CDP Application Library. They are preconfigured in the sense that they query the necessary configuration options in a wizard driven manner and already "know" the basic schema of the data that is exchanged from the system to be integrated. Let’s look at an example: the integration of an Emarsys system. The configuration provides that an instance of the Emarsys connector is created (Name, Authentication Type, User Name and Password as well as the URL for access), then the actions of the connector are edited (Process Purposes, Trigger and Data Model). Behind the scenes, however, there is a lot going on: the SAP CDP handles authentication, Automation Processes are started, http calls are exchanged, and the extensive technological integration is pre-installed and packaged into the form we call a connector, which is then listed in the Application Library. If you are interested in the current list of connectors provided by SAP CDP, you can find it here.

Of course, it is possible to modify these connectors from the schema side, as it is to be expected that the instances of IT systems will not always send the standard data, but may also provide data that deviates from the standard schema.

The SAP CDP is designed as a “No code” interface for configuring an end-to-end integration with a particular provider and facilitating the process of mapping data by a non-tech user.

Connector Studio - Build your own connector

If no suitable connector is currently available in the Application Library, one can be configured in the Connector Studio. This process is also wizard driven and queries the essential parameters for the configuration. The prerequisite is that the system to be integrated provide an open and accessible REST API that can be reached via a URL.

If this is the case, all parameters for addressing this REST API can be captured, including, among others:

The purpose of the data collection: the SAP CDP marks the data collection from the relevant target systems with the appropriate purpose attribute. This ensures that the data from the relevant target system is subject to purpose binding. This is an important requirement from various data protection directives such as the GDPR.

The authentication method against the API (you can use a basic UserID/Password combination, or Oauth2 or even a totally custom authentication method by passing a free configurable list of parameters to the relevant authentication method).

The BaseURL, which is the protocol, domain, and common path for APIs to connect to the REST interface.

The details of the endpoints such as the HTTP method (Get/Post/PUT or DELETE).

The schema of the incoming data stream.

The endpoint definition (is it an event or an action?).

As discussed above, these connectors can only "pull" the data from a source system. However, if a source system can push the data, it must address the CDP via the APIs. The inclusion of webhooks sent from a source system into an SAP CDP that transmit updates of customer information to the CDP in the form of JSON objects is currently under discussion.

In terms of configurability, the SAP CDP Connector Studio provides the full range of addressability fort the REST APIs, including integration into SAP Business Technology Platform via SAP Cloud Platform Integration (CPI).

Summary

The SAP CDP is designed with openness in mind. It is not SAP-centric, but agnostic to the heterogeneous IT infrastructures of many enterprises. This is a significant advantage over competitor solutions, some of which are difficult to integrate from their own "home turf." With its CDP, SAP thus taps the customer data of a multitude of data silos and uses it to form customer profiles, segments, and audiences across all manufacturers for all participating IT platforms. It is high-performing, highly flexible and highly scalable, all while considering all relevant data protection requirements.

Want to go to the next stage? Here we discuss the integration scenarios in more detail...