Logic of CDC (Change Data Capture) in ABAP CDS View

[ashish_dey84sap]

Background

There was requirement to extract closed POs as part of Daily Delta from S4 and load to BW. We are extracting Data from S4 via creating Custom Abap CDS Views. But here we had the challenge to bring only Delta records which was closed on a particular Day.

We went ahead to implement Change Data Capture (CDC) Delta: This delta method makes use of real-time database triggers on table level. The Change Data Capture recording mechanism uses database triggers to record any changes to the tables that belong to an ABAP CDS view. To do this, the key fields of all underlying tables need to be mapped to the fields of the CDS view. 

• Below is the annotation used for CDC Delta Functionality

@analytics.dataExtraction.delta.changeDataCapture.mapping

CDC Mechanism in Delta Enabled ABAP CDS View

We successfully did the technical go-live of this deliverable to Production Environment and daily deltas in terms of closed PO’s was successfully getting captured in ODQMON and was flowing to BW as part of Daily Loads.

Problem Statement

One Particular Day (For example 5th Oct 2023), we noticed that as part of Daily Delta loads for closed Po’s, zero records were fetched. The same happened in next day as well. We checked in ODQMON and saw that last TSN Requested Date was not changing, it was still showing the date when the last delta was captured successfully i.e. 4th Oct 2023.

Root Cause Analysis

After exploring, got to know that there are basically 2 Jobs in the S4 Side which drives the entire flow of delta records based on CDC mechanism.

• Observer job (/1DH/OBSERVE_LOGTAB)
• Transfer job (/1DH/PUSH_CDS_DELTA) which's started by Observer job.

In this case when we checked in SM37, Observer Job was running as per logs, But Transfer Job last run was on the day (04-OCT-2023) when Delta records were extracted successfully in BW side. After that there was no run.

Observer job /1DH/OBSERVE_LOGTAB is scheduled hourly.

Transfer Job /1DH/PUSH_CDS_DELTA in SM37 in S4. No Run after 4th Oct 2023

If a record is changed, then database triggers write it into a master logging table /1DH/ML... (e.g./1DH/ML000000001).  Afterwards, the Transfer job copies the records from master logging table to subscriber logging table /1DH/SL... (e.g./1DH/SL000000001).

After the Observer job copies the logging table entries, the view reconstruction is executed. The view reconstruction selects the latest after images from the CDS view if there exists a corresponding key in at least one subscriber logging table. The result is written into unit of Operational Delta Queue (ODQ).

In our case, we could see that database triggers have written data correctly into master logging table (/1DH/ML000000001) but it's not being copied from master logging table to subscriber logging table /1DH/SL000000001 & /1DH/SL000000002 because Transfer job is not running.

Master Logging Table – Having Entries

Subscriber logging Table - No Entries

Resolution

GOTO Transaction code DHCDCMON to check the status of Observer & Transfer Job. It must be green. Either Red or yellow, probably no delta can be pushed into ODQMON unit.

To correct the status, click button “Dispatcher job” (starts SAP_DHCDC_DISPATCHER job) which will schedule all the required and missing jobs.

/1DH/OBSERVE_LOGTAB (Observer Job)

/1DH/PUSH_CDS_DELTA (Transfer Job)

/1DH/CDC_HEALTH_CHECK (Health Check Job)

For more information, please refer SAP Note – 2930269: ABAP CDS CDC: Issues & Troubleshooting