Loading Data from SAP DI to HANA Data Lake Files

[aghoshal]

Hello Community,

We are trying to implement Data Tiering by loading data from S/4 HANA tables to SAP HANA Data Lake. We have certain clarity on loading data from S/4 HANA to SAP DI as middle layer but now we need to establish load between SAP DI and HANA Data Lake. Can anyone suggest how to achieve the same in order to load data from DI to HANA Data lake.

[AlexGourdet]

Thank you for visiting SAP Community to get answers to your questions.

Since you're asking a question here for the first time, I'd like to recommend you with the following steps so you can get the most out of your community membership:

• Please see: https://community.sap.com/resources/questions-and-answers as this resource page provides tips for preparing questions that draw responses from our members
• Consider to take our Q&A tutorial at: https://developers.sap.com/tutorials/community-qa.html. as that will help you when submitting questions to the community.

I also recommend that you include a profile picture. By personalizing your profile, you encourage readers to respond: https://developers.sap.com/tutorials/community-profile.html.

I hope you find this advice useful, and we're happy to have you as part of SAP Community!

All the best,
-Alex

[markmumy]

Aporup,

Can you explain a bit more about what you want to do with DI and the source systems? How are you capturing changes? This is important as we released the Replication Management Service with DI Cloud recently. If you are using that to capture changes, it does not currently support HANA Cloud, data lake files (HDLFS) as a target. That is a Q2 2022 (shortly) feature that is being released.

If you are simply using DI pipelines to capture data changes, then you can certainly create a StructuredFileProducer to write data to HDLFS, given the proper configuration and connection of HDLFS in your DI landscape.

As a reference, look at this video starting about the 27 minute mark. If explains, at a high level, on the StructuredFileProducer. In this case it reads data from HANA and a file, joins it, then writes it out to a file.

https://www.youtube.com/watch?v=7V7W0Fj2BuQ

[aghoshal]

Hello Mark,

Currently we are trying to design an architecture where we need to unload historical FPSL data from FPSL tables in S/4 HANA and dump it into HANA Data lake. Furthermore we need to pull out data from HANA Data lake to HANA Data Lake Files for storing cold data. In order to pull out data from S/4 to HANA Data Lake we are left with only approach of using SAP DI due to high data volume nearly 20TB. We are planning to capture changes using CDC in CDS Views, but still unsure if this approach will capture changes also about the performance. Secondly we ar also exploring into various ways by which we can load data into HANA Data lake files from HANA Data lake. We are currently in design phase and have came up with this approcah but are stuck with the last two parts:

1) How to load data from DI to HANA Data Lake.

2) How to load and unload data out of HANA Data Lake and HANA Data Lake Files for Historical data reporting.

Appreciate your suggestion on the same.

Thank you