Technology Blogs by SAP
Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
cancel
Showing results for 
Search instead for 
Did you mean: 
FrankRiesner
Product and Topic Expert
Product and Topic Expert

Dear SAP BW/4HANA enthusiasts!

As opposed to SAP NetWeaver BW in the past, BW/4HANA provides a new write interface which was introduced in Gordon Witzel´s earlier blog about Data Integration with SAP BW/4HANA 2.0 .
This new interface provides the possibility to integrate data into the ADSO inbound table (SAP BW/4HANA 2.0 SP00) or InfoObject Characteristics (SAP BW/4HANA 2021 SP04) without the need of the normal standard SAP BW/4HANA objects like Source System, DataSource, Transformation and Data Transfer Process which pull the data into SAP BW. Instead data can be pushed into these BW/4HANA objects from outside by tools such as SAP Data Services, SAP NetWeaver Process Integration, SAP Data Intelligence, SAP BTP Integration Suite, or similar 3rd party solutions. For this reason, the whole scenario is sometimes referred to as Push ADSO (e.g. in SAP notes).See also more details in the SAP BW/4HANA application help.

The new write interface can also serve as interface for SAP and non-SAP cloud sources, more details in my follow-up blog post https://blogs.sap.com/2021/09/22/sap-cloud-integration-into-sap-bw-4hana/.

This SAP BW/4HANA feature provides a clear successor interface for the former source systems of SAP BW 7.5 called “Web Service”, “Data Services”, and “External System”:

SAP recommends leveraging the new write interface during conversion projects for sources which are connected to SAP BW 7.x on these obsolete source system types. There is no automated conversion path, but the manual steps are well documented in following SAP notes:

 **** UPDATE 06/2023 ****
During a customer conversion project, we ran into challenges replacing a BW 7.5 Web Service providing mass tracking data. We discussed the matter with the SAP development team and in the end, RSPM processing (i.e. the new BW/4HANA TSN-based request mgmt) can now be adjusted for this Push scenario by setting of RSADMIN parameter PUSH_ADSO_PROCESS_TRACE = N which suppresses the creation of RSPMLOG entries. So if the standard push scenario does not meet your performance expectations, consider following notes:

  • 3283799 - Push ADSO: Messaging Request enablement
  • 3290288 - Failure in function module RSDRI_INFOPROV_READ
  • 3339770 - BW4:RSPM:DATA_PUSH:Performance Issue:Frequent Checks:Traces
  • 3340847 - Push ADSO: Performance issues with internal RSPM Access (parallel loading)

**** end of update ****

 A fact not very widely known is, that the write interface is not limited to SAP solutions but can also be addressed by 3rd party tools (although stated differently in some sources in the past). In the following scenario we will describe how this can be accomplished based on the SEEBURGER Business Integration Suite (BIS):

In our recent project, the customer uses SEEBURGER BIS as its general strategic platform for integration requirements in its IT landscape. During a shell conversion project of their SAP BW 7.5 to SAP BW/4HANA 2.0, the former SEEBURGER BIS connection via the “Web Service” source system had to be redesigned in a way that it is able to address the new ADSO write interface of SAP BW/4HANA.
The following is a step-by-step guide to how this was achieved:

Step 1: In the SAP BW/4HANA Modeling Tools, create a new ADSO (here: technical name ZADSO) as “Standard DataStore Object” (or “Staging DataStore Object”). For these two types the modeling parameter “Write Interface-Enabled” is available to be switched on.
Step 2: After activation of this new ADSO, template URIs (Uniform Resource Identifiers) are generated in the Properties window.
These URIs have the following purpose:

Get Structure:

https://server:8443/sap/bw4/v1/push/dataStores/zadso

--> Get the JSON schema and get CSRF token
Get Sample Data:

https://server:8443/sap/bw4/v1/push/dataStores/zadso/sampleData{?records,seed}

--> Get sample data generated by the server
Send Data:

https://server:8443/sap/bw4/v1/push/dataStores/zadso/dataSend{?request,datapid}

--> Send data using the CSRF token
Open Request:

https://server:8443/sap/bw4/v1/push/dataStores/zadso/requests

--> Open request (TSN) and get CSRF token
Close Request:

https://server:8443/sap/bw4/v1/push/dataStores/zadso/requests/{request}/close{?error}

--> Close request (TSN)


Step 3: For sending data a CSRF token is needed, otherwise an error 403 HTTP Forbidden with message “CSRF token validation failed” will be returned. To get this token, an HTTP GET needs to be done on the Open-Request-URL with key “x-csrf-token” in the HTTP header. In the response, this key will be filled with the token. In the HTTP GET request, basic authentication needs to be set with a valid user and password.

 


 Step 4: Data can be sent to the inbound table of DataStore object ZADSO doing an HTTP POST on the Send-Data-URL with the JSON data as raw body and the CSRF token in the HTTP header. Basic authentication needs to be set as well. If the data is sent from another IP than the open request, the cookie from the Open-Request response needs to be in the HTTP Header,
e. g. Cookie =
sap-usercontext=sapclient=001;path=/;SAP_SESSIONID_BWP_001=Ui3914ArIMefp4pXhIjg9Z;path=/;

The response from SAP is empty.

 

Final Step: After the data is sent successfully, a new request TSN is listed in the SAP BW/4HANA Cockpit app "Manage DataStore". From this point onward, this request TSN and the related data behaves like any other request TSN in an ADSO inbound table. For example, you can activate it to provision it to other InfoProviders or consume it with a BW query.

If SEEBURGER BIS detects an error (e.g. wrong syntax in a DATE field), no request TSN is created until the error is corrected in the source system.
Explicit Request Handling
By following the above steps no explicit request TSN is used in SAP BW/4HANA. This means, every Data-Send creates its own implicit request TSN. If there is the need to control the request TSN from outside, it needs to be opened explicitly to be able to obtain the request TSN.

To do so, after step 3 the Open-request-URL needs to be called again, this time with HTTP POST to receive the request TSN e. g. 20200721105757000003000.


This request TSN and the corresponding data package IDs have then to be added to the Data-Send call as a parameter, e. g. …/zadso/dataSend?request=20200721105757000003000&datapid=1. This way several Data-Send calls can be done using the same request TSN.

 At the end, the request TSN needs to be closed explicitly by calling HTTP POST on the Close-request-URL with the related CSRF token and request TSN, e. g. …/zadso/requests/20200721105757000003000/close.

So in summary, the steps are:

    1. GET   …/zadso/requests [Obtain CSRF token]
    2. POST …/zadso/requests [Open request and get TSN]
    3. POST …/dataSend?request=<TSN>&datapid=1 [Send the data several times, based on  increasing data package ID (x+1)]
    4. POST …/zadso/requests/<TSN>/close [Close request with TSN]


Additional technical details

  • To retrieve the JSON schema a plain HTTP GET can be performed on the Get-Structure-URL.
  • To generate sample data the Get-Sample-Data-URL can be called:
    GET …/zadso/sampleData?records=<number of records>&seed=<seed>
  • If the sent data is not correct, it is possible to close the request and mark it as Error. Use the CLOSE-Request-URL with parameter error=true to achieve this. The data is then excluded from the ADSO activation queue.
  • The data activation in the target ADSO can be triggered automatically by means of a Process Chain in ‘Streaming’ mode, which is executed when a Request in the inbound queue is closed and if the “streaming” Process Chain contains a Process Variant ‘Activate DSO Data’ specifying the DataStore name.“ See more details in my separate blog covering BW process chains in streaming mode.

Conclusion
In SAP BW/4HANA, the write interface-enabled DataStore object / InfoObject Characteristic provides the interface of choice for sources of data which were connected to SAP BW 7.x via source system types “Web Service”, “Data Services”, and “External System”. As these types have been decommissioned in SAP BW/4HANA, a manual conversion is required as documented in the two SAP notes mentioned at the beginning.

The example described in this blog showed, that also 3rd party tools like SEEBURGER BIS can be leveraged to provision data to SAP BW/4HANA this way. In our customer project this resulted in following benefits:

  • The SEEBURGER platform was already connected and integrated to several source systems, and it manages and secures access to a broad range of SAP and non-SAP data sources. No changes were required in this infrastructure.
  • This relieved SAP BW/4HANA developers from tasks that are not explicitly in their focus, such as managing and maintaining source systems of different types, managing the data quality and data transfer processing and monitoring.
  • Based on the new write interface, integration of SAP BW/4HANA into the customer environment is fast and easy. Existing connections could be reused in one-step, reducing integration time and cost.
  • Thanks to the new ADSO write interface and the capabilities of the SEEBURGER platform, the customer was able to execute and to achieve these objectives.
36 Comments
DirkO
Participant
0 Kudos

Hello Frank,

thank you for sharing. I really appreciate level of the provided details by you and the provided function as such, well done. Hope to see more such articles, soon. ?

Best regards,

Dirk

anilani999
Member

Hi Frank,

Thanks for sharing, this will definitely help me since I have a similar requirement. Well Done ?. 

Regards,

Anil.

0 Kudos
Hi Frank,

Very well explained and useful blog. Appreciated 🙂

 

Regards,

Shruti
johnny999
Participant
0 Kudos
Thanks for the blog. Is there a way to activate the request TSN automatically? Since this isn't a process chain, I tried and activate the direct update but then I lost the option of activating the "write interface".

Please advise.
FrankRiesner
Product and Topic Expert
Product and Topic Expert

Dear Jonathan, 
please see my updated answer below.
KR Frank

JoergBoeke
Explorer
0 Kudos
Hi Frank,

just a quick question cause we do use the old WebService integration in Process chain

E.G. a specific user uploads data ( for correction purpose e.g. Masterdata) and if the package/data is fine the MD activation will be triggered via upload in old fashioned PSA

 

Is there a new Process chain variant available or planned, to detect the new Write interface functionality like data is being pushed into ADSO==> trigger the further upload to target ADSO?
FrankRiesner
Product and Topic Expert
Product and Topic Expert
0 Kudos

Hi Jonathan, hi Jörg!

I checked this detail again with some SAP colleagues and good news is that there is actually a way to achieve this: The data activation in the target ADSO can be triggered automatically by means of a Process Chain in ‘Streaming’ mode, which is executed when a Request in the inbound queue is closed and if the “streaming” Process Chain contains a Process Variant ‘Activate DSO Data’ specifying the DataStore name.“ See more details in my separate blog covering BW process chains in streaming mode (part D).

 

Regards Frank

FrankRiesner
Product and Topic Expert
Product and Topic Expert
0 Kudos
Hi,
please see my answer below.
KR Frank
JaySchwendemann
Active Contributor
0 Kudos
This might be a stupid question but we are evaluating integrating IBP via CPI-DS and wondered if ADSO might be a way to read data from the BW/4 system, so BW/4 is the source.

Is that possible (from an ADSO API POV) and (by the chance that someone here knows by accident) CPI-DS support this as REST or the like datat store connection?

Many thanks and kind regards

Jens
FrankRiesner
Product and Topic Expert
Product and Topic Expert
Dear Jens,

there are two ways in general to extract BW data:

  1. Open Hub: Data is pushed from BW e.g. via process chain into a file or DB table (local or external DB)

  2. ODP: Data is pulled from external sources like BW, DataServices, HANA SDI or OData. I think the last option is the one most interesting for you if I get your question right.
    Please check further details here in the SAP help:
    https://help.sap.com/viewer/107a6e8a38b74ede94c833ca3b7b6f51/2.0.4/en-US/11853413cf124dde91925284133...


Regards,

Frank
0 Kudos
While using a ADSO with the option "write interface enabled" in the "general" preferences of the ADSO, with this option set the rest services for push into this ADSO are generated. Using this rest service we push the data from our SAP PO system into ADSO.

What I need to know is where I can find detailed error logs in case that the push from SAP PO via rest service into the ADSO is failing .  In this case there is no request created in ADSO  and in SAP PO there is only a very general error message "Send not successful. Internal server error".
masovice
Discoverer
0 Kudos
we have the same challenge

any suggestions would be highly appreciated
virenp_devi
Contributor
0 Kudos
Hello Frank,

Will SAP create same API feature in SAP 7.5 on HANA so that external apps can write into Direct update ADSO?

Thank you.
FrankRiesner
Product and Topic Expert
Product and Topic Expert
Dear Devi,

the ADSO write-interface is only available for BW/4HANA and there are no plans to downport this feature. Sorry. Most probably will have to leverage the existing APIs for classic DSOs as long as you are on BW 7.5

KR Frank
virenp_devi
Contributor
Thats is bit unfortunate as clients generally prefer to use B4HANA compatible objects . In our case we will have to use Classic DSO's to be written from middleware tool. Thank you for the information and great blog.
FrankRiesner
Product and Topic Expert
Product and Topic Expert
0 Kudos
Hi Devi,

here is the documentation of the ADSO APIs in SAP BW/4HANA.
https://help.sap.com/viewer/107a6e8a38b74ede94c833ca3b7b6f51/2.0.9/en-US/72e16c936fb94cffb71ce90edd5...

Maybe those are available in your BW 7.5 release as well? At least worth to take a little time to check before you go for classic InfoProviders.

KR Frank

 
virenp_devi
Contributor
Hello Frank,

This is true. We did find API's in 7.5 as well. Appearently work same way.   Thank you.
0 Kudos
Hi Frank,

 

I have the same requirement and followed all the steps but still receiving X-CSRF Token Validation Failed error.

Could you please help me what is missing? I have attached all the snapshots in below blog.

 

https://answers.sap.com/questions/13464600/x-csrf-token-validation-failed-when-save-data-in-b.html

 

thanks
FrankRiesner
Product and Topic Expert
Product and Topic Expert
0 Kudos
Hi Ednan,

please consider SAP note "2907345 - Push aDSO: Aborted processes changed to status Finished"

or open an incident on component BW4-DM-ADSO

Regards - Frank
FrankRiesner
Product and Topic Expert
Product and Topic Expert
0 Kudos
Hi Rajni,

there not enough details provided for me to be able to assess what is wrong.

Please check this blog, which describes CPI integration in detail:https://blogs.sap.com/2019/08/23/sap-cloud-platform-integration-cpi-to-push-data-into-sap-bw4...

or open an incident on component BW4-DM-ADSO

Regards - Frank

 
masovice
Discoverer
0 Kudos

Hi Frank,

thanks a lot for the replay but the S-Note is not applicable as we are already on SP08 (SAPK-20008INDW4CORE).

When receiving data from SAP PO the BW-System’s response in case of data format errors is sent as plain text. These messages can be found when sending faulty data with POSTMAN.

Our PO team is struggling to get these messages. So the question is: Is there a way of receiving these error messages in SAP PO?

Thanks and regards,

Ednan

former_member770779
Discoverer
0 Kudos

Hi Frank,

 

Thanks a lot for this blog. We were able to replicate in another 3rd party tool - Talend in our case . But the problem is performance .

We have a file with size approximately 1GB and we tried loading it with ADSO (REST API) method as per recommendation (implicitly).
However the job took 10.5 hours to load this file.
We have tried the following method with the intention to improve the performance but it does not work:
  1. Splitted the files into small sizes and do REST POST.
  2. Parallel execution (Multiple jobs to process the REST API request for splitted files)

Do you have any option to improve the performance while posting the data into ADSO through rest api way ?

Just sample Job design below for your better understanding

 

Regards,

Mani.

FrankRiesner
Product and Topic Expert
Product and Topic Expert
0 Kudos
Dear Mani,
the purpose of my blog post was to highlight, that the ADSO write interface is open for non-SAP tools as well, although stated differently in some sources. The customer scenario above gave a good example for that.

Unfortunately I cannot comment on any other 3rd party tool. My general feeling is that sending 1 GB of data via a WebService is a challenge in general and maybe for this amount of data this might not be the best approach. One idea we have to check via Postman or similar tools if SAP BW/4HANA can handle the data properly. This might provide some details which explain the problem. Maybe there is an index missing in the target or some Jobs are not setup properly.

Regards, Frank
michael_metzner2
Explorer
Hi Frank,

we are integrating Qualtrics survey data via CPI into BW/4HANA following you approach with streaming chains in BW/4HANA.

It works as expected, nevertheless since some days we do not receive CPI data for some SEND DATA request (only some, most data is transferred). The CPI team can't find any error message on their side.

So I'm wondering is there any way to trace the SEND DATA calls in BW/4HANA?

Best regards,
Michael
FrankRiesner
Product and Topic Expert
Product and Topic Expert
Hi Michael,

SAP BW/4HANA, i.e. the write-enabled ADSO, does not save/provide any logs.
However, the error message is part of the response body and should be made accessible in the sender system.

Unfortunately, there is no additional monitoring function available on BW/4HANA side.

Regards, Frank
0 Kudos
Hi Frank,

 

Can you please let me know how to update entries in active table instead of inbound table. we are able to post in inbound table. we would like to know to post in active table.

 

Thanks,

Gopal KS
FrankRiesner
Product and Topic Expert
Product and Topic Expert
0 Kudos

Hi Gopal,

the data cannot directly be pushed into the ADSO active table. It has to go via the inbound table, and then the activation process will push it to the active table. See my prior comment from June 07, 2021 how to achieve this.

Regards, Frank

DirkO
Participant
0 Kudos
Hello Frank,

many thanks for your immediate confirmation.

Br.

Dirk
0 Kudos
Hi Frank,

 

Thanks for the quick response. Now I am able to open request and close request from CPI. But BW team confirmed that they cannot activate the request since few of the below requests are in amber. Can you please let me know how to proceed here now.


BW4HANA screenshot

paulvatter
Participant
hi Gopal:

do you have implemented the following OSS note? https://launchpad.support.sap.com/#/notes/3129312

We had a similar situation - need to confirm if this was the only point, will let you know...

Best regards Paul
sgharat83
Discoverer
0 Kudos
I am trying to load data from BODS to ADSO in BWH 2.0. In BODS should I create a Data Store connection to load Inbound table directly( which i did but can not see any request created in ADSO). Or I should use Nested Schemas to load data from BODS to BW.
FrankRiesner
Product and Topic Expert
Product and Topic Expert
0 Kudos
Hi Gharat,

as long as you have enabled the write-interface for the specific ADSO in BW/4HANA, you should be able to address that object from BODS side and push a data request into the ADSO inbound table.

There are two ADSO flavors which support this interface
1) Standard with Change Log
2) Staging with Inbound Queue only

Maybe try both approaches to find the most suitable for your scenario.

KR Frank
JaySchwendemann
Active Contributor
0 Kudos

Hi Frank

your continued support / answering comments is much appreciated 🙂

I'd like to follow up upon this. Is this still true in 2023, that Write-enabled ADSO does not support error logging?

We have the following situation and BW team hesitates to switch to write-enabled ADSO because they fear they lose monitoring capabilities they currently have in their process chains.

  • Current Setup: ABAP Report calling a Middleware (PI/PO or CI) which calls external service synchronously and uses FM to populate ADSO. This Report is used in a process chain. On Error this error situation (be it in the leg from AS ABAP to Middleware or from Middleware to external service) is visible
  • To-Be Setup: External Service (via Webhook) calls Middleware (CI) which then calls write-enabled ADSO. In this situation we do not have process chains.

How could we give the BW team their "Monitoring via process chains" back in an write-enabled ADSO?

 

If it makes sense I'll gladly ask this as a question in the community, of course 🙂

 

Many thanks and kind regards

Jens

 

PS: Oh and by the way, I'm more an integrations guy, not a BW guy, so this might be a painstakingly stupid question or there might be an painstakingly simple answer 😉

FrankRiesner
Product and Topic Expert
Product and Topic Expert
Hi Jens,

a process chain in streaming mode is capable to see that new data has been pushed into the ADSO inbound queue to be processed further. But this is how far a process chain reaches.
Normally external ETL tools will push data into this ADSO, normally without any ABAP relation. So this cannot not be controlled or monitored by the means of Process Chains.

Regards - Frank
JaySchwendemann
Active Contributor
0 Kudos
Hi Frank,

sorry, for the delay and thanks for the input.

My takeaway then would be: Monitoring in the "old" world of process chains would be possible to such a degree, that the chain could tell if data was written (into the inbound queue) and would then consequently fail on the next step (activation?)

It would however be unable to tell that the data was not written because e.g. there was an authentication error from middleware to BW.

Is that the gist of it?

Thanks and kind regards

Jens
FrankRiesner
Product and Topic Expert
Product and Topic Expert
0 Kudos
Hi Jens - yes this is the correct understanding
KR Frank