Introduction:
With the increasing level of interface complexity, it’s very important to log payloads at different stages of build. Payloads can be logged using groovy scripts in message processing logs (MPL), data store and persist message. SAP has added a circuit breaker to stop MPL attachments if 1 GB data is stored in 24hrs. Currently SAP doesn’t provide any UI to download Persisted messages. Logging payloads using data store is a better option, but with the increase in payload logging the memory consumption will also be high. So, data store entries should be cleared regularly, especially in production tenant. SAP has provided standard API to extract data store entries. In this blog I will explain how to delete data store payloads/entries using SAP API, groovy script and delete data store operation.
Scenario:
Retrieve the data store entries from SAP API, filter the entries to be deleted using groovy script and delete the data store entries using delete operation.
Integration Artifact Details:
Step 1:
Step 3:
Configure the above iflow, schedule it as per business requirement and deploy it.
Conclusion:
We just saw how to configure the interface to automatically delete the logged payload in data store. The interface schedule can be altered based on payload retention period as per the business requirement.
References:
https://blogs.sap.com/2020/01/29/sap_cpi-demonstrating-the-datastoreoperations-feature/
https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/Cloud/en-US/8c35f3fa3b9c42c5b810332eccb...
With the increasing level of interface complexity, it’s very important to log payloads at different stages of build. Payloads can be logged using groovy scripts in message processing logs (MPL), data store and persist message. SAP has added a circuit breaker to stop MPL attachments if 1 GB data is stored in 24hrs. Currently SAP doesn’t provide any UI to download Persisted messages. Logging payloads using data store is a better option, but with the increase in payload logging the memory consumption will also be high. So, data store entries should be cleared regularly, especially in production tenant. SAP has provided standard API to extract data store entries. In this blog I will explain how to delete data store payloads/entries using SAP API, groovy script and delete data store operation.
Scenario:
Retrieve the data store entries from SAP API, filter the entries to be deleted using groovy script and delete the data store entries using delete operation.
Integration Artifact Details:
Step 1:
StorePayload
Main iFlow
- Start Timer: Schedule the interface to recur daily.
- Set Header: Configure the header details as below
- Request Reply SAP API call:
https://tenantID-tmn.xxxxx.ondemand.com/api/v1/DataStores(DataStoreName='DS_Payloads',IntegrationFlo...
- Format API Response: Refer the below xslt code to format API response
<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output indent="yes" method="xml" encoding="utf-8" omit-xml-declaration="yes"/>
<!-- Stylesheet to remove all namespaces from a document -->
<!-- template to copy elements -->
<xsl:template match="*">
<xsl:element name="{local-name()}">
<xsl:apply-templates select="@* | node()"/>
</xsl:element>
</xsl:template>
<!-- template to copy attributes -->
<xsl:template match="@*">
<xsl:attribute name="{local-name()}">
<xsl:value-of select="."/>
</xsl:attribute>
</xsl:template>
<!-- template to copy the rest of the nodes -->
<xsl:template match="comment() | text() | processing-instruction()">
<xsl:copy/>
</xsl:template>
</xsl:stylesheet>
- Filter Entries ToBeDeleted: Refer the below groovy script to filter data store entries to be deleted and create a deletion entries list - EntryID. Here I am deleting the payloads which were saved 3 days before the current time. Payload’s retention period can be changed based on the business requirement.
import com.sap.gateway.ip.core.customdev.util.Message
import java.util.HashMap
import com.sap.it.api.ITApiFactory
import com.sap.it.api.mapping.ValueMappingApi
import groovy.xml.MarkupBuilder
import groovy.time.*;
def Message processData(Message message)
{
def a = ITApiFactory.getApi(ValueMappingApi.class, null);
def body = message.getBody(java.lang.String) as String;
def Test = new XmlParser().parseText(body);
def writer = new StringWriter();
def builder = new MarkupBuilder(writer);
def ToBeDeleted = []
def ID = []
def TimeDuration = []
String Create = '';
builder.Target
{
'Record'
{
Test.entry.each {
it.properties.each{
def mappedValue = "${it.Id[0].text()}";
if ( mappedValue.equals("") || mappedValue ==null )
{
ToBeDeleted.push("No")
}
else
{
Create = "${it.CreatedAt[0].text()}"
def CreatedAt = Create.substring(0,19)
CreatedAt = CreatedAt.replace("T", " ")
def Newdate = Date.parse("yyyy-MM-dd HH:mm:ss",CreatedAt)
def Currentdate = new Date()
TimeDuration duration = TimeCategory.minus(Currentdate, Newdate);
TimeDuration.push(duration)
if(duration.getDays() > 3){
ToBeDeleted.push("Yes")
ID.push(mappedValue)
}
}
}
}
}
'Id'(ID)
}
// Generate output
message.setBody(writer.toString())
message.setProperty("ToBeDeleted", ToBeDeleted)
message.setProperty("EntryID", ID)
message.setProperty("TimeDuration", TimeDuration)
return message;
}
- Looping Process Call: Process call to delete messages. This process repeats for each entry in the deletion list.
- Set DelEntryID: Refer the below groovy script to set delete EntryID and set indicator if any valid entries are present to delete.
import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import groovy.json.*;
def Message processData(Message message) {
def body = message.getBody(java.lang.String) as String
def DelEntryID = message.getProperties().get("EntryID")[0]
def ToBeDeleted
if ( DelEntryID.equals("") || DelEntryID ==null )
{
ToBeDeleted= 'No'
}
else{
ToBeDeleted= 'Yes'
}
message.setProperty("DelEntryID", DelEntryID)
message.setProperty("ToBeDeleted", ToBeDeleted)
return message;
}
- Router: Decides if there is any valid entry to delete. If not ends the call.
- Delete: Deletes data store entry:
- Remove Deleted Entry Id: Refer the below groovy script to remove the deleted EntryID from the deletion list.
import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import groovy.json.*;
def Message processData(Message message) {
def body = message.getBody(java.lang.String) as String
//Remove EntryID from lookup list
def packages = message.getProperties().get("EntryID").reverse() as Stack
packages.pop()
packages = packages.reverse() as Stack
message.setProperty("EntryID", packages)
return message;
}
Step 3:
Configure the above iflow, schedule it as per business requirement and deploy it.
Conclusion:
We just saw how to configure the interface to automatically delete the logged payload in data store. The interface schedule can be altered based on payload retention period as per the business requirement.
References:
https://blogs.sap.com/2020/01/29/sap_cpi-demonstrating-the-datastoreoperations-feature/
https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/Cloud/en-US/8c35f3fa3b9c42c5b810332eccb...
- SAP Managed Tags:
5 Comments
