[SAP Cloud Platform-Integration] Content Filter in Detail

[Sriprasadsbhat]

Introduction:

This blog is to demonstrate usage of content Filter step of message Transformation in SAP Cloud Platform Integration.

Content Filter step filters required node or list of nodes or value of specific node from the incoming XML message.

Content Filter consist of below

Name	Valid Logical name for Filter
XPath	Expression Enter an XPATH to extract a node in the message
Value Type	Specify the type for the value you want to extract from XML Node

Below table provides the different type of Value Type returned from Filter.

String	Returns the String value from Node	INPUT: <NODE>VALUE1</NODE> XPATH: //NODE OUTPUT: VALUE1
Integer	Returns the Integer value from Node	INPUT:<NODE>1</NODE> XPATH: //NODE OUTPUT: 1
Boolean	Returns the Boolean value from Node	INPUT: <NODE>VALUE1</NODE> XPATH: //NODE OUTPUT: True
Node	Returns the Node value from input xml	INPUT: <NODE>VALUE1</NODE> XPATH: //NODE OUTPUT: <NODE>VALUE1</NODE>
Nodelist	Returns list of nodes from incoming XML	INPUT: <Root> <NODE>VALUE1</NODE> <NODE1>VALUE1</NODE1> </Root> XPATH: //NODE OUTPUT: <Root> <NODE>VALUE1</NODE> <NODE1>VALUE1</NODE1> </Root>

Use Cases:

Since the Value Types with Sting,Integer,Boolean and Node are self explanatory will get into Nodelist which we use more often in Custom Integration.

Case#1:

Filter the incoming XML containing <ErrorNode>

Input:



XPATH:

/Root/Record[Error]

Output:

Case#2:

Filters the Record node which contains <Status> node with value S [ Success ] or SK [ Skipped ].

Input:



XPATH:

/Root/Record[(Log/Status/text()= ‘S’ ) or (Log/Status/text() = ‘SK’ )]

Output:

Case#3:

Filters the Record node which contains <Data1> node with value having string length greater than or equal to 2 and <Data2> value not equal to blank.

Input:



XPATH:

/Root/Record[((string-length(Data1))>=2) and (Data2!='')]

Output:

Case#4:

Filter the incoming XML not containing <ErrorNode>

Input:



XPATH:

/Root/Record[not(Error)]


Output:

Case#5:

Filter the incoming XML record containing  only values which are present in Property .


Input:



Content Modifier:



XPATH:

/Root/Record[contains($PropData1,Data1)]

Output:

Design Use Case:

Lets take a example  that you need to perform action on input data containing both valid and invalid data and need to take further action on only valid set of data.Whenever we get this kind of requirement we use to go with Splitter and Gather ( Since we are expecting group of records post validation ) and in between we will try to route the invalid data but you will end up in error ( Since Router step is not allowed between Splitter and Gather ). Content Filter comes to your help when you have these kind of requirement ( Same will be demonstrated in below image ) where 2 difference branches will have set of valid and invalid data .

Limitation:

• It works only for XML messages:


• Content Modifier to be used post Filter step to rebuild the Valid XML since Filter removes the Root node.

References:

• [Filter]- SAP HELP -Filter

Conclusion :

Very useful step which can be leveraged for data validation or filter and group incoming XML based on some condition.

Thanks to my friend Vinay for helping out in few use cases.:)

Regards,

Sriprasad Shivaram Bhat

 

[former_member203045]

Excellent Explanation  and Thanks Sriprasad for sharing.

[former_member113361]

Well explained. Is it possible to use multiple and complex x-path expressions?

For example - consider a scenario where employees without " Compensation" information and "nationalID should be filtered out from the Successfactors "Compound Employee" API XML. Is it possible achieve this with content filter?

 

[Sriprasadsbhat]

Hello Biplab,

Its possible to have complex condition also ,but you need to make sure these conditions are grouped together in proper manner.

I have just created a condition for above case including few more cases.

queryCompoundEmployeeResponse/CompoundEmployee[((person/logon_user_is_active = 'true') and ((person/employment_information/job_information/emplStatus ='A') and (person/employment_information/job_information/emplStatus/event_reason ='HIRNEW')) and (person/employment_information/compensation_information) and (person/national_id_card)) ]

Regards,

Sriprasad Shivaram Bhat

[Former Member]

Good one Sri and keep blogging!

[adityawsharma]

Very nicely explained with use cases, it would be a lot of benefit for developers in HCI to implement requirements based on your use cases. Keep blogging and sharing

[former_member340751]

Thanks Shri, Very nice blog.

 

Thanks for sharing.

 

[gagandeep_batra]

Great Sri...

Thanks for Sharing...

Hope more to come 🙂 🙂

 

Regards

Gagan

 

[gayathri_narayana]

Great amount of detail !! keep blogging:)

[Former Member]

Helps me a lot. Great Job Sriprasad.

 

[former_member216636]

 

Hi sriprasadshivaramabhat , sriprasad.shivarambhat ,

Its very informative.One query Is it possible to use multiple expressions to filter SF response of queryCompoundEmployeeResponse XML ?

Example: Consider a scenario where employees job_information records in response of SuccessFactors query need to be sorted in HCI and only one record job_information(Latest and needs to be sort by end_date) should be send to receiver payload XML by filtering/removing all other Job information records of the employee from the Successfactors “Compound Employee” API XML. Is it possible achieve this with content filter?

Thanks,
Rajesh N

[former_member45775]

Very nice blog. Explained with all possible scenarios

[former_member554379]

How Can I do this?

 

• Content Modifier to be used post Filter step to rebuild the Valid XML since Filter removes the Root node.

 

[nitindeshpande]

Hello Roland,

You can do it in the below way, under your content modifier tab -

<root>

${in.body}

</root>

 

Regards,

Nitin Deshpande

 

[axelalbrecht]

Update: Router step is meanwhile allowed between Splitter and Gather.

[aalapati]

Hi sriprasad.shivarambhat ,

I have a requirement where I need to filter parent node depending on the child node  attribute value . I tried the below attached logic , but result is not as expected . Looking for helpful inputs.

Logic in Filter Node:

Payload: 

Thanks in advance,

Aswini.

[former_member82872]

Hi Siva,

I need to use complex filter on CE response.

 

Job Code contains 'RJC' (and) exclude Job Title is equal to 'Cleaner'

(OR)

Job Title contains 'Regional Manager'

(OR)

Job Title contains 'Regional Manager'

 

Filter I have provided:

queryCompoundEmployeeResponse/CompoundEmployee[((person/employment_information/job_information[contains($PropCode1,job_code)] and (person/employment_information/job_information/job_title!='Cleaner')) or (person/employment_information/job_information[contains($PropTitle1,job_title)]))]

This is not giving the expected results, please advice.

Nag

 

[former_member612246]

I have tried putting a router step between Splitter and Gather but there seems to be some anomaly.

The Gather is put after 2nd branch of Router, Hence ideally its supposed to be collecting the records which suffice only the condition satisfied in the 2nd branch.

But here Gather step is collating the records from both the router branches.

Any idea how to solve this?

[former_member612246]

Hi sriprasadshivaramabhat ,

Very effective blog... Thanks for the detailed explanation!

Here I have seen that the Error node appears as a child node of the main Record node. I had a situation where Record & Error nodes both are generated as child nodes of the Parent Root node.

Root --> Record

--> Error

So will the filter step be applicable here if I want to log the Error details in one branch & carry forward the Records in another branch.

Appreciate your response.

 

[Dhaladhuli]

My incoming XML is as below. As per the requirement we must exclude ‘LEE_000001’ and retain ‘LEE_000029’ based on the values mentioned in the property as externalized parameters in content modifier



<FOLocation>

<externalCode>LOC_000045</externalCode>

<nameTranslationNav>

<FoTranslation>

<value_ja_JP/>

</FoTranslation>

</nameTranslationNav>

<addressNavDEFLT>

<FOCorporateAddressDEFLT>

<state>18915</state>

</FOCorporateAddressDEFLT>

</addressNavDEFLT>

<name>California - Fremont</name>

<companyFlxNav>

<FOCompany>

<externalCode>LEE_000001</externalCode>

</FOCompany>

<FOCompany>

<externalCode>LEE_000029</externalCode>

</FOCompany>

</companyFlxNav>

</FOLocation>

 

I  followed the steps mentioned in the blog







But am get the output as:



Whereas the expected output  is

<FOLocation>

<externalCode>LOC_000045</externalCode>

<nameTranslationNav>

<FoTranslation>

<value_ja_JP/>

</FoTranslation>

</nameTranslationNav>

<addressNavDEFLT>

<FOCorporateAddressDEFLT>

<state>18915</state>

</FOCorporateAddressDEFLT>

</addressNavDEFLT>

<name>California - Fremont</name>

<companyFlxNav>

<FOCompany>

<externalCode>LEE_000001</externalCode>

</FOCompany>

</companyFlxNav>

</FOLocation>

Can you  please suggest  what  am I missing??

[axelalbrecht]

Hi, sorry for the late reply.

The behavior is correct. The gather will get all items of the splitter, no matter if you end them into the Gather step or not.

My proposal is to add some empty entries for the items that you don't need, and after the Gather step you can remove them or ignore them.

Hope that helps!

[former_member685202]

Really helpful and informative blog! Thanks

[former_member737300]

<root>

<sub>

<empID>123</empID>

<Name>ABC<//Name>

</sub>

<sub>

<empID></empID>

<Name>ABC</Name>

</sub>

</root>

I am using a content Filter to filter the nodeList with empty empID.

I am using the below xPath expression in Filter.

/root[(sub/empID/text() = '')]

Deploying which , the artifact throws the below error.

NoTypeConversionAvailableException: No type converter available to convert from type: net.sf.saxon.dom.DOMNodeList to the required type: javax.xml.transform.sax.SAXSource with value net.sf.saxon.dom.DOMNodeList@6957b780]

Any idea why am I getting this error?

Thanks!

[amit777]

Thanks man! Nice Blog!

[former_member650765]

Hello Sivaprasad,

Thank you for the blog. Request you to kindly update the same for exists function as well.

Thanks in advance!!

Regards,

Prashanth

[former_member509573]

Thanks a lot for your time and good article. I follow many of your posts.

[former_member799247]

Hi Sir,

I have tried these examples in SAP Cloud Integration (CPI) but I am not getting the output. Please help me on it.

 

Thanks,

Saritha

[former_member829995]

Hi,

below condition is not working is there changes required.

/A_PurOrdAccountAssignment/A_PurOrdAccountAssignmentType[(not(NewLineItem)) or (not(ItemDeletionCode))]

 

Thanks,

Akhilesh

[former_member813323]

Hi @sriprasad.shivarambhat,

This blog was very helpful for me.

I am stuck at one point.

My xmlpayload after a message mapping is shown as below, which the filter condition is unable to find since the xml payload is coming in text format.

(below message is multi-casted after a specific step)

Since the xml payload is not in the required format(see below) the filter condition above is failing.

My filter condition : 

Can you please help me?

[former_member813323]

see above

[former_member600627]

Hi Sriprasad,

I have a query.

I am putting a filter where Employee which has contract_Type = permanent should move to next step and Contract_Type=Vendor should be filtered out.

But as per the below xml one employee has multiple contract/Contract_Type and due to which filter is unable to read it and throwing error.

I have made a property for Contract Type = Vendor.

Also, Contract which does not have end date can be considered out of multiple contract for an employee record.

How to handle such situation through Filter Condition.

<Employee> <Contract> <Position_ID>111</Position_ID> <Contract_Type>Vendor</Contract_Type> <Start_Date>2015-10-01</Start_Date> <End_Date>2020-09-30</End_Date> <Contract_Status>Closed</Contract_Status> </Contract> <Contract> <Position_ID>112</Position_ID> <Contract_ID>12345</Contract_ID> <Contract_Type>Vendor</Contract_Type> <Start_Date>2020-10-01</Start_Date> <End_Date>2021-01-02</End_Date> <Contract_Status>Closed</Contract_Status> </Contract> <Contract> <Position_ID>112</Position_ID> <Contract_Type>Vendor</Contract_Type> <Start_Date>2021-01-04</Start_Date> <Contract_Status>Closed</Contract_Status> <Description>xyz</Description> </Contract></Employee><Employee>
<Contract> <Position_ID>115</Position_ID> <Contract_Type>permanent</Contract_Type> <Start_Date>2021-06-09</Start_Date> <Contract_Status>Open</Contract_Status> </Contract></Employee>

[ankit_mishra09]

Were you able to achieve this?