Technology Blogs by Members
Explore a vibrant mix of technical expertise, industry insights, and tech buzz in member blogs covering SAP products, technology, and events. Get in the mix!
cancel
Showing results for 
Search instead for 
Did you mean: 
MortenWittrock
Active Contributor

Introduction


One of the core functions of any integration platform, is to translate between different message formats. These transformations are colloquially known as mappings, and every integration platform offers its own set of tools for building them.

Cloud Integration is SAP’s integration platform in the cloud. In this blog post, I’d like to introduce you to the mapping options provided by Cloud Integration, namely:

  • Message mapping

  • XSLT

  • JavaScript and Groovy scripting


I will give you an overview of each option, and discuss their pros and cons. Plus a one-sentence TL;DR summary for the busy reader 🙂

Message mapping




The TL;DR version: Message mapping is easy to learn, and can be a very effective tool, but beware that if you apply it to the wrong problem, things can get ugly.

Overview


Message mapping is a graphical mapping tool, that doesn’t require any coding. Unlike the other two mapping options, the message mapping tool is built right into the Cloud Integration web application. You construct a message mapping by connecting source structure fields to target structure fields via drag and drop, and possibly inserting functions in between.

The image above is an example of fields being mapped in a message mapping. When the mapping runs, the concat function will concatenate the contents of the FirstName and LastName source fields, and the result will be copied into the FullName target field.

The tool has a library of built-in functions for doing calculations, string handling, conversions etc. In addition to those, you can add your own, custom functions written in the Groovy scripting language.

A very similar message mapping tool ships with SAP PI, so developers coming from that background will feel right at home with Cloud Integration’s version. The Cloud Integration web application also supports importing existing message mappings from an on-premise SAP PI system via Cloud Connector.

Initially, message mapping supported only XML to XML transformations, but support for JSON (specifically Open API service definitions) was added recently. Support for JSON in general via JSON Schema is on the roadmap.

Pros and cons


The message mapping tool is exceedingly easy to pick up and get started with; you can learn the basics in a very short amount of time. It excels at solving mapping problems of low to medium complexity. For that category of problems, message mapping can be a very effective tool.

However, solving highly complicated mapping problems with message mapping tends to result in mappings that are overly large, messy and disproportionately hard to test and maintain. In other words, be mindful of whether message mapping is a good fit for the task at hand.

XSLT




The TL;DR version: For XML to XML mapping, XSLT is the most powerful of the three mapping options, but that power comes at the cost of a steep learning curve.

Overview


Extensible Stylesheet Language Transformations (XSLT for short) is an XML language for transforming XML documents into other XML documents, HTML or text. An XSLT transformation is called a stylesheet (not to be confused with Cascading Style Sheets, which describe the presentation of HTML documents). XSLT is a standard maintained by the World Wide Web Consortium (W3C). Cloud Integration supports XSLT 3, the latest version of the standard.

XSLT development revolves around writing templates. A template creates a part of the stylesheet’s output. It can match parts of the input document, or it can be called by name from another template. The XPath query language (another W3C standard) is used to navigate the input XML document. XPath enables the developer to pinpoint locations in the input in a very succinct way. Using XPath, extracting a value from a deeply nested, complicated XML structure can be trivial.

The stylesheet in the image above performs a so-called identity transformation, i.e. its output is identical to its input.

The Cloud Integration web application offers a rudimentary XSLT editor, but its functionality is very limited. If you want conveniences like debugging and stylesheet validation, you need a separate tool (typically an XML editor such as XMLSpy or Oxygen).

Pros and cons


Of the three mapping options offered by Cloud Integration, XSLT is without a doubt the most powerful for XML to XML mappings. Faced with a complicated transformation, like mapping a large IDoc to a deeply nested XML structure, XSLT should be your first choice.

However, the language has a steep learning curve, that must be climbed, before one becomes productive. Also, if you want to get serious about XSLT development, you will need an XML editor, and that means one more tool to learn.

One additional benefit of XSLT, is that stylesheets are text files, that you can store in a version control system like Git.

JavaScript and Groovy scripting




The TL;DR version: If the developer is willing to manage all aspects of a mapping herself, scripting offers more flexibility and more control over input processing and output generation than message mapping and XSLT.

Overview


The Script step lets you add scripts written in Groovy or JavaScript to your integration flows. Script code can access and modify the contents of the exchange, like properties, attachments, headers and the message body. In addition to that, scripts can use built-in and external class libraries.

Scripts serve multiple purposes in Cloud Integration, and one of those purposes is mapping. The image above is an example of a simple mapping performed in Groovy. It transforms a CSV-formatted message into an XML document. While the code is simplified for the sake of brevity, it does demonstrate the two main tasks performed by all mapping scripts:

  • Process the input message

  • Generate the output message


If you are handling plain text or XML, the built-in classes will go a long way. To process, say, an Excel spreadsheet attachment, you need an external library.

As is the case with XSLT, the Cloud Integration web application offers a very basic script editor. However, most developers opt for external editors or IDEs with a much richer feature set.

For a very comprehensive look at Groovy mappings in Cloud Integration, please see the blog post I *heart* Groovy mapping by SAP Mentors alumnus engswee.yeoh.

Pros and cons


Mapping in script code gives you full control over how the input is processed and how the output is generated. This is obviously great, and it lets you write, for instance, advanced text processing that goes well beyond the built-in capabilities.

The other side of that coin, though, is that the developer needs to manage every single aspect of the mapping herself. This is due to the fact that JavaScript and Groovy are general-purpose programming languages, whereas message mapping and XSLT are specialised tools.

Also, if you are new to JavaScript, Groovy or indeed programming in general, there is a sizeable learning curve to overcome, before you become productive.

An added benefit is that scripts are text files, that you can store in your version control system.
14 Comments
engswee
Active Contributor
Hi Morten

 

Thanks for the great post on the comparison between the different mapping options available.

 

A few points from my own experience:-

Message mapping - yes, complex mappings can get real ugly as the developer needs to understand the queues & context concept that is inherited from PI. I've seen too many PI mappings gone horribly wrong, that I sometimes wish CPI would start afresh without this legacy baggage! Plus currently there's no way to run automated unit tests on these mappings - the simulate functionality in WebUI is very rudimentary.

 

XSLT - totally agree with the steep learning curve. I've always struggled with it (and still do), and only use it sparingly when it seems to be the best choice for a specific use-case. Do any of the tools support running a suite of unit test cases?

 

Scripting (Groovy in particular) - given that I have a bias towards coding, this is my favorite. But IMHO, any integration developer will at some point need to write some codes, so why not pick up this lovely-to-use language early on? Plus, there's a trick to unit test the scripts locally on your computer 🙂 Furthermore, XML is no longer the de-facto medium for data transfer - JSON's popularity is growing, as well as other binary or unstructured data formats. So there are times where this is the only option available.

 

Regards

Eng Swee
MortenWittrock
Active Contributor
Hi Eng Swee

Thank you for your always interesting comments 🙂 You bring up a very good point about non-XML formats, and scripting being pretty much the only option in many of those cases.

Regards,

Morten

 
vadimklimov
Active Contributor
Hi Morten,

 

Very well written and structured blog, thank you for this summary and comparison as well as for valuable highlights of advantages and disadvantages of each of techniques!

 

In my opinion, history repeats itself to some extent here: PI / dual stack started with support of variety of mapping techniques - graphical mappings, XSLT (with ABAP and Java based XSLT processors), ABAP and Java mapping programs. This was followed by elimination of ABAP based techniques due to migration to Java-only installation options. And now, in CPI, remaining techniques experience second breath:

  • PI/PO's graphical mapping > CPI's message mapping,

  • PI/PO's Java mapping program > CPI's Groovy and JavaScript scripts,

  • PI/PO's XSLT program > CPI's XSLT mapping.


 

Nice thing here is that CPI comes with native support of XSLT 2.0 processor, while in PI/PO, we need to make use of custom XSLT transformer - good step forward for those who have to deal with many XSLT mappings on a regular basis and need some more recent and advanced capabilities.

 

I'm yet uncertain about clear segregation of use cases between Groovy and JavaScript scripts and positioning these two - putting aside such aspects as skill set and experience of the particular developer or the team / organization adherence to specific languages / frameworks and accumulated experience in corresponding fields, but focusing purely on technical development, performance / robustness and maintenance aspects. Looking into CPI roadmap, both Groovy and JavaScript scripts' support is declared in a longer term, and they both share very common use cases and satisfy space of requirements that is usually covered by Java mapping programs in PI/PO world. I might only assume here that few motivations to have both Groovy and JavaScript being supported by CPI, can be:

  • Smooth transition between SAP integration products, such as PI/PO, CPI and API Management. PI/PO developer with Java skills can become productive with Groovy scripting relatively quickly - most difference will come from Groovy-specific libraries and language shortcuts, but the developer can dive into Groovy gradually, starting from Java-style coding and utilization of already known Java native libraries, and gracefully moving into Groovy-style and leveraging Groovy-specific libraries. Good point is that compiler and runtime will finely accept both styles, as well as techniques and principles of program performance troubleshooting and optimization on lower level / JVM remain the same for Java, Groovy or other JVM language. Further progression with usage of both CPI as an integration platform and API Management as an API platform in a single landscape and their collaboration might also add to this from JavaScript perspective in the same way as PI/PO adds to CPI from Java/Groovy perspective, as a major piece of custom policies in API Management is developed using JavaScript.

  • Common popularity of JVM languages on the one hand side and JavaScript (with all its frontend and backend / Node.js backing ecosystem) on the other hand. I observe trend in various other cloud services, where they aim provisioning of runtimes and tools for both JVM languages and JavaScript.


 

To me, this might also cause a bit of confusion in mixed heterogeneous landscapes, where integration developers can come from different backgrounds and be more experienced with different languages and frameworks. Technically, it will be hard to justify usage of Groovy over JavaScript in custom flow steps, or vice versa, so it seems that the choice might be done based on various non-technical factors, or there will be a mix of both of alternatives.

 

Given I personally come from ABAP and Java background, Groovy is more comfortable for me, so choosing between Groovy and JavaScript script (mapping or any other integration flow step), in absence of any other non-technical factors and given parity in availability of external dependencies/libraries (should those be required) for the assessed task, my preference will be Groovy, but I recognize there shall be a reasonably large number of developers, whose choice might be in favor of JavaScript.

 

Regards,
Vadim
MortenWittrock
Active Contributor
0 Kudos

Hi Vadim

Thank you for your insightful comment!There’s probably an interesting blog post there, waiting to be written, about migrating SAP PI skills to Cloud Integration. On that note, I much prefer CPI’s scripts to PI’s Java mappings. They’re much more versatile, and you can script wherever you want.

Thanks again,

Morten

camejia24
Explorer
Hi Morten

 

Thank you, it is a nice explanation about maping.

 

Best Regards

Cesar Mejia
MortenWittrock
Active Contributor
0 Kudos
Hi Cesar

Thank you, that's very kind.

Regards,

Morten

 
Florian_Kube
Participant
Hello Morten,

great overview of the different mapping options.

On the PI/PO I used to use XSLT Mapping when it was complex. However, as I just tried it is not possible to upload a ZIP XSLT mapping with multiple XSL-files. I like to separate segments etc. in different files. Therefore, you need to have the whole mapping in one file.

 

Regards Florian

MortenWittrock
Active Contributor
Hi Florian

Thank you! You can have multiple stylesheets that refer to each other, but you do need to upload them separately, unfortunately. I've written about xsl:import in Cloud Integration here.

Regards,

Morten
Florian_Kube
Participant
Thank you! Now I switch 🙂
rasjoshi
Active Contributor
Thanks7a519509aed84a2c9e6f627841825b5a for sharing this document. I am new to SAP CPI, any leads to learn Groovy Scripting for SAP CPI?

 

Regards,

Rashmi

 

 

 

 
MortenWittrock
Active Contributor
0 Kudos
Hi Rashmi

Sorry about the (very) late reply. Check out this blog post by Eng Swee Yeoh about Groovy mappings.

Regards,

Morten
jasonwilliams
Explorer
Hi Morten,

 

For simple mappings (XML to JSON) I've been using the Content Modifier (ie. use XPATH to retrieve value into a Property then use that property in the Body). Understand that there are significant limitation here eg. repeating elements. But for a simple static output message format I think it is (perhaps) the simplest method.

Is there a reason this method isn't mentioned as one of the options here?

Thanks,

Jason
MortenWittrock
Active Contributor
0 Kudos
Hi Williams

That's a perfectly viable approach. It's not in this blog post because it's not generally considered one of the supported mapping options, in the sense that it's not transforming an input payload to an output payload.

But as you say, for simple, static messages it's absolutely fine.

It's been on my list of topics to blog about for a while by now. I should probably get around to it soon 🙂

Regards,

Morten
tuh
Explorer
0 Kudos
Hi Morten,

Thanks for this fine blog. I am wondering if there has been any performance considerations since this was written?. The state of the cloud system is maybe always up and with an infinite memory recourse. But that is not the case in clound on onpremise systems.

I can see it is often StringWriter that is used to move the output of the groovy mapping to the Camel message. But to me this looks like a potential memory problem if the payload is very big > 10 MB.

Do you have any ideas how to adress the memory problem on big size payloads ?.
StringWriter sw = new StringWriter();

// here  you would have XmlSlurper og some  other Sax-parser code

//and then.. this ugly statement
message.setBody(sw.getBuffer().toString()) 
//or like yours
message.setBody(sw.toString())

Looking forward to your reply.

Thanks
Torsten

 
Labels in this area