Solving Special Character Issue while sending data to Receiver System using HTTP Adapter

[ahmed_t]

Introduction:

While sending data to target system using HTTP adapter, all the special characters in the data turns into  "?".Languages like Hebrew, Turkish, Russian contains special characters and these data after sending to target system with HTTP receiver channel gets corrupted and does not reflect as expected. Reason behind this is the HTTP receiver adapter needs a byte array and uses the defined charset for the encoding (or ASCII if nothing is defined). Characters that do not exist in the used encoding are replaced by "?" signs.

 

Let's follow some simple steps to overcome this issue:

 

We will define the required encoding in Content Modifier step. Add a new property in Exchange Property as below:

Action            Name                                  Type                          Value

Create           CamelCharsetName           Constant                    UTF-8

 



 

While providing payload in JSON structure, we provide in the below way:



Now, the issue is Hebrew most of the times contains a (") Double Quote in between strings.

Example : **עובד כ"א זמני**--Here is one " in between string. While sending payload in JSON structure this " creates an issue. Because JSON payload always come in between a "<string>".



Here, CPI accepts this as a double quote in between the JSON payload string that does not have an ending double quote. So, we need to escape this double quote. We will try to escape this using a Groovy Script.

Please find the Sample escape sequence in below code snippet:



 

The replaceAll() function escapes all the double quotes in strings and shapes the payload in a good structure.

 

By taking care of these simple steps we can avoid data corruption in target systems while using HTTP receiver channel.

 

Reference:

I found this KBA very helpful.

https://launchpad.support.sap.com/#/notes/2533303

 

 

Thanks 🙂

 

 

 

[irijsdijk]

Hi Toushif Ahmed,

Seems simple enough. I'm sending a JSON message containing the special character ë (Cliënt). I added the exchange property, and during the trace all looks OK right up to the point where the special character is sent to the receiving system. Then it changes into a ? (Cli?nt). I added the exchange profile to a content modifier shape right before the HTTP receiver, but it still messes up the character.

Any ideas?

Iddo

[ahmed_t]

Hey Iddo,

 

Please add the exchange profile in the same content modifier where your request payload is in JSON format in the content modifier's body with Content-Type as "application/json" in the content modifier's header.

 

Regards,

Toushif.

[edumarcelino]

Same problem here.

Could you tell me what do you do to corret the encode ?

[ahmed_t]

Please add a Content Modifier before you insert data using HTTP adapter. Here in my case Payload in JSON is a content modifier.

Now, in the content modifier’s Message Header section, add a new header as:

Add the content encoding property in the Content Modifier’s exchange property as:

and in the Message Body section, please add an expression like below:

Hope the issue will be resolved.

Kind Regards,

Toushif.

[edumarcelino]

No way, thats something wrong.

Seems it loses the charset inside second HTTP step.

error in second step of HTTP

 

[ahmed_t]

Hi,

There may be something wrong you are doing in the content modifier's configuration. Please check it again.

And if there are two separate HTTP adapter for data insertion place two separate content modifiers with the same set of configuration.

Another thing, please log the final payload to insert using a groovy script and place that groovy script just before the HTTP adapter.

 

Regards,

Toushif.

[edumarcelino]

The message is ok in the first HTTP segment.

The charset error occurs in the second segment of HTTP.

[ahmed_t]

Please share the raw payload with me.

[edumarcelino]

I'm sending from postman.

<?xml version="1.0" encoding="utf-8"?>

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">

<soap:Header></soap:Header>

    <soap:Body>

        <movies>

            <movie>

                <id>1</id>

                <name>Ação</name>

            </movie>

            <movie>

                <id>2</id>

                <name>Animação</name>

            </movie>

            <movie>

                <id>3</id>

                <name>Aventura</name>

            </movie>

            <movie>

                <id>4</id>

                <name>Cinema de arte</name>

            </movie>

            <movie>

                <id>5</id>

                <name>Chanchada</name>

            </movie>

            <movie>

                <id>6</id>

                <name>Comédia</name>

            </movie>

        </movies>

    </soap:Body>

</soap:Envelope>

[ahmed_t]

Hi,

Sorry for the delayed response. I am getting the payload properly encoded with the said configurations:

P.S.: You can try without using a message mapping step after the SOAP input as the payload is in the message body, only the JSON conversion from it will work.

Regards,

Toushif.

[dirugadr]

Hi!

Did you find a way to solve this issue? I'm facing the same! It's very frustrating!

Regards

Adrian

[ahmed_t]

Hi Adrian,

Is it solved ? or you are still having the issue.

Please share a screenshot of your flow and let's see if I can help you anyway.

 

Regards,

Toushif.

[dirugadr]

Hi! Yes, I'm still having this issue. I'll prepare some screenshots to share and I'll post them here.

Thanks!