How to Connect Azure using SCPI - Upload file to azure blob

[former_member595526]

Many of you aware that there is no standard adapter like SFTP, SOAP available to connect Azure Cloud with SCPI. So I am writing this post to share how we can get connected with Azure Cloud using SCPI

In this post you will learn how to get connected with Azure Cloud and uploading file to Azure

Azure provides SDK in Java to get connected. As SCPI supports groovy, we can write a simple groovy script to upload any file to azure blob container

For example, I have created a simple flow which uploads csv file to azure folder "trial"



And the script to upload the file is below

import com.sap.gateway.ip.core.customdev.util.Message

import java.io.*

import com.microsoft.azure.storage.*

import com.microsoft.azure.storage.blob.*



def Message processData(Message message) {

    def body = message.getBody(java.lang.String) as String



    String accountName = "your_azure_account_name"

    String accountKey = "your_azure_account_key"

    String storageConnectionString = "DefaultEndpointsProtocol=http;" + "AccountName=" + accountName+ ";" + "AccountKey=" + accountKey



    CloudStorageAccount account = CloudStorageAccount.parse(storageConnectionString)

    CloudBlobClient serviceClient = account.createCloudBlobClient()



    CloudBlobContainer container = serviceClient.getContainerReference("azurefoldername/containername")



    String fileName = "filename.csv"

    String fileContent = body

    byte[] fileBytes = fileContent.getBytes()



    CloudBlockBlob blob = container.getBlockBlobReference(fileName)

    blob.uploadFromByteArray(fileBytes, 0, fileBytes.length)



    blob.getProperties().setContentType("text/csv;charset=utf-8")

    blob.uploadProperties()



    message.setBody("OK")



    return message;

}

 

you have to replace the accountname, account key and container's name with your azure's credentials.

File name can be either static or dynamically populated.

In my groovy script, I have replaced all the credentials and file name as test.csv and azure container name as trial( which is already available in my azure cloud)

Now we have all set to upload a file to Azure cloud. Let's test it via Postman tool



It is Status 200, which indicates success of uploading the file to Azure. Now let's check the file in Azure using Azure Storage Explorer



 

Good Luck !

 

Questions/Comment and Suggestions are welcome ?

[mmcisme1]

My head is going to explode this morning.  I've read so many good technical blogs.  Is there a good business/technical reason for connecting to Azure.

I know this is the how.   I always ask for more....  What was your business case?

[former_member595526]

SAP Commerce(Hybris) uses Hot Folder Concept which is now in Azure. SAP Commerce process the file immediately whatever been uploaded to Azure blob. We had business requirement to load huge csv data from ERP to Commerce. Hence we took this approach

[mmcisme1]

That is a great reason.

Thank you for taking the time to reply!

Michelle

[former_member526466]

Hi Ansari,

Is that possible to connect CPI to MS SQL db or Microsoft Azure ,to read data from tables ?. If yes ,do you want to share your thoughts.

Thanks,

Venkat

[smansari32]

Hi Venkat

Yes you can connect, you have to add required jar files to resources tab of integration flow, and write a groovy script to connect with any Data base

[Bais]

Do you have an example ? Github?

[Muniyappan]

Just curious, did you not find any REST APIs to connect to Azure?

[former_member643281]

Where in Postman do I add that Groovy script? When I copy-paste it into the pre-requisite script box I get a bunch of errors saying the code is invalid.

[MortenWittrock]

Hi Ansari

There is an Azure Blob service REST API, it seems. I would really suggest to use that, instead of a binary library. The obvious place to store the required configuration is inside a receiver channel - not a Groovy script.

Regards,

Morten

[vijay3773]

Dear Ansari

Thanks for the insight.

Can you please suggest the JAR Files required in order to implement the same via Groovy Script?

Regards,

Vijay Devulapalli

[jonprow]

Thank you for the blog.  Anyone get this to work?

 

-Jon

[former_member556842]

Hi Ansari,

 

I've been having some issues that you encountered previously looking at the SCN - with integrating CPI and Commerce Cloud;

 

"com.sap.gateway.core.ip.component.odata.exception.OsciException: Bad Request : 400 : HTTP/1.1"

And not having contact data /adr3mas in ERP for the standard B2B Customer replication iflow.

Would you be able to share you insights on how you solved those problems?

 

Kind Regards.

[former_member595526]

Hi Daniel,

It is difficult to handle when you don't have adr3mas. What I did was, I commented out adr3mas link in both receiver and sender side of the integration flow and created dummy adr3mas data in the property along with adrmas03 and injected the same with payload. To do that, you have to go though the entire flow and also should be careful with how data is stored in data store and pulled back. requires customization

[philippeaddor]

Of course you don't add it in postman, but in SAP CPI (Cloud Platform Integration), as a script step in the iflow.

[former_member556842]

Thanks I appreciate your reply!

We ended up removing the branch for ADR3MAS, editing the xslt which checked to see if adrmas3, adr3mas and debmas had been received, and adding a conversion to remove the xml utf metadata at one stage where it was storing in the datastore for the missing adr3mas (as it is a reserved keyword in xml).

We then encountered issues with the standard odata adapter for the B2B Unit as it added URL parameters which couldnt be changed, which was resolved by deleting it and using a new OData V2 adapter, loading a fresh instance of the EDMX, and using ATOM only - as the conversion to JSON added an additional comma which caused incorrect syntax.

Bit if a nightmare but got a working solution in the end... Not quite the plug and play solution we were hoping for 😄

Thanks for the tips and guidance. All the best

[former_member656542]

Hi, We have a similar requirement and working towards the same. We need to fetch the data from Azure . We are getting an exception while trying to read blob from Azure and the exception says cause: java.lang.NoClassDefFoundError: javax/crypto/Mac. I belive this is related the JCE policy and not sure if we have the appropriate jar files are present in the JVM. Please provide guidelines in this regards

[former_member625050]

Hello Ansari,

We are getting the following exception after deployment of iflow.

javax.script.ScriptException: java.lang.Exception: com.microsoft.azure.storage.StorageException: Host not found@ line 44 in script1.groovy, cause: com.microsoft.azure.storage.StorageException: Host not found

Can you please guide on this.

Regards,

Irfan

[former_member625050]

Hello Ansari,

 

I had added Endpointsuffix  to the storageConnectionString, and it got resolved.

Thank you for this blog.

 

[former_member26001]

Hi smansari

Great blog, thanks, it helped me to setup scenario CPI-->Azure File Storage

Did you succeded with fetching files from Azure in this way?

BR,

Piotr

[former_member26001]

Actually, it was easier then I thought. I'm processing CSV file from Azure File Storage placed in directory to CPI as text string. To make it even better without fixing a filename this method need to be used: CloudFileDirectory.ListFilesAndDirectories

code snippet:

    //prepare file

    String fileName = "Test_response.txt" 



    CloudFile file = sampleDir.getFileReference(fileName)

    

    //get a file

    String fileContent = file.downloadText()

    file.delete()

    message.setBody(fileContent)

[gregory_p_richardson]

Hi Ansari,

I found your blog very helpful!  I used it in conjunction with the SAP Commerce ‘azurecloud’ extension to write a PNG file to a cloud hot folder (CHF) from a PNG image URL in a digital asset manager (DAM).  I used a different API, but your sample Groovy script was a great starting point!

Thanks,
Greg Richardson

[anandabalan_s2]

Hi Irfan,

 

I am also facing similar issue. "Noclass def found"

can you elaborate the solution which worked for you.

 

[former_member673677]

Hi Ansari,

 

My Question is about Replicate  -B2B Customer From ERP To SAP Commerce Cloud-.

 

We too have to same scenario where ADR3MAS will not be posted by SAP.

Can you please help how to customize the flow.

I saw your post with same issue.

https://answers.sap.com/questions/12664439/replicate-b2b-customer-from-erp-to-commerce-cloud-.html

Regards,

Sandeep y.

[former_member711149]

Hi, I'm trying the same , connecting from CPI to Azure,

and getting error while calling

CloudStorageAccount account = CloudStorageAccount.parse(storageConnectionString)

Error :unable to resolve class CloudStorageAccount 

do you have some idea to resolve this.

thanks and regards,

Anup

[harshithayadav2021]

Hi,

I have used the groovy script mentioned in the blog and changed the Azure account details and created an iflow in CPI.

I am getting the below error:

Error Details

org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[ID-c4050bbe-be45-482f-5677-2cd5-1619827319770-188-1], cause: java.lang.NoClassDefFoundError: javax/crypto/Mac

Can some one help me with this.

[former_member761029]

Hello Prabushankar,
Were you able to solve that issue?

[former_member761029]

Hello Harshitha Yadav,
I have the same error. Did you by any chance find a solution?

[harshithayadav2021]

Hi igalapagos

I wasn't able to resolve this error using script. So, instead I have used Azure REST APIs to connect and  read and write data into to Azure blob storage using http adapter in CPI through request-reply pattern.

 

Please refer to: https://docs.microsoft.com/en-us/rest/api/storageservices/blob-service-rest-api

for more info on Blob service REST APIs.

You can either use SAS key or Azure Active directory for authorization based on your requirements.

 

Additional References:

https://docs.microsoft.com/en-us/azure/storage/common/storage-auth-aad

https://docs.microsoft.com/en-us/azure/storage/common/storage-auth-aad-rbac-portal

https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-porta...

Please let me know if you tried this and faced any issue.

 

Thank you,

Harshitha Yadav.

[former_member761029]

Hi Harshitha,

I also wrote to Suman, who wrote this other blog that uses this same solution via Groovy, to see if he knows how to work around this error. If not, we'll have to try the REST api. Thank you for your reply and references.

[former_member761202]

Have you successfully used this method, if not would you be so kind as to point me in the direction of a guide you did use?

Many thanks.

[julian_baldner2]

How is it with PDF ? I tried, but PDF is empty or missing pictures.

[David_Oexner]

Hi Piotr,

I am trying to read csv files dynamically from the blob and the requirement is to move the file to different directory in the blob after reading. Could you share the complete code for pulling the csv files from the blob as a sample?

[former_member726476]

Hi,

We are trying to get attachments to Fi document in  SAP archived to azure blob. Unfortunately, this is not possible with usual content server config or are we missing something. Can you please put some light to this topic.