SAP Continuous Integration and Delivery for SAP In...

former_member3947 · ‎08-27-2021

With its latest release, SAP Continuous Integration and Delivery provides a predefined pipeline for SAP Integration Suite Artifacts. In this blog post I would like to show you the steps how to use this new functionality and configure a CI/CD pipeline for the development of integration content in SAP Cloud Integration.

Prerequisites

You have enabled SAP Continuous Integration and Delivery. See Enabling the Service.

You are an administrator of SAP Continuous Integration and Delivery. See Assigning Roles and Permissions.

You have provisioned the Cloud Integration capability as part of SAP Integration Suite. See Initial Setup of SAP Cloud Integration in the Cloud Foundry Environment.

You've created the Process Integration Runtime service instance of plan api and a service key to authenticate your pipeline against SAP Cloud Integration. See Setting Up OAuth Inbound Authentication with Client Credentials Grant for API Clients, Cloud Foundry....

Procedure

First, you have to transfer your integration artifacts from SAP Cloud Integration to a repository of your source code management system. To do this, perform the following steps:
- In SAP Cloud Integration, open your integration package and choose  Artifacts.
- Choose Actions→ Download.
- Save the *.zip file to your choice of destination.
- Extract the file and upload its contents into your repository.
  
  Note: Repeat the procedure in case you make changes in your integration flow in SAP Cloud Integration.

In SAP Continuous Integration and Delivery, configure a new job as described in Create a Job. As Pipeline, choose SAP Integration Suite Artifacts.

In this demo, I will use Job Editor from the Configuration Mode dropdown list.

In the Stages tab, specify the general parameters. Then switch on and configure the stages you want to execute.

Don't forget to save your job.

Of course, if you prefer, you can use the Source Repository Option for configuring your job.
In that case you can follow our documentation instead.

After you have created the job, you can either run it manually, or you can create Webhook to automate your builds.

The picture below shows the result of a job run. In this view, you can monitor its outcome.

Documentation & Further Readings

If you have found this article useful and want to learn more and use the SAP Continuous Integration and Delivery to manage your SAP Integration Suite Artifacts, take a look at the following resources:

Our Product Documentation - SAP Continuous Integration and Delivery Documentation

A very useful blog post containing information for the different scenarios - CI/CD for SAP Integration Suite? Here you go!

Not what you have expected?

If you think that we’ve missed important use cases, please help us improve our offering by leaving a note to this blog or submit your idea in SAP Business Technology Platform Continuous Influence Program.

srini_reddy · ‎08-27-2021

Hello irina.kirilova

I have a few questions after going through the documentation.

Does this use any integration artifacts from the source code repository or pulling the artifact from the Integration Suite instance?

Do I need to create multiple pipelines to deploy multiple iflows or Can I deploy multiple iflows from the same pipeline?

How to handle iflow configuration parameters in the pipeline?

Any plans to support Azure Repos for the webhooks?

Thanks,

Srini

former_member3947 · ‎08-30-2021

Hi Srini,

The service takes the artifatcs from a Git Repository. In the procedure above, I have described one possibility how to do that.
You can deploy one flow per pipeline. In case you have multiple flows in one package, you have to create multiple pipeline for them.
Currently not supported in the service. In case you need more flexibility in your pipeline, you could consider using Project "Piper" which covers this feature already - https://www.project-piper.io/steps/integrationArtifactUpdateConfiguration/
We get frequent customer requests for this, so we are considering the feature for 2022.

Best regards,

Irina

RSAB · ‎08-30-2021

Hi,

I have one question related to SAP CPI DS.

Question : CPI-DS option is getting covered in SAP BTP Integration suite?

Please clarify.

Thanks, Ramoji

mayurbelur_mohan · ‎08-30-2021

No CPI-DS is not part of SAP BTP Integration suite.

ManikandanR · ‎10-13-2021

Hi Irnia,

Upload step is failing when I include the Upload step in the Job. If I only include the Deploy step it is running successfully.

I tried with and without having the Integration Flow in the package in my trial CF account. In both cases Upload step is failing.

<error>

<code>Bad Request</code>

<message xml:lang="en">INVALID_INTEGRATION_PROJECT_NO_MANIFEST</message></error>

Log looks like as follows:

fatal integrationArtifactUpload - step execution failed - HTTP PUT request to https://****/api/v1/IntegrationDesigntimeArtifacts(Id='testflow',Version='Active') failed with error: <?xml version='1.0' encoding='UTF-8'?><error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"><code>Bad Request</code><message xml:lang="en">INVALID_INTEGRATION_PROJECT_NO_MANIFEST</message></error>: Request to https://****/api/v1/IntegrationDesigntimeArtifacts(Id='testflow',Version='Active') returned with response 400 Bad Request

[2021-10-13T09:46:41.471Z] + ./piper readPipelineEnv



But the zip in the Git Hub has the correct Manifest file.

mayurbelur_mohan · ‎10-14-2021

Hi Mani,

Upload Steps works only with integration flow archive but not integration package archive. are you

sure you are using right archive(i.e. Integration flow)?

Best Regards,

Mayur

former_member75928 · ‎10-14-2021

Hi Mayur,

Yes thats the Integration Flow archive only, not the package archive. Still it is failing as above.

fatal integrationArtifactUpload - step execution failed - HTTP PUT request to https://****/api/v1/IntegrationDesigntimeArtifacts(Id='testflow',Version='Active') failed with error: <?xml version='1.0' encoding='UTF-8'?><error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"><code>Bad Request</code><message xml:lang="en">INVALID_INTEGRATION_PROJECT_NO_MANIFEST</message></error>: Request to https://****/api/v1/IntegrationDesigntimeArtifacts(Id='testflow',Version='Active') returned with response 400 Bad Request

[2021-10-13T09:46:41.471Z] + ./piper readPipelineEnv



But the zip in the Git Hub has the correct Manifest file.

mayurbelur_mohan · ‎10-14-2021

is this only iflow archive this issue is happening or for all iflow archives same issue exist?

virender123 · ‎11-16-2021

Hello Irina,

As discussed on our call last month, do we have some video handy for the same. Its not very clear with the steps and getting problem in replicating the same.

Thanks in advance !

Virender Singh

Peter99084 · ‎08-15-2022

The link create Webhook is broken.

Thanks, Peter

rasjoshi · ‎09-03-2022

Hi,

Thanks for the blog. I wanted to know how to pass input file for testing?

Also how upload feature works? does it import iflow to prod tenant?

is there any method to upload iflows to github repository rather than manual download n upload?

Regards,

RJ

LindaSieb · ‎09-05-2022

Hi Peter,

Here is the correct link to the documentation for creating a Webhook:

https://help.sap.com/docs/CONTINUOUS_DELIVERY/f3d64e9188f242ffb7873da5dfad4278/a273cffe863b4663b2394...

I will talk to Irina about fixing the link in the blog post.

Best regards,

Linda

former_member3947 · ‎09-12-2022

Updated. Thanks, Linda!

Jitendra_Jeswan · ‎03-02-2023

Hello irina.kirilova mayurbelur.mohan lindasieb Please advise.

Pt. 1- I understand that behind CICD service, SAP is calling Piper library steps.

Pt. 2- My assumption is that Input file for Integration Test Step, need to be maintained in configured repository itself for test to get successful.

Pt. 3-I understand we need to create multiple pipelines (Jobs) per IFlow & per Environment.

Example: If we have 1 IFlow & we need to execute the pipeline in 3 Environments, we need to create 3 Jobs (Pipelines) for each environment as the service key (/authentication) would be different for each Tenant. Similarly additional Jobs for additional IFlows.

Pt. 4-My understanding is there is no OOTB feature provided as of today to Push the Tenant Artefacts to the Repository directly and we need to upload / Push manually any changes from Tenant into repository.

Looking forward to your response. Thx.

LindaSieb · ‎03-08-2023

Hi Rashmi and Jeswani,

I can't answer all of these questions but I can answer a few! 🙂

To pass the input file to the test you can use the "Message Body File Path" parameter in the Job Editor. You can put the path to your file there. This file needs to be in your source repository.

Yes, 1 job is needed per each integration flow. And right now we have to manually download the integration flow and upload it into GitHub, there are no separate tools for this.

Hope that helps!

Linda Siebert

ankush10990 · ‎03-27-2023

Hello linda.abell , irina.kirilova mani.rajendran :Could you please help here, I am trying to follow the same steps however getting below issue in upload step:

[2023-03-27T12:47:08.881Z] info  integrationArtifactUpload - [DEBUG] GET https://trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesigntimeArtifacts(Id='hg',Version='Active')

[2023-03-27T12:47:08.893Z] error integrationArtifactUpload - Requested resource ('https://trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesigntimeArtifacts(Id='hg',Version='Active')') could not be found

[2023-03-27T12:47:08.905Z] info  integrationArtifactUpload - [DEBUG] POST https://trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesi...

[2023-03-27T12:47:08.916Z] error integrationArtifactUpload - Requested resource ('https://trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesigntimeArtifacts') could not be found

[2023-03-27T12:47:08.929Z] error integrationArtifactUpload - a HTTP error occurred! Response body: [52 48 52 32 78 111 116 32 70 111 117 110 100 58 32 82 101 113 117 101 115 116 101 100 32 114 111 117 116 101 32 40 39 116 114 105 97 108 116 101 115 116 45 111 120 102 97 52 114 51 97 46 105 116 45 99 112 105 116 114 105 97 108 48 53 45 114 116 46 99 102 97 112 112 115 46 117 115 49 48 45 48 48 49 46 104 97 110 97 46 111 110 100 101 109 97 110 100 46 99 111 109 39 41 32 100 111 101 115 32 110 111 116 32 101 120 105 115 116 46 10], Response status code: 404

[2023-03-27T12:47:08.942Z] info  integrationArtifactUpload - fatal error: errorDetails{"category":"undefined","correlationId":"n/a","error":"HTTP POST request to https://trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesi... failed with error: 404 Not Found: Requested route ('trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com') does not exist.\n: request to https://trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesi... returned with response 404 Not Found","library":"SAP/jenkins-library","message":"step execution failed","result":"failure","stepName":"integrationArtifactUpload","time":"2023-03-27T12:47:08.773521894Z"}

[2023-03-27T12:47:08.953Z] fatal integrationArtifactUpload - step execution failed - HTTP POST request to https://trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesi... failed with error: 404 Not Found: Requested route ('trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com') does not exist.

[2023-03-27T12:47:08.965Z] : request to https://trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesi... returned with response 404 Not Found

[2023-03-27T12:47:08.978Z] info  integrationArtifactUpload - Step telemetry data:{"StepStartTime":"2023-03-27 12:47:08.593669706 +0000 UTC","PipelineURLHash":"50f013532a9770a2c2cfdc38b7581dd01df69b70","BuildURLHash":"50f013532a9770a2c2cfdc38b7581dd01df69b70","StageName":"Upload","StepName":"integrationArtifactUpload","ErrorCode":"1","StepDuration":"180","ErrorCategory":"undefined","CorrelationID":"n/a","PiperCommitHash":"8084ce1a9464d8c63ea866c0750a07e54fc8e147","ErrorDetail":{"category":"undefined","correlationId":"n/a","error":"HTTP POST request to https://trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesi... failed with error: 404 Not Found: Requested route ('trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com') does not exist.\n: request to https://trialtest-oxfa4r3a.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesi... returned with response 404 Not Found","library":"SAP/jenkins-library","message":"step execution failed","result":"failure","stepName":"integrationArtifactUpload","time":"2023-03-27T12:47:08.773521894Z"}}



I am stuck at this point, could someone please help with this.

Appreciate your help!



Thanks

former_member3947 · ‎04-03-2023

Hello Ankush,

please use Support Channel to request official support for your issue.

Thanks and regards,

Irina

former_member37947 · ‎04-03-2023

Check the service key type.

You may be using a "integration-flow" SK instead of an "api" SK.

Also check the SK has the following roles:

"MonitoringArtifactsDeploy",
"WorkspaceArtifactLocksDelete",
"WorkspaceArtifactLocksRead",
"WorkspaceArtifactsDeploy",
"WorkspacePackagesConfigure",
"WorkspacePackagesEdit",
"WorkspacePackagesRead",
"WorkspacePackagesTransport",
"MonitoringDataRead"

Some of them may be unnecessary, but it gets the pipeline working

Best regards

Mikel

samudralasreenivas2132 · ‎12-12-2023

Hi Irina Kirilova,

Thanks for making the blog.

I am facing the below error while build is triggered. Can you help me out!

[2023-12-11T10:12:07.928Z] ----------------------------------------------------------

[2023-12-11T10:12:07.928Z] --- An error occurred in the library step: integrationArtifactUpload

[2023-12-11T10:12:07.928Z] ----------------------------------------------------------

[2023-12-11T10:12:07.928Z] 

[2023-12-11T10:12:07.929Z] The following parameters were available to the step:

[2023-12-11T10:12:07.929Z] ***

[2023-12-11T10:12:07.929Z] *** to show step parameters, set verbose:true in general pipeline configuration

[2023-12-11T10:12:07.929Z] *** WARNING: this may reveal sensitive information. ***

[2023-12-11T10:12:07.929Z] ***

[2023-12-11T10:12:07.929Z] 

[2023-12-11T10:12:07.929Z] The error was:

[2023-12-11T10:12:07.929Z] ***

[2023-12-11T10:12:07.929Z] hudson.AbortException: [integrationArtifactUpload] Step execution failed (category: undefined). Error: HTTP POST request to https://3fdebddftrial.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesigntim... failed with error: 404 Not Found: Requested route ('3fdebddftrial.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com') does not exist.

[2023-12-11T10:12:07.929Z] : request to https://3fdebddftrial.it-cpitrial05-rt.cfapps.us10-001.hana.ondemand.com/api/v1/IntegrationDesigntim... returned with response 404 Not Found

[2023-12-11T10:12:07.929Z] ***

[2023-12-11T10:12:07.929Z] 

[2023-12-11T10:12:07.929Z] Further information:

[2023-12-11T10:12:07.929Z] * Documentation of library step integrationArtifactUpload: https://sap.github.io/jenkins-library/steps/integrationArtifactUpload/

[2023-12-11T10:12:07.929Z] * Source code of library step integrationArtifactUpload: https://github.com/SAP/jenkins-library/blob/master/vars/integrationArtifactUpload.groovy

[2023-12-11T10:12:07.929Z] * Library documentation: https://sap.github.io/jenkins-library/

[2023-12-11T10:12:07.929Z] * Library repository: https://github.com/SAP/jenkins-library/

[2023-12-11T10:12:07.929Z] 

[2023-12-11T10:12:07.929Z] ----------------------------------------------------------

[2023-12-11T10:12:07.929Z] --- End library step of: integrationArtifactUpload ---





Thanks,

Sreenivas.