Cloud Integration: How to access nextLink using message flow

[THEN]

Hello all,

I've developed an integration flow which fetches the data from the CRM system and writes it to a query log. However, the source system only provides 5000 records and a nextLink that points to the next x-thousand records. Unfortunately, I have no idea how to access the nextLink from the XML to fetch the next x-thousand records until there are none left. Can anyone tell me how to proceed here?

Cloud integration - message flow:

ODATA.NextLink

[katherresh_shanmugasundar]

Hi Torben,

This kind of paginated extract is quite common now a days. One way of resolving this would be

1. Have 2 Main integration processes
2. First process should execute the initial query, fetch first 5000 records, process and post to the target. In addition this process should extract the next link URL & update it to a Data Store
3. The second process should have a Data Store sender adapter which would extract data using this next link & fetch next 5000 records, process and post to the target. And also extract the next link URL & update it to the Data Store
4. Step 3 will be recursive and it will write to the data store until next link has a value
5. The common steps should be in a local integration process which should be called in both Main processes

And the length of next link should be less than 255

Best Regards,

Katherresh

[THEN]

Dear Katherresh,

thanks for the information. This helped me a lot - I was able to write a script which passes identifies the nextLink. However, if I try to copy that Link into the ODATA query, I can't execute the process (I think the issue ist the $skiptoken) anymore. My code looks like this:

Everything before the "$skiptoken" is the same and is executable. Am I doing something wrong here or does the cloud integration service doesn't support the $skiptoken-parameter?

[katherresh_shanmugasundar]

Hi Torben,

Hopefully you were able to resolve this already. Skip Token is basically a pointer for the next set of records to be selected & it is supported by SAP Cloud Integration as it is just a parameter.

If the issue still persists then it might be because of session/ cookie handling and re use.

Best Regards,

Katherresh