SAP DI Structure File Consumers; specifying dynamic input filename/url for S3 retrieval

[former_member819188]

Hi all,

I'm fairly new to SAP DI; we're currently looking at options of retrieving S3 objects that are created by a Table Replicator (CDC) within a flow (which have timestamp added within their name), and joining these against a HANA table using a Data Transform. I think to do this we would need to read the created S3 objects first via a Structured File Consumer, but I cannot see a way of specifying the S3 source object path/filename here; it appears to require a constant Source path to be defined.

Can anyone advise whether its possible to specify source object paths programmatically in Structured File Consumer please, or if their may be alternative solutions to what we are trying to accomplish (performing a join of a HANA table against multiple outputs generated from CDC)?

Thanks,

Bryce

[michal_majer]

The structure (fields) of the files will be always the same?

[former_member609638]

hi michal,Bryce

Our requirement is to read multiple files with same format through structure file consumer operator.

Followed the above mentioned steps, and at runtime we are providing a wild card character *.csv so that all the files can be read from the same location.

Getting error, can we pass wild card characters to read multiple files?

Thanks,

Shubha

[patrobibeksap]

Hi Shubha, You can use Monitor file operator to read all file paths under directory and that can be fed into an Read FIle / Binary FIle consumer operator to read the content.

You can also use Python or Java script to feed the dynamic file name as input to Read operator which can get you the content.

Thanks