SAP Analytics Cloud – Monitoring All Model Import/Export Jobs

[mo_ezzat]

Introduction:

If you have a huge number of import models on your SAP Analytics Cloud tenant and wondering how you can monitor all of their import jobs from one place, you came to the right blog post.

In this blog post I will be explaining how you can create a one stop shop story, from which you check:

• When the job was executed.


• By which user it was executed.


• Whether it failed or succeeded.


• Which queries are used in which models (By following a certain logic).

Sample Story

Before we start, please note that there is already a notification feature available per each model, it can be switched on from the "Data Management" tab, through the toggle button labeled "Notify me of refresh failures by Email" at the bottom of the page, by activating this, you will be recieving an email for each import job that failed.

There is also the "Schedule Status" page that shows the status of all the import jobs, it can be accessed through from "Connections" then "Schedule Status" tab.

Schedule Status

Step by Step Guide:

Activate SAP Analytics Cloud Usage Tracking Content

Ignore this step if it is already active.

The SAP Analytics Cloud Usage Tracking Content provides useful insights about the activities, objects, files, and objects on your tenant. In our scenario we will be using the Activities and Objects data models.


The content can be activated by navigating to Content Network > SAP Analytics Cloud Usage Tracking Content > Adjust the import settings > Import.

This content is very useful in general, I have already used it before in a scenario where the customer wanted to check which models are used in which stories.

Create a new Story

Once the content is successfully imported, it will be available in Public/SAP_Content/SAP Analytics Cloud Usage Tracking Content.

An interesting feature that has been out for some time is that you now can choose a model and directly create a story based upon it, instead of creating a new story and choosing the initial model when an object is added.

Go to the specified path and choose the "SAP__SAC_USAGE_ACTIVITIES" model, navigate as per the screenshot below:

Choose between Responsive and Canvas based on your Preference.

Insert a table, in the Buildler Panel, click add Linked Models, navigate to the same path and choose the "SAP__SAC_USAGE_OTHER_OBJECTS" model, create the link based on Object ID, make sure you have selected description.

Linked Models

 

The story design is entirely based on what you need, I will be referencing the SAP Analytics Cloud objects and features I used in my scenario in the upcoming steps.

I used a responsive page, created 3 lanes, 1 top horizontal lane where a chart is placed, 2 vertical lanes with 1 table inside of each.

Story Filters

Model "SAP__SAC_USAGE_ACTIVITIES"

Timestamp

If you need the story to fetch data based on the current date, simply create a story filter based on "Timestamp", filter by range, set the granularity for Day.

Timestamp Range

In case you want to do a comparison between different time periods, do not set this as a story filter.

Object Type

Set its value to "Model".

Activity

Set its value to "Import", you can also select "Export" in case monitoring export jobs was desired.

Model "SAP__SAC_USAGE_OTHER_OBJECTS"

Description

Select "All Members", the usage of this filter can also extend to searching for the used queries as well, currently there is no way to retrieve the query related to a model, however I had the inspiration for this idea from a very clever customer I had, simply suffix or prefix the model description by the query name that is used, by doing this you can easily search for the query name in the description filter, and isolate all the models that use it.

Model Description

Description Filter

Comparison - Stacked Bar/Column

Placed in the top horizontal lane, based only on model "SAP__SAC_USAGE_ACTIVITIES", having in measures "Count of Activities" and "Status" in the Color, using a new pallet consisting of 2 Swatches Red, and Green.

Activated Linked Analysis based with Interactions apply to "All widgets on the Page", the purpose of this is to filter all other widgets based on the status in a visual manner.

The Models by Status Table

This table shows the Timestamp, Model description, User ID, Status, and the calculated measure flag.

 Model By Status

Note that:

• Only the Model Description is coming from the "SAP__SAC_USAGE_OTHER_OBJECTS" Model.


• Flag is a calculated measure, having the formula"IF([d/"SAP__SAC_USAGE_ACTIVITIES":STATUS_TYPE]="Success",
  1,
  IF([d/"SAP__SAC_USAGE_ACTIVITIES":STATUS_TYPE]="Fail",
  0,
  99
  )
  )"

It is created in order to be able to use the "Thresholds" feature, in order to show red in case of failure and green in case of success.

Thresholds

 

• From the "Styling Panel" you can choose how the thresholds can appear, in my case I left it as symbol, then set the font to transparent in order to hide the values.


• The Count of Objects measure from the other model is hidden by using the "Hide Column" feature.


• Linked Analysis has been applied to only affect the Navigate to Model table.

Navigate to Model Table

This model contains only the Description and Name ID, it was created to be able to directly jump to the relevant model using the hyperlinking feature, the reason why it needed a separate table is a current limitation on linking external content on a blended table.

This feature is very interesting, it can be used to jump to go to any link that has a static format until a certain value that you have in your report, lets have the current scenario as an example.

You can directly navigate to any SAP Analytics Cloud model by using this format https://<TENANT LINK>/sap/fpa/ui/app.html#/modeler&/m/model/<NAME ID>  to achieve this, right click any Dimension in the relevant table> Table Functions > Add > Hyperlink

Place the static part of the link, then in the relevant position, in our case the last part of the URL, use the drop down list to select the dimension supplying the values, in our case it's available in IN USE, choose Name ID.

External Hyperlink

The Label indicates the text that will appear when you choose the relevant row and click on "Navigate to External URL".

Navigate to External URL

Navigating to the URL can either be done through selecting a row, clicking Navigate to External URL, then selecting the Description that appears, or simply adding the relevant Dimension, in this case "Name ID" to the main table, then "Show Hyperlinks in Table", then it will turn into what is exactly shown in the screenshot above, and can be directly consumed.

Conclusion:

The story design is entirely based on your needs, it can be simpler or more complicated than this.

I hope this was informative, it was also a good chance to emphasize and introduce some features that exist and can be used in different ways on SAP Analytics Cloud.

Thank you for your time, please feel free to add any suggestions or feedback in the comments.

[ThomasK]

Great blog!

Two questions arise by myself:

a) Why is nothing similiar available out-of-the box?

b) Is there any possibility to get actively informed in case an import job fails?

[mo_ezzat]

Hi Thomas,

Thank you for your feedback!

Currently what is available out of the box is the "Schedule Status" available in the "Connections".

As for getting actively informed when an import job failes, toggeling the "Notify me on refresh failures by email" will send you an email like the one in the screenshot below:

Refresh failure in model XYZ

This will only be recieved if a specific job failes, and you can directly go to the model using the "View Model" button.

Best Regards,

Mohamed Ezzat

 

[ThomasK]

Thanks Mohamed!

Where can I find the option "Notify me on refresh failures by email"?

Is it just available for new models only or also for classic model (where I´ve currently imports running)? Further I also couldn´t find the toggle for dimension imports.

BR,

Thomas

[Henry_Banks]

Hi thkarner

you can find that setting at the bottom of the 'data management' screen (where the import jobs are scheduled). if memory serves me right, it exists for both old and new models.

Regards,

H

[ThomasK]

Thanks! How could I have missed that 🙂

[former_member432984]

Thanks Mohamed. Nice Job!!

I have a question: What do you mean with "Refresh Now"  button in Action col?  is this to relaunch the import  job again manually?

I'm so interested in being able to launch a specific import job from a button manually in case it fails..

Thanks.

[mo_ezzat]

Hi Jose,

Thanks for your feedback! Yes, this is possible from the "Schedule Status" window, in the Action Column you will find a button labeled "Refresh Now", it will manually trigger the import job once it is clicked.

Schedule Status

[former_member775956]

Hi Mohamed,

 

After creating all the steps we cannot able to see the Model ID's in the instead we can see Description, even turning Display options to ID and Description both are same no ID is showing as C********. Where as when we open the model we can see in the URL Model is called by C********.

 

Appreciate your suggestions here.

 

Regards,

Venkat.

[mo_ezzat]

Hi Venkat,

Please make sure you are using the "Name ID" Dimension from the model

"SAP__SAC_USAGE_OTHER_OBJECTS"

Table Details

Hope this helps.

Regards,

Ezzat

[former_member775956]

Hi Mohammed,

 

Thank you for your suggestions.

 

I have taken the model SAP__SAC_USAGE_OTHER_OBJECTS and didn't applied any link as you shown in screenshot, even though we are not able to see the ID's exactly starting with C*******.

 

We are facing a strange issue the models we created ourselves are showing as a members in NameID, but when we try to access log is there is null records we tried with linking both models too.

Log_Error

 

Appreciate your suggestions in advance.

 

Best Regards,

Venkata.

[former_member775552]

Hello Mohamed,

can you tell me WHO is going to receive the notification?

My import failed, but although toggled the button, there was no email.

Now, it is poosible, that we didnt't get one or we just looked at the wrong mailbox.

 

Thanks in advance:)

[mo_ezzat]

Hi Louisa,

It will go to the email assigned to the user you are using, you can see it by navigating to Security > Users.

Hope this helps.

BR,

Mohamed Ezzat

[mo_ezzat]

Hi Venkata,

Apologies for the late reply, I am not sure I understand your issue as you are covering the model IDs, can you please shed more light on the issue you are having?

BR,

Mohamed Ezzat

[tanisha_gupta20]

Is it possible to set up a Generic email id which will get a notification when the schedule job fails. A user may end up being on holiday sometimes and miss the job fail notification.

Thanks

Regards

Tanisha

[mo_ezzat]

Hi Tanisha,

There is no standard way to do this, however one workaround can be that you create a dummy user with a generic email address, use it to logon to SAP Analytics Cloud and enable the notification setting for the desired model.

Another solution is to check wheather you can use a distribution list email to create a user, use it to logon to SAP Analytics Cloud then enable the notification setting for the desired model. I have not checked it before, I believe that would be an alternative solution as you can add/remove whoever needs to be notified fromt he distribtution list itself.

Hope this helps!

Regards,

Ezzat

[tanisha_gupta20]

That is what I thought myself. Thank you for confirming Ezzat

Regards

Tanisha

 

[alebus]

Hi Mohammed,

do you know, if in the meantime there is some standard way how to send an automatic notification to several recipients in case an import job fails?

Alternatives you have already described to tanisha.gupta20...

Thanks & BR

Aleksej

[sk_acharjya]

Hello Mohammed Izzat ,

May i know why you have written the IF statement as

• Flag is a calculated measure, having the formula”IF([d/”SAP__SAC_USAGE_ACTIVITIES”:STATUS_TYPE]=”Success”,
  1,
  IF([d/”SAP__SAC_USAGE_ACTIVITIES”:STATUS_TYPE]=”Fail”,
  0,
  99
  )
  )”

Coz this formula throwing an error . 

What is the significance of 99 here ? We could write as simple

IF([d/"SAP__SAC_USAGE_ACTIVITIES":STATUS_TYPE]="Success", 1, 0 )

and then while creating threshold , will use greater than equal to 1 means success and <1 means fail .

Could you please help us to understand ? Thanks

[theofanis_hatzis]

Hello Mohamed,

This is a very interesting and inspiring article. What I would like to ask, is if it is possible to retrieve information in a similar dashboard as the one you have described for all the information that are appeared under Connections -> Schedule Status.

For example, I want in a story to retrieve, not only  all the Jobs that have an active/enabled schedule but also the Import/Export Job Name that has either Failed or Completed successfully, plus the Schedule timing per Import/Export job (which is shown under Data Management in each Model). The last one I understand that may need to be retrieved in another story. For example, we need to retrieve in a story all the Scheduled timings from all Models per Import/Export Job grouped by Model.

[mo_ezzat]

Hi Theofanis,

Thanks for your feedback. Right now this whole idea can be replaced by the "Admin Cockpit". What you are looking for is located on the "DM Job Statistics and Analysis Tab", navigate to "Statistics" you will find the time (this will be the last loading time) and load frequency.

Job Statistics

BR,

Mohamed Ezzat

[mo_ezzat]

Hi Suraj,

Sorry for the delayed reply, if i remember correctly there were other statuses that were irrelevant to either success of fail, 99 was to represent a unique value to exclude what I do not need.

 

BR,

Mohamed Ezzat

[mo_ezzat]

Hi Aleksej Busch,

Sorry for the delyaed reply, unfortunately so far I think not, I will reply back to your comment in case anything comes up.

BR,

Mohamed Ezat

[theofanis_hatzis]

Hello Mohamed,

 

This is really really helpful. But while looking at it, I noticed that column "Frequency" does not include information about the timing. It only includes info on how often the job is executed (e.g. Daily) and I would expect to say: Daily - Every day at 07:00:00 (as it is shown on my second screenshot of my initial question). This info that seems to be missing from this table, is something that can be retrieved from the Cockpit or somewhere else?

 

Thanks a lot

 

Fanis

[theofanis_hatzis]

Hello,

 

Additionally, to the above, I just noticed that the values of the Frequency column, seem not be aligned with what is shown in the Schedule of a Job, but seem to be produced automatically.

 

E.g. if a Job is executed either manually either on schedule every week, then this column shows Weekly, if it s executed either manually either on schedule every day, then this column shows Daily.

 

What I actually need, is a list of all the Jobs per model that are about to Run / or have already Run ONLY on Schedule (including timing) excluding those that run manually. Is that possible?

 

Thanks again.

 

Fanis

[mo_ezzat]

Hi Fanis,

Check the datasource itself, I would say check the schedule related fields first, unfortunately I am using a test tenant so I do not have any scheduled jobs, let me know if it works.

Model DM_SCHEDULE

BR,

Ezzat