Purpose of the blog
Intelligent Scenario Lifecycle Management (ISLM), available as of S/4HANA2020, could unleash the potential value of Machine Learning (ML) capability in S/4HANA
because ISLM overcomes the obstacles which have blocked Machine Learning!
- Creating new ML model and adjusting standard ML model is possible with ISLM (successor of Predictive Analytics Integrator).
- Automatic ML model creation is possible, thanks to Automated Predictive Library (PAL).
- All toolset for ML model and App are prepared in S/4HANA: tools for cleansing the data for ML, for creating analytical app for ML model as well as for creating and training ML model leveraging S/4HANA embedded analytics capability.
The goal of this blog is to convince you of how easy you can enjoy standard / custom embedded ML app in S/4HANA!
Table of Content
- embedded ML in S/4HANA overview
- What can be predicted and how is it possible with embedded ML?
- Total picture of S/4HANA ML and embedded ML
- Main Functions of S/4HANA embedded ML
- Tools / Functions of ISLM
- Available Machine Learning Libraries and Algorithms in S/4HANA embedded ML
- Skillset to enjoy embedded ML APL scenario
- Concrete processes to create ML model and app with ISLM
- Technical objects of the ABAP Apply View (Output View) of the ML model
- Enjoy Standard embedded ML app
- Others
- Preparation of the Dataset for training ML is the key for success
- How to train the ML model?
- Further possible scenarios
1. embedded ML in S/4HANA overview
What can be predicted and how is it possible with embedded ML?
What can be predictied?
You can use standard app or create analytical app including
- Predicted number (e.g. predicted purchase amount of unknown customer)
- Predicted attribute (e.g. win/lost the deal).
How can it be predicted?
ML can predict by applying the tendency learned from the past data. You can find many web sites to understand Machine Learning. This means good data quality is mandatory for getting expected result of predicted value.
Case:
In the following case, Score and Pass/Fail for driver license test are predicted. Training data set is used to consider the logic to predict, and this logic is applied to “Apply Dataset” to predict the Score and Pass/Fail for the data.
Total picture of S/4HANA ML and embedded ML
Options of S/4HANA ML
When to use which option?
- Embedded ML: Data is in S/4HANA, Regression/Classification is enough
- Side by Side ML on BTP: ML services / Algorithms beyond APL/PAL are needed. Large data volume, external data sets from the hyperscalers such as AWS, GCP and Azure etc.
- Explorative Analytics via SAC: Smart Feature is needed, e.g. Smart Discovery, Smart Insight, Smart Predict, Search to Insight. Additional data sources apart from S/4 such as SFSF or Ariba and build quick explorative predictive models with SAC.
2 scenarios in S/4HANA ML
- Predefined standard ML Apps
- Custom ML model and app
Main Functions of S/4HANA embedded ML
- Automatic ML modeling without deep ML knowledge is possible with APL. More sophisticated ML model can be created with PAL. In the case of “How can ML predict?” above, Input fields (Gender, Region, Age, duration) have impact on the prediction, and normally how each input field impact has to be modelled manually with deep learning knowledge. But Thanks to APL, it is modeled automatically!
- Fiori analytical app can be used as the viewer of Predictive data
- Available with other apps in Fiori launchpad
- Data Transformation / Cleansing is possible in CDS View
- Standard ML model and app based on S/4HANA data model are provided.
- Virtual Data Model (VDM), CDS View template for S/4HANA data model is also provided
Tools / Functions of ISLM
- Intelligent Scenarios: To create Intelligent Scenario and ML model including CDS View data source (for training / for output view) and ML method (APL or PAL, Regression or Classification, etc..)
- Intelligent Scenario Management: To train ML Model (and output CDS View is generated automatically)
- What’s new in ISML, as of S/4HANA2020, compared to Predictive Models
- Successor of PAI, ISLM is used to create ML model or adjust standard ML model.
- Side by Side Intelligent Scenario can be created (Data Intelligence is needed) or additionally AI business services can be leveraged
Available Machine Learning Libraries and Algorithms in S/4HANA embedded ML
- Automated Predictive Library (APL) and Predictive Analysis Library (PAL) in HANA are available in ISML embedded ML.
- APL is far easier to use as deep knowledge of ML is not needed as APL designs the appropriate ML model automatically (amazing!).
- Many more algorithms are available in PAL, but deep technical and machine learning knowledge is needed.
- 2 ML algorithms available in APL.
- Regression: predicting number (e.g. test score predicted by the attributes of the student). This algorithm can take additional variable factors while doing the predictions.
- Classification: predicting characteristic value (e.g. pass/fail a test by the attribute of the student)
Skillset to enjoy embedded ML APL scenario
Understanding of S/4HANA data model and skill for S/4HANA Analytics (CDS View and Analytical app) are enough, and deep ML knowledge and complex technical knowledge are no longer needed. But you may have to do try and error to win more precise predicted value. It doesn’t depend on knowledge or expertise, but it is rather the issue of “Do it or not”. Analytics expert would be the best as they have already known S/4HANA analytics and many understand data model of S/4HANA.
2. Concrete processes to create ML model and app with ISLM
This blog uses the scenario in Best Practice Scope Item Embedded Machine Learning in S/4HANA (55Z) to explain how to create embedded ML model. This blog explains total picture of the sample scenario of this Scope Item. Please see detail about each step in the Configuration Guide of this Scope Item,
In the CAL instance, they have already been set up!
Other than this exercise, this movie includes demos of ISLM on embedded scenario and side by Side scenario using DI (from 21:40), as well as the lecture of S/4HANA ML.
Goal
The analytical app including predicted “Cleaning Days” (Document Date – Cleaning Date) can be created.
Preparation
To work with ISLM ensure that all pre-requisites are complete.
Please refer details in following document:
https://help.sap.com/viewer/product/SAP_S4HANA_ON-PREMISE/ (Versions: 2020)
Search Keywords: Introduction to Intelligent Scenario Lifecycle Management
Go to Introduction to Intelligent Scenario Lifecycle Management ® Pre-Requisites , which includes the following configurations
- Initial Setup
- Installing components (APL, AFL (incl PAL), PAi,
- Assign Roles to HANA DB User or ABAP technical user
- Run technical Job (Transaction SJOBREPO)
- Configuration Information
- Setting up the OData Services
- Setting up the bgRFC Queue
- Configuring Business Roles for the Backend
- Configuring Business Roles for the Frontend
- Access the SAP FIORI apps for ISLM
Overview of the process
For training ML model, data source, called Training Dataset, is needed, and the data to which the prediction is applied has to be prepared, which is called Apply Dataset. Both have to be created as CDS View manually.
Then the ML data model, called Intelligent Scenario, is created in which Library (APL), algorithm (Regression) is defined as well as setting Training Dataset and Apply Dataset with the tool Intelligent Scenarios.
After that, the model is trained to create output view, called ABAP Apply View. it is also CDS View.
In the final step, the Fiori application (KPI Tile Generic Drill down app) is created. For doing that, the CDS view data source for the app is created in which ABAP Apply View is used as the source.
Steps in detail
- Create Custom CDS Views as the data source of Intelligent Scenario / ML Model.
- Create Intelligent scenario / ML model with Intelligent Scenarios using the created CDS Views.
- Train the ML model with Intelligent Scenario Management and output CDS View is generated.
- Create Custom CDS View as the source of Custom Fiori app.
- Create Fiori app using the created Custom CDS View.
Detail steps are described in the configuration guide of the Scope Item 55Z. Here the brief steps are shown.
1. Create Custom CDS Views as the data source of Intelligent Scenario / ML Model.
For creating ML model, 2 CDS Views are needed, for Training and for Apply (the source of the output view).
- Training Dataset: Z_OPACDOIT_TRAIN
- Apply Dataset: Z_OPACDOIT_APPLY
Both views must have the same structure (the same keys and fields) except Key (Cleaning Days), which doesn’t have to be in Apply Dataset.
In this example, the data source of both views is VDM I_OperationalAcctgDocItem. Difference between both views is only WHERE condition. Z_OPACDOIT_TRAIN includes “DocumentDate < 20200301” and Z_OPACDOIT_APPLY includes “DocumentDate >= 20200301” instead. For predicting Cleaning Days, the model learns from “Training Dataset” (DocumentDate < 20200301), and predict the Cleaning Days of the “Apply Dataset” (DocumentDate >= 20200301). In this example, in reality, Apply Dataset has real Cleaning Days, but in the ML App, it is not used but predicted Cleaning Days is used instead.
Sample source code of the CDS Views are in the Configuration guide of Scope Item 55Z.
They do not have to be published as OData service.
2. Create Intelligent Scenario / ML model with Intelligent Scenarios using the created CDS Views.
2 tools are provided in ISLM, Intelligent Scenarios and Intelligent Scenario Management, and Intelligent Scenarios is used to create Intelligent scenario / ML model. The role SAP_BR_ANALYTICS_SPECIALIST has to be assigned to use the tools.
Create ML model “Z_PAYMENT_FORECAST as below.
Intelligent Scenario type: Regression
Add Model: APL
Model:
Training Dataset: Z_OPACDOIT_TRAIN
Apply Dataset: Z_OPACDOIT_APPLY
Target: CLEANINGDAYS (The value to be predicted with ML algorithm)
Keys: (Keys in the Training Dataset are set automatically)
Apply Output configuration: check Score, Confidence, Percentile (all)
Score is the predicted target value (Predicted CLEANINGDAYS)
After running “Publish”, the model is generated.
Input:
Same as Training Dataset. Input means the fields which impacts the target value.
Output:
Data source is the keys of Apply Dataset CDS View, in which Keys, (target), Score, Confidence, Percentile are included. As Score, Confidence and Percentile are checked when creating the model, those the field of Score, Confidence and Percentile are added in the output view.
Score: RR_CLEARINGDAYS
Confidence: BAR_RR_CLEARINGDAYS
Percentile: QUANTILE_RR_CLEARINGDAYS_100
Model:
Note: New Model can be added (“Add Model”) in which different setting is possible, e.g. changing the Training Dataset, Apply Dataset.
Apply Settings:
ABAP Apply Class is generated automatically for the ML model.
ABAP Apply View is the CDS View of the output. It will be generated after training the model, so now it says “Apply View not available”.
3. Train the ML model with Intelligent Scenario Management and output CDS View is generated.
Train the model with Intelligent Scenario Management.
Select the created Intelligent scenario. You can filter the data for training, e.g. ~2020/12.
When running training, the status is changed to scheduled -> Training -> Ready. Now the model version is created. It takes some time.
After trained (Status = Ready), you can see the result.
You can find
Quality is 5 stars (good!). The influential fields are GLACCOUNT, NETNEWDUE, POSTINGKEY…
Then this model version can be activated. Status is changed to Activating -> Active.
After Activated, ABAP Apply View is generated. You can find it in Intelligent Scenarios.
Definition of the ABAP Apply View Z_PAYMENT_FORECAST_CDS01 (CDS View):
This means the definiton of the CDS View is changed after the training. The training can be run in Production system.
4. Create Custom CDS View as the source of Custom Fiori app.
Subsequent steps are the same as the normal way to create analytical app. In this blog, the app to be created is Fiori KPI tile Generic Drilldown app.
Firstly, the backend service CDS View has to be created for the app using the generated ABAP Apply View as the source.
Some fields are added in the view: DocumentDate, GLAccount, CompanyCodeCurrency.
As the app is Generic Drilldown app, it has to be exposed as OData Service.
@AbapCatalog.sqlViewName: 'ZCPAYMNTFRCST'
@AbapCatalog.compiler.compareFilter: true
@AbapCatalog.preserveKey: true
@AccessControl.authorizationCheck: #CHECK
@EndUserText.label: 'Predictive: Payment Forecast'
@OData.publish: true
@Analytics.dataCategory: #CUBE
define view ZC_PAYMENT_FORECAST as select from Z_PAYMENT_FORECAST_CDS01 F
association[0..1] to I_CompanyCode as _CompanyCode on F.CompanyCode = _CompanyCode.CompanyCode
{
@ObjectModel.foreignKey.association: '_CompanyCode'
CompanyCode,
AccountingDocument,
FiscalYear,
AccountingDocumentItem,
_CompanyCode,
_CKE_toBase.DocumentDate,
_CKE_toBase.GLAccount,
_CKE_toBase.CompanyCodeCurrency,
@Semantics.amount.currencyCode: 'CompanyCodeCurrency'
@DefaultAggregation: #SUM
_CKE_toBase.AmountInCompanyCodeCurrency,
ClearingDays,
Rr_ClearingDays,
Bar_Rr_ClearingDays,
Quantile_Rr_ClearingDays_100,
/* Associations */
_CKE_toBase
}
if you want to use Custom CDS View to create CDS View, the generated CDS View has to be released. See more in the blog about the release.
The OData Service for this view has to be added with Transaction /iwfnd/maint_service in SAPGUI.
Select “Add Service”, then add this OData service. See more in this blog.
5. Create Fiori app using the created Custom CDS View.
In this blog, the Table view of the Generic Drilldown app is created using the OData Service created in the previous step with the tool Manage KPIs and Reports. See more in this blog.
The created app:
This is only an example. Many other app can be created using this custom CDS View, e.g. List Report, Analytical List Page, Web Dynpro Grid app. For creating Web Dynpro Grid app, Interface view (@Analytics.dataCategory:#CUBE or #DIMENSION) and Consumption View (@Analytics.query: true) have to be created.
3. Technical objects of the ABAP Apply View (Output View) of the ML model
Apply View (output view) “Z_PAYMENT_FORECAST_CDS01” is based on Table Function “Z_PAYMENT_FORECAST_TF01” (source AMDP Class-Method is “Z_PAYMENT_FORECAST_TF01_CL=>GET_DATA”), and ABAP Apply class “/1ISLM/CL_Z_PAYMENT_FORECAST=>APPLY_PRIMARY” is called in it. In the class, HANA Function, Procedures and APF procedure are generated and called internally.
Apply View
Table Function Z_PAYMENT_FORECAST_TF01
AMDP for the Table Function: Z_PAYMENT_FORECAST_TF01_CL=>GET_DATA
AMDP (Apply ABAP Class): /1ISLM/CL_Z_PAYMENT_FORECAST=>APPLY_PRIMARY
HANA Function: /1BCAMDP/ISLM_PS_TF_Z_PAYMENT_FORECAST
This case uses PAL. But Scope Item 55Z includes the exercise on embedded ML using PAL. ABAP Class has to be created in which PAL procedure in HANA is called.
4. Enjoy Standard embedded ML app
Standard Fiori Analytica apps are provided in S/4HANA, in which value predicted by ML is included (analytical ML apps).
How to use the analytical ML app? Only after training with Predictive Models or Intelligent Scenario Management, the predicted figure is displayed in the corresponding analytical app.
The list of the ML apps is in the Configuration Guide of Best Practice Scope Item 55Z and some are in this blog. Best Practice Scope Item is provided for each ML app in which normally only Test Script is included, and in the test script, you can understand how to operate and relevant objects, e.g. Name of Model, app, role, etc..
Example:
- App: Quotation Conversion Rates (F1904)
- Scope Item: Predictive Analytics Model Training - Sales (2YJ)
- the app Predicted Delivery Delay - No. of Order Items (F3408) also included.
- Predictive/Intelligent Scenario: SLSQTANPREDICTION
- Role: SAP_BR_INTERNAL_SALES_REP
- Data source: C_SLSQTANCONVERSIONRATEQ
- The data source in which predicted figure is included: P_SLSQUOTATIONITEM_PREDICT
- ABAP Class: CL_SLS_QUOTATION_PREDICTION
Quotation Conversion Rate (F1904): Predicted Value: Conversion Rate.
As the design of the generated output view (ABAP Apply View) depends on the data, which is different in each company, you would have to adjust the standard ML model with ISLM to get more precise predicted value.
See detail about this app in this blog.
In S/4HANA2020, predefined predictive scenarios can be found with the tool “Predictive Scenarios” (older version of “Intelligent Scenario”) as below. Predictive Scenarios can be trained with the tool “Predictive Models”. The standard ML models will be migrated to Intelligent Scenarios in S/4HANA2021.
5. Others
Preparation of the Dataset for training ML is the key for success
Excellent teacher is mandatory for training!
- Data for training would have to be changed for making ML model understood.
- In addition, dirty data would have to be cleansed.
Nothing could be born from dirty data. Data for training would have to be transformed and cleansed, dynamically in CDS View, or physically and stored in the custom table.
The most part of the effort would have to be taken for this task for embedded ML with APL.
Possible Cases to make the data ready for ML:
Change the attribute value in Training Dataset to make ML model understood
- From Calendar Year/Month to Calendar Month (e.g. 202005 -> 05) to reflect seasonal change to the predicted value.
- From birthdate to age (e.g. 19850607 to 36)
- From document data and shipping data to shipping days (20200501 and 20200520 to 20)
Cleanse the dirty data
- Remove records of exceptional value in Training Dataset View
- Master data code identification (3 different code for one customer -> grouping into one code).
- Remove some input fields in Training Data set View (remove customer name field)
You may have to do Try & Error and recreate Training Dataset view as you could not find how the Training Dataset should be designed and adjusted only after checking the result of the trained data model.
Create Training dataset -> create Model and get the result -> adjusting training data -> create Model and get the result ->…
Just for this background, it is important to integrate data when moving to S/4HANA, and have the policy and process to keep the data consistent.
How to train the ML model?
In which system (DEV, QAS or PRD) should the training be done?
Training can be done on any system. Please access the ISLM app – “Intelligent Scenario Management” as an “Analytics_Specialist” to create training data for a particular ML model. You can then train and activate a version. Whichever version is active is leveraged in the actual application when called from the business process.
How frequently should the training be done? Monthly? Weekly?
This depends on how good the data is and how good the models are. Whenever you train your data, you will get a predictive power and predictive confidence. These should be greater than 0.9 but less than 1 for better results.
Data size for the training
Data size for the training depends on the quality. The data quality, tendency, condition of the training dataset are expected to be similar to those of the "Apply dataset". If you want to predict the year 2021, but the data quality of year 2019 is so dirty, it would be better to use the data of year 2020 only for the test.
Can the training be run as job (not manual operation)?
For embedded ISLM approach, we are currently running the training using the “Intelligent Scenario Management” app. This is mainly a manual operation but you could also run it as a training job.
Further possible scenarios
- Create SAC Story based on the CDS View with predicted value.
- Use external data with S/4HANA ISML by uploading the data into custom table or BW provider with staging function of embedded BW in S/4HANA.
- Extract data from the CDS View with predicted value and send other systems, e.g. BW/4HANA.
- Read the predicted value (CDS View) in ABAP Program.
- Create API including predicted value and call it in the Side by Side App in BTP.
Enjoy Machine Learning in S/4HANA!
- SAP Managed Tags:
6 Comments
