Process mining is increasingly becoming the holy grail when it comes to finding solutions to meet current business challenges such as unreliable supply chains, rising costs for energy, and more sustainable resource management. But as in any analysis project, the success depends on the data foundation and how fast, clean, and reliable data is available.
This blog is about how SAP Signavio Process Intelligence (Process Mining) works together with and SAP Data Warehouse Cloud and how Process Mining projects can be accelerated and optimized in several aspects, like:
Additionally, this blog uses a simplified real-life scenario to also give the necessary business context and use cases which hopefully makes it easier to understand, crisper and more enjoyable to read.
Last remark – if you are new to either SAP Signavio or SAP Data Warehouse Cloud you´ll find at the end of this blog links to other resources and short introduction videos.
The scenario:
First let´s have a look on to the use case and the goals of the scenario. One part of the SAP Signavio Transformation Suite is the so-called Process Intelligence module. With this module you can analyze data which is produced through business processes like Order-to-Cash, Hire-to-Retire, Plan-to-Fulfill and many more. These processes are the life blood of a company because they are directly connected with the value chain, the better they are managed the more efficient and resource-saving they are.
For our example we will focus on the Order-to-Cash process of a fictitious T-Shirt Selling Company, which you might can relate to the SAP ERP Sales and Distribution (SD) module.
Let´s assume the End-to-End-Process on high level looks like the following:
Behind the outbound delivery step there is at least one important decision to take: either they can directly ship an ordered t-shirt from the warehouse (if in stock), or they need to produce the imprint of the t-shirt first. You can imagine that a printing process will have an impact on the delivery time for the customer and on the cycle time of a process at all. Such decisions are leading to commonly named process variants, because each decision opens new paths – like on a street map you can take various roads to get to a destination.
One of the goals of process mining and especially in this scenario is to analyze the process variants and identify bottlenecks, loops, or other inefficiencies during the Order-to-Cash process. This helps on the one hand to understand why cycle times are too high and on the other hand where lots of process “energy” is wasted due to wrongly used paths or other waste of time.
How this looks like for our T-Shirt company you´ll see on following pictures:
The pictures on the left-hand side show the so-called Happy Path – this means most of the cases (orders) are following this route and they need an average consumption of time (cycle time) between the steps. On the right-hand side we can see a variant caused by
the decision if a T-Shirt can directly be shipped from the warehouse or if the T-Shirt needs to be printed first. Of course, we can also see the impact on the cycle times. Another inefficiency seems to happen in the shipping process, sometimes it takes several attempts to deliver the goods.
Ok, so now that we have an overview about the scenario lets peek under the hood how the data for process mining looks like and how SAP Data Warehouse Cloud accelerates Process Mining Projects.
Speed up the data acquisition and connectivity setup
Of course, first we must put our nozzle into the systems in which the processes are executed, and these are the ERP-, CRM-, HR- and many other systems. In our scenario we need to connect to SAP ECC on premise to get the respective data. SAP Data Warehouse Cloud offers out-of-the-box connectors to different applications and data sources (current list see here).
With the SAP EEC connection you can get direct data access to your SAP system and can decide in which mode you want to retrieve data. Either you will replicate the necessary tables to SAP Data Warehouse Cloud, or you can use the remote tables (virtual access) to reduce data movements and separate storage. Sometimes you even get a free ride because it is more than likely that in the context of a SAP Data Warehouse Cloud project this connection has already been used for other analytical questions. In such case the sharing concept within the spaces in SAP Data Warehouse Cloud will reduce effort and additionally speed up the data acquisition.
For detailed information I recommend reading these nice blogs.
SAP Data Warehouse Cloud: Introduction to the ABAP-Adapter
SAP Data Warehouse Cloud: Capabilities of the ABAP-Adapter Connection to on-premises S/4HANA & ECC
Simplify data preparation and increase low/no-code usage
Another challenge you might face in Process Mining project is to prepare the data into the right format (process data model). As described in the scenario we want to analyze an Order-to-Cash process and you can image there are many steps to perform from creating an order, organize the shipment until you can do an invoice clearing after you have received the payment from the customers. These steps are called events, which are necessary to execute a case (one order) from beginning to end and are the foundation to calculate cycle times or identifying loops and other inefficiencies. The second ingredient are the attributes (in BI Projects you might also say dimensions) which describes a case to which organization, country, customer, customer group, product, supplier etc. it belongs. These attributes help to categorize and subsume similar cases to find dependencies like all orders of a specific type, product group etc. performing better than form other groups.
Data is the keyword - now the SAP Data Warehouse Cloud comes again into play. In SAP Data Warehouse Cloud you can not only benefit from the connectivity, but with the Data Builder many useful functions are available to prepare and wrangle the data. That includes e.g., joins, unions, calculations, filters, intermediate results, and repeatable processes such as data flows and graphical views.
Here in this example an events-view is created in the Data Builder by using the connected tables from an SAP EEC (left list) and they are then joined, filters, calculated in a low-code sequence. Sure, you could also program that in a SQL View, but imagine you want to clarify with the business if you are calculating the right columns – such a visual representation is much easier to understand.
Help to increase the data quality of the raw data
Anyone who has ever managed Analytics, Business Intelligence or Mining projects knows the phrase: “Sh... in - sh... out”, meaning if you have wrong or unclean data, the results of and analysis might be wrong or lead to bad decisions. Therefore, it is fundamental to have strong capabilities to influence the data quality. Sometimes it´s already beneficial to set the right filters like company code or the document types. But in other cases, you need to cleanse the data in terms of calculations, standardizations, rules, and other operations. SAP Data Warehouse Cloud provides various features and functions from simple case-when statements up to sophisticated data matching capabilities with the Intelligent Lookup (more information). All these little helpers will reduce the effort to get correct and meaningful data for process mining.
Provide agility and flexibility for changes or adjustments
Now that the data, the format, and the data quality is adequate for process mining, you can start investigating and exploring variants and possible drawbacks in the order to cash process (like shown in the scenario). There is a chance that you might be already happy with the results. But as soon as you present your findings to process owners, businesspeople or to the management, I´m pretty sure there will be questions. Most of the time those questions lead to adjustments like adding or changing attributes, including other columns in calculations or even you need to replace the one or other source table of your data views. Another advantage of being agile and flexible is to transport your complete data extraction from a test or development system to retrieve real-life data from productive system. All the requirements of an iterative approach are covered in SAP Data Warehouse Cloud. In a graphical view you can easily change or replace a table by dragging another one on to the current one (left picture blow), or you can ex-, and import (transport) complete data acquisition pipelines with all dependent objects from e.g., a test system to a productive one (right picture).
Increase transparency and governance in the data acquisition process
Process Mining really x-rays how business processes are executed in real-life and offer with insight the chance to optimize them. If we now look left and right in terms of analytics at all you will see that the same data is often used in existing business intelligence reports – like the value of all orders from yesterday, the number of outstanding invoices, revenue reports comparing last year with the current one. Instead of extracting the same data over and over for different purposes it totally makes sense to reuse the data efficiently and intelligent and reduces the extracting efforts. How can this be tackled? First, you need transparency about the available data, where it is coming from and where it is used. Secondly, if you know what is available you need to able to share or grant access to the data. Data Lineage and the Space Sharing concept of SAP Data Warehouse Cloud can exactly do this. With Data Lineage you can track and trace the origin of an information, you can see how it is transformed and combined and finally you can also see where it is consumed – see below
The second aspect, which is also visible on the picture is the Space Sharing concept. Once you have data available in a space (grey boxes) you can share the objects to other spaces where other users or departments can access and use the data. One the one hand the physical storage of data can be reduces, because only at runtime the data is queried. On the other hand, of course data access controls, role-depended interactions etc. can be set. A typical scenario is that the IT department cares about the connectivity to sources like SAP ECC and then distribute the necessary objects to the other consuming department or groups. Even a governed self-service approach is possible in such a setup.
Open the door for holistic End-to-End Process Analysis over SAP and Non-SAP applications
Today in many companies End-to-End Processes are distributed over several applications/systems. This means the process starts in an CRM system where leads are managed. As soon as there is an order the process continues in an ERP application and if there are problems with the payment there might be dunning management system involved. Sure, the easiest would be if all process steps are covered by one system like in SAP ERP but facing the reality it isn´t so. This brings us back to the various connectors SAP Data Warehouse Cloud covers. With these connectors it is possible to acquire more process data from other sources and with the data management capabilities they can be correctly combined, formatted, and cleansed.
Takeaways and conclusion
Synergy is king and better together! On a recent public event I talked to people who are already familiar with Process Mining, with SAP Signavio and other competitor tools and many of them gave the feedback that data acquisition and data management is the most time- and resource-consuming part. So there´s is clear opportunity to reduce these efforts with the combination of SAP Data Warehouse Cloud and SAP Signavio Process Intelligence. Also, the combination of Process Intelligence Accelerator and SAP Data Warehouse Cloud is quite valuable – the accelerators are great in picking the events out of the process data and SAP Data Warehouse Cloud is the perfect toolbox to tackle adjustments or non-standard data. Overall, the reuse of data, the reduction of copies and storage, the access control/management and the transparency are big advantages – and both are full SaaS-Applications born in the cloud.
Maybe one sentences to the scenario with the T-shirt company – Yes, we did not go into all the details to learn how to do an investigation in process mining and why there is loop in the delivery process. But that leaves room for another blogpost to go into more detail.
What are your thoughts? Just leave a comment or join the our SAP Signavio community.
Join the community and for future blogs you are more than welcome to follow me here.
Further resources and links:
What is SAP Signavio in a nutshell – Video
SAP Data Warehouse Cloud Overview
This blog is about how SAP Signavio Process Intelligence (Process Mining) works together with and SAP Data Warehouse Cloud and how Process Mining projects can be accelerated and optimized in several aspects, like:
- Speed up the data acquisition and connectivity setup
- Simplify data preparation and increase low/no-code usage
- Help to increase the data quality of the raw data
- Provide agility and flexibility for changes or adjustments
- Increase transparency and governance in the data acquisition process
- Open the door for holistic End-to-End Process Analysis over SAP and Non-SAP applications
Additionally, this blog uses a simplified real-life scenario to also give the necessary business context and use cases which hopefully makes it easier to understand, crisper and more enjoyable to read.
Last remark – if you are new to either SAP Signavio or SAP Data Warehouse Cloud you´ll find at the end of this blog links to other resources and short introduction videos.
The scenario:
First let´s have a look on to the use case and the goals of the scenario. One part of the SAP Signavio Transformation Suite is the so-called Process Intelligence module. With this module you can analyze data which is produced through business processes like Order-to-Cash, Hire-to-Retire, Plan-to-Fulfill and many more. These processes are the life blood of a company because they are directly connected with the value chain, the better they are managed the more efficient and resource-saving they are.
For our example we will focus on the Order-to-Cash process of a fictitious T-Shirt Selling Company, which you might can relate to the SAP ERP Sales and Distribution (SD) module.
Let´s assume the End-to-End-Process on high level looks like the following:
Order to cash process - high level overview
Behind the outbound delivery step there is at least one important decision to take: either they can directly ship an ordered t-shirt from the warehouse (if in stock), or they need to produce the imprint of the t-shirt first. You can imagine that a printing process will have an impact on the delivery time for the customer and on the cycle time of a process at all. Such decisions are leading to commonly named process variants, because each decision opens new paths – like on a street map you can take various roads to get to a destination.
One of the goals of process mining and especially in this scenario is to analyze the process variants and identify bottlenecks, loops, or other inefficiencies during the Order-to-Cash process. This helps on the one hand to understand why cycle times are too high and on the other hand where lots of process “energy” is wasted due to wrongly used paths or other waste of time.
How this looks like for our T-Shirt company you´ll see on following pictures:
Process Mining scenario of the T-shirt company
The pictures on the left-hand side show the so-called Happy Path – this means most of the cases (orders) are following this route and they need an average consumption of time (cycle time) between the steps. On the right-hand side we can see a variant caused by
the decision if a T-Shirt can directly be shipped from the warehouse or if the T-Shirt needs to be printed first. Of course, we can also see the impact on the cycle times. Another inefficiency seems to happen in the shipping process, sometimes it takes several attempts to deliver the goods.
Ok, so now that we have an overview about the scenario lets peek under the hood how the data for process mining looks like and how SAP Data Warehouse Cloud accelerates Process Mining Projects.
Speed up the data acquisition and connectivity setup
Of course, first we must put our nozzle into the systems in which the processes are executed, and these are the ERP-, CRM-, HR- and many other systems. In our scenario we need to connect to SAP ECC on premise to get the respective data. SAP Data Warehouse Cloud offers out-of-the-box connectors to different applications and data sources (current list see here).
Connection to SAP ECC and connection properties
With the SAP EEC connection you can get direct data access to your SAP system and can decide in which mode you want to retrieve data. Either you will replicate the necessary tables to SAP Data Warehouse Cloud, or you can use the remote tables (virtual access) to reduce data movements and separate storage. Sometimes you even get a free ride because it is more than likely that in the context of a SAP Data Warehouse Cloud project this connection has already been used for other analytical questions. In such case the sharing concept within the spaces in SAP Data Warehouse Cloud will reduce effort and additionally speed up the data acquisition.
For detailed information I recommend reading these nice blogs.
SAP Data Warehouse Cloud: Introduction to the ABAP-Adapter
SAP Data Warehouse Cloud: Capabilities of the ABAP-Adapter Connection to on-premises S/4HANA & ECC
Simplify data preparation and increase low/no-code usage
Another challenge you might face in Process Mining project is to prepare the data into the right format (process data model). As described in the scenario we want to analyze an Order-to-Cash process and you can image there are many steps to perform from creating an order, organize the shipment until you can do an invoice clearing after you have received the payment from the customers. These steps are called events, which are necessary to execute a case (one order) from beginning to end and are the foundation to calculate cycle times or identifying loops and other inefficiencies. The second ingredient are the attributes (in BI Projects you might also say dimensions) which describes a case to which organization, country, customer, customer group, product, supplier etc. it belongs. These attributes help to categorize and subsume similar cases to find dependencies like all orders of a specific type, product group etc. performing better than form other groups.
Events collectors of a standard order to cash process
Data is the keyword - now the SAP Data Warehouse Cloud comes again into play. In SAP Data Warehouse Cloud you can not only benefit from the connectivity, but with the Data Builder many useful functions are available to prepare and wrangle the data. That includes e.g., joins, unions, calculations, filters, intermediate results, and repeatable processes such as data flows and graphical views.
Graphical View in Data Builder
Here in this example an events-view is created in the Data Builder by using the connected tables from an SAP EEC (left list) and they are then joined, filters, calculated in a low-code sequence. Sure, you could also program that in a SQL View, but imagine you want to clarify with the business if you are calculating the right columns – such a visual representation is much easier to understand.
Help to increase the data quality of the raw data
Anyone who has ever managed Analytics, Business Intelligence or Mining projects knows the phrase: “Sh... in - sh... out”, meaning if you have wrong or unclean data, the results of and analysis might be wrong or lead to bad decisions. Therefore, it is fundamental to have strong capabilities to influence the data quality. Sometimes it´s already beneficial to set the right filters like company code or the document types. But in other cases, you need to cleanse the data in terms of calculations, standardizations, rules, and other operations. SAP Data Warehouse Cloud provides various features and functions from simple case-when statements up to sophisticated data matching capabilities with the Intelligent Lookup (more information). All these little helpers will reduce the effort to get correct and meaningful data for process mining.
Provide agility and flexibility for changes or adjustments
Now that the data, the format, and the data quality is adequate for process mining, you can start investigating and exploring variants and possible drawbacks in the order to cash process (like shown in the scenario). There is a chance that you might be already happy with the results. But as soon as you present your findings to process owners, businesspeople or to the management, I´m pretty sure there will be questions. Most of the time those questions lead to adjustments like adding or changing attributes, including other columns in calculations or even you need to replace the one or other source table of your data views. Another advantage of being agile and flexible is to transport your complete data extraction from a test or development system to retrieve real-life data from productive system. All the requirements of an iterative approach are covered in SAP Data Warehouse Cloud. In a graphical view you can easily change or replace a table by dragging another one on to the current one (left picture blow), or you can ex-, and import (transport) complete data acquisition pipelines with all dependent objects from e.g., a test system to a productive one (right picture).
Table replacement (left) / Transport (right)
Increase transparency and governance in the data acquisition process
Process Mining really x-rays how business processes are executed in real-life and offer with insight the chance to optimize them. If we now look left and right in terms of analytics at all you will see that the same data is often used in existing business intelligence reports – like the value of all orders from yesterday, the number of outstanding invoices, revenue reports comparing last year with the current one. Instead of extracting the same data over and over for different purposes it totally makes sense to reuse the data efficiently and intelligent and reduces the extracting efforts. How can this be tackled? First, you need transparency about the available data, where it is coming from and where it is used. Secondly, if you know what is available you need to able to share or grant access to the data. Data Lineage and the Space Sharing concept of SAP Data Warehouse Cloud can exactly do this. With Data Lineage you can track and trace the origin of an information, you can see how it is transformed and combined and finally you can also see where it is consumed – see below
Data Lineage for events view
The second aspect, which is also visible on the picture is the Space Sharing concept. Once you have data available in a space (grey boxes) you can share the objects to other spaces where other users or departments can access and use the data. One the one hand the physical storage of data can be reduces, because only at runtime the data is queried. On the other hand, of course data access controls, role-depended interactions etc. can be set. A typical scenario is that the IT department cares about the connectivity to sources like SAP ECC and then distribute the necessary objects to the other consuming department or groups. Even a governed self-service approach is possible in such a setup.
Open the door for holistic End-to-End Process Analysis over SAP and Non-SAP applications
Today in many companies End-to-End Processes are distributed over several applications/systems. This means the process starts in an CRM system where leads are managed. As soon as there is an order the process continues in an ERP application and if there are problems with the payment there might be dunning management system involved. Sure, the easiest would be if all process steps are covered by one system like in SAP ERP but facing the reality it isn´t so. This brings us back to the various connectors SAP Data Warehouse Cloud covers. With these connectors it is possible to acquire more process data from other sources and with the data management capabilities they can be correctly combined, formatted, and cleansed.
Takeaways and conclusion
Synergy is king and better together! On a recent public event I talked to people who are already familiar with Process Mining, with SAP Signavio and other competitor tools and many of them gave the feedback that data acquisition and data management is the most time- and resource-consuming part. So there´s is clear opportunity to reduce these efforts with the combination of SAP Data Warehouse Cloud and SAP Signavio Process Intelligence. Also, the combination of Process Intelligence Accelerator and SAP Data Warehouse Cloud is quite valuable – the accelerators are great in picking the events out of the process data and SAP Data Warehouse Cloud is the perfect toolbox to tackle adjustments or non-standard data. Overall, the reuse of data, the reduction of copies and storage, the access control/management and the transparency are big advantages – and both are full SaaS-Applications born in the cloud.
Maybe one sentences to the scenario with the T-shirt company – Yes, we did not go into all the details to learn how to do an investigation in process mining and why there is loop in the delivery process. But that leaves room for another blogpost to go into more detail.
What are your thoughts? Just leave a comment or join the our SAP Signavio community.
Join the community and for future blogs you are more than welcome to follow me here.
Further resources and links:
What is SAP Signavio in a nutshell – Video
SAP Data Warehouse Cloud Overview
- SAP Managed Tags:
2 Comments
