The rising of the IoT has connected to the cloud lots of devices and made available a huge amount of data.
The next-gen scenario, when we are talking about IoT, is the possibility to make analysis and prediction over the big amount of data that we are receiving through our IoT enabled devices.
The prediction, in the SAP ecosystem, is possible both in the cloud and at the Edge.
In some particular environments, such as low connectivity scenarios or real-time scenarios, it became really important and useful to have the possibility to run the predictive algorithm at the edge.
You can natively create such a scenario by leveraging the SAP Cloud Platform Internet of Things and Edge Services capabilities.
In the following article, I’m going to explain how you can physically implement a realistic industrial scenario that makes a prediction at the edge.
Our story tells about Paint Ink, a company that carries out a professional painting service for several industries. They have for example orders from several automotive industries.
Even if the technologies permit to replicate the color used to paint an object, based on a color code, the company used for this example would like to have a high-quality level of their industrial process and objects.
The validation consists of the analysis of the painting, piece by piece. This process is composed of a conveyor belt with a sensor, like a camera, that reads the color components (RGB) of the pieces that are in the belt.
The painting of each piece is not completely uniform, and the color could change in the different areas of it due, for example, to the nature of the paint or to the form of the object.
They have already implemented a code to validate the analysis of the painting, based on the KNN (K-Nearest Neighbor) algorithm, that analyzes in several points of the object and establishes if the object has an enough high painting quality before proceeding with the packaging of it. In case the quality is not enough high the piece is automatically discarded.
As you can imagine, in this scenario it’s important to have a real-time analysis of the colors in order to speed up all the post-production tasks with no blocks on the belt.
They have decided to implement the analysis at the edge to satisfy this goal.
The training of the base KNN algorithm has been done in the cloud within SAP Data Intelligence.
Inside my tenant of SAP Data Intelligence, I have used Jupyter to implement in a Notebook a python algorithm that trains a KNN model with a big amount of already categorized data that saves the output in a file as a PMML model.
It could also expose an endpoint to recompute the model to improve the recognition with additional trained data.
To implement the predictive analysis at the edge we have inherited the features exposed by SAP Cloud Platform Internet of Things Edge Platform with the creation of a new OSGi Bundle that runs in parallel into our Edge Platform and by leveraging the features of SAP Edge Services Persistence Service.
The implemented module imports into the OSGi environment the libraries required to use in Java the PMML model, such as the JPMML library.
In this scenario, we need to analyze several measures for our painted object, that represents the RGB color in different areas of it. Due to improving the system performances, since the required time to complete the acquisition of the measurements is around 10 seconds, we have decided to get all the required measurements in a single bunch.
It’s very easy and native with the module of SAP Edge Services Persistence Service that is installed on the top of the Edge Platform.
Persistence Service stores all the measurements into a local SQLAnywhere database and permits to fetch them within some REST/Java APIs or within a custom SQL query.
All the data is passed to the evaluator module to get a KNN prediction, based on the input values.
Each single analyzed point contributes as a result of the analysis to build a prediction index that passed on certain policies to validate the overall quality of the painting.
Both the punctual prediction and the overall index are streamed to SAP Cloud Platform Internet of Things, to permits further analysis, to improve the KNN model, and to permits to apply rules on the top of the streamed values by using the capabilities of SAP Edge Services Streaming Service.
You can easily extend this scenario to the ones described in this other blog post. In the setup of the rule, you have to apply it, for example to the prediction index, instead of the temperature used for that example.
This example is using JPMM but you can easily replace this library with your own preferred library or custom implementation.
As the first operation, create the device model inside your tenant of SAP Cloud Platform Internet of Things. If it's possible, in order to have an immediate mapping between the model and the input used for your KNN pmml model file, I suggest using the same property name specified as a label for the input parameters.
For this example, we have created into SAP Cloud Platform Internet of Things one sensor type with inside 3 capabilities.
Open Edge Services Policy Service and click the tile Edge Services Management tile, and in the list search and click on your gateway.
Now you can deploy Persistence Service into your gateway, to permit the automatic storage of the measurements.
Click the tab named Services and then the plus button to deploy a new service into your SAP Edge Platform.
In the next window select Persistence Service in the dropdown and Save to start the deployment. When the deployment of the service has been completed the status of the service changes from Requested to Installed. You can also configure the user management and the User Identity Provider for this service accordingly with certain invocation rights policies.
Now you can also deploy Streaming Service and Essential Business Functions and their configuration accordingly with this blog post. In this example, I've used the validity color score capability into the streaming rule and, if it's over a certain threshold (e.g. 100), an SAP Field Service Management Service Call will be created.
Now we can proceed with the custom logic that implements the Predictive Analytics Service.
The code implementation for this example is available in the following link of the SAP official Github repository.
If you import it in your favorite IDE tool (such as Intellij IDEA or Eclipse) the process looks like in this picture
Now I'll start to add more details about the most important classes:
The required logic to implement the scenario is inside the classes defined in the custom package:
First, at the startup of the bundle, the initialize method is invoked, you need to import the saved PMML model with your java library. I’m loading it within the classloader, since the PMML model is saved as a resource of the project. You can make the load of the model dynamically by changing the implementation in this method.
After that, we are loading it into a KNN evaluator implementation. This is the object that I’m going to use to make the prediction over the streamed data.
Now let’s have a look at the run method that is automatically scheduled every 10 seconds (the time required to analyze each single painted object).
This method is going to fetch the data within persistence client in a 10 seconds time window. In case there are some data, for each sample, we are going to evaluate within the KNN evaluator if the data is near enough to the values used to train the model.
It will return the distance from the top 3 neighbors and the prediction with the label that contains the predicted value.
Then we are going to build an index that came from a mathematical formula that makes aggregation of the calculated prediction for all the existing samples.
Finally, we can stream the punctual prediction and the calculated index (that represents the overall prediction) back to SAP Cloud Platform Internet of Things.
This operation permits to make additional computation to the predicted values such as send them back to a 3rd party system in the cloud, or, like in this scenario, with a Rule computed within Streaming Service.
In case you need to use additional libraries don't forget to add them into the pom.xml Maven file like I've done for the PMML libraries:
The scope provided means that you need to provide the dependencies, satisfy them by sideloading them and all the inherited dependencies to your Edge Platform within the SAP Cloud Platform Internet of Things Cockpit.
Open Cockpit and go to the Gateways list, and open your gateway, go to Bundle Management and use the upload button to start the upload of a new jar OSGi bundle into the Edge Platform.
Repeat the following for all the dependencies (marked with scope provided) that you have in your pom file (except for the PersistenceService that is already provided by Edge Services).
After you have uploaded one single jar, please have a look into the OSGi console of your Edge Platform, you can have errors like the following:
It means that one inherited dependency is missing into the OSGi environment. My suggestion is to search for the library that contains the missing Import-Package specified in the error string and load it with the same procedure into the Edge Platform
You might need to create your own OSGi library for some projects that are not OSGi compatible. To transform a non-OSGi project into an OSGi project, I generally add into the pom.xml file these plugins:
Now we can proceed with the compile of the customized service.
First I need to import the Edge Service persistence java library.
Copy from the custombundles folder of your Edge Platform the file osgi-bundle-PersistenceService-3.1909.0.jar to the project root of your project.
Use the following command from the bash console in the root of your project to import the library into your local maven repository.
Replace the version in the file name and in the pom.xml file of the project in case it’s different from the version you are using to.
Build the project with maven: go in the root of your project and type in a bash console:
It will create in the target folder of your project the OSGi bundle of our service, his name is PredictiveModel-1.0.0.jar
It is possible to load the service from the SAP Cloud Platform Internet of Things Cockpit into your Edge Platform, like I have done for the dependencies, but I suggest to use this method only in development and upload the “released” version within Edge Services; the main advantage of it is that you can easily deploy it to several gateways, or group of gateways, and it’s always available in the cloud.
Open Edge Services and click Settings; ensure that the flag Allow upload of Custom Services (OSGi bundles) to my IoT Edge Platforms is checked and press Save.
Open Edge Service Management, in the left menu click on Services and press the plus button in the button bar of the services list to create a new service.
Specify a name such as RGB Predictive Analytics and any name for the configuration topic (we are not using it).
The File Name field contains the OSGi bundle we are going to load with this service, which is the PredictiveModel-1.0.0.jar we have built in the previous section.
Now go to the Groups and Gateways screen and open your gateway, switch to the Services tab and press the plus button to start the deployment of a new service.
In the dropdown is now available a custom service with the same name you have specified during the creation of it. Select it and press Save.
It took a while to be installed, after some time the service status in the list became Installed. Go to the OSGi console of your Edge Platform and check that:
We have used a REST Edge Platform, so I've used Postman to test it.
I'm going to send one measurement that is R:235, G:64, B 52
this value should be classified as valid with an overall index lower than the threshold that is 100, with the label brown.
Now we can check the predicted value in SAP Cloud Platform Internet of Things Cockpit. Open the device and check Data Visualization:
The bundle has streamed back to SAP Cloud Platform Internet of Things, as expected, the prediction that is brown, the 3 top neighbors, that are with a euclidean distance below 100, and the overall evaluation that lower than 100.
If we send a value that is out of range, that generates the validity index out of range a new Service Call into SAP field Service management is created,. This will notify the employees of the Paint Ink company, that something goes wrong in the painting, so they have to check manually the object and, in case, search for the root of the problem and solve it (i.e. the presence of some dirty).
With this example, I have explained how to build a Service that uses the predictive model at the edge in a pure Java Implementation.
In this other blog post, you can find also a Python implementation that could be easily extended to several different languages.
The technologies used and the services are several, and it will also explain how to put all together to create a realistic scenario, and how to grant the continuum from the cloud to the edge.
The next-gen scenario, when we are talking about IoT, is the possibility to make analysis and prediction over the big amount of data that we are receiving through our IoT enabled devices.
The prediction, in the SAP ecosystem, is possible both in the cloud and at the Edge.
In some particular environments, such as low connectivity scenarios or real-time scenarios, it became really important and useful to have the possibility to run the predictive algorithm at the edge.
You can natively create such a scenario by leveraging the SAP Cloud Platform Internet of Things and Edge Services capabilities.
In the following article, I’m going to explain how you can physically implement a realistic industrial scenario that makes a prediction at the edge.
0. Story
Our story tells about Paint Ink, a company that carries out a professional painting service for several industries. They have for example orders from several automotive industries.
Even if the technologies permit to replicate the color used to paint an object, based on a color code, the company used for this example would like to have a high-quality level of their industrial process and objects.
The validation consists of the analysis of the painting, piece by piece. This process is composed of a conveyor belt with a sensor, like a camera, that reads the color components (RGB) of the pieces that are in the belt.
The painting of each piece is not completely uniform, and the color could change in the different areas of it due, for example, to the nature of the paint or to the form of the object.
They have already implemented a code to validate the analysis of the painting, based on the KNN (K-Nearest Neighbor) algorithm, that analyzes in several points of the object and establishes if the object has an enough high painting quality before proceeding with the packaging of it. In case the quality is not enough high the piece is automatically discarded.
As you can imagine, in this scenario it’s important to have a real-time analysis of the colors in order to speed up all the post-production tasks with no blocks on the belt.
They have decided to implement the analysis at the edge to satisfy this goal.
1. Implementation pre-requirements
The training of the base KNN algorithm has been done in the cloud within SAP Data Intelligence.
Inside my tenant of SAP Data Intelligence, I have used Jupyter to implement in a Notebook a python algorithm that trains a KNN model with a big amount of already categorized data that saves the output in a file as a PMML model.
It could also expose an endpoint to recompute the model to improve the recognition with additional trained data.
2. Edge Implementation schema
To implement the predictive analysis at the edge we have inherited the features exposed by SAP Cloud Platform Internet of Things Edge Platform with the creation of a new OSGi Bundle that runs in parallel into our Edge Platform and by leveraging the features of SAP Edge Services Persistence Service.
The implemented module imports into the OSGi environment the libraries required to use in Java the PMML model, such as the JPMML library.
In this scenario, we need to analyze several measures for our painted object, that represents the RGB color in different areas of it. Due to improving the system performances, since the required time to complete the acquisition of the measurements is around 10 seconds, we have decided to get all the required measurements in a single bunch.
It’s very easy and native with the module of SAP Edge Services Persistence Service that is installed on the top of the Edge Platform.
Persistence Service stores all the measurements into a local SQLAnywhere database and permits to fetch them within some REST/Java APIs or within a custom SQL query.
All the data is passed to the evaluator module to get a KNN prediction, based on the input values.
Each single analyzed point contributes as a result of the analysis to build a prediction index that passed on certain policies to validate the overall quality of the painting.
Both the punctual prediction and the overall index are streamed to SAP Cloud Platform Internet of Things, to permits further analysis, to improve the KNN model, and to permits to apply rules on the top of the streamed values by using the capabilities of SAP Edge Services Streaming Service.
You can easily extend this scenario to the ones described in this other blog post. In the setup of the rule, you have to apply it, for example to the prediction index, instead of the temperature used for that example.
This example is using JPMM but you can easily replace this library with your own preferred library or custom implementation.
3. Implementation
As the first operation, create the device model inside your tenant of SAP Cloud Platform Internet of Things. If it's possible, in order to have an immediate mapping between the model and the input used for your KNN pmml model file, I suggest using the same property name specified as a label for the input parameters.
For this example, we have created into SAP Cloud Platform Internet of Things one sensor type with inside 3 capabilities.
- color: this capability permits to receive as measurement the RGB values captured by RGB sensor
- color prediction: contains the label of the computed prediction (point by point) with the 3 closest neighbors that are used to compute the affinity index
- validity color score: it's one per object analyzed (1 measurement each 10 seconds), and give a proximity index based on all the measurement received in the last 10 seconds. It permits us to identify if the painting has enough high quality.
4. Deploy Persistence Service
Open Edge Services Policy Service and click the tile Edge Services Management tile, and in the list search and click on your gateway.
Now you can deploy Persistence Service into your gateway, to permit the automatic storage of the measurements.
Click the tab named Services and then the plus button to deploy a new service into your SAP Edge Platform.
In the next window select Persistence Service in the dropdown and Save to start the deployment. When the deployment of the service has been completed the status of the service changes from Requested to Installed. You can also configure the user management and the User Identity Provider for this service accordingly with certain invocation rights policies.
Now you can also deploy Streaming Service and Essential Business Functions and their configuration accordingly with this blog post. In this example, I've used the validity color score capability into the streaming rule and, if it's over a certain threshold (e.g. 100), an SAP Field Service Management Service Call will be created.
5. Create the pure Java Service
Do you mind to use a Python implementation? Let's have a look at this alternative Python implementation
Now we can proceed with the custom logic that implements the Predictive Analytics Service.
The code implementation for this example is available in the following link of the SAP official Github repository.
If you import it in your favorite IDE tool (such as Intellij IDEA or Eclipse) the process looks like in this picture
Now I'll start to add more details about the most important classes:
- Engine.java class represents the entry point to invoke the main loop that makes the prediction with a ScheduledExecutorService.
- DataStreamer class contains just the methods to stream the results to SAP Cloud Platform Internet of Things based on a simple HTTP client
The required logic to implement the scenario is inside the classes defined in the custom package:
- QueryData.java is our interface to execute queries of the SQLAnywhere database within the Persistence java client.
- The core of the logic that allows us to collect the data, to make the final prediction and send the data to SAP Cloud Platform Internet of Things is inside our PredictValues class.
First, at the startup of the bundle, the initialize method is invoked, you need to import the saved PMML model with your java library. I’m loading it within the classloader, since the PMML model is saved as a resource of the project. You can make the load of the model dynamically by changing the implementation in this method.
After that, we are loading it into a KNN evaluator implementation. This is the object that I’m going to use to make the prediction over the streamed data.
Now let’s have a look at the run method that is automatically scheduled every 10 seconds (the time required to analyze each single painted object).
This method is going to fetch the data within persistence client in a 10 seconds time window. In case there are some data, for each sample, we are going to evaluate within the KNN evaluator if the data is near enough to the values used to train the model.
It will return the distance from the top 3 neighbors and the prediction with the label that contains the predicted value.
Then we are going to build an index that came from a mathematical formula that makes aggregation of the calculated prediction for all the existing samples.
Finally, we can stream the punctual prediction and the calculated index (that represents the overall prediction) back to SAP Cloud Platform Internet of Things.
This operation permits to make additional computation to the predicted values such as send them back to a 3rd party system in the cloud, or, like in this scenario, with a Rule computed within Streaming Service.
In case you need to use additional libraries don't forget to add them into the pom.xml Maven file like I've done for the PMML libraries:
<dependency>
<groupId>org.jpmml</groupId>
<artifactId>pmml-evaluator</artifactId>
<version>1.4.13</version>
<type>bundle</type>
<scope>provided</scope>
</dependency>
The scope provided means that you need to provide the dependencies, satisfy them by sideloading them and all the inherited dependencies to your Edge Platform within the SAP Cloud Platform Internet of Things Cockpit.
- Satisfy the dependencies
Open Cockpit and go to the Gateways list, and open your gateway, go to Bundle Management and use the upload button to start the upload of a new jar OSGi bundle into the Edge Platform.
Repeat the following for all the dependencies (marked with scope provided) that you have in your pom file (except for the PersistenceService that is already provided by Edge Services).
After you have uploaded one single jar, please have a look into the OSGi console of your Edge Platform, you can have errors like the following:
org.osgi.framework.BundleException: The bundle "<BUNDLE NAME HERE>" could not be resolved. Reason: Missing Constraint: Import-Package: <MISSING PACKAGE HERE>
It means that one inherited dependency is missing into the OSGi environment. My suggestion is to search for the library that contains the missing Import-Package specified in the error string and load it with the same procedure into the Edge Platform
You might need to create your own OSGi library for some projects that are not OSGi compatible. To transform a non-OSGi project into an OSGi project, I generally add into the pom.xml file these plugins:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifestFile>META-INF/MANIFEST.MF</manifestFile>
</archive>
</configuration>
</plugin>
<!-- the versions plugin can be used to check for new plugin versions
like this: mvn versions:Display-plugin-updates -->
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<executions>
<!-- Configure extra execution of 'manifest' in process-classes
phase to make sure SCR metadata is generated before unit test runs -->
<execution>
<id>scr-metadata</id>
<goals>
<goal>manifest</goal>
</goals>
<configuration>
<supportIncrementalBuild>true</supportIncrementalBuild>
</configuration>
</execution>
</executions>
<configuration>
<exportScr>true</exportScr>
<manifestLocation>META-INF</manifestLocation>
<instructions>
<!-- Enable processing of OSGI DS component annotations -->
<_dsannotations>*</_dsannotations>
<!-- Enable processing of OSGI metatype annotations -->
<_metatypeannotations>*</_metatypeannotations>
</instructions>
</configuration>
</plugin> - Build the Service
Now we can proceed with the compile of the customized service.
First I need to import the Edge Service persistence java library.
Copy from the custombundles folder of your Edge Platform the file osgi-bundle-PersistenceService-3.1909.0.jar to the project root of your project.
Use the following command from the bash console in the root of your project to import the library into your local maven repository.
mvn install:install-file -Dfile=PersistenceService-3.1909.0.jar -DgroupId=com.sap.iot.edgeservices -DartifactId=PersistenceService -Dversion=3.1909.0 -Dpackaging=jarReplace the version in the file name and in the pom.xml file of the project in case it’s different from the version you are using to.
Build the project with maven: go in the root of your project and type in a bash console:
mvn clean installIt will create in the target folder of your project the OSGi bundle of our service, his name is PredictiveModel-1.0.0.jar
6. Deploy the Service
It is possible to load the service from the SAP Cloud Platform Internet of Things Cockpit into your Edge Platform, like I have done for the dependencies, but I suggest to use this method only in development and upload the “released” version within Edge Services; the main advantage of it is that you can easily deploy it to several gateways, or group of gateways, and it’s always available in the cloud.
Open Edge Services and click Settings; ensure that the flag Allow upload of Custom Services (OSGi bundles) to my IoT Edge Platforms is checked and press Save.
Open Edge Service Management, in the left menu click on Services and press the plus button in the button bar of the services list to create a new service.
Specify a name such as RGB Predictive Analytics and any name for the configuration topic (we are not using it).
The File Name field contains the OSGi bundle we are going to load with this service, which is the PredictiveModel-1.0.0.jar we have built in the previous section.
Now go to the Groups and Gateways screen and open your gateway, switch to the Services tab and press the plus button to start the deployment of a new service.
In the dropdown is now available a custom service with the same name you have specified during the creation of it. Select it and press Save.
It took a while to be installed, after some time the service status in the list became Installed. Go to the OSGi console of your Edge Platform and check that:
- No errors have been generated during the installation and the start of the service
- The status of the service is STARTED (type “lb” command into the OSGi console)
7. Test the Service
We have used a REST Edge Platform, so I've used Postman to test it.
I'm going to send one measurement that is R:235, G:64, B 52
this value should be classified as valid with an overall index lower than the threshold that is 100, with the label brown.
Now we can check the predicted value in SAP Cloud Platform Internet of Things Cockpit. Open the device and check Data Visualization:
The bundle has streamed back to SAP Cloud Platform Internet of Things, as expected, the prediction that is brown, the 3 top neighbors, that are with a euclidean distance below 100, and the overall evaluation that lower than 100.
If we send a value that is out of range, that generates the validity index out of range a new Service Call into SAP field Service management is created,. This will notify the employees of the Paint Ink company, that something goes wrong in the painting, so they have to check manually the object and, in case, search for the root of the problem and solve it (i.e. the presence of some dirty).
8. Conclusions
With this example, I have explained how to build a Service that uses the predictive model at the edge in a pure Java Implementation.
In this other blog post, you can find also a Python implementation that could be easily extended to several different languages.
The technologies used and the services are several, and it will also explain how to put all together to create a realistic scenario, and how to grant the continuum from the cloud to the edge.
- SAP Managed Tags:
1 Comment
