Hey folks! ๐โโ๏ธ๐
Another AI blog post, right? Nope, not this time โ let's finally stop talking and start walking! ๐ฌ๐ฅ We've got some real goodies to share with you, whether you're an SAP customer, SAP partner, or just one of those strange but adorable enthusiasts like me, who can't get enough of SAP. ๐ค๐ผ
At TechEd 2023, SAP dropped some major news on the Artificial Intelligence front, including our new friend Joule, the SAP HANA Cloud Vector Engine, and the star of today's show, the generative AI hub in SAP AI Core, wrapped up in a real-world use-case. ๐๐ก
With our fresh-off-the-press reference architecture guidance for Generative AI scenarios, you'll be more than ready to kickstart your own AI journey on SAP BTP. ๐๐ Whether you're building extensions, standalone solutions, or selling your own SaaS apps on SAP BTP, you can count on the generative AI hub for infusing your architecture with AI superpower. ๐๏ธ๐ผ๐
And it doesn't stop there! You'll also get to leverage best practices like a great SAPUI5 app, a robust SAP HANA Cloud back-end, a single and multitenant CAP application, and an API for your SaaS customers using the SAP Service Broker Framework. This sample scenario has got it all, folks!๐๐ช๐ Great news - our sample is available for the SAP BTP, Kyma Runtime and the the SAP BTP, Cloud Foundry Runtime!
This blog post is gonna shine a spotlight ๐ต๏ธโโ๏ธ on how customer and partner apps on SAP BTP can take advantage of SAPโs freshest service offering โ the new generative AI hub as part of SAP AI Core. ๐ This is all about helping you to add some serious AI capabilities to your single- and multitenant solutions.
To make things a bit easier and to lay down some solid groundwork for all you developers ๐งโ๐ป diving headfirst into the SAP AI space, we're gonna show off a few neat AI features. These are part of a new SAP-samples GitHub repository all built around a make-believe travel agency ๐ฎ we're calling ThorTours. ๐โ๏ธ
And here's the fun part: we're gonna make ThorTours' customer support processes even better using Generative AI by reducing customer response efforts for faster handling of incoming requests and at the same time boosting the quality for higher consistency in support through reproducible responses. ๐ค๐ So hold on to your hats and let's dive in! ๐โโ๏ธ๐ฉโ๐ป
Our GenAI Mail Insights sample solution is the new best friend of all ThorTours ๐๏ธโฑ๏ธcustomer support employees ๐. It's all about giving you top-notch mail insights and automation in a comprehensive, (multitenant) SAP BTP solution enhanced with a bunch of cool AI features ๐ค. How does it do it? It uses Large Language Models (LLMs) via the generative AI hub in SAP AI Core to analyze incoming mails, offering categorization, sentiment analysis ๐, urgency assessment, and even extracts key facts to add a personalized touch ๐.
What's the cherry on top? ๐ It's the super innovative feature of Retrieval Augmented Generation (RAG) using mail embeddings. This helps in figuring out how similar requests and infuses those confirmed responses as additional context to reduce hallucinations ๐ป, ensuring you always get consistent service quality ๐ฏ. It doesn't stop there, it also summarizes and translates simple mail enquiries for But wait, there's more!
Our sample setup boosts automation by generating potential answers for customer inquiries, enhancing response accuracy and speed like a boss ๐ช. And the best part? It's not just for the travel industry โ it's adaptable for various scenarios using custom schemas. So, whatever your need, we've got you covered! ๐
Why does this matter, you ask? ๐ค Well, companies are often struggle with customer support headaches, everything from reading through long mails manually ๐ง to language hurdles ๐ and a lack of automation ๐ค. What our demo solution brings to the table is quicker, more personalized, and consistent customer service ๐ฅ, savings on cost ๐ธ, flexibility, and an edge over the competition ๐ผ. And we're not stopping there! In upcoming releases, we're even thinking about an integration with SAP Cloud Solutions such as SAP Concur or a Microsoft Exchange Online Inbox. This could supercharge operations and give a boost to data-driven decision-making ๐.
Alright, enough talking for now โ let's roll up our sleeves and dive into some of the techie highlights you can uncover in our SAP-samples repository ๐๏ธ. Ready to kick off with the first steps of processing new mail enquiries? This will lead us straight into our first challenge and the GenAI spotlight of our use-case ๐ฏ. First things first, we're going to see how our solution, built on SAP BTP, actually links up to the Large Language Models using the generative AI hub capabilities โ the latest and greatest service offering as part of SAP AI Core announced at TechEd 2023 ๐ฅ.
If you've had a go at existing LLMs like Azure OpenAI, you'll find the setup pretty familiar. ๐ง When an app needs to hook up with a Large Language Model (we're talking Azure OpenAI in our scenario), it links up with the generative AI hub and dials up the URL or what we often call the inference endpoint of a tenant-specific deployment. ๐ฅ๏ธ๐ Each tenant in a multitenant setup gets their own dedicated so-called SAP AI Core resource group onboarded, which is awesome because it offers the flexibility for individual model deployments and upcoming metering features. ๐๐
When it comes to our sample scenario, we're sending payloads to the Inference URL for Chat Completion and creating Embeddings. This process is a piece of cake ๐ฐ because it's just like the Azure OpenAI API specification, which means requests and custom modifications are a breeze ๐ฌ๏ธ. Just a heads up though, this is still an early-bird ๐ฆ release of the generative AI hub, and we might simplify the integration even more in the future. Why, you ask? Well, it's all thanks to the rising popularity of contributions to Open-Source resources like LangChain. ๐๐ฅ
For the time being, we've got some handy sample wrappers ๐ that make connecting to the generative AI hub deployments super easy. They take care of handling multitenancy and can be swiftly adapted for other Large Language models available. Plus, these wrappers ensure compatibility with available LangChain features once the LLM object is instantiated. This allows you as a developer ๐ฉ๐ก to keep their eyes on the prize - the actual business process!
So, we've got a customer that's just a big block of text, right? ๐ Our goal is to wrangle that into some useful info we can store it away in our database. ๐ We're using the brainpower of a Large Language Model, LangChain and custom schemas (courtesy of the npm zod package) to make this happen. It's all about having a flexible and hands-on setup with the Large Language Model. ๐ง ๐ฌ
We're using custom schemas and auto-generated format instructions (that's some prompt engineering magic right there! ๐ฎ) to get a nice and structured JSON result. ๐ This we can process further and easily store in our SAP HANA Cloud Database using our beloved CAP framework making that task a childโs play. โ๏ธ๐พ
The parsing features of LangChain and the custom schema package features are like two peas in a pod ๐ฑ They make sure that the LLM response fits into a properly typed object without any further type conversion hustle. ๐ก๏ธ Pretty cool!? Sounds almost too good to be true! ๐ฒ๐ Using these kind of custom schema definitions will allow you to adapt the solution to any other kind of support scenario, by simply updating the respective schema details!
Apart from extracting a summary, insights and relevant key-facts from a new customer enquiry, we've also made life easier by automating the drafting of reliable response proposals for new mails. ๐ฉ This brings us to our second reveal, a cool mix of Large Language Models (LLM) magic and the capabilities of what's known as Retrieval Augmented Generation . ๐ฉโจAlso check the following blog post to learn more.
The concept is as straightforward as it is genius โ we use an LLM to create a new response for a customer enquiry, but here's the twist: we add in some extra context from previously answered similar mails. ๐
While this might sound like a lot of effort, a very simple implementation is as easy as pie. ๐ฅง All you need is access to generative AI hub that is generating so-called embeddings for incoming mails and a suitable vector store to hold these embeddings. ๐๏ธ Embeddings are vector representations of versatile input formats (e.g., text, images, videos) in a high-dimensional space, capturing semantic relationships for natural language processing tasks ๐ค. The SAP HANA Cloud Vector Engine will also support this requirement soon!
Once you've got these two features ready to go, you only need a few lines of custom code to spot similar mails in real time and inject the previous answer as additional context. All part of the regular prompt engineering process. ๐
Last but not least, let us try to handle the language barrier! As you all know, many of our fabulous SAP customers and partners are global rockstars ๐๐ค. This means dealing with international customers and team members from various countries. So, getting reliable translations is super crucial, right? ๐๐
Only for our simple and non-productive demo scenario, we're mixing things up a bit and using Large Language Model capabilities - just for the sake of demonstration, mind you ๐. The LLM translates incoming customer demands into the user's preferred language. And it doesn't stop there! When you reply to those queries in your comfy working language, guess what? The recipient gets your response plus an automated translation whipped up by the Large Language Model. Another interesting use-case for using Generative AI ๐ฅ๏ธ๐, showing you about the endless possibilities this new technology has to offer.
Hungry for more knowledge? ๐ง Get the ball rolling by diving into our Generative AI Reference Architecture ๐๏ธ and exploring our SAP-samples GitHub repository or related Discovery Center mission. ๐ Once the generative AI hub in SAP AI Core goes public by end of , you'll be able to navigate our step-by-step guide ๐บ๏ธ to deploy our sample solution to your own SAP BTP account. ๐ค
The generative AI hub as part of SAP AI Core is generally available since Q4/2023 and the planned availability for the SAP HANA Cloud Vector Engine is scheduled for the Q1/2024 release (see Road Map Explorer). Please check the available SAP-samples GitHub repository to learn more!
Big virtual high-five ๐ to all the team members who played a part in putting together this sample use-case for TechEd 2023, including kay_, Iyad Al Hafez, Adi Pleyer, Julian Schambeck, anirban.majumdar, and the entire SAP AI Core team around z1133666 and Andreas Roth. ๐๐
Further links
Another AI blog post, right? Nope, not this time โ let's finally stop talking and start walking! ๐ฌ๐ฅ We've got some real goodies to share with you, whether you're an SAP customer, SAP partner, or just one of those strange but adorable enthusiasts like me, who can't get enough of SAP. ๐ค๐ผ
At TechEd 2023, SAP dropped some major news on the Artificial Intelligence front, including our new friend Joule, the SAP HANA Cloud Vector Engine, and the star of today's show, the generative AI hub in SAP AI Core, wrapped up in a real-world use-case. ๐๐ก
Too busy to read but still want the essential bits? ๐โโ๏ธ๐จ
Take a quick peek at our SAP-samples repository and Generative AI reference architecture๐ต๏ธโโ๏ธ
Make sure you catch the brand-new openSAP course, "Generative AI at SAP"โ๏ธ
With our fresh-off-the-press reference architecture guidance for Generative AI scenarios, you'll be more than ready to kickstart your own AI journey on SAP BTP. ๐๐ Whether you're building extensions, standalone solutions, or selling your own SaaS apps on SAP BTP, you can count on the generative AI hub for infusing your architecture with AI superpower. ๐๏ธ๐ผ๐
And it doesn't stop there! You'll also get to leverage best practices like a great SAPUI5 app, a robust SAP HANA Cloud back-end, a single and multitenant CAP application, and an API for your SaaS customers using the SAP Service Broker Framework. This sample scenario has got it all, folks!๐๐ช๐ Great news - our sample is available for the SAP BTP, Kyma Runtime and the the SAP BTP, Cloud Foundry Runtime!
This blog post is gonna shine a spotlight ๐ต๏ธโโ๏ธ on how customer and partner apps on SAP BTP can take advantage of SAPโs freshest service offering โ the new generative AI hub as part of SAP AI Core. ๐ This is all about helping you to add some serious AI capabilities to your single- and multitenant solutions.
| If you're looking to deep dive into building multitenant applications on SAP BTP, head over to our SAP-samples GitHub repo "Develop a multitenant Software as a Service application in SAP BTP using CAP" that'll steer you through the whole process. ๐ |
To make things a bit easier and to lay down some solid groundwork for all you developers ๐งโ๐ป diving headfirst into the SAP AI space, we're gonna show off a few neat AI features. These are part of a new SAP-samples GitHub repository all built around a make-believe travel agency ๐ฎ we're calling ThorTours. ๐โ๏ธ
And here's the fun part: we're gonna make ThorTours' customer support processes even better using Generative AI by reducing customer response efforts for faster handling of incoming requests and at the same time boosting the quality for higher consistency in support through reproducible responses. ๐ค๐ So hold on to your hats and let's dive in! ๐โโ๏ธ๐ฉโ๐ป
Our GenAI Mail Insights sample solution is the new best friend of all ThorTours ๐๏ธโฑ๏ธcustomer support employees ๐. It's all about giving you top-notch mail insights and automation in a comprehensive, (multitenant) SAP BTP solution enhanced with a bunch of cool AI features ๐ค. How does it do it? It uses Large Language Models (LLMs) via the generative AI hub in SAP AI Core to analyze incoming mails, offering categorization, sentiment analysis ๐, urgency assessment, and even extracts key facts to add a personalized touch ๐.
What's the cherry on top? ๐ It's the super innovative feature of Retrieval Augmented Generation (RAG) using mail embeddings. This helps in figuring out how similar requests and infuses those confirmed responses as additional context to reduce hallucinations ๐ป, ensuring you always get consistent service quality ๐ฏ. It doesn't stop there, it also summarizes and translates simple mail enquiries for But wait, there's more!
Our sample setup boosts automation by generating potential answers for customer inquiries, enhancing response accuracy and speed like a boss ๐ช. And the best part? It's not just for the travel industry โ it's adaptable for various scenarios using custom schemas. So, whatever your need, we've got you covered! ๐
Why does this matter, you ask? ๐ค Well, companies are often struggle with customer support headaches, everything from reading through long mails manually ๐ง to language hurdles ๐ and a lack of automation ๐ค. What our demo solution brings to the table is quicker, more personalized, and consistent customer service ๐ฅ, savings on cost ๐ธ, flexibility, and an edge over the competition ๐ผ. And we're not stopping there! In upcoming releases, we're even thinking about an integration with SAP Cloud Solutions such as SAP Concur or a Microsoft Exchange Online Inbox. This could supercharge operations and give a boost to data-driven decision-making ๐.
Alright, enough talking for now โ let's roll up our sleeves and dive into some of the techie highlights you can uncover in our SAP-samples repository ๐๏ธ. Ready to kick off with the first steps of processing new mail enquiries? This will lead us straight into our first challenge and the GenAI spotlight of our use-case ๐ฏ. First things first, we're going to see how our solution, built on SAP BTP, actually links up to the Large Language Models using the generative AI hub capabilities โ the latest and greatest service offering as part of SAP AI Core announced at TechEd 2023 ๐ฅ.
If you've had a go at existing LLMs like Azure OpenAI, you'll find the setup pretty familiar. ๐ง When an app needs to hook up with a Large Language Model (we're talking Azure OpenAI in our scenario), it links up with the generative AI hub and dials up the URL or what we often call the inference endpoint of a tenant-specific deployment. ๐ฅ๏ธ๐ Each tenant in a multitenant setup gets their own dedicated so-called SAP AI Core resource group onboarded, which is awesome because it offers the flexibility for individual model deployments and upcoming metering features. ๐๐
When it comes to our sample scenario, we're sending payloads to the Inference URL for Chat Completion and creating Embeddings. This process is a piece of cake ๐ฐ because it's just like the Azure OpenAI API specification, which means requests and custom modifications are a breeze ๐ฌ๏ธ. Just a heads up though, this is still an early-bird ๐ฆ release of the generative AI hub, and we might simplify the integration even more in the future. Why, you ask? Well, it's all thanks to the rising popularity of contributions to Open-Source resources like LangChain. ๐๐ฅ
| To learn more about the generative AI hub (incl. the latest preview) and how SAP is leveraging Generative AI, please check out the latest openSAP course Generative AI at SAP. |
For the time being, we've got some handy sample wrappers ๐ that make connecting to the generative AI hub deployments super easy. They take care of handling multitenancy and can be swiftly adapted for other Large Language models available. Plus, these wrappers ensure compatibility with available LangChain features once the LLM object is instantiated. This allows you as a developer ๐ฉ๐ก to keep their eyes on the prize - the actual business process!
So, we've got a customer that's just a big block of text, right? ๐ Our goal is to wrangle that into some useful info we can store it away in our database. ๐ We're using the brainpower of a Large Language Model, LangChain and custom schemas (courtesy of the npm zod package) to make this happen. It's all about having a flexible and hands-on setup with the Large Language Model. ๐ง ๐ฌ
We're using custom schemas and auto-generated format instructions (that's some prompt engineering magic right there! ๐ฎ) to get a nice and structured JSON result. ๐ This we can process further and easily store in our SAP HANA Cloud Database using our beloved CAP framework making that task a childโs play. โ๏ธ๐พ
The parsing features of LangChain and the custom schema package features are like two peas in a pod ๐ฑ They make sure that the LLM response fits into a properly typed object without any further type conversion hustle. ๐ก๏ธ Pretty cool!? Sounds almost too good to be true! ๐ฒ๐ Using these kind of custom schema definitions will allow you to adapt the solution to any other kind of support scenario, by simply updating the respective schema details!
Using a custom schema (zod + LangChain) for a programmatic LLM interaction
Apart from extracting a summary, insights and relevant key-facts from a new customer enquiry, we've also made life easier by automating the drafting of reliable response proposals for new mails. ๐ฉ This brings us to our second reveal, a cool mix of Large Language Models (LLM) magic and the capabilities of what's known as Retrieval Augmented Generation . ๐ฉโจAlso check the following blog post to learn more.
The concept is as straightforward as it is genius โ we use an LLM to create a new response for a customer enquiry, but here's the twist: we add in some extra context from previously answered similar mails. ๐
While this might sound like a lot of effort, a very simple implementation is as easy as pie. ๐ฅง All you need is access to generative AI hub that is generating so-called embeddings for incoming mails and a suitable vector store to hold these embeddings. ๐๏ธ Embeddings are vector representations of versatile input formats (e.g., text, images, videos) in a high-dimensional space, capturing semantic relationships for natural language processing tasks ๐ค. The SAP HANA Cloud Vector Engine will also support this requirement soon!
Once you've got these two features ready to go, you only need a few lines of custom code to spot similar mails in real time and inject the previous answer as additional context. All part of the regular prompt engineering process. ๐
Leveraging the power of Retrieval Augmented Generation
Last but not least, let us try to handle the language barrier! As you all know, many of our fabulous SAP customers and partners are global rockstars ๐๐ค. This means dealing with international customers and team members from various countries. So, getting reliable translations is super crucial, right? ๐๐
| Our advice for top-notch translations in the SAP universe hasn't shifted a bit. The SAP Translation Hub is our go-to recommendation for translating texts and documents in productive SAP scenarios. (Need a refresher? Click here!) ๐๐ก |
Only for our simple and non-productive demo scenario, we're mixing things up a bit and using Large Language Model capabilities - just for the sake of demonstration, mind you ๐. The LLM translates incoming customer demands into the user's preferred language. And it doesn't stop there! When you reply to those queries in your comfy working language, guess what? The recipient gets your response plus an automated translation whipped up by the Large Language Model. Another interesting use-case for using Generative AI ๐ฅ๏ธ๐, showing you about the endless possibilities this new technology has to offer.
Explore further exciting use-cases and scenarios for Generative AI
Hungry for more knowledge? ๐ง Get the ball rolling by diving into our Generative AI Reference Architecture ๐๏ธ and exploring our SAP-samples GitHub repository or related Discovery Center mission. ๐ Once the generative AI hub in SAP AI Core goes public by end of , you'll be able to navigate our step-by-step guide ๐บ๏ธ to deploy our sample solution to your own SAP BTP account. ๐ค
The generative AI hub as part of SAP AI Core is generally available since Q4/2023 and the planned availability for the SAP HANA Cloud Vector Engine is scheduled for the Q1/2024 release (see Road Map Explorer). Please check the available SAP-samples GitHub repository to learn more!
Big virtual high-five ๐ to all the team members who played a part in putting together this sample use-case for TechEd 2023, including kay_, Iyad Al Hafez, Adi Pleyer, Julian Schambeck, anirban.majumdar, and the entire SAP AI Core team around z1133666 and Andreas Roth. ๐๐
Further links
- openSAP Course GenerativeAI at SAP
- Artificial Intelligence / Machine Learning
- SAP AI Core
- SAP AI Launchpad
- SAP AI Business Services
- SAP AI Ethics
- Document Information Extraction (SAP AI Business Services)
- Data Attribute Recommendation (SAP AI Business Services)
- Personalized Recommendation (SAP AI Business Services)
- SAP Managed Tags:
9 Comments
