Technology Blogs by SAP
Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
cancel
Showing results for 
Search instead for 
Did you mean: 
mandy_krimmel
Advisor
Advisor
This blog describes how to send automated notifications (after the June-10-2018 update) in case the JMS resources are critical or exhausted using an integration flow.

Automated Notification for Critical or Exhausted JMS Resources


As mentioned in the blog 'JMS Resources and Size Limits' limited resources are available on the JMS broker. To get notified as soon as the resources get critical is required to take early actions to avoid runtime issues. This blog describes how to enable the alerting using a simple integration flow.

How to use the Alert Notification Service for Alerting with the OData APIs is described in the blog Automated Notifications via SAP Alert Notification Service for Critical or Exhausted JMS Resources.

Scenario Description


To enable the notification we use the OData APIs for JMS queue monitoring in an integration flow.. These APIs can be consumed via https://<APIUrl>/api/v1 where <tmn> is the address of the tenant management node(Neo) or the address of the API service instance (CF). Here we will use the API for the JMS broker monitoring via https://<APIUrl>/api/v1/JmsBrokers('Broker1') .

Overall scenario looks like this:


The flow is triggered by a timer, fetches the JMS Resource data via OData adapter, evaluates the JMS resource status in a script and send a notification mail if the resources are critical or exhausted.

Let's create the flow.

Create Integration Flow with Timer Start Event and OData Receiver


Create an integration flow with Start Timer Event. Using Request-Reply step call OData APIs to fetch details of the JMS broker via OData receiver channel. Below find the configuration of OData Receiver channel:


Note, that you need to use Operation Read(Get), enter JmsBrokers('Broker1') as Resource Path and $expand=QueueStates as Query Options.

Read JMS Resources into Properties in Content Modifier


In a Content Modifier step you now read all the JMS resources via XPath into properties:



Create the following properties:

QueueCapacityOK: XPath to /JmsBrokers/JmsBroker/CapacityOk

QueueCapacityError: XPath to /JmsBrokers/JmsBroker/CapacityError

QueueCapacityWarning: XPath to /JmsBrokers/JmsBroker/CapacityWarning

MaxQueues: XPath to /JmsBrokers/JmsBroker/MaxQueueNumber

Queues: XPath to /JmsBrokers/JmsBroker/QueueNumber

Transactions: XPath to /JmsBrokers/JmsBroker/IsTransactedSessionsHigh

Providers: XPath to /JmsBrokers/JmsBroker/IsProducersHigh

Consumers: XPath to /JmsBrokers/JmsBroker/IsConsumersHigh

Capacity: XPath to /JmsBrokers/JmsBroker/Capacity

MaxCapacity: XPath to /JmsBrokers/JmsBroker/MaxCapacity

 

Evaluate Overall JMS Resource Status via Groovy Script


In a Groovy Script we now calculate the overall JMS resource status. Use the following script code:
import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;

def Message processData(Message message) {
// Get Properties
map = message.getProperties();
MaxCapacity = new BigDecimal(map.get("MaxCapacity"));
Capacity = new BigDecimal(map.get("Capacity"));
BigDecimal PercentCapacity = Capacity * 100 / MaxCapacity;
BigDecimal CriticalLimit = MaxCapacity / 100 * 80;
BigDecimal ExhaustedLimit = MaxCapacity / 100 * 95;
MaxQueueNumber = new BigDecimal(map.get("MaxQueues"));
QueueNumber = new BigDecimal(map.get("Queues"));
BigDecimal CriticalLimitQueues = MaxQueueNumber - 1;

if (Capacity >= ExhaustedLimit) {
CapacityStatus = "Exhausted";
}
else if (Capacity >= CriticalLimit) {
CapacityStatus = "Critical";
}
else {
CapacityStatus = "OK";
}

if (QueueNumber >= CriticalLimitQueues) {
QueueStatus = "Critical";
}
else {
QueueStatus = "OK";
}

int QueueCapacityWarning = new Integer(map.get("QueueCapacityWarning"));
int QueueCapacityError = new Integer(map.get("QueueCapacityError"));
if (QueueCapacityError >= 1) {
QueueCapacityStatus = "Exhausted";
}
else if (QueueCapacityWarning >= 1) {
QueueCapacityStatus = "Critical";
}
else {
QueueCapacityStatus = "OK";
}

int Consumers = new Integer(map.get("Consumers"));
if (Consumers == 1) {
ConsumerStatus = "Critical";
}
else {
ConsumerStatus = "OK";
}

int Providers = new Integer(map.get("Providers"));
if (Providers == 1) {
ProviderStatus = "Critical";
}
else {
ProviderStatus = "OK";
}

int Transactions = new Integer(map.get("Transactions"));
if (Transactions == 1) {
TransactionStatus = "Critical";
}
else {
TransactionStatus = "OK";
}

message.setProperty("PercentCapacity", PercentCapacity);
message.setProperty("CapacityStatus", CapacityStatus);
message.setProperty("QueueCapacityStatus", QueueCapacityStatus);
message.setProperty("QueueStatus", QueueStatus);
message.setProperty("ConsumerStatus", ConsumerStatus);
message.setProperty("ProviderStatus", ProviderStatus);
message.setProperty("TransactionStatus", TransactionStatus);

if ((CapacityStatus == "Exhausted") || (QueueCapacityStatus == "Exhausted")) {
message.setProperty("JMSStatus", "Exhausted");
}
else if ((CapacityStatus == "Critical") || (QueueCapacityStatus == "Critical") || (QueueStatus == "Critical") || (ConsumerStatus == "Critical") || (ProviderStatus == "Critical") || (TransactionStatus == "Critical")) {
message.setProperty("JMSStatus", "Critical");
}
else {
message.setProperty("JMSStatus", "OK");
}
return message;

}

Check Queue Capacity Status in Router


If you want to get additional information about dedicated message queues in case the queue capacity is critical or exhausted in a Router step check the Queue capacity Status and route to a script doing the detailed analysis for the dedicated queues that are critical or exhausted.


Define Non-XML Expression ${property.QueueCapacityStatus} = 'OK' for the branch going to the second Router. Default branch is the branch to the Filter step which is executed if the status is not 'OK'. In the Filter you define an XPath to the //QueueStates to get the details for all the message queues available in the tenant. Note that you have to select Nodelist as Value Type.


In a Script step evaluate the message queues and return all critical and exhausted message queues as properties QueuesCriticalNames and QueuesExhaustedNames,
import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
import groovy.xml.*;
import java.util.*;
import java.text.SimpleDateFormat;

def Message processData(Message message) {
def body = message.getBody(String.class);

def SourceMessage = new XmlParser().parseText(body);
def QueuesExhausted = "";
def QueuesCritical = "";

SourceMessage.QueueState.each {
value = it.text();
state = it.State.text()
name = it.Name.text()
if ("1".equals(state)) {
QueuesCritical = QueuesCritical +' '+ name
}
}

SourceMessage.QueueState.each {
value = it.text();
state = it.State.text()
name = it.Name.text()
if ("2".equals(state)) {
QueuesExhausted = QueuesExhausted +' '+ name
}
}

message.setProperty("QueuesCriticalNames", QueuesCritical);
message.setProperty("QueuesExhaustedNames", QueuesExhausted);
return message;
}

Check JMS Status in Router


In the second Router step check the JMS Status and route to the mail receiver in case the JMS resources are critical or exhausted:



Define Non-XML Expression ${property.JMSStatus} = 'OK' for the branch going to the End event. Default branch is the branch to the Mail receiver which is executed if the status is not 'OK'.

Configure Mail Receiver


Now we configure the Mail receiver channel sending out the notification. Configure the mail server and authentication you want to use. In the Mail Attributes define the mail receiver and sender mail address and the Mail Subject and Mail Body:

Subject: JMS Ressources ${property.JMSStatus}

Mail Body:

The JMS Resources on the Cloud Integration tenant are ${property.JMSStatus}.
Overall Queue Capacity Status: ${property.CapacityStatus} (${property.PercentCapacity} Percent)
Capacity Status for Queues: Error: ${property.QueueCapacityError} Queues; Warning: ${property.QueueCapacityWarning} Queues; OK: ${property.QueueCapacityOK} Queues
Queues with Warnings:${property.QueuesCriticalNames}
Queues with Errors: ${property.QueuesExhaustedNames}
Queue Status: ${property.QueueStatus} (${property.Queues} / ${property.MaxQueues} )
Transactions Status: ${property.TransactionStatus}
Consumer Connections Status: ${property.ConsumerStatus}
Provider Connections Status: ${property.ProviderStatus}

Deploy and Run the Integration Flow


Configure the Timer start event to run the flow every day or any other interval that suits your requirements. Deploy the integration flow.

Now you will get a Email as soon as the JMS resources get critical.
36 Comments
markus_villiger
Explorer
Hello Mandy

 

Great blog!

Is it possible also to get the name of the queue per API which is exhausted?

 

thanks and regards,

Markus

 

regards, Markus
mandy_krimmel
Advisor
Advisor
0 Kudos
Hello Markus,

currently this is not possible with the released APIs, we will check internally if we can enhance the API. the problem is that such queue-specific calls are much more expensive and are not supported in high frequency be the messaging system. Let's see what we can achieve.

Best regards

Mandy
markus_villiger
Explorer
0 Kudos
Hi Mandy

Thanks allot for feedback and checking this topic:)

regards,

Markus
mandy_krimmel
Advisor
Advisor
Hello Markus,

with the March update the API is extended so that you can also get the names of the queues in error/warning.

I will update the blog shortly before the change will get released.

Best regards

Mandy
markus_villiger
Explorer
0 Kudos
Hi Mandy

 

This are great news! Thanks alot!

 

regards,

Markus
0 Kudos
HI Mandy,

Thanks for documentation. But i am getting error of 401 as you can see in the attachments.




I am using the Oauth2 client credentials.


Oauth2cred


i have also created the Security credentials and it is deployed.

 


 

So could you please help me to get the issue resolved?

 

Thanks in advance!

 

T&R,

Prabhat

 

 
mandy_krimmel
Advisor
Advisor
0 Kudos
Hello

the error means that something in the credentials is not correct. Please check carefully, maybe redeploy the credentials and restart the integration flow. If you do not get it resolved please open a ticket on LOD-HCI-PI-CON-OD.

Best regards

Mandy
0 Kudos
Thanks mandy.krimmel for the info but unfortunately, it is not working after changing and redeployment as well.

So i will raise a OSS ticket as suggested

 

T&R,

Prabhat.
markus_villiger
Explorer
0 Kudos
Hi Mandy

Is the Update with the queue names already released to the customers?
Somehow we couldn't fetch the queue names at the moment.

 

regards,

Markus
markus_villiger
Explorer
0 Kudos
Hello Mandy

 

Sorry forget my question. I mistaked:(

regards,

Markus
valter_oliveira
Active Contributor
0 Kudos
Hi.

Thanks, great info.

Is it predicted to add the message count and capacity together with the names of the queues in error/warning for a more refined monitoring?

Regards

Valter
mandy_krimmel
Advisor
Advisor
Unfortunately this is not possible because those details cannot be retrieved easily as we are using eventing mechanism with is only able to provide basic details.

Those details would need to be checked by the alerted tenant admin via the queue monitor.

Best regards

Mandy
markus_villiger
Explorer
0 Kudos
Hello Mandy

 

We are searching for the Data Store also a Monitoring / Alerting functionality , similar to the JMS Resources in this blog.

Is there something available on this topic?

We would be interested:

  • used memory per Data Store

  • Total memory used ober all Data Stores


Unfortunetly we couldn't find any API which provides this kind of information.

thanks and regards,

Markus

 

 

 

 

 

 
mandy_krimmel
Advisor
Advisor
Hello Markus,

there is not yet an API available to get similar information for the different Data stores. We have this on our ToDo list, but I cannot give you an estimation by when this would be provided.

Best regards

Mandy

 
markus_villiger
Explorer
0 Kudos
Hi Mandy

Thanks for your feedback!
Good to hear that SAP already identifed this as a feature:)

 

Would be great if there will be a notification when the feature is available.

Thanks and regards,

Markus

 

 
valter_oliveira
Active Contributor
0 Kudos
Hi Prabhat. Was this solved? Facing same issue.
fareshs1
Explorer
0 Kudos
Hi Mandy

 

i got the below error when i deploy the iflow :

 

[CAMEL][IFLOW][EXCEPTION] : org.apache.camel.FailedToCreateRouteException: Failed to create route Process_1 at: >>> SetProperty[QueueCapacityOK, xpath{java.lang.String}] <<< in route: Route(Process_1)[[From[direct:Automated_Notification_for_Cri... because of java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
[CAMEL][IFLOW][CAUSE] : Cause: org.apache.camel.RuntimeCamelException: java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
[CAMEL][IFLOW][CAUSE] : Cause: java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
mandy_krimmel
Advisor
Advisor
0 Kudos
this sounds like an error in your script, would suggest to check this.

Best regards

Mandy
fareshs1
Explorer
0 Kudos
thank you Mandy for your reply, I did double check the script many times but still it seems that there are something missing.

just to confirm do I need to import any sort of additional library to the script in your original post ?

 

below is my script :

 
import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
import java.util.*;
import groovy.xml.*;

def Message processData(Message message) {
// Get Properties
map = message.getProperties();
MaxCapacity = new BigDecimal(map.get("MaxCapacity"));
Capacity = new BigDecimal(map.get("Capacity"));
BigDecimal PercentCapacity = Capacity * 100 / MaxCapacity;
BigDecimal CriticalLimit = MaxCapacity / 100 * 80;
BigDecimal ExhaustedLimit = MaxCapacity / 100 * 95;
MaxQueueNumber = new BigDecimal(map.get("MaxQueues"));
QueueNumber = new BigDecimal(map.get("Queues"));
BigDecimal CriticalLimitQueues = MaxQueueNumber - 1;
QueueCapacityStatus = "";

if (Capacity >= ExhaustedLimit) {
CapacityStatus = "Exhausted";
}
else if (Capacity >= CriticalLimit) {
CapacityStatus = "Critical";
}
else {
CapacityStatus = "OK";
}

if (QueueNumber >= CriticalLimitQueues) {
QueueStatus = "Critical";
}
else {
QueueStatus = "OK";
}

int QueueCapacityWarning = new Integer(map.get("QueueCapacityWarning"));
int QueueCapacityError = new Integer(map.get("QueueCapacityError"));
if (QueueCapacityError >= 1) {
QueueCapacityStatus = "Exhausted";
}
else if (QueueCapacityWarning >= 1) {
QueueCapacityStatus = "Critical";
}
else {
QueueCapacityStatus = "OK";
}

int Consumers = new Integer(map.get("Consumers"));
if (Consumers == 1) {
ConsumerStatus = "Critical";
}
else {
ConsumerStatus = "OK";
}

int Providers = new Integer(map.get("Providers"));
if (Providers == 1) {
ProviderStatus = "Critical";
}
else {
ProviderStatus = "OK";
}

int Transactions = new Integer(map.get("Transactions"));
if (Transactions == 1) {
TransactionStatus = "Critical";
}
else {
TransactionStatus = "OK";
}

message.setProperty("PercentCapacity", PercentCapacity);
message.setProperty("CapacityStatus", CapacityStatus);
message.setProperty("QueueCapacityStatus", QueueCapacityStatus);
message.setProperty("QueueStatus", QueueStatus);
message.setProperty("ConsumerStatus", ConsumerStatus);
message.setProperty("ProviderStatus", ProviderStatus);
message.setProperty("TransactionStatus", TransactionStatus);

if ((CapacityStatus == "Exhausted") || (QueueCapacityStatus == "Exhausted")) {
message.setProperty("JMSStatus", "Exhausted");
}
else if ((CapacityStatus == "Critical") || (QueueCapacityStatus == "Critical") || (QueueStatus == "Critical") || (ConsumerStatus == "Critical") || (ProviderStatus == "Critical") || (TransactionStatus == "Critical")) {
message.setProperty("JMSStatus", "Critical");
}
else {
message.setProperty("JMSStatus", "OK");
}
return message;

}
mandy_krimmel
Advisor
Advisor
0 Kudos
Hello,

looks ok so far. Do you already get the error when you deploy the flow or in the monitor message processing? What is the status of the integration flow in the artifact monitor?

Best would be you create a ticket on LOD-HCI-PI-CON-SOAP and attach the integration flow.

Please also share the ticket here.

Best regards

Mandy
fareshs1
Explorer
0 Kudos

it failed to deploy the iflow (failed to start).

Question from where I can create this ticket ?

mandy_krimmel
Advisor
Advisor
0 Kudos
I added your script in my tenant and deployed without problem. There is no message processing log for this integration flow?

Another idea: check the declaration of the properties in the content modifier, maybe this is where the error is. Could you share a screen shot?

Thank you,

Mandy
fareshs1
Explorer
0 Kudos


 

here
mandy_krimmel
Advisor
Advisor
0 Kudos
hm, looks good. Could you also share the configuration of the OData channel to fetch the details?

Thank you

Mandy
fareshs1
Explorer
0 Kudos


 

message protocol: Odata V2
mandy_krimmel
Advisor
Advisor
0 Kudos
Hello ,

could you please change to Read(Get) operation in the OData channel instead of Query(Get)? See this comment in the blog:

--

Note, that you need to use Operation Read(Get), enter JmsBrokers(‘Broker1’) as Resource Path and $expand=QueueStates as Query Options.




Thank you

Mandy
fareshs1
Explorer
0 Kudos
still getting the same error :

[CAMEL][IFLOW][EXCEPTION] : org.apache.camel.FailedToCreateRouteException: Failed to create route Process_1 at: >>> SetProperty[QueueCapacityOK, xpath{java.lang.String}] <<< in route: Route(Process_1)[[From[direct:Automated_Notification_for_Cri... because of java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
[CAMEL][IFLOW][CAUSE] : Cause: org.apache.camel.RuntimeCamelException: java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
[CAMEL][IFLOW][CAUSE] : Cause: java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
mandy_krimmel
Advisor
Advisor
0 Kudos
Hello,

please open a ticket on LOD-HCI-PI-CON-SOAP, attach the integration flow and error message

Thank you

Mandy
fareshs1
Explorer
0 Kudos
i did some digging and found the below service in SAP cockpit should i configure it ? if yes please provide me with required data to enter.

keep in mind that i did use the information available in the article but it gave me an error "400 bad request"

 

mandy_krimmel
Advisor
Advisor
0 Kudos
No, OData Provisioning is not required.

Are you running in CF? Then you need to activate the API plan to call the OData APIs, see: Setting Up Inbound HTTP Connections (for API Clients) | SAP Help Portal

Best regards

Mandy
fareshs1
Explorer
0 Kudos
i'm running on neo environment.
mandy_krimmel
Advisor
Advisor
0 Kudos
Could you please open a ticket on LOD-HCI-PI-CON-SOAP attach the integration flow and error message. It is really impossible to trouble shoot this via the blog comments.

Thank you

Mandy
0 Kudos
Hi Mandy,

 

Thanks for this blog!

How about if we want to access fields like /QueueStates/Broker/IsTransactedSessionsHigh?

Querying using $expand=QueueStates returns us the overall status of JMS but whenever we try to access QueueStates sub-entity, we encounter Bad Request - An exception of type 'OData2Exception' occurred.

Using the same JmsBrokers(Key='Broker1').

 

Cheers,

Dan
mandy_krimmel
Advisor
Advisor
0 Kudos
Hello Dan,

please only use the described/released APIs. To get more details is not supported as this would raise too many calls to the broker. You need to stick to the released APIs.

Best regards

Mandy
kevindass
Participant
0 Kudos
mandy.krimmel Thank you for the blog.

Would like to know how these API's could be used when enterprise/customer has activated "Cloud Identity Services" ?

With "Cloud Identity Services" we get 200 response with "sign in with" page just like the way in web browser when logging into BTP


View from Postman:


 
markus_beier
Explorer
0 Kudos
Hi Kevin,

the posted response hints that a SAML workflow was to be started, which should never be the case for calls to the API hostname. This could happen if you didn't use the API hostname, but an Integration Suite UI hostname. The hostname of the Integration Suite UI have ".integrationsuite." as part of the domain. Did you try it with that URL?

This is the documentation on how to retrieve the API hostname and construct the complete API URL: https://help.sap.com/docs/cloud-integration/sap-cloud-integration/http-calls-and-uri-components

The API access in case SAP Cloud Identity Services are activated should still be possible, however, if you want to use Basic Authentication, you have to do some additional configuration, so that the users can still be authenticated:

https://help.sap.com/docs/cloud-integration/sap-cloud-integration/setting-up-sap-identity-authentica...

Best regards,
Markus