Integrating Twitter tweets into SAC

[hai_murali_here]

Hello All,

As we see that there are more Cloud Adoptions in the Analytics area, I was exploring the possibilities of integrating Twitter tweets into SAP Analytics Cloud(SAC), and this blog explains the architecture and the steps involved to achieve the same.

The workflow is getting the Tweets into SAP HANA and exposing them through the Open ODS View to SAC.

The following diagram, explains the architecture of the same.

 



 

Here we created a Twitter Application which holds the information about,

Consumer_Key

Consumer_Secret

Access_Token

Access_Token_Secret

 

These information are given to the Configuration.Java file of the TwitterAnalysis app.

The Twitter #HashTag for the search also maintained in this file.



The HDBConnection.java file ensures that the connection to the HANA DB works as expected.

The TwitterConnection.java file ensures that the connection to Twitter works as expected.

Executing the SearchTweets.java file brings the data from Twitter into SAP HANA DB. You should have created the table in HANA, before executing this file to get the data and store in HANA.

The following blog explains, how this can be achieved.

https://blogs.sap.com/2017/06/07/integrating-twitter-with-sap-hana-for-text-analysis/

 

Once the Tweets are into HANA, you can expose this as a native HANA View(Analytics/Calc View) or via BW (through Open ODS View or so) to SAC.

 



 

To get the Live data, a Live Connection can be created in SAC to HANA directly or to the BW on HANA system(if the data exposed via Open ODS View).

Getting Live data involves configuring CORS(Cross Origin Resource Sharing) in the backend(either HANA or BW).You can refer the SAC Help document about these CORS configurations.

In this example, I have a BW on HANA system, where the Tweets(which resides on HANA table) are exposed via Open ODS View to the SAC.

The steps involved here are,

1. Make sure that the tweets are stored in the HANA table, after executing the SearchTweets.java file


2. In SAP HANA Studio, go to the BW Modelling perspective, and create an InfoArea


3. Create a Open ODS View on top of the HANA Data source


4. Create a Composite Provider on top of the Open ODS View


5. Create a BEx Query on top of a Composite Provider


6. Create an Import Connection to the BW on HANA system, in SAC (You need the Cloud Connector and Cloud Agent configured. For more details on this, you can find SAC Help document)


7. Using this connection, create a Model and then the Story on top of it

 

Steps 2,3 and 4 can be done with BW Modelling perspective of HANA Studio

Step 5, needs BEx Query Designer

Steps 6 and 7 need to be done in SAC

 



 

Since we are using the Import type Connection, you can schedule the Data import every hr or so, to get the latest data Or you can also trigger import just by clicking the “Update Model Now” option in the Data Model->Data Sources tab



Once you refresh the Story, the tweets would be shown in SAC.

You can modify the SearchTweets.java file based on the need, to get more fields from tweets like location where its tweeted, etc.

 

Hope you would have found this topic interesting and thanks for reading it.

Please feel free to leave your Comments/Feedbacks.

 

Rgds,

Murali

[paulcarney]

That's a really informative blog. Thanks Murali.

[harshil_joshi]

Hello Murali, can you help me in this? I am still getting an error even after importing twitter4j* reference library.