WSO2 Internship - IV

In this series of posts, I provide some key points and Ideas that I report to my university during my internship. I arrange these posts as a set of reports, where each report provides the summary of tasks I have done during those four weeks of my training and the experience I gained during that period. This post is the fourth post in the series of posts about my internship @WSO2.

These four weeks, I mainly worked towards designing dashboards, the third and last milestone of my project. I followed the following order while developing the dashboard: first prepare the spark scripts required to display the analytics related to the HL7. Then I developed the dashboard with existing gadgets using gadget generation wizards. Finally, I started work on the custom dashboard. For all, I tried following the ESB analytics, which has nearly similar aspects to my scenario. By the end of these four weeks, I had nearly completed the dashboard’s summary parts with some fixes regarding chart zooming, and chart values displaying inversely. And I was working on the search part based on the BAM’s management console.

Problems Encountered

ARBITRARY ATTRIBUTES

These are attributes used in Event Stream, in addition to usual meta, correlation, and payload data. These are specific for different scenarios; for HL7, we can obtain more than 600 Arbitrary attributes. We can get this through Event Stream in DAS, store it in the event store, and use it for analysis with Spark Scripts. When using these specific HL7 arbitrary attributes, I faced issues in SELECT and INSERT queries. For explicit references, look at the below link.

Unusual Arbitrary Attribute

Things I Learned

INCREMENTAL PARAMETERS

When analyzing with Spark scripts in WSO2 DAS, we may need to do analysis based on repetitive time, which means analyzing data periodically to get historical stats like PER MINUTE, PER HOUR, PER DAY, PER MONTH analysis. For this, we may need to go through all data, to avoid that, we can add the incremental parameter with WINDOW and TIMES. Consider If you want to analyze data by PER HOUR using PER MINUTE table, we can set the incremental parameter in PER MINUTE as the table which contains PER MINUTE based analysis contents, and add a Window; windows are one step above the level of the time unit, for this case it is HOUR. and the third parameter is optional which is the number of records we need to look back. By default, it is 1 if we not set it. It means it will go to the last record of the particular table (PER HOUR) and update it based on the PER MINUTE table by taking the values which are after the set Incremental Parameter Value and do analysis. By this, we can omit redundant data analysis and increase the efficiency of data analysis in a large block of data.

JAGGERY CONTROLLER

[NOTE THIS NOT DEPRECIATED, BUT I USED IT JUST AS A CONTROLLER, INSTEAD OF THIS, THEY ARE NOW USING UUF]

The JavaScript executing on the client-side cannot fetch the data from the server, so they give an ajax request which will direct to the URL of jaggery controllers. (generally for purpose of dashboards these Jaggery controllers are located in /repository/deployment/server/jaggeryapps/portal/controllers/apis/ and when we used for URL we often use portal/apis/nameofjaggerycontroller by this we can give an ajax request which will direct to the jaggerycontroller on the server-side, which gets the request and fetches the data from the datastore based on the requirement of the ajax request and gives the response back to the client, then the client-side callback function will execute actions on the received response based on its function in the case of success, and in the case of failure it will direct to the error case function. This is how a jaggery controller is used to fetch the details from the database in the server by the client-side application.

References

1. http://sachith.withana.net