Friday, January 21. 2011
Improving RGraph Plot with Real Data via Web Services
In a previous entry I talked about how to implement a scatter graph using javascript and the new HTML5 canvas element. This post was just the first step towards a more robust solution. If you remember my intention was studying the feasibility of a migration from an old applet application to a new HTML5 one. Today I will try to continue the PoC plotting real data inside the graph instead of the fixed sine wave (javascript generated) RGraph drew before.
First of all the real samples must be retrieved from somewhere and, obviously, a relational database is the easiest solution (besides the current application uses a database too). It is important to remember that the scatter plot represents a physical magnitude (wind strength, sea level, high of waves,...) in time. For the PoC some JPA (Java Persistence API) entities were created, these classes are very simple. Every sample only contains the timestamp and the measured value. It is clear that in a real scenario tables would usually have more columns (current devices measure several magnitudes at the same time) but I think this is enough for this example. Three classes are presented:
- PlotType.java: Enumeration that handles all the plots the application can draw (my idea is simple, if a new plot is needed it has to be added here). SEA_LEVEL and WAVE are the current plot types (Sea Level is measured one sample per minute and the High of Waves one per hour).
- PlotSample.java: Abstract entity class that represents any sample (as I said, a timestamp and the measured value).
- SeaLevel.java and Wave.java: Two examples of real plots, both extend PlotSample and define a <PLOT_TYPE_NAME>-findSamplesBetweenDates query which is always used to query for the samples between two specified dates.
Once the data is organized a web service is needed. It is quite clear this technique is perfect to retrieve the data for plotting. Nowadays there are a lot of JavaScript libraries to easily call a server-side Web Service (prototypejs, jquery,...) and managing JSON (JavaScript Object Notation). For this part I decided to use JAX-RS (Java API for RESTful Web Services) to implement the service and JAXB (Java Architecture for XML Binding) for generating the JSON data. JAX-RS (which I had rarely used) is incredible easy and very good services are implemented in a little time. Besides JSON can be specified as the result to be produced and this, using JAXB at the same time, lets you return a Java object avoiding all JSON generating stuff. Only two classes are presented here:
- PlotData.java: This is the JAXB element that represents the data to be plotted by the browser. The data returned when a scatter plot is going to be drawn is just a simple class with some properties (start and end timestamps, plot type name, description,...) and a map with the samples (value keyed by the timestamp).
- PlotResource.java: The JAX-RS resource which receives the name of the plot type to graph, start and end timestamp. With this information it queries the database and creates the PlotData to be returned.
Two important tips here. First, JavaScript (in my opinion) handles the timezone in dates not very good. The date object is created using the timezone of the browser/system (GMT+1 in my case) but it is quite difficult to convert from one timezone to another. The date object has a getTimezoneOffset method that returns the offset in minutes between your zone and GMT, but your code needs to add and subtract offsets all the time. For this reason I decided to manage timestamps (milliseconds since standard epoch of 1/1/1970) instead of string dates. A timestamp will be represented in JSON with a String containing the big number of milliseconds prefixed by a character "T" (for example the epoch 1/1/1970 would be the string "T0").
Second, I wanted the JSON response was (more or less) the following:
{ "type":"WAVE", "start":"T1295095138172", "end":"T1295181538172", "description":"High of the waves", "legend":"Meters (m)", ... "data": { "T1295096400000":"3.4", "T1295100000000":"3.16", "T1295103600000":"3.52", ... } }
The JSON data part (the list/map of samples) should be the pair timestamp and value, which is the natural way to represent a map in JSON. Nevertheless JAXB has some problems with maps and they are not represented in this natural way (see RFE JERSEY-551 for more info). So I added the JsonMapAdapter.java class this enhancement manages in order to obtain the JSON representation I showed before.
So now the browser can query via the restful web service in order to obtain the data to plot. The final part is performing some little changes in the JavaScript developed for the previous post, now the array of samples has to be retrieved calling the Web Service. The presentation part are the following files:
- PlotTypeBean.java: A extremely simple request scoped JSF bean only used to list the plot types defined in the application (the combo box at the right) and select one of them.
- TestRGraphChart.xhtml: XHTML page very similar to the one used in the first entry but using facelets/JSF in order to call the previous bean and so on.
- TestRGraphChart.js: The JavaScript file that uses prototype.js and JavascriptToolbox data.js to call the web service and parse/manage JSON and timestamps.
So the PoC is ready, RGraph plots a scatter XY graph of which data is retrieved via Web Service. The application was deployed in a Java EE 6 compatible application server (Glassfish v3) using its default database engine (derby/javadb). I personally think this is an elegant solution for my problem, all the plotting part is done by the browser and the server only sends the samples to draw. The solution works quite well in browsers with decent canvas performance (which are all the current browsers except IE8, check the previous post). In the video I first call restful web service with WAVE plot type (high of waves), I use no parameters so it returns 24 hours of data (WAVE plot type has one sample per hour). I change to the graph page in order to display the whole day of the sea level plot (one sample per minute). I also move the graph forward and reverse in time. Then I increase the range of time to display a whole week of data (less than 10,000 points cos I did not insert in my database the whole week of samples). And then I choose the time range to plot selecting directly over the canvas element. Finally I select the other plot type, high of waves again. As you can see it works very well in my chromium 6 browser. The application is also very smooth in iceweasel 3.5 and epiphany 2.30 (the other two browsers I have installed in my debian box).
At the moment the JavaScript always requests all the samples to plot. It is clear that some data can be reused (for example when the plot is moved forward or reverse) and the JavaScript could be improved to only request the missing samples and after join all of them. You can improve that part if you need and want it.
As a summary this entry goes one step further in the scatter graph problem. Now the solution plots real data. Data which is read from a database and requested by the browser using a RESTful Web Service and JSON. Here I present the complete NetBeans project (without any jar file as usual). As I said in the first entry of this series the PoC is currently unusable (or at least without a second plan for IE8), but the time for triggering HTML5 features is getting closer day by day...
Please plot a smile in your face!
Comments