Saturday, November 15. 2014
Compiling Servlet (part II)
In the previous post I started the implementation of a little PoC about in-line compiling in a Java application. The first entry showed how a Servlet compiled another Servlet from source and used the resulting class to instantiate a new object that actually served the request. This idea is useful for hot replacement of certain classes without re-deploying the whole application bundle. Nevertheless using a Servlet java file as the replacement unit is quite radical, when I thought about this hot-replacement I was thinking in Java Server Pages (JSP) or any other modern Java technology.
A JSP is a way of generating dynamic HTML pages embedding java code inside an HTML template. This is an old standard in the JavaEE specification and I am sure that all of you know about them. Obviously storing JSP pages outside the WAR file for a later use is much simpler and easier to maintain. The JSP could be stored in another repository (DDBB, NoSQL, Cache system,...) and the application would use that JSP if it has been modified. As I said with JSP the in-line compilation idea is less cumbersome and easier to understand.
If you are familiar with JavaEE you already know that any Servlet container parses JSP files into a Java Servlet source file and then that file is compiled and executed. So, more or less, the container already performs the idea I am trying to explore. The main difference is that, by default, the container always expects the JSP files located in the File System (after de-archiving the deployment file). Usually they use temporary or working directories where intermediate Java source and class files are generated. Besides all the containers check for modifications in those JSP files and, if the requested JSP is modified, it compiles the file again before processing the request (this behavior can be disabled and usually it is not recommended for production environments, compiling is a heavy task).
At this point I managed two ideas:
Just relay in common container behavior and put something (a filter for instance) that previously reads the JSP from the repository and updates the current JSP in the file system if necessary. Doing that the container then would detect that the JSP had been modified and it would compile the file into a Servlet again.
Follow the idea presented in the previous entry, do all the job by myself.
The first point was the easiest solution but it has two main drawbacks: first, the container should be configured to detect changes (usually the container is only configured in development mode for that task) and, second, this idea manages files in the file system (thing that I tried to avoid).
So finally I decided to explore the solution of parsing and compiling the JSP by my own. The second part is already done (see the previous post) but the first part is a new one (parsing the JSP into a Servlet). Following the idea started in the previous entry, I have tried to use the jasper 2 JSP engine, the engine that tomcat container uses. Looking the code of the jasper implementation, the main ideas are the following:
Tomcat uses the JDTCompiler which is a implementation that internally uses the JDT / ECJ (the same Java compiler that I used in the previous entry).
The JDTCompiler extends the abstract Compiler class. The latter class performs the parsing phase (from the JSP to the Java Servlet Source) in the generateJava method, the second phase (compiling Java to class) is not implemented, the method generateClass is declared abstract (it is the JDTCompiler class the one that implements it using the ECJ compiler).
So I tried to implement another Compiler that extended the abstract class and overrided both methods (generateJava for generating the Java in memory, and generateClass using the SimpleCompiler developed in the previous entry). Then my class would substitute the default JDTCompiler. The main problem is that the generateJava method cannot be directly reused because of the fact that it uses a private method setupContextWriter to create the file writers. If that method could have been overridden the writer could be replaced by a memory one without more modifications. Instead of copying the whole method and replacing the writer inside it, I decided to compile my own version of the jasper.jar with that method declared as protected (I know it is a big shoddy job but this is just a PoC and it is much easier to code).
Once I could override that method I developed my MemoryCompiler that implements a jasper compiler but overrides both methods: the now protected setupContextWriter which construct a Writer from a ByteArrayOutputStream (in memory) and the generateClass which uses the SimpleCompiler to insert the resulting java class into memory.
I also needed to implement a MemoryJspCompilationContext which extends the default JspCompilationContext. It is very similar to the default one but there are minor differences to obtain the JSP source from a simple string instead from a file.
Finally I created a DefaultCompilerOptions which is a bunch of options that the jasper compiler uses but defaulted to the values needed by my compilations. The important method is the getCompilerClassName which returns the new and bright MemoryCompiler class which I implemented.
With all this changes my old CompilerServlet was extended in order to generate a Servlet class from a JSP using this method:
protected void processCompileJsp(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { try { String jspSource = readJspFile(request.getParameter("name")); DefaultCompilerOptions opts = new DefaultCompilerOptions(this.getServletContext()); JspRuntimeContext rtctx = new JspRuntimeContext(this.getServletContext(), opts); JspCompilationContext clctxt = new MemoryJspCompilationContext( "/" + request.getParameter("name") + ".jsp", jspSource, opts, this.getServletContext(), null, rtctx); org.apache.jasper.compiler.Compiler clc = clctxt.createCompiler(); clc.compile(true, true); Class clazz = ((MemoryCompiler) clc).getGeneratedClass(); if (javax.servlet.jsp.JspFactory.getDefaultFactory() == null) { // initialize in case Tomcat has not done yet javax.servlet.jsp.JspFactory.setDefaultFactory( new org.apache.jasper.runtime.JspFactoryImpl()); } HttpServlet servlet = (HttpServlet) clazz.newInstance(); servlet.init(this.getServletConfig()); servlet.service(request, response); } catch (Exception e) { throw new ServletException(e); } }
The idea is almost the same which I used to compile the Servlet source, now the JSP is read (again it is retrieved from the WAR file) and a compilation context is created using my MemoryJspCompilationContext. Besides the DefaultCompilerOptions sets the compiler to my MemoryCompiler class. This way the JSP is parsed and compiled using my own classes and only using memory arrays for both processes.
I tested the JSP compilation inside a tomcat 8 container and I needed to put some packages inside the general lib folder. The reason is that the internal classes of the Tomcat infrastructure also uses the Jasper engine, so there were conflicts between the both levels (application and tomcat system level). Finally I solved this problem placing some classes in the system level (not in the application /WEB-INF/lib directory), all the package es.rickyepoderi.compiler (the compiler classes used in the previous entry and the new created for this post) had to be moved from the application to a JAR in the general tomcat lib folder. Remember also to replace the original jasper.jar with my modified version (the one with the setupContextWriter method defined as protected)
In summary this entry is again a Frankenstein implementation which makes a Servlet to dynamically load, parse, compile and execute a JSP file (in the entry the file was obtained from the same WAR file but they could be retrieved from a database, a cache or any other repository). The solution is very botched cos the jasper JSP engine was modified in order to extend the setupContextWriter method and not generate any file during JSP processing. Besides the solution conflicted with Tomcat internal jasper library and some application classes should have been moved to a JAR in the main lib directory. But finally the CompilerServlet can execute both, Servlet source files or JSP, re-compiling them on the fly and with no intermediary files in the File System. Take in mind that this PoC is very bound to the Tomcat container (changes would be needed in order to work in any other JavaEE server).
Again (and this time it makes even more sense), don't try this at home!
Comments