Sunday, April 7. 2013
Glassfish HA using EJB
I have received some comments about the old entry that installed a full glassfish HA solution using debian, people complain that the post did not deal with Stateful EJBs. A Stateful EJB is a enterprise bean that acts as a server-side extension of the client maintaining its state during calls (the bean object maintains its properties). Obviously this type of EJB has to also be replicated between the application server instances in case of an HA architecture. The reason for that oversight is clear, I am not a fan of remote EJBs, I always recommend to use local ones (which are deployed and called inside the same JVM and their performance is much better) or Web Services.
Stateful EJBs should work smoothly in glassfish and today I am going to present a little example (and, as you will see later, I faced a very nasty issue). The first thing I did was installing the solution with glassfish 3.1.2.2, I repeated the same steps explained in the previous entry one by one. When the (in)famous clusterjsp was running I started to develop and deploy a simple stateful EJB following Markus Eisele's blog entry.
The remote interface was created. The sample bean is just a counter with the count as the state to maintain/replicate:
@Remote public interface Sample { public String increment(); public String reset(); public String stop(); }
The real EJB implementation is coded. The increment method returns the instance name which is processing the call and the current value. The stop method finishes the count and removes the EJB (the @Remove annotation means that, when the client calls this method, the EJB can be destroyed).
@Stateful(name="SampleBean") @Remote(Sample.class) public class SampleBean implements Sample, Serializable { private int count = 0; @Override public String increment() { return new StringBuilder(System.getProperty("com.sun.aas.instanceName")) .append(": ") .append(++count) .toString(); } @Override public String reset() { count = 0; return new StringBuilder(System.getProperty("com.sun.aas.instanceName")) .append(": reset to ") .append(count) .toString(); } @Override @Remove public String stop() { return new StringBuilder(System.getProperty("com.sun.aas.instanceName")) .append(": releasing count on ") .append(count) .toString(); } }
The EJB is packed in a JAR and the JAR inside an EAR, then it is deployed with availability enabled (exactly as a web WAR application file).
$ ./asadmin deploy --availabilityenabled=true --target cluster1 ~/clusterejb.ear Application deployed with name clusterejb. Command deploy executed successfully.
At this point the JNDI name for the EJB should be available (the relative path depends on the names used in ear and ejb-jar files):
$ ./asadmin list-jndi-entries --context "java:global/clusterejb/clusterejb-ejb" cluster1 debian1-gf: SampleBean__3_x_Internal_RemoteBusinessHome__: javax.naming.Reference SampleBean: javax.naming.Reference SampleBean!es.rickyepoderi.sample.Sample: javax.naming.Reference debian2-gf: SampleBean__3_x_Internal_RemoteBusinessHome__: javax.naming.Reference SampleBean: javax.naming.Reference SampleBean!es.rickyepoderi.sample.Sample: javax.naming.Reference
Finally I created a simple test client that connects to the remote EJB and calls the increment method. The client performs two iterations, the internal one increments the EJB and the outer loop just creates another EJB (so you can repeat the counting loop as many times as you want).
public class Test { static public void main(String[] args) throws Exception { InitialContext ic = null; Sample sample = null; Properties env = new Properties(); String ans; BufferedReader in = new BufferedReader(new InputStreamReader(System.in)); do { try { ic = new InitialContext(env); String lookup = "java:global/clusterejb/clusterejb-ejb/" + "SampleBean!es.rickyepoderi.sample.Sample"; System.out.println("Retrieving " + lookup); sample = (Sample) ic.lookup(lookup); do { System.out.println(sample.increment()); System.out.print("More (Y/n): "); ans = in.readLine(); } while (!ans.equalsIgnoreCase("n")); } finally { if (sample != null) { System.out.println("Stoping count!!!"); try {System.out.println(sample.stop());} catch (Exception e) {} } if (ic != null) { System.out.println("Closing context!!!"); try {ic.close();} catch (Exception e) {} } } System.out.print("New EJB (Y/n): "); ans = in.readLine(); } while (!ans.equalsIgnoreCase("n")); } }
The client is executed in the following way:
$ java -Dcom.sun.appserv.iiop.endpoints=debian1.demo.kvm:23700,debian2.demo.kvm:23700 \ -cp .:/home/ricky/glassfish3/glassfish/lib/gf-client.jar es.rickyepoderi.client.Test
In theory the glassfish provider gets the specified property (com.sun.appserv.iiop.endpoints) and constructs a specific environment for the JNDI lookup similar to the following (that environment can be used directly in the code too):
Properties env = new Properties(); env.put("java.naming.factory.initial", "com.sun.enterprise.naming.impl.SerialInitContextFactory"); env.put("java.naming.factory.state", "com.sun.corba.ee.impl.presentation.rmi.JNDIStateFactoryImpl"); env.put("com.sun.appserv.ee.iiop.endpointslist", "corbaloc:iiop:1.2@debian1.demo.kvm:23700,iiop:1.2@debian2.demo.kvm:23700");
It uses the SerialInitContextFactory and constructs the corbaloc URL using both servers. Besides it randomizes the order (sometimes debian1 will be the primary and debian2 the alternative and sometimes the other way around) so the solution gets some balancing over the servers (take in mind the order is assigned at context creation).
At this point the client should just work but I faced a very weird problem. When the test application was connected to the debian1-gf instance everything worked as expected, but when the client connected to the debian2-gf one no failover was done. The client application hung trying to always connect to the second instance even thought that server was down. I spent all the morning trying to find what the hell was happening and finally I understood the situation.
In RMI/IIOP it seems that the server architecture is managed during conversation, I mean, although I set both servers in the property (and in turn in the corbaloc URL), when the client contacts with the primary server the servers are defined again using the IOR field (Interoperable Object Reference). And my problem was that debian1 returned both (debian1 and debian2) in the reference but debian2 returned itself twice (no debian1 was answered as a valid alternative server). Finally (and with a big headache as a result) the problem was that glassfish uses nodes as the corba servers and, if you re-check how they were created, I created debian1-ssh node as localhost and debian2-ssh with the real hostname (I thought the nodes were only used for management purposes). So debian1 used localhost and debian2 (debian1 and debian2) and debian2 localhost and debian2 (debian2 and debian2), that produced the annoying issue explained before. So I changed the node directly in the domain.xml configuration file:
<node node-host="debian1.demo.kvm" name="debian1-ssh" type="SSH" install-dir="${com.sun.aas.productRoot}"> <ssh-connector> <ssh-auth></ssh-auth> </ssh-connector> </node>
But obviously the correct solution would have been creating the debian1-ssh node using the real hostname:
$ ./asadmin create-node-ssh --nodehost debian1.demo.kvm debian1-ssh
I confess that I do not know if those names can be forced somehow (maybe some configuration in glassfish let use specific names and not the names of the nodes, but it does not matter much right now). As usual it was much more complicated realizing what was going on than fixing the issue.
After that I successfully tested the EJB. Iterating over the first loop the server changes (randomly the client provider change the primary server from debian1 to debian2). When I was being served by a specific server (debian1) and that server was shutting down, the alternate (debian2) continued the counting. The count is preserved without problem cos replication is maintaining the state of the EJB. Here it is the video that shows the previous explanation.
Today's entry complements a previous one of the blog (simple but full glassfish HA using debian), that entry tested only the web part but some people have commented to me what happened with a Stateful EJB. Usually that type of EJB should work in the HA scenario smoothly but a weird problem happened to me, using localhost instead of the real hostname for node creation complicated my solution. Once that issue was fixed the solution worked smoothly and EJB HA was assured by the glassfish cluster. Here I upload the clusterejb project I used in the entry. The moral is glassfish replication also works with stateful EJBs.
Replicated regards!
Comments