Friday, May 4. 2012
I am sure that some of you have already guessed that I am investigating how to create a couchbase session manager for glassfish. In previous posts I studied the memcached-session-manager (a session manager for tomcat) and then you saw me integrating new bundles to glassfish. All those steps were done to implement a new couchbase session manager for glassfish. This post presents the ideas I am following and the first bits of couchbase-manager project which is hosted in git-hub.
The main ideas behind the couchbase-manager are the following:
Couchbase is the responsible of managing sessions (locks, expirations, values,...), the glassfish local sessions rely completely in the external memory repository. This is the axiom for the implementation and it determines all the following ideas.
Sessions will be loaded and saved in every request. When a request comes, the manager usually finds the session, locks it, performs all the stuff and releases it back to the pool of sessions. So now, all that process is exactly the same but using couchbase. Session is found (locally first, if not found couchbase one is got), locked (using getAndLock method), modified and saved (unlock is performed via cas or unlock if session not modified).
This new idea of processing means that local sessions in the internal map cannot be trusted, they only are real when locked. For this reason sessions are cleared (attributes are empty to save space and to test if it is really working) when unlocking.
Besides all the expiration process is also relying in couchbase, all the puts, sets or adds are always done using the expiration time parameter. A session is expired if it does not exist in the external repository. Cos expiration checks are performed anytime during the request lifecycle (not only when session is locked) it is clear that couchbase should be checked everytime, but for saving calls local timestamps are used when not locked (continue reading for details).
So following this ideas the request lifecycle in couchbase manager performs the following steps (I added a little diagram of my ideas):
A request is initiated by the client browser. If the request has a session id (it was created before) the findSession method is called. This method searches the session locally in the internal Map but if the session is not found this id is got from couchbase (the session was created by another server). If not found anywhere null is returned. On the other hand if the session is new and the createSession is called (first request of the client), the session is created and the add method is called. In the couchbase manager this method adds the session in the internal map and also in couchbase.
After the session is retrieved (created or found) the manager is called to lock the session. It is important to remark that until locking the session is not reliable cos, as it is not locked, it can be modified by any other server . There are two lock methods in the manager interface, lockForeground and lockBackgroung, the first one is the method called in normal request processing, the second in internal actions (for example when performing expiration checking task). Besides the foreground method lets several clients to lock the session concurrently (in the same server) and not the background. In any case, the new manager always locks the session in couchbase (accesses from other servers are prevented).
Once the session is locked the session is modified by the request lifecycle (user code manages session). In this stage the session can be modified, read or deleted (invalidated) and implementation assures this server is the only one modifying the session.
If the session is not deleted the unlock process performs the saving in the couchbase. In order to do that an access status of the session is maintained. If the session was modified (dirty) it is saved (cas method is used, session is stored and unlocked in couchbase). If the session is only accessed (local timestamps are refreshed for expiration) the session is touched and unlocked (the pity here is that couchbase client does not have a touchAndUnlock or unlockAndTouch method, so two calls are made). Nevertheless if the session is accessed but not saved for a long time (when a session is touched the internal couchbase expiration is refreshed but not local timestamps of the sessions) a save is executed instead of a touch/unlock.
If the session is manually invalidated the manager is called via remove method. This method removes the session from couchbase (if not already deleted or expired before). The expiration is the most dangerous point of the implementation, the hasExpired method of the session is called everywhere (not only when locked) so, being meticulous, the session should always be checked against couchbase (if the session exists there it still lives, if not it is expired). But, obviously, saving calls is necessary. For this reason if the session is not locked local timestamps are checked and, if session is ok, no access to the couchbase is done (a kind of cache response is done when not expired). There is no high risk cos as soon as the session is locked (it is always locked in a request processing) a real check is done.
Expiration task (a task that manages expiration by inactivity in the internal map) now follows the same idea, the session expiration is first checked locally (internal timestamps) and if it is expired here a lockBackground is performed (it is checked if the session is really deleted in couchbase).
I think the basis are quite simple, but I have experienced several problems to set the manager in glassfish. I summarized here my problems:
In theory glassfish uses an OSGi architecture and a new manager is just a class that implements the PersistenceStrategyBuilder interface. But in reality the SessionManagerConfigurationHelper class limits the manager implementations to the ones that glassfish gives. There is a CUSTOM implementation but this is not thought to be distributable, so I was forced to re-use the coherence-web name (that implementation is a memory repository implementation for the so-named Oracle product).
More or less the same problem happens with the Manager interface. I started extending the ManagerBase abstract class but finally I needed to extend the StandardManager. If the manager is a StandardManager session expiration is controlled by a internal processExpires method. If not you need to implement a PersistentManagerBase (this is managed inside StandardContext class). So finally I decided to extend the StandardManager (some methods are overriden just for cleaning purposes).
My personal opinion is that there is so much noise in the whole manager implementation that it is quite difficult to understand it. A lot of cross calls between manager and session, misunderstanding in what to extend or implement, lots of tests and intensive code browsing are needed to know just where to put your classes and so on. It seems that tomcat and glassfish uses more or less the same classes (I am not completely sure).
Right now couchbase-manager only works with couchbase server 2.0 development preview 4 and current git branch of spymemcached and couchbase Java client (features like lock/unlock are new in all couchbase software).
Finally I present a little video of the current status. The famous clusterjsp is functional in just one server, some attributes are added to the session. Because of using external couchbase server sessions are fully persistent (the session remains after glassfish and couchbase restart).
In summary the implementation is very poor nowadays. I only developed the manager classes and it was only tested in one server and manually (so it is currently very bad tested and, probably, it does not work with two servers). Besides there is another important point, the performance and latency of the manager. It performs always (at least) two calls to the couchbase server (getAndLock and then cas or touch/unlock), in other cases can be more (initial creation or deletion). Besides some snags in couchbase Java client add some extra calls (no touchAndUnlock, no deleteAndUnlock,...). So some performance tests of the manager would be necessary. These two points (add multi-server, non-sticky architecture and performance tests) are in my mind for the next weeks. But I have such little time, you know, I am...
Always outnumbered, never outgunned.