Saturday, September 28. 2013
WBXML stream: Version 0.1.0
If you remember I was developing a wbxml parser and encoder for Java. This was motivated by another idea that I think is not going to be productive, but, at least, I wanted to complete this first step. The wbxml-stream is the result of all this work, it is a StAX (Streaming API for XML) for the JavaSE to deal with WBXML (WAP Binary XML). This week the first version 0.1.0 was released and it is a complete StAX implementation.
The WBXML is a standard which is defined in this document by the Open Mobile Alliance. In order to develop the library the java implementation is based on the C project libwbxml, which is the only implementation that I know of the WBXML format. Nevertheless the java library is quite tricky and different, its main features are the following:
In WBXML all the common elements of XML (element, attributes,... ) are mapped to an integer token (that is why it is called binary). This mapping data and some other information compose what is called a language (examples of languages are SyncML or Wireless Village).
The main feature of this implementation is to be generic, i.e., valid for any WBXML language. This way the wbxml-stream project loads a language from a properties definition file and the implementation does not deal with any special language tricks (thing that happens in libwbxml all the time). The definition part is managed in the library by the es.rickyepoderi.wbxml.definition package.
By default some language definitions are located and loaded at initialization in version 0.1.0 (ActiveSync, CONML, DevInf 1.1, DevInf 1.2, DMDDF 1.2, drmrel 1.0, EMN 1.0, OTA, PROV 1.0, SI 1.0, SL 1.0, SyncML 1.1, SyncML 1.2 and WV CSP 1.1). If you need to add a new definition the project wiki explains how a definition file is organized and presents a working example of a dummy new one.
The main idea of parsing and decoding is that an intermediary java structure is used. This way, when a WBXML is read (parsed), those intermediary objects are first created (memory representation of the WBXML document) and then the StAX reader iterates over the structure. In a writing (encoding) process the operations are reversed, the StAX writer constructs the object representation and it writes the binary stream at the end.
I know this idea means more memory and not a real streaming (StAX) way of working. But WBXML is a quite weird format (please check the pdf presented before) and there are several characteristics that convinced me to do it simple at first (this handicap could be a good improvement for next versions).
The objects of this intermediary representation are placed inside the es.rickyepoderi.wbxml.document package. The WbXmlEncoder and the WbXmlParser are the main classes to encode and parse a WBXML document respectively.
Finally the package es.rickyepoderi.wbxml.stream contains the StAX implementation. I tried to fully cover the standard, there are implementations for XMLStreamReader, XMLStreamWriter, XMLEventReader, XMLEventWriter, XMLInputFactory and XMLOutputFactory. In order to create the readers and writers the factories are recommended, see some examples in the wiki pages.
For some time I decided to only implement the stream classes and use internal java implementation for event reader and writer (a event reader can use a stream reader) but I finally decided to go all the way. Now the library only depends in JavaSE, no other dependency is needed. Even the logging is performed using the java.util.logging classes.
I think this first version is more or less usable. It contains several languages (only a few of the languages that libwbxml supports are not defined in wbxml-stream) and it is quite well tested (all the xml and wbxml files that libwbxml uses for testing are also tested with wbxml-stream). Besides I have been using it against a Microsoft Exchange Server (ActiveSync language) without problems in the communication.
If you need to use it please go ahead and share your feelings with me. Remember that WBXML is a very odd format and it is almost impossible to deal with it without understanding its specification. So although wbxml-stream lets you manage WBXML documents exactly as a XML one, it is convenient that you read the standard before. In case you have to define a new language it is compulsory. If any bug or problem is detected please inform me using this blog or github.
Enjoy wbxml-stream!
Sunday, September 22. 2013
Copy-n-Paste in KVM
A quick entry this time, it is just a tip I had been looking for for a long time. Sometimes I need to use a windows box inside the KVM virtualization of my debian box. I know, I do not like it either, but sometimes you have no other choice. I always complained about the copy-n-paste thing, what I copied inside the virtual windows guest cannot be pasted to my debian host and vice-versa. I always supposed that there should be any solution but I had never found any, although I have to admit that I did not spend much time on it. These last weeks I have had to use the windows guest more continuously and finally I was forced to find a proper solution.
The solution is based on spice, a project to provide a high-quality remote access to QEMU and, therefore, KVM. In theory there are a lot of advantages in using spice instead normal VNC protocol and one of those is the copy-n-paste feature. The steps to provide this functionality between KVM guest and host are more or less explained inside the kvm documentation, but the problem is that the example is for a fedora guest, not windows. My dummy how-to is presented below.
First the spice virtual drivers were installed inside the windows OS. I tried first the spice-guest-tools version 0.59 (the version that is listed in the downloads page) but virtual IO serial device for copy-n-paste hanged (exactly what I wanted to use, you know, Murphy's law). Finally I discovered that there was also a version 0.65 but it is still not listed as a download, and with that one I got it working.
Once the spice tools are installed in the guest box I configured my KVM machine to use spice (using graphical tool, virt-manager). The display was changed from VNC to SPICE.
Then a channel should be added in order to provide the copy-n-paste feature.
After that configuration I checked that the following options had been added to the kvm command line:
- -spice port=5900,addr=127.0.0.1,disable-ticketing
- -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
- -chardev spicevmc,id=charchannel0,name=vdagent
The first one activates the SPICE server. Now you can connect to the host using the spice client:
$ spicec -h localhost -p 5900
But it is not needed cos virt-manager already has an integrated spice client. The other two options are for the copy-n-paste (very weird options indeed). All this options are also explained in the KVM page I linked before.
The last step is booting again the windows guest. At startup time it discovered the new hardware, I selected not to connect to windows update or the internet and then to install the software automatically. The system found the drivers previously installed and chose two of them (the QLX driver for the display and a VirtIO for the character device).
As I explained in the first point I had several problems with the virtIO serial driver for copy-n-paste. It hanged at installation, exactly at this point. After a lot of hangs and reboots it let me replace 0.59 version for the new 0.65 and finally it booted without any issue.
So That is all. I finally has copy-n-paste in my windows guest using KVM and virt-manager. This entry is a little summary of what I did just to remember the next time I need it again. I hope that it helps someone else too.
Cheerio!
Tuesday, September 3. 2013
OpenDNIe integrated into OpenSC
Today's entry is dedicated again to the Spanish electronic ID card (DNIe). This subject has been talked several times in the blog, but this week the final act of this drama has been played. Finally the code of OpenDNIe has been merged into OpenSC. I suppose that some of you do not understand why that integration is so important and that is the reason for the entry.
The Spanish eID card project started several years ago and the first card was issued in march 2006. The needed software was ready very quick and (to my surprise) the Spanish government not only developed it for windows, a DNIe module for OpenSC was ready in a few months. OpenSC is the project that provides access to smart cards and their cryptographic operations in the linux / open source world, it also provides a PKCS#11 library for external applications. This code was initially not published but in some months the source code could be downloaded from the DNIe page. The problem was that the module was released using GPLv3 license while OpenSC uses LGPL, this issue did not let a direct integration (see this bug for more information). Besides the code for the module was never maintained, therefore it became deprecated very quickly and soon it could not be integrated with the new OpenSC versions in a straightforward way. My previous entry about DNIe and the java applet was of that time and talked about some of the issues.
After that messy situation a Spanish guy (Juan Antonio Martinez) started a new implementation from scratch. It called the new module OpenDNIe and it was presented in the OpenSC mailing lists in the beginning of 2011. For several months this second attempt was said to be included in the main OpenSC project (initially for version 0.12.1, then for 0.12.2) but it was not merged until this week. During that time the new module lived inside a Spanish site which provided packages for OpenSC versions.
The new version 0.13.0 of OpenSC was released at the end of 2012. Again the integration of the OpenDNIe module was not there and the situation was getting more and more complicated. Currently it seems that both OpenSC and OpenDNIe are under some important changes. The original developer of OpenDNIe seems to not continue with the project and OpenSC is moving to github, but I think that this point is only the visible part of a complete change in the project organization and management.
During the Christmas vacation I decided to check what was the status of OpenDNIe and its integration in the new OpenSC 0.13.0 version. I checked the differences and I successfully compiled a working OpenDNIe/OpenSC 0.13.0. But I realized that another guy (Germán Blanco) was also doing the same effort and he was being tidier and smarter than me. He was talking with people of the OpenSC project in order to try to finally merge the code. He did an incredible job and this week there was a big commit that included OpenDNIe changes into OpenSC main branch.
So finally the Spanish DNIe is integrated in OpenSC. This milestone is incredibly important, as soon as the new version of this project was released all the major linux distributions will start to support DNIe by default. Therefore any Spanish linux user will be able to use his national eID card with minimal configuration (only the browser/application configuration will be needed). But the main question is still in place. Why did not the Spanish government follow this way since the beginning? In my humble opinion because they did not understand how open source works, an effort to include DNIe in linux/UNIX was done (which is quite important I must say) but it was done absolutely in the wrong way. If they had worked hand in hand with OpenSC since the beginning all the following pain would not have been necessary.
At this moment the only chance to get OpenSC/OpenDNIe working is compiling the sources directly from github (let's see if we are lucky and OpenSC guys release the new version in a short time). I present a simple compilation in debian testing which follows step by step the project instructions:
First Clone the project.
$ git clone https://github.com/OpenSC/OpenSC.git
Then check that your system (debian in my case) has the needed packages.
# apt-get install pcscd libccid libpcsclite-dev libssl-dev libreadline-dev autoconf automake build-essential docbook-xsl xsltproc libtool
Compile the project with the needed OpenDNIe options (dni-ui and sm are compulsory for OpenDNIe to work).
$ ./bootstrap $ ./configure --prefix=/home/ricky/apps/opensc --enable-dnie-ui --enable-sm $ make $ make install
Now the pcscd daemon can be started. I started it in the foreground and with debug just for the first time.
# pcscd -f -d
This point is optional and obviously not necessary when OpenSC is used normally (it will be started by the system by default). At this moment and with no configuration the OpenSC commands can be executed (the card should be inserted in your reader).
$ /home/ricky/apps/opensc/bin/opensc-tool -D | grep DNI dnie DNIe: Spanish eID card $ /home/ricky/apps/opensc/bin/dnie-tool -d Using reader with a card: Broadcom Corp 5880 [Broadcom USH w/swipe sensor] (0123456789ABCD) 00 00 DNIe Number: ********* SurName: MARTIN Name: RICARDO
The final part is configuring your browser to work with the PKCS#11 module that OpenSC provides. In Edit → Preferences → Select Advanced → Then Encryption.
Click the Security Devices button and add the compiled OpenSC PKCS#11 library (the library is opensc-pkcs11.so placed in the lib directory). Click Load button and add the library.
The new OpenSC module appears and now you can login in the DNI slot.
Now when the certificates are requested (clicking the View Certificates in the same preferences window) iceweasel shows the two certs that our DNIe contains.
Besides I tested the module with the testing DNIe page. The secure access to the page was done correctly but the signing test did not work, but my feeling is that the problem is in the Java/applet part (I am using openjdk with the icedtea-web plugin) and not with the OpenSC PKCS#11 module.
The conclusion of this entry is quite important and a big lesson to learn by anybody who wants to work in the open source world. Because Spanish government decided to follow a crazy path with the linux implementation of the DNIe (but please think that at least a linux solution was thought and done, which is much more than the usual only windows solution) all OpenDNIe module was developed by unselfish people in their spare time (mainly Juan Antonio). So please, when developing for linux environments think that linux is not windows, rules are different, and you have to work properly in order to be in all the major distributions. For me DNIe is a clear example of the misunderstanding that exists around the FOSS world.
Special thanks to Juan Antonio and Germán for their work!
Comments