Saturday, February 24. 2018
OCSP in the Apache web server
OCSP (Online Certificate Status Protocol) is a recurring topic of the blog. I am certainly not an expert, but I fear that there are very few people that really know about this protocol deeply so, sometimes, I am just the best suited around. There have been two or three times already that I needed to configure the Apache httpd server to deal with OCSP for verifying client certificates (TLS mutual authentication) and, in almost all of them, I have had issues with the setup. In all the occasions we were trying to remove the CRL (Certificate Revocation List) configuration for another one using OCSP. And it's usually not so easy.
The CRL configuration is mainly managed by the option SSLRevocationCheck since version 2.4 which controls how the openssl is called to perform the verification of the client certificate. This way you can configure to check the CRL only over the final client certificate (leaf) or over all the chain of certificates (root and intermediate CAs and the final certificate). Besides there is a very useful flag to let pass a certificate for which the CRL is not found (no_crl_for_cert_ok option). In general the option for configuring CRL is very complete and let you setup almost any situation.
Nevertheless the OCSP configuration is much less rich. The revocation is enabled with the option SSLOCSPEnable (only on or off values) and then there are more options to locate the URL of the responder to call (options SSLOCSPDefaultResponder and SSLOCSPOverrideResponder let you configure a default OCSP responder in case the certificate does not have the proper extension, you can even override that extension and always call the same responder no matter what the certificate says), dealing with signature responses and timeouts. But none of the options controls when the OCSP verification is done, if you enable it all the chain is always validated (no leaf of no_crl_for_cert_ok equivalents here).
This difference in configuration makes sometimes impossible to change from CRL to OCSP. The first time the issue was that the customer just wanted to validate the certificates that had the OCSP extension in it (it's not possible because if you enable OCSP you need a valid responder, if no one is found the certificate is not validated and that's an error). In the next times the problem was that the OCSP responder was only for final certificates, the intermediate CA has no extension to validate it (CRL or OCSP) and, therefore, the CRL was configured to only check leaf but the same cannot be configured using OCSP (the intermediate CA was validated, unknown returned and the certificate was not admitted). So, in summary, if the configuration for revocation is a bit complex (several CAs, intermediates, different responders, certificates with extensions and without them,...) the OCSP is a understatement.
The last weekend I decided to check how difficult was configuring OCSP exactly in the same way than CRL (none, leaf and chain with a flag no_ocsp_for_cert_ok to allow the certificates that has no responder associated to go in). The modifications finally resulted in just a few lines of code so I decided to submit a bugzilla just to know what the apache guys think about it. With the changes the CRL behavior can be more easily mapped to the OCSP one, at least I think that I would have configured all my failed previous attempts. I do not know how many people are interested in OCSP revocation for client certificates (maybe it's a minor issue) but, if somebody is using it, the changes can be interesting.
Let's see. Best regards!
Monday, April 21. 2014
Kerberos, Apache 2.4 and Solaris 10 (Part II)
Continuing the previous post today's entry is dedicated to the integration of the mod_auth_gss module into an Apache 2.4 web server in a Solaris 10 box. If you remember the MIT kerberos implementation that is bundled in Solaris 10 is version 1.4.0 but modified, and, besides, the exposed API is restricted. For that reason, the typical Apache module (mod_auth_kerb) for handling kerberos authentication is not usable in a straight way. Reading an entry about this subject inside the Oracle blogs only two options are available: compiling your own kerberos implementation (then configuring mod_auth_kerb) or compiling a special mod_auth_gss module developed by Sun to interact with the Solaris implementation. The previous entry was dedicated to the former option and this one continues implementing the latter.
The mod_auth_gss is a an authentication module for Apache based on SPNEGO and GSSAPI. While the mod_auth_kerb uses kerberos directly, this module instead works in an upper layer. The GSSAPI is a security interface that supports different underlying mechanisms but, actually, the dominant GSSAPI mechanism implementation in use is Kerberos. In my humble opinion GSSAPI is the reason that makes Solaris hide the Kerberos API. At that time Solaris engineers preferred GSSAPI (which is standardized) over Kerberos (its API has never been standardized and various existing implementations use different APIs). The mod_auth_gss module is bundle in the Apache 2.2 that Solaris 10 provides by default (it is placed in the /usr/apache2/libexec directory).
After knowing this information the new library seems to be a much proper solution inside a Solaris / Apache box. So continuing with the installation which was setup in the previous entry this module is going to be added.
The module can be retrieved from its web page.
wget https://kenai.com/projects/mod-auth-gss/downloads/download/mod_auth_gss.c
If you try to compile it following its instructions, it fails (actually it compiles with warnings but then it crashes the apache process). But the reason is simple, it is prepared for Apache 2.2 and not 2.4 (remember that the new Apache 2.4.9 is being used through this series). It seems that now the log_rerror function needs more parameters (If you check the patch provided in the previous post by PLD Linux distribution for mod_auth_kerb, there were few lines changed for this same reason). So I repeated the changes to obtain a little patch (mod_auth_gss_apache24.patch).
gpatch -p1 mod_auth_gss.c < mod_auth_gss_apache24.patch
Now you can safely compile the module and install it inside the modules directory.
/export/home/ricky/httpd/bin/apxs -S CC=gcc -l gss -c mod_auth_gss.c cp .libs/mod_auth_gss.so /export/home/ricky/httpd/modules/
Finally I configured the module just reusing some items which were created in the previous entry. First the same krb5.conf file is used, but now it should be placed in the default Solaris location.
cp /export/home/ricky/httpd/etc/krb5.conf /etc/krb5/krb5.conf
The httpd.conf is modified to add the new module too.
LoadModule auth_gss_module modules/mod_auth_gss.so
And the secured location was modified with the new module parameters (the same keytab of the previous entry was reused) instead of previous mod_auth_kerb module.
<Location /secured> AuthType GSSAPI AuthGSSServiceName HTTP AuthGSSKeytabFile /export/home/ricky/httpd/conf/httpd.keytab AuthGSSForceCase upper AuthGSSDebug Off require valid-user </Location>
And it also works well. If you try to enter in the login.pl CGI the user logged in the Windows 8.1 box is shown. Below it is a video showing what I have just commented. Remember that AJP protocol (the protocol to join Apache with some JavaEE servers like Glassfish or Tomcat) can send this user to the underlaying JavaEE server in order to propagate the logged user to the application (see the tomcatAuthentication property). Therefore authentication can be delegated from the JavaEE layer to the Web layer. Obviously a lot of features, like managing groups, are lost but for some cases it could be a much simpler solution.
As a summary this entry implements exactly the same than in the previous one (the integration of kerberos into an Apache Web Server 2.4 in a Solaris 10 box) but using the specific Solaris mod_auth_gss module. This module manages the standard GSSAPI instead of a specific kerberos API and it seems to be a better solution in a Solaris box (think that this module is installed by default in the Apache 2.2 that Solaris bundles). I have just realized that, although kerberos is quite unknown to me, a lot of entries in this blog talk about it.
Kerberized regards!
Saturday, April 5. 2014
Kerberos, Apache 2.4 and Solaris 10 (Part I)
Some months ago I was asked for advise in the integration of the mod_auth_kerb module for Apache web server in a Solaris 10 box. The engineer assigned to the task was having problems with the compilation of the module. The mod_auth_kerb is a kerberos authentication module for Apache, it is the same idea that was shown in the old series about spnego/kerberos in JavaEE but using Apache instead of Glassfish.
I prepared a virtual environment with an old Solaris 10 box and I reproduced the same symptoms my colleague was commenting. In theory Solaris 10 provides the MIT kerberos implementation, but the configure script of the module did not work, when the script checked the krb5 installation it always said no kerberos was available:
checking for krb5_init_context in -lkrb5... no checking for krb5_init_context in -lkrb5... (cached) no checking for krb5_init_context in -lkrb5... (cached) no checking for krb5_init_context in -lkrb5... (cached) no
Finally I understood that Kerberos in Solaris 10 is a modified version of the MIT implementation 1.4.0 but, as it is said in this blog entry, it does not expose the same API and, therefore, the mod_auth_kerb module cannot be compiled against it. So I sent an email referencing this entry and explaining that the MIT kerberos implementation should be also compiled, because Solaris implementation was not valid for the module.
The present entry shows exactly the process which I recommended to follow. I decided to use Apache 2.4.9 and the current MIT kerberos 1.12.1. The Active Directory was installed in a Windows 2012 box (KVM virtual machine) and the client is going to be a Windows 8.1 (another KVM box). As always I am going to explain the installation in the Solaris 10 box step by step.
First the PATH env variable should be set to get access to compilers and other utilities in the Solaris box.
export PATH=$PATH:/usr/sfw/bin:/usr/ccs/bin
Apache 2.4 needs PCRE library first. So current version was compiled and installed (all the needed software is going to be installed in ${HOME}/httpd).
gtar jxvf pcre-8.34.tar.bz2 cd pcre-8.34 ./configure --prefix=/export/home/ricky/httpd make make test make install
The next step is compiling a proper OpenSSL. Solaris 10 bundles version 0.9.7d and applies the important security fixes to it, but it is never upgraded to a new version (solaris 11 has a more recent package). Apache 2.4 needs version 0.9.8a or newer so, again, it should be compiled and installed apart.
gtar zxvf openssl-1.0.1f.tar.gz cd openssl-1.0.1f ./config zlib threads shared --prefix=/export/home/ricky/httpd make make test make install
Once all the requirements are in place the Apache itself is compiled and installed (version 2.4 needs apr libraries, but they can be just placed in the srclib directory).
gtar jxvf httpd-2.4.9.tar.bz2 cp apr-util-1.5.3.tar.bz2 apr-1.5.0.tar.bz2 httpd-2.4.9/srclib/ cd httpd-2.4.9/srclib gtar jxvf apr-util-1.5.3.tar.bz2 gtar jxvf apr-1.5.0.tar.bz2 mv apr-1.5.0 apr mv apr-util-1.5.3 apr-util cd .. ./configure --prefix=/export/home/ricky/httpd --with-ssl=/export/home/ricky/httpd --with-mpm=worker --with-included-apr --with-pcre=/export/home/ricky/httpd make make install
Now it is the turn for the Kerberos MIT implementation.
gtar zxvf krb5-1.12.1.tar.gz cd krb5-1.12.1/src ./configure --prefix=/export/home/ricky/httpd make make check make install
When the web server and the kerberos implementation are ready the module itself is added. The module is prepared to Apache 2.2 and it needs a little patch for the newer version (for example this one provided by the PLD Linux distribution team).
gtar zxvf mod_auth_kerb-5.4.tar.gz gpatch -p0 < mod_auth_kerb-apache24.patch cd mod_auth_kerb-5.4 ./configure --with-krb4=no --with-krb5=/export/home/ricky/httpd --with-apache=/export/home/ricky/httpd make make install
Now all the software is ready to be used. First of all, we need to configure the kerberos in order to be used by the module. A very similar krb5.conf file to the one used in the previous series is used.
The only difference is the name of the KDC (the AD Windows 2012 machine, I did not remember to change the name before adding the AD role so the DC has a horrible name) and the addition of arcfour-hmac-md5 as an admitted encryption type. It seems that Windows 2012 has deprecated the old des-cbc-md5 and now this one should be used.
In this point you can check your configuration with a user of the domain (for example Administrator):
/export/home/ricky/httpd/bin/kinit Administrator@DEMO.TEST
Setting the proper password, the command should be executed successfully. Therefore a valid ticket for the user is created.
/export/home/ricky/httpd/bin/klist Ticket cache: FILE:/tmp/krb5cc_1000 Default principal: Administrator@DEMO.TEST Valid starting Expires Service principal 03/27/14 02:06:48 03/27/14 12:06:48 krbtgt/DEMO.TEST@DEMO.TEST renew until 03/28/14 02:06:45
Now it is necessary to create a user for the apache server. A simple httpd user was created in the AD (using common Active Directory Users and Computers snap-in).
Then the service principal name is added to the user (it is also done in the Windows 2012 AD machine).
setspn -A HTTP/solaris10.demo.test httpd
And finally the keytab file is created for the user.
ktpass -princ HTTP/solaris10.demo.test@DEMO.TEST -mapuser DEMO\httpd -crypto RC4-HMAC-NT -pass <password_httpd_user> -ptype KRB5_NT_PRINCIPAL -out httpd.keytab
I think that, with the windows ktpass command, both things are done at the same time (setting the spn and creating the keytab, but I did the setspn before, so I am not completely sure). It is important to use the same crypto that was added in the krb5.conf file. As it was commented before, I had problems setting the typical DES-CBC-MD5 because Windows 2012 has disabled it by default, so RC4-HMAC-NT was used. This file should be moved to the Solaris / Apache box. Once we have the httpd.keytab file in there the keytab can be checked, it should have the proper name inside.
/export/home/ricky/httpd/bin/klist -k httpd.keytab Keytab name: FILE:httpd.keytab KVNO Principal ---- --------------------------------------- 5 HTTP/solaris10.demo.test@DEMO.TEST
With that file the connection should work properly.
/export/home/ricky/httpd/bin/kinit -k -t httpd.keytab HTTP/solaris10.demo.test@DEMO.TEST /export/home/ricky/httpd/bin/klist Ticket cache: FILE:/tmp/krb5cc_1000 Default principal: HTTP/solaris10.demo.test@DEMO.TEST Valid starting Expires Service principal 03/27/14 02:18:38 03/27/14 12:18:38 krbtgt/DEMO.TEST@DEMO.TEST renew until 03/28/14 02:18:38
Finally the httpd.conf file is prepared to use the module. The important parts are the module addition.
LoadModule auth_kerb_module modules/mod_auth_kerb.so
And the location which is going to request the kerberos authentication.
<Location /secured> AuthType Kerberos AuthName "Kerberos Login" KrbMethodNegotiate On KrbMethodK5Passwd On KrbDelegateBasic Off KrbAuthRealms DEMO.TEST KrbServiceName HTTP Krb5KeyTab /export/home/ricky/httpd/conf/httpd.keytab require valid-user </Location>
Then Firefox and IE11 browsers were prepared to send the kerberos token. The Firefox application was prepared exactly as it was in the previous entry (check that entry for further details) but with IE11 (the native browser in windows 8.1) I spent a lot of time until I found this microsoft note. I think the problem is that now the box that contains the Apache web server is not registered in the AD and, for that, it is not considered as an intranet site. In the previous entry the glassfish server was inside the samba / DC box, so, it was part of the intarnet for sure. Once I added the Solaris fqdn in the local intranet sites it started to work.
Finally I prepared a little perl cgi which shows the authenticated user and it can be checked that the user displayed is the one logged in the Windows 8.1 box. Here it is a little video that shows the whole process.
Two final tips. The first one is that at first tries the solution did not work because the module was searching the user HTTP/solaris10 in the keytab instead the fully qualified name HTTP/solaris10.demo.test. I think that I solved this issue just setting the proper hostname and the fqdn first in /etc/host file. The second point is already known, in order to have a working kerberos login the time in all the machines (Solaris10 apache box, Windows 2012r2 AD and Windows 8.1 client) should be the same (ntp is compulsory in a production environment). I had problems with the time in the Solaris 10 box but a simple ntpdate against the AD host adjusted the time properly.
This entry is a bit long and tedious but it complements the previous Kerberos series presented in this blog. Now the Apache is the one that performs the Windows authentication (not the JavaEE container). In this entry there are very specific details: AD is a Windows 2012r2, the client is Windows 8.1 and the Apache is installed in a Solaris 10 box. The latter is the main characteristic for the entry, because it makes me compile a lot of different software to make Apache 2.4 work. The moral here is that, in Solaris 10, the mod_auth_kerb module cannot use the internal Solaris implementation, one solution is compiling and installing your own MIT Kerberos5 implementation, this task was presented throughout this entry. Besides another conclusion for me is that Solaris 10 is a distinguished OS but it is getting older, Solaris 11 gives much more recent versions of the same software. Finally, if you recheck the link provided at the beginning, the Oracle engineer provides two different solutions, the one performed in the entry (compiling your MIT kerberos implementation) and using a specific module for the Solaris Apache, mod_auth_gss. There will be a second entry in this series that will implement the other option.
Cheerio!
Comments