Saturday, September 12. 2015
SPMLv2 - Part II: XSD profile
In the previous entry the SPMLv2 provisioning standard was introduced and its DSML profile explained. The DSML profile manages object classes and attributes in a very similar way to an LDAP server. Basic CRUD methods were shown in a little PoC that implemented the core SPML capabilities with a demo server and client. Both ends were developed using the outdated Java library openspml. This library is very old and I am sure that now it would be developed in a very different way. Besides openspml does not cover the XSD profile that the SPML standard also provides. This is the second entry of the SPML series and it will extend the openspml library to minimally cover the XSD profile. The basic CRUD examples that were explained in the previous entry will be repeated with this new profile.
If you remember, the SPML server exposes several targets and each target should manage the data interchanged using one of the schema profiles. In the XSD profile the data is exchanged using XML language, data which is compliant to a XSD file previously defined. That schema file is accessible through the initial listTargets operation.
As I said in the previous introduction the openspml library is too old to implement the XSD profile (the XSD profile was added to the standard later than the DSML one and the library was never updated to include it). So I was forced to add some little changes in the library to add that second profile. The library manages a interface called OpenContentElement to transform any object to XML language. (As you see the library is very old and it does not manage current frameworks like JAXB. In general it uses DOM/Xerces for everything.) So I added a XSDDocElement class that wraps any DOM XML object exchanged storing the DOM document itself as an OpenContentElement. This way any XML data that is sent or received in the XSD profile is maintained as a Java object with the XSDDocElement. In openspml the library itself manages its own unmarshallers in order to understand the data interchanged. The library uses an interface called OCEUnmarshaller (Open Content Element unmarshaller) to give the possibility to be extended to new profiles. By default there is a DSMLUnmarshaller to support the DSML profile. So I added a XSDUnmarshaller that, following the same idea, converts the data to a XSDDocElement. I basically used the namespace of the XSD element to guess that the data is a XSD XML chunk. Finally I created the XSDProfileRegistrar which is a registration class for my new XSD unmarshaller. I know my implementation is a bit sloppy but I just wanted to show in a quick way how the XSD profile works.
In this second entry I will also show some parts of my server implementation. Remember that the server is based on the openspml provided Servlet and a simple SPMLExecutor I developed. This executor just maps a Request class from openspml (ListTargetRequest, AddRequest, LookupRequest,...) to the real executor that handles this kind of requests. This way my server project added the executors to cover the core capabilities of the SPML framework.
ListTargets
As in the previous entry the listEntries method should be called first by the client. There is no difference with the DSML profile except that in this example the profile is set to the XSD URI.
ListTargetsRequest ltr = new ListTargetsRequest();
ltr.setRequestID("ltr1");
ltr.setProfile(new URI(XSDProfileRegistrar.PROFILE_URI_STRING));
ltr.setExecutionMode(ExecutionMode.SYNCHRONOUS);
Response res = client.send(ltr);
The executor implemented for this operation just creates the response loading a fixed response with the DSML, XSD or both schema. Depending the URI requested by the client the correct prefixed XML is returned.
ListTargetsResponse res = null;
try {
XMLUnmarshaller unmarshaller = new ReflectiveDOMXMLUnmarshaller();
if (req.getProfile() == null) {
res = (ListTargetsResponse) unmarshaller.unmarshall(TARGET_DEFINITION_BOTH);
} else if (XSDProfileRegistrar.PROFILE_URI_STRING.equals(req.getProfile().toString())) {
res = (ListTargetsResponse) unmarshaller.unmarshall(TARGET_DEFINITION_XSD);
} else if (DSMLProfileRegistrar.PROFILE_URI_STRING.equals(req.getProfile().toString())) {
res = (ListTargetsResponse) unmarshaller.unmarshall(TARGET_DEFINITION_DSML);
} else {
SpmlUtils.fillError(res, ErrorCode.UNSUPPORTED_PROFILE,
"Profile '" + req.getProfile() + "' is not supported");
}
res.setRequestID(req.getRequestID());
} catch (UnknownSpml2TypeException e) {
e.printStackTrace();
res = new ListTargetsResponse(new String[] {e.getMessage()}, StatusCode.FAILURE,
TARGET_DEFINITION_XSD, ErrorCode.CUSTOM_ERROR, null);
}
return res;
In the XSD case the returned schema is the XSD definition file of the different entities the target manages. In my little PoC the XSD schema is just a single element with the definition for the user type.
<listTargetsRequest xmlns='urn:oasis:names:tc:SPML:2:0' requestID='ltr1' executionMode='synchronous' profile='urn:oasis:names:tc:SPML:2.0:profiles:XSD'/>
<listTargetsResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='success' requestID='ltr1'>
<target targetID='ddbb-spml-xsd' profile='urn:oasis:names:tc:SPML:2.0:profiles:XSD'>
<schema>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns="http://www.w3.org/2001/XMLSchema" xmlns:spml="urn:oasis:names:tc:SPML:2:0" elementFormDefault="qualified" targetNamespace="urn:ddbb-spml-dsml:user" version="1.0">
<element name="user">
<complexType>
<sequence>
<element maxOccurs="1" minOccurs="1" name="uid" type="string"/>
<element maxOccurs="1" minOccurs="0" name="password" type="string"/>
<element maxOccurs="1" minOccurs="1" name="cn" type="string"/>
<element maxOccurs="1" minOccurs="0" name="description" type="string"/>
<element maxOccurs="unbounded" minOccurs="0" name="role" type="string"/>
</sequence>
</complexType>
</element>
</xs:schema>
<supportedSchemaEntity entityName='user'/>
</schema>
<capabilities/>
</target>
</listTargetsResponse>
Add
The add request for the XSD profile just sends the correct XML data for the desired element to be created specifying the XSD target returned by the previous listTargets. Instead of the attributes and values sent with the DSML profile, the XSD profile just send a complete XML that represents a supported entity. In this profile the openspml client just needs to add the User object to the Extensible class. For the XSD profile the User class has been modified to implement the OpenContentElement interface. Any user object can be transformed to XML using JAXB, that resulting XML is the data sent to the server.
AddRequest ar = new AddRequest();
ar.setRequestID("ar1");
ar.setTargetId("ddbb-spml-xsd");
ar.setExecutionMode(ExecutionMode.SYNCHRONOUS);
Extensible data = new Extensible();
User u = new User();
u.setUsername("ricky");
u.setPassword("ricky123");
u.setCommonName("Ricardo Martin");
u.setDescription("me");
u.getRoles().add("Admin");
u.getRoles().add("User");
data.addOpenContentElement(u);
ar.setData(data);
Response res = client.send(ar);
In the server end, the add executor receives the data and it is converted back to a user object. Depending the target specified the user object is created using DSML attributes or the DOM document. Once the user is recovered it is inserted in the ddbb and the data added to the response (again, depending the profile, DSML attributes or XML data is appended).
AddResponse res = new AddResponse();
res.setStatus(StatusCode.SUCCESS);
if (SpmlUtils.checkSynch(req, res) && SpmlUtils.checkTargetId(req.getTargetId(), res)) {
try {
User u = new User();
if (ListTargetsExecutor.TARGET_ID_DSML.equals(req.getTargetId())) {
// convert the data in a User
u = SpmlUtils.spml2User(u, req.getData());
} else {
// convert from DOC to User
List docs = (List) req.getData(
).getOpenContentElements(XSDDocElement.class);
if (docs.size() == 1) {
u = SpmlUtils.doc2User(docs.get(0).getDocument());
} else {
SpmlUtils.fillError(res, ErrorCode.MALFORMED_REQUEST,
"The XML data is not properly received");
}
}
// if the psoId is set assign the id
if (req.getPsoID() != null && req.getPsoID().getID() != null) {
u.setUsername(req.getPsoID().getID());
}
if (SpmlUtils.checkUser(u, res, true)) {
// add the user normally
um.create(u);
if (ReturnData.IDENTIFIER.equals(req.getReturnData())) {
res.setPso(SpmlUtils.user2Spml(u, false,
ListTargetsExecutor.TARGET_ID_DSML.equals(req.getTargetId())));
} else if (req.getReturnData() == null
|| ReturnData.DATA.equals(req.getReturnData())
|| ReturnData.EVERYTHING.equals(req.getReturnData())) {
res.setPso(SpmlUtils.user2Spml(u, true,
ListTargetsExecutor.TARGET_ID_DSML.equals(req.getTargetId())));
}
}
} catch (Exception e) {
e.printStackTrace();
SpmlUtils.fillErrorException(res, e);
}
}
return res;
Finally the conversation is as follows (please pay attention to the XML data used instead of the DSML attributes).
<addRequest xmlns='urn:oasis:names:tc:SPML:2:0' requestID='ar1' executionMode='synchronous' targetId='ddbb-spml-xsd' returnData='everything'>
<data>
<usr:user xmlns:usr="urn:ddbb-spml-dsml:user">
<usr:uid>ricky</usr:uid>
<usr:password>ricky123</usr:password>
<usr:cn>Ricardo Martin</usr:cn>
<usr:description>me</usr:description>
<usr:role>User</usr:role>
<usr:role>Admin</usr:role>
</usr:user>
</data>
</addRequest>
<addResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='success' requestID='ar1'>
<pso>
<psoID ID='ricky' targetID='ddbb-spml-xsd'/>
<data>
<usr:user xmlns:usr="urn:ddbb-spml-dsml:user">
<usr:uid>ricky</usr:uid>
<usr:password>ricky123</usr:password>
<usr:cn>Ricardo Martin</usr:cn>
<usr:description>me</usr:description>
<usr:role>User</usr:role>
<usr:role>Admin</usr:role>
</usr:user>
</data>
</pso>
</addResponse>
LookupRequest
The lookup request is only different in the response, because the data is returned as XML instead of DSML attributes. The request is created setting the psoID to be read but using the XSD target.
LookupRequest lr = new LookupRequest();
lr.setRequestID("lr1");
lr.setExecutionMode(ExecutionMode.SYNCHRONOUS);
PSOIdentifier psoId = new PSOIdentifier("ricky", null, "ddbb-spml-xsd");
lr.setPsoID(psoId);
lr.setReturnData(ReturnData.DATA);
Response res = client.send(lr);
The server gets the psoID to retrieve the user object from the ddbb. Then, depending the profile specified in the pso, the returned data is constructed using DSML attributes or XML.
LookupResponse res = new LookupResponse();
res.setStatus(StatusCode.SUCCESS);
if (SpmlUtils.checkSynch(req, res) && SpmlUtils.checkPSO(req, res)) {
try {
User u = um.read(req.getPsoID().getID());
if (u != null) {
if (ReturnData.IDENTIFIER.equals(req.getReturnData())) {
res.setPso(SpmlUtils.user2Spml(u, false,
ListTargetsExecutor.TARGET_ID_DSML.equals(req.getPsoID().getTargetID())));
} else if (ReturnData.DATA.equals(req.getReturnData())
|| ReturnData.EVERYTHING.equals(req.getReturnData())) {
res.setPso(SpmlUtils.user2Spml(u, true,
ListTargetsExecutor.TARGET_ID_DSML.equals(req.getPsoID().getTargetID())));
}
} else {
SpmlUtils.fillError(res, ErrorCode.INVALID_IDENTIFIER,
"The user does not exist in the ddbb");
}
} catch (Exception e) {
e.printStackTrace();
SpmlUtils.fillErrorException(res, e);
}
}
return res;
The conversation is similar to the previous entry except that the returned pso contains the XML data corresponding to the user object looked up.
<lookupRequest xmlns='urn:oasis:names:tc:SPML:2:0' requestID='lr1' executionMode='synchronous' returnData='data'>
<psoID ID='ricky' targetID='ddbb-spml-xsd'/>
</lookupRequest>
<lookupResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='success' requestID='lr1'>
<pso>
<psoID ID='ricky' targetID='ddbb-spml-xsd'/>
<data>
<usr:user xmlns:usr="urn:ddbb-spml-dsml:user">
<usr:uid>ricky</usr:uid>
<usr:cn>Ricardo Martin</usr:cn>
<usr:description>me</usr:description>
<usr:role>User</usr:role>
<usr:role>Admin</usr:role>
</usr:user>
</data>
</pso>
</lookupResponse>
Modify
A modify request contains several modification operations but, in the XSD profile, a XPath expression is used to define where the operation take place. So the modify (as in DSML) can add, delete or replace a part of the XML document but the expression marks where that operation takes place. This is an example of how to construct a complex modify that adds a new role Other, remove role Admin and replaces the common name.
ModifyRequest mr = new ModifyRequest();
mr.setRequestID("mr1");
mr.setExecutionMode(ExecutionMode.SYNCHRONOUS);
PSOIdentifier psoId = new PSOIdentifier("ricky", null, "ddbb-spml-xsd");
mr.setPsoID(psoId);
// add role Other
Modification wrapper = new Modification();
wrapper.setModificationMode(ModificationMode.ADD);
Selection sel = new Selection();
sel.setNamespaceURI("http://www.w3.org/TR/xpath20");
sel.setPath("/user");
sel.addOpenContentElement(new XSDDocElement(
"<usr:role xmlns:usr=\"urn:ddbb-spml-dsml:user\">Other</usr:role>"));
wrapper.setComponent(sel);
mr.addModification(wrapper);
// remove role Admin
wrapper = new Modification();
wrapper.setModificationMode(ModificationMode.DELETE);
sel = new Selection();
sel.setNamespaceURI("http://www.w3.org/TR/xpath20");
sel.setPath("/user/role[text()='Admin']");
wrapper.setComponent(sel);
mr.addModification(wrapper);
// replace cn
wrapper = new Modification();
wrapper.setModificationMode(ModificationMode.REPLACE);
sel = new Selection();
sel.setNamespaceURI("http://www.w3.org/TR/xpath20");
sel.setPath("/usr:user/usr:cn");
sel.addOpenContentElement(new XSDDocElement(
"<usr:cn xmlns:usr=\"urn:ddbb-spml-dsml:user\">Ricardo Martin Camarero</usr:cn>"));
wrapper.setComponent(sel);
mr.addModification(wrapper);
Response res = client.send(mr);
In this point I have two comments. First I have coded the server in such a way that the xpath expression is evaluated with the default prefix for the namespace (last prefixed expression /usr:user/usr:cn) and not considering them (the previous examples, for example /user/role[text()='Admin'] to select the Admin role). Second the data can be a fragment or a complete user XML document (the examples just show three fragments), so it is complicated to parse that information in the XSDDocElement. I have decided to use the namespace (which I think is not in the standard). That is why this implementation is not very trustable. The server part in this request is more complicated, the current user is read from the ddbb and then the modifications are applied to that user. In the case of DSML the different values are added, deleted or replaced but in the case of XSD the user is transformed into a DOM document, the xpath element retrieved and the DOM elements sent in the request applied (deleted, added or replaced). The resulting user is updated in the ddbb and returned again using DSML or XSD.
ModifyResponse res = new ModifyResponse();
res.setStatus(StatusCode.SUCCESS);
if (SpmlUtils.checkSynch(req, res) && SpmlUtils.checkPSO(req, res) &&
SpmlUtils.checkTargetId(req.getPsoID().getTargetID(), res)) {
try {
User u = um.read(req.getPsoID().getID());
if (u == null) {
SpmlUtils.fillError(res, ErrorCode.INVALID_IDENTIFIER, "The user does not exist in the ddbb");
} else {
if (ListTargetsExecutor.TARGET_ID_DSML.equals(req.getPsoID().getTargetID())) {
// DSML modification
u = SpmlUtils.spmlDsml2User(u, req.getModifications());
} else {
// XSD modification
u = SpmlUtils.spmlXsd2User(u, req.getModifications(), res);
}
if (StatusCode.SUCCESS.equals(res.getStatus())) {
if (SpmlUtils.checkUser(u, res, false)) {
// add the user normally
if (um.update(u)) {
res.setStatus(StatusCode.SUCCESS);
if (ReturnData.IDENTIFIER.equals(req.getReturnData())) {
res.setPso(SpmlUtils.user2Spml(u, false,
ListTargetsExecutor.TARGET_ID_DSML.equals(req.getPsoID().getTargetID())));
} else if (ReturnData.DATA.equals(req.getReturnData())
|| ReturnData.EVERYTHING.equals(req.getReturnData())) {
res.setPso(SpmlUtils.user2Spml(u, true,
ListTargetsExecutor.TARGET_ID_DSML.equals(req.getPsoID().getTargetID())));
}
} else {
SpmlUtils.fillError(res, ErrorCode.INVALID_IDENTIFIER, "The user does not exist in the ddbb");
}
}
}
}
} catch (Exception e) {
e.printStackTrace();
SpmlUtils.fillErrorException(res, e);
}
}
return res;
The communication for the previous example between both is presented.
<modifyRequest xmlns='urn:oasis:names:tc:SPML:2:0' requestID='mr1' executionMode='synchronous' returnData='everything'>
<psoID ID='ricky' targetID='ddbb-spml-xsd'/>
<modification modificationMode='add'>
<component path='/user' namespaceURI='http://www.w3.org/TR/xpath20'>
<usr:role xmlns:usr="urn:ddbb-spml-dsml:user">Other</usr:role>
</component>
</modification>
<modification modificationMode='delete'>
<component path="/user/role[text()='Admin']" namespaceURI='http://www.w3.org/TR/xpath20'/>
</modification>
<modification modificationMode='replace'>
<component path='/usr:user/usr:cn' namespaceURI='http://www.w3.org/TR/xpath20'>
<usr:cn xmlns:usr="urn:ddbb-spml-dsml:user">Ricardo Martin Camarero</usr:cn>
</component>
</modification>
</modifyRequest>
<modifyResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='success' requestID='mr1'>
<pso>
<psoID ID='ricky' targetID='ddbb-spml-xsd'/>
<data>
<usr:user xmlns:usr="urn:ddbb-spml-dsml:user">
<usr:uid>ricky</usr:uid>
<usr:cn>Ricardo Martin Camarero</usr:cn>
<usr:description>me</usr:description>
<usr:role>User</usr:role>
<usr:role>Other</usr:role>
</usr:user>
</data>
</pso>
</modifyResponse>
As you can see the example presents the three possible operations but obviously the whole modification (sending a complete user) can always be executed sending a replace modification with the full XML for the user and specifying the XPath to point to root user element (/user in my PoC example).
Delete
The delete operation has no difference between profiles because no data is exchanged (just the pso identifier is needed). So the client part is exactly the same but specifying the XSD target.
DeleteRequest dr = new DeleteRequest();
dr.setRequestID("dr1");
dr.setExecutionMode(ExecutionMode.SYNCHRONOUS);
PSOIdentifier psoId = new PSOIdentifier("ricky", null, "ddbb-spml-xsd");
dr.setPsoID(psoId);
Response res = client.send(dr);
In the server implementation the executor has also no difference between profiles. The user is just deleted in the ddbb using the manager.
DeleteResponse res = new DeleteResponse();
res.setStatus(StatusCode.SUCCESS);
if (SpmlUtils.checkSynch(req, res) || SpmlUtils.checkPSO(req, res)) {
try {
if (!um.delete(req.getPsoID().getID())) {
SpmlUtils.fillError(res, ErrorCode.INVALID_IDENTIFIER,
"The user does not exist in the ddbb");
}
} catch (Exception e) {
e.printStackTrace();
SpmlUtils.fillErrorException(res, e);
}
}
return res;
The conversation is the same than in the previous entry.
<deleteRequest xmlns='urn:oasis:names:tc:SPML:2:0' requestID='dr1' executionMode='synchronous' recursive='false'>
<psoID ID='ricky' targetID='ddbb-spml-xsd'/>
</deleteRequest>
<deleteResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='success' requestID='dr1'/>
This is the second entry about the SPMLv2 provisioning standard. The entry has explained the XSD profile in which the data is sent in XML language defined by a XSD schema exchanged in the initial listTargets operation. The openspml library has been slightly modified in order to marshall and unmarshall the new XML data. You can download both projects from the following links: the modified openspml library with XSD support and the SpmlServlet project which I have used to present the test cases. In general SPMLv2 is a good standard that can be suitable for a lot of provisioning software but, sadly, it is quite unknown and rarely included (only some identity solutions are exposing SPML in the last years). The openspml library is quite old and developing a new implementation (based on JAXB or any new modern technology) would be much better. If I have time I will try to start the development of a new SPMLv2 library from scratch, using JAXB I suppose that all XML management should be quite easier.
Regards!
Sunday, August 23. 2015
SPMLv2 - Part I: DSML profile
Today's entry is about another standard: Service Provisioning Markup Language (SPML) Version 2. It is the only standard I know that models how to communicate with a provisioning system. SPML is almost unknown and rarely implemented by applications although it was defined long time ago. The standard is an XML-based framework, developed by OASIS, for exchanging any resource information (users, groups, containers,...) between cooperating organizations. If this framework finally got some traction it would be an excellent new for anyone related with provisioning or identity solutions. This series is my modest contribution to help spreading this unknown standard.
The framework defines several operations separated in capability groups. The core capability includes the functions listTargets, add, lookup, modify and delete. Over this basic CRUD functionality some other capabilities can be exposed by the server: async (asynchronous execution), batch (multiple operations sent in the same request), password (password change, reset and so on), search (searching over the objects), suspend (enables / disables users), updates (obtaining changed records) and custom capability (custom extensions). As you see SPML is thought to be open and extensible, for this reason, it defines two profiles that controls the schema of the interchanged data.
The DSML schema as defined in the SPMLv2 DSMLv2 Profile. In this profile all the resources are managed as a bunch of attributes similar to an LDAP server.
- The XML Schema as defined in the SPMLv2 XSD Profile. In this profile all the interchanged format is XML data formatted using an XSD schema file which is exposed in the listTargets operation.
A SPML server can define several targets, each target should use one of the previous two profiles. This first entry is going to show the DSML profile. One of the problems of SPML is the absence of good implementations in any language (you know, if the standard is not used, there are no implementations). The old Sun openspml is the only library that handles SPML in Java. The project only covers the DSML profile and it is not maintained at all since Oracle acquisition. In spite of these disadvantages I am going to use the library to implement a sample server and client in order to show how SPML works. Only the core capabilities and synchronous execution will be examined.
My simple PoC uses SPML to provision a user inside a fake database application (just a simple table with a one to many relation to a role table). The tables that are provisioned were defined in MySQL like this:
CREATE TABLE USER (
USERNAME varchar(50) NOT NULL,
PASSWORD varchar(50) NOT NULL,
COMMON_NAME varchar(256) NOT NULL,
DESCRIPTION varchar(1024) DEFAULT NULL,
PRIMARY KEY (USERNAME)
);
CREATE TABLE USER_ROLES (
USERNAME varchar(50) NOT NULL,
ROLENAME varchar(50) NOT NULL,
PRIMARY KEY (USERNAME,ROLENAME),
FOREIGN KEY (USERNAME) REFERENCES USER(USERNAME)
);
Obviously a User POJO class and a UserManager were developed. A class that models the user tables in Java and the manager with CRUD operations.
The openspml library gives a Servlet (RPCRouterServlet) and a SOAP dispatcher (SPMLViaSoapDispatcher) that can be used to implement an empty SOAP/SPML server (SPMLv2 does not force any transport protocol, i.e., any protocol can be used to exchange SPML data, but SOAP is usually used cos it is easy to include SPML/XML messages inside a SOAP conversation). The Servlet receives a SPMLExecutor, which is an interface that just receives an SPML Request and returns a Response. The openspml library uses the command pattern which fits quite well in the SPML protocol (each operation uses a specific Request and Response object). My server implementation just manages a SimpleSPMLExecutor which receives the SPML requests, transforms to a intermediate User class, interacts with the ddbb with the UserManager and returns a SPML response.
Once the PoC is explained the different core methods will be shown.
ListTargets
The listTargets in SPML is intended to be the first request any client should send. The command is used to identify how many targets the server has and their schema definition. Using openspml library a listTargets is easily created:
Spml2Client client = new Spml2Client("http://localhost:8080/SpmlServlet/rpcrouter2");
ListTargetsRequest ltr = new ListTargetsRequest();
ltr.setRequestID("ltr1");
ltr.setProfile(new URI(XSDProfileRegistrar.PROFILE_URI_STRING));
ltr.setExecutionMode(ExecutionMode.SYNCHRONOUS);
ListTargetsResponse res = client.send(ltr);
My PoC supports both profiles (XSD and DSML) and the standard says that the server should just answer with the targets of the requested profile that was specified. So the implementation checks if the request asks for a specific profile and responds accordingly. The conversation is shown below:
<listTargetsRequest xmlns='urn:oasis:names:tc:SPML:2:0' requestID='ltr1' executionMode='synchronous' profile='urn:oasis:names:tc:SPML:2:0:DSML'/>
<listTargetsResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='success' requestID='ltr1'>
<target targetID='ddbb-spml-dsml' profile='urn:oasis:names:tc:SPML:2:0:DSML'>
<schema>
<spmldsml:schema xmlns:spmldsml='urn:oasis:names:tc:SPML:2:0:DSML'>
<spmldsml:objectClassDefinition name='user'>
<spmldsml:memberAttributes>
<spmldsml:attributeDefinitionReference required='true' name='uid'/>
<spmldsml:attributeDefinitionReference required='true' name='password'/>
<spmldsml:attributeDefinitionReference required='true' name='cn'/>
<spmldsml:attributeDefinitionReference name='description'/>
<spmldsml:attributeDefinitionReference name='role'/>
</spmldsml:memberAttributes>
</spmldsml:objectClassDefinition>
<spmldsml:attributeDefinition description='User name' name='uid'/>
<spmldsml:attributeDefinition description='Password' name='password'/>
<spmldsml:attributeDefinition description='Common name' name='cn'/>
<spmldsml:attributeDefinition description='description' name='description'/>
<spmldsml:attributeDefinition description='roles (multiple)' name='role' multivalued='true'/>
</spmldsml:schema>
<supportedSchemaEntity entityName='user'/>
</schema>
<capabilities/>
</target>
</listTargetsResponse>
It is clear that the listTargets answers with the DSML target of the server. In the DSML profile this operation answers with the schema of the object classes and attributes managed in each target. In my PoC there is only one objectclass (user) which has five attributes (uid, password, cn, description and role). So the listTargets operations gives to the client enough information about the objects and the attributes the target manages.
Add
The add request performs the creation of an object, in this PoC a user in the database. The client part is just the construction of the add request, adding all the attributes of the user (as specified in the schema returned in the listTargets response) and sending it. The openspml library gives a generic Extensible class that can handle the DSML attributes.
AddRequest ar = new AddRequest();
ar.setRequestID("ar1");
ar.setExecutionMode(ExecutionMode.SYNCHRONOUS);
ar.setTargetId("ddbb-spml-dsml");
Extensible data = new Extensible();
data.addOpenContentElement(new DSMLAttr("uid", "ricky"));
data.addOpenContentElement(new DSMLAttr("objectclass", "user"));
data.addOpenContentElement(new DSMLAttr("password", "ricky"));
data.addOpenContentElement(new DSMLAttr("cn", "Ricardo Martin"));
data.addOpenContentElement(new DSMLAttr("description", "me"));
data.addOpenContentElement(new DSMLAttr("role",
new DSMLValue[]{new DSMLValue("Admin"), new DSMLValue("User")}));
ar.setData(data);
Response res = client.send(ar);
The server part recovers the data in DSML attributes and convert again into a User, then the user is saved into the tables using the manager. The conversation is as follows:
<addRequest xmlns='urn:oasis:names:tc:SPML:2:0' requestID='ar1' executionMode='synchronous' targetId='ddbb-spml-dsml' returnData='everything'>
<data>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='uid'>
<dsml:value>ricky</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='objectclass'>
<dsml:value>user</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='password'>
<dsml:value>ricky</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='cn'>
<dsml:value>Ricardo Martin</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='description'>
<dsml:value>me</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='role'>
<dsml:value>Admin</dsml:value>
<dsml:value>User</dsml:value>
</dsml:attr>
</data>
</addRequest>
<addResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='success' requestID='ar1'>
<pso>
<psoID ID='ricky' targetID='ddbb-spml-dsml'/>
<data>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='uid'>
<dsml:value>ricky</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='password'>
<dsml:value>ricky</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='cn'>
<dsml:value>Ricardo Martin</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='description'>
<dsml:value>me</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='role'>
<dsml:value>User</dsml:value>
<dsml:value>Admin</dsml:value>
</dsml:attr>
</data>
</pso>
</addResponse>
The request just specifies in which target the user is going to be created and then all the data is specified as DSML attributes and values. The returnData attribute defines what data the response will include. Everything means thet the server should return all the data of the object after the creation. Inside SPML every object (pso) has a ID (psoID) which is used in all the following operations.
Lookup
The lookup operation is the method of reading a user inside a target. The data is returned inside the pso as it was done in the add request. The client lookup using openspml is easy to be done.
LookupRequest lr = new LookupRequest();
lr.setRequestID("lr1");
lr.setExecutionMode(ExecutionMode.SYNCHRONOUS);
PSOIdentifier psoId = new PSOIdentifier("ricky", null, "ddbb-spml-dsml");
lr.setPsoID(psoId);
Response res = client.send(lr);
The request specifies which object is requested (ID and the target are specified, the second argument is used to define a container, some objects in SPML can act as containers, folders where other objects can be created like in a File System or in an LDAP server). By default the return data is everything, so the pso is returned exactly as in the add operation.
<lookupRequest xmlns='urn:oasis:names:tc:SPML:2:0' requestID='lr1' executionMode='synchronous' returnData='everything'>
<psoID ID='ricky' targetID='ddbb-spml-dsml'/>
</lookupRequest>
<lookupResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='success' requestID='lr1'>
<pso>
<psoID ID='ricky' targetID='ddbb-spml-dsml'/>
<data>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='uid'>
<dsml:value>ricky</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='cn'>
<dsml:value>Ricardo Martin</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='description'>
<dsml:value>me</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='role'>
<dsml:value>User</dsml:value>
<dsml:value>Admin</dsml:value>
</dsml:attr>
</data>
</pso>
</lookupResponse>
Modify
The modify request let us change or update any resource in the target server. In DSML profile some attribute values can be added, deleted or replaced (very similar to an LDAP server). With openspml the implementation is very similar to the add request but sending all the modifications you want.
ModifyRequest mr = new ModifyRequest();
mr.setRequestID("mr1");
mr.setExecutionMode(ExecutionMode.SYNCHRONOUS);
PSOIdentifier psoId = new PSOIdentifier("ricky", null, "ddbb-spml-dsml");
mr.setPsoID(psoId);
Modification wrapper = new Modification();
// delete description attribute
DSMLModification dsmlmod = new DSMLModification("description", "me", ModificationMode.DELETE);
wrapper.addOpenContentElement(dsmlmod);
// replace the password attribute
dsmlmod = new DSMLModification("password", "ricky123", ModificationMode.REPLACE);
wrapper.addOpenContentElement(dsmlmod);
// remove Admin role
dsmlmod = new DSMLModification("role", "Admin", ModificationMode.DELETE);
wrapper.addOpenContentElement(dsmlmod);
// Add Test role
dsmlmod = new DSMLModification("role", "Test", ModificationMode.ADD);
wrapper.addOpenContentElement(dsmlmod);
mr.addModification(wrapper);
Response res = client.send(mr);
The conversation is like this:
<modifyRequest xmlns='urn:oasis:names:tc:SPML:2:0' requestID='mr1' executionMode='synchronous' returnData='everything'>
<psoID ID='ricky' targetID='ddbb-spml-dsml'/>
<modification>
<dsml:modification xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='description' operation='delete'>
<dsml:value>me</dsml:value>
</dsml:modification>
<dsml:modification xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='password' operation='replace'>
<dsml:value>ricky123</dsml:value>
</dsml:modification>
<dsml:modification xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='role' operation='delete'>
<dsml:value>Admin</dsml:value>
</dsml:modification>
<dsml:modification xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='role' operation='add'>
<dsml:value>Test</dsml:value>
</dsml:modification>
</modification>
</modifyRequest>
<modifyResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='success' requestID='mr1'>
<pso>
<psoID ID='ricky' targetID='ddbb-spml-dsml'/>
<data>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='uid'>
<dsml:value>ricky</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='password'>
<dsml:value>ricky123</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='cn'>
<dsml:value>Ricardo Martin</dsml:value>
</dsml:attr>
<dsml:attr xmlns:dsml='urn:oasis:names:tc:DSML:2:0:core' name='role'>
<dsml:value>User</dsml:value>
<dsml:value>Test</dsml:value>
</dsml:attr>
</data>
</pso>
</modifyResponse>
The modify request contains several DSML modifications (I have sent four different modified attributes in the previous example) to a specified psoID user. The response (just like add or lookup operations) contains all the data of the user as everything was set as return data.
Delete
The last delete operation needs the psoID of the object to be deleted.
DeleteRequest dr = new DeleteRequest();
dr.setRequestID("dr1");
dr.setExecutionMode(ExecutionMode.SYNCHRONOUS);
PSOIdentifier psoId = new PSOIdentifier("ricky", null, "ddbb-spml-dsml");
dr.setPsoID(psoId);
Response res = client.send(dr);
And the conversation is the following:
<deleteRequest xmlns='urn:oasis:names:tc:SPML:2:0' requestID='dr1' executionMode='synchronous' recursive='false'>
<psoID ID='ricky' targetID='ddbb-spml-dsml'/>
</deleteRequest>
<deleteResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='success' requestID='dr1'/>
This entry is a little introduction about a rare provisioning standard called SPMLv2. This standard defines an interesting framework to provision any object or piece of data or information inside an application. Like any standard it gives the ability to separate the client and the server (any SPML compliant server should work with a compatible client and vice-versa). It would be incredible to deploy your automatic provisioning system independently of the chosen software (it is a quite ambitious idea but a provisioning standard like this one is always needed as a first step). Until now SPML is uncommon and rarely implemented by provisioning or identity solutions. Consequently there are no production level SPML implementations in any of the typical development languages. For example in Java only openspml library exists, which is an old try developed by Sun Microsystems that only covers the DSML profile and is currently unmaintained. This entry uses this outdated library to exemplify how the DSML profile works and how to perform basic CRUD operations with SPML/DSML. I am trying to extend the openspml library to cover the XSD profile in order to also show both profiles. You can expect a second part in the series if I can get it working. I will link the sources then cos now it is a bit messy and not complete.
Be standard my friend!
Saturday, June 1. 2013
HTTP Headers and JavaEE
Today I am going to present a little entry about an issue I found some weeks ago. One Web Single SingOn (SSO) solution, which is already using the old and nice opensso, was planning to move agents from the application layer (JavaEE containers) to the web layer. Until that moment the opensso agents just intercept the requests, perform all the SignOn stuff and insert some information to the application using headers (common user information like identifier, name, email,... And two multi-valued attributes which act as a profile, data to inform the application what the user can and cannot do). Besides a JAR library is used in the applications to isolate the part of the code that interacts with the SSO system.
At first sight the story seemed very easy. Just installing a new agent in the desired web server and removing any agent from the application server. In theory the agent in the front layer can perform exactly the same actions (authentication and header injection) and the library in the applications should work properly without any change. But, as usual, theory and practice are never the same thing.
Two problems came out and needed to be resolved:
The multi-valued attributes are passed differently. In the JavaEE layer they were just several values and they are retrieved using the getHeaders method. The following example shows this idea:
List<String> values = new ArrayList<>(); Enumeration<String> e = request.getHeaders(headerName); while (e.hasNext()) { values.add(e.next()); }
That is because the agent in the JavaEE container just intercepts the request and adds the pre-configured headers in the same JVM (in case of opensso using a wrapper request object). Nevertheless in the web layer all the values are added in only one header using a separator. The standard HTTP 1.1 says the following about multiple header values with the same name:
Multiple message-header fields with the same field-name MAY be present in a message if and only if the entire field-value for that header field is defined as a comma-separated list [i.e., #(values)]. It MUST be possible to combine the multiple header fields into one "field-name: field-value" pair, without changing the semantics of the message, by appending each subsequent field-value to the first, each separated by a comma. The order in which header fields with the same field-name are received is therefore significant to the interpretation of the combined field value, and thus a proxy MUST NOT change the order of these field values when a message is forwarded.
So there is a limitation and, for that, opensso web agents just add a unique value separated by a pre-configured separator (| or pipe by default). This way the agents avoid the problem of a comma being part of a value. Obviously the application library had to be modified to split values in order to get multi-valued attributes properly.
The second (and the main) issue was non-ascii characters in the header values. As you now I am Spanish and as in any non-English country our names can contain non-ascii characters (acutes, character ñ,...). All those characters were shown garbled with the new configuration.
I spent some time trying to guess what the standard says about non-ascii headers and it seems that headers should be in ISO-8859-1 encoding but there is a way of sending other charsets:
The TEXT rule is only used for descriptive field contents and values that are not intended to be interpreted by the message parser. Words of *TEXT MAY contain characters from character sets other than ISO- 8859-1 only when encoded according to the rules of RFC 2047.
So, because of the standard, headers accept any encoding and it should be marked like it is done in the email system (RFC 2047). Java mail for example has methods to encode and decode text in that format. Here it is two header examples for my common name using Q (quoted printable) and B (Base64) encodings respectively:
COMMON_NAME_Q: =?UTF-8?Q?Ricardo_Mart=C3=ADn?= COMMON_NAME_B: =?UTF-8?B?UmljYXJkbyBNYXJ0w61u?=
For what I understand from the spec a encoded header is intended to be parsed by the local application and not by the JavaEE server. If you check the code of tomcat or glassfish there is always a default charset which is ISO-8859-1 and no special parsing is performed. I even tried to recompile a tomcat 7 changing the default from ISO-8859-1 to UTF-8 just to certify my suspicion. And it indeed worked (the characters were not garbled any more), but obviously that test was just a proof and not a solution at all. In my opinion the problem here is that the opensso webagents cannot send headers in RFC 2047 format. I found that at least there is a property, called com.sun.am.policy.agents.config.convert_mbyte.enable, that let encode the headers in the system locale (opensso in UNIX uses iconv to perform the conversion). So if the locale had been set to any ISO-8859-1 language I suppose it would have worked. But obviously that property limits the header values to only the characters allowed by that encoding.
So I think that the problem is as follows: the agent in the web layer inserts the headers in UTF-8 (if the commented property is not set) and the JavaEE container receives it as ISO-8859-1. Garbling is assured. It worked before because the agent was inside the JVM and a common UTF-8 string was placed as a header (remember headers in the previous situation just live inside the JVM and never traveled from one server to another). Finally I reached the conclusion that the best thing I could do was converting the headers by my own inside the library:
String value = request.getHeader(headerName); value = new String(value.getBytes(Charset.forName("ISO-8859-1")), Charset.forName("UTF-8"));
I know it is a crap but I defined some configuration parameters in the library (I am a bit sloppy but elegant). Everybody should be aware that there are some bugs involved here. I think that I decided to follow the least bad solution.
The conclusion of this entry is that chaos theory is a matter of fact, that any little change can break your complete solution up, that shit happens. And as I am going to forget this situation very quickly I thought it was nice to write a little summary here. You know that sometimes this blog is just my personal logbook.
Sometimes just fix it and run away!
Comments