Thursday, April 8. 2010
Setting Routes Using C
Last week I was on eastern vacation and I decided to do some cleaning in my laptop. Doing that I almost deleted an old virtual image of OpenSolaris which would have been a real pity because some C examples of great worth were in it. One of the reasons to start this blog was exactly this, try to not lose or forget things I have done. So here it is, a new entry.
Some months ago a customer was migrating a project developed in C from linux to Solaris/OpenSolaris. Briefly the application was a client/server solution with a communication high availability requirement (data flow had to resist a link drop). Customer availability consisted of having two different lines between server and clients. When there was a link drop over current line the client software assigned another IP on another interface and added a new route to reach the server using the second line (new net route, not host, using gateway and netmask). But when they tested their client software on OpenSolaris routes were not assigned as expected. Software was using SIOCADDRT ioctl to add the new route.
When I faced this problem (I have experience in C programming but I had never used such ioctl) the first thing I did was developing a little example using linux. It was extremely easy, SIOCADDRT ioctl uses a rtentry structure with three fields:
Once I had the linux test I tried to compile it under OpenSolaris. After changing or adding various header includes first issue was that you need more libraries to make compilation of ioctl-solaris.c work:
And second, rtentry structure did not have any rt_genmask field. As a result of that the 10.35.7.0 route was created with A class netmask (255.0.0.0) following IETF/IANA private network restrictions, which was obviously wrong. In Solaris the /etc/netmasks file contains custom masks for subnetting, so I added the following line:
The line is quite clear and means the net 10.35.7.0 has a netmask 255.255.255.0. This is very pretty but ioctl-solaris did not work, it kept assigning A class netmask (ioctl was apparently not reading this file). So SIOCADDRT way was for me depleted and I needed another completely different solution.
OpenSolaris is the open source version (and a kind of test bench) of Solaris. So the solution was easy: route (the typical command of any UNIX to add/remove network routes) source code had to be be revised. Studying the code was crystal clear route command does not use any ioctl but a kind of socket. With this valuable information I discovered Solaris has something called routing sockets. These special sockets can be used to get the actual list of routes, add a new route or delete a current one. This functionality is quite simple, some special data structures are written into the socket and results are read from it. Very recommended lectures about routing sockets are TCP/IP Illustrated, Volume 2: The Implementation and Unix Network Programming, Volume 1: The Sockets Networking API both of Richard Stevens (some chapters are dedicated to this issue).
Following this new way the routing-socket-solaris.c was created. This example added successfully the same route commented before but using a routing socket.
In order to complete the circle I also created rtnetlink-linux.c file. Linux also has a special routing sockets but it calls the technique rtnetlink. It works very similar than Solaris but using different structures and constants. Manipulating the Networking Environment Using RTNETLINK is a very good entry about rtnetlink published in linuxjournal.
Compilation in Linux again does not need any library:
Knowing this information is now safe here (or at least safer) makes me feel comfortable. Now I can delete my OpenSolaris image, I need the space.
cheerio!
Some months ago a customer was migrating a project developed in C from linux to Solaris/OpenSolaris. Briefly the application was a client/server solution with a communication high availability requirement (data flow had to resist a link drop). Customer availability consisted of having two different lines between server and clients. When there was a link drop over current line the client software assigned another IP on another interface and added a new route to reach the server using the second line (new net route, not host, using gateway and netmask). But when they tested their client software on OpenSolaris routes were not assigned as expected. Software was using SIOCADDRT ioctl to add the new route.
When I faced this problem (I have experience in C programming but I had never used such ioctl) the first thing I did was developing a little example using linux. It was extremely easy, SIOCADDRT ioctl uses a rtentry structure with three fields:
- rt_dst. Destination net (for example 10.35.7.0).
- rt_genmask. Netmask for the previous net (255.255.255.0).
- rt_gateway. Gateway to send packets to when reaching this net (192.168.122.1).
# gcc -o ioctl-linux ioctl-linux.c
Once I had the linux test I tried to compile it under OpenSolaris. After changing or adding various header includes first issue was that you need more libraries to make compilation of ioctl-solaris.c work:
# gcc -o ioctl-solaris -lsocket -lnsl ioctl-solaris.c
And second, rtentry structure did not have any rt_genmask field. As a result of that the 10.35.7.0 route was created with A class netmask (255.0.0.0) following IETF/IANA private network restrictions, which was obviously wrong. In Solaris the /etc/netmasks file contains custom masks for subnetting, so I added the following line:
10.35.7.0 255.255.255.0
The line is quite clear and means the net 10.35.7.0 has a netmask 255.255.255.0. This is very pretty but ioctl-solaris did not work, it kept assigning A class netmask (ioctl was apparently not reading this file). So SIOCADDRT way was for me depleted and I needed another completely different solution.
OpenSolaris is the open source version (and a kind of test bench) of Solaris. So the solution was easy: route (the typical command of any UNIX to add/remove network routes) source code had to be be revised. Studying the code was crystal clear route command does not use any ioctl but a kind of socket. With this valuable information I discovered Solaris has something called routing sockets. These special sockets can be used to get the actual list of routes, add a new route or delete a current one. This functionality is quite simple, some special data structures are written into the socket and results are read from it. Very recommended lectures about routing sockets are TCP/IP Illustrated, Volume 2: The Implementation and Unix Network Programming, Volume 1: The Sockets Networking API both of Richard Stevens (some chapters are dedicated to this issue).
Following this new way the routing-socket-solaris.c was created. This example added successfully the same route commented before but using a routing socket.
# gcc -o routing-socket-solaris -lsocket -lnsl routing-socket-solaris.c
In order to complete the circle I also created rtnetlink-linux.c file. Linux also has a special routing sockets but it calls the technique rtnetlink. It works very similar than Solaris but using different structures and constants. Manipulating the Networking Environment Using RTNETLINK is a very good entry about rtnetlink published in linuxjournal.
Compilation in Linux again does not need any library:
# gcc -o rtnetlink-linux rtnetlink-linux.c
Knowing this information is now safe here (or at least safer) makes me feel comfortable. Now I can delete my OpenSolaris image, I need the space.
cheerio!
Comments