Super Sparrow: Global Load Balancing Solution for Linux [Sparrow]
| English | Japanese || Main | Code | Installation | Configuration | Paper | Contacts |

Super Sparrow Code

Super Sparrow provides a mechanism for BGP information to be retrieved from Route Servers. The core functionality to do this is implemented in a library, libsupersparrow. Two applications have been built using libsupersparrow: A stand alone application, supersparrow. And a Dents driver module, mod_supersparrow.

This code is available for download with this release of Super Sparrow in both binary and source forms. Alternatively the latest code may be obtained via CVS.

Library: libsupersparrow

libsupersparrow implements the core functionality for Super Sparrow applications. All exported functions, data structures and macros with descriptions can be found in supersparrow.h, when libsupersparrow is installed. The source for both supersparrow and mod_supersparrow are intended as reference implementations of Super Sparrow applications.

Dents Driver Module: mod_supersparrow

The Dents driver module, mod_supersparrow enables Super Sparrow to return information on what point of presence the a client should connect to through DNS.

[Network Diagram]

Suppose that www.slarken.org.au is mirrored between POP X and Y and mod_supersparrow is being used to distribute traffic between these two POPs.

  1. Client Makes DNS Request to local DNS Server in Network C for www.slarken.org.au

  2. The DNS Server makes a recursive query on behalf of the Client. In doing so it queries POP X for the IP address of www.slarken.org.au. Both POP X and Y are authorative for the slarken.org.au domain, the Network C DNS Server happens to query POP X this time around. POP Y would do equally well.

  3. The DNS server in POP X is running Dents and www.slarken.org.au is in a zone handled by mod_supersparrow. The local route server is queried and it is determined that POP Y is the closest POP to the Network C DNS Server. The best route to the Network C DNS Server is queried and not the best route to the Client, as the IP address of the Client is not known to the DNS server in POP X. It is assumes that Clients use DNS servers that are network-wise close to them. The IP address of the web server or Ultra Monkey web farm in POP Y is returned in response to the DNS query by the Network C DNS Server.

    If POP X had been down then Network C DNS Server would have queried POP Y and returned the IP address of the web server or farm within itself, thus the result would be the same.

    If POP Y was down then the query to the route server in POP X would have shown that POP X was the closest POP to the Network C DNS Server and the IP address of the web server or farm in POP X would be returned.

    If both POPs were down then there would be no result and the DNS lookup would fail.

  4. Network C DNS Server responds to the Client's DNS request with the answer obtained from POP X.

  5. The client has the IP address of a server in POP Y as the IP address of www.slarken.org.au and makes an HTTP request to this server.

  6. The server responds to the Client's HTTP request.

Standalone Application: supersparrow

supersparrow is a standalone application intended to provided access to the functionality provided by libsupersparrow. It may be used as a test tool, or to enable any application that can use stdio to use Super Sparrow. The sample invocations were run on the POP X configured as per the configuration section..

$ supersparrow \
  --peer 65751=192.168.192.13,64750=192.168.193.11\
  --debug \
  --host localhost \
  --password frub \
  --route_server zebra \
  --self 192.168.192.13 \
  --source 192.168.193.15
192.168.193.11
This shows that Super Sparrow things that the closest peer is 192.168.193.11. The command line options used are:

A full list of command line options can be found by running supersparrow --help. Both short and long versions of options are listed. Command line options may also be specified in the configuration file, /etc/supersparrow.conf. The Format is <key> <value> where key is either a short or long option, without the leading - or --. Blank lines are ignored, as is anything including and after a # (hash) on a line.

Options that do not make sense in the configuration file such as h|help and f|config_file are ignored. Options specified on the command line override the options in this file.

By running the supersparrow command again, with the --verbose command line option the query to and result from the route server can be observed.

$ supersparrow \
  --peer 65751=192.168.192.13,64750=192.168.193.11 \
  --debug \
  --host localhost \
  --password frub \
  --route_server zebra \
  --self 192.168.192.13 \
  --source 192.168.193.15 \
  --verbose
supersparrow version 0.0.0pre0 Copyright Horms
Logs sent to syslog


Hello, this is zebra (version 0.89.horms.pre.2)
Copyright 1996-2000 Kunihiro Ishiguro


User Access Verification

"Password: 
jasmine> sh ip bgp  192.168.193.15
BGP routing table entry for 192.168.193.0/24
Paths: (2 available, best #2, table Default-IP-Routing-Table)
  64754 64755 64752
    192.168.192.1 from 192.168.192.1 (192.168.192.12)
      Origin IGP, metric 1, localpref 100, valid, external
      Last update: Thu Oct  5 00:52:59 2000

  64750
    192.168.193.11 from 192.168.193.11 (192.168.193.11)
      Origin IGP, metric 1, localpref 100, valid, external, best
      Last update: Thu Oct  5 00:09:12 2000

jasmine> 
PEERS: 6571=192.168.192.13 64750=192.168.193.11
ASPATH: 64750
192.168.193.11
This shows that there are two prefixes of equal specificity covering 192.168.195.14 known by the route server and the second one listed is preferred. This prefix has the AS path 64750. The --peer command line option configured 64750 as a peer with IP address 192.168.193.11. This IP address is returned by supersparrow.

As well as linking against libsupersparrow, supersparrow requires libpopt for command line argument option parsing. If you do not have this library it is available from ftp://ftp.redhat.com/pub/redhat/code/popt/ and mirrors.

Using supersparrow with Apache

The flexibility of The Apache HTTP Server and in particular Apache's mod_rewrite allows supersparrow to be tied directly into Apache. An example follows:

[Network Diagram]

Suppose once again that www.slarken.org.au is mirrored between POP X and Y and supersparrow is being used in conjunction with Apache's mod_rewrite to distribute traffic between these two POPs.

  1. Web servers or Ultra Monkey web farm in POP X and Y are both listed as IP addresses for www.slarken.org.au. The client happens to send an HTTP request to POP X this time around. POP Y would do equally well.

  2. The Apache server in POP X is running supersparrow in conjunction with mod_rewrite. A rewrite rule matches the request made by Client and passes the Client's IP address to supersparrow. The local route server is queried and it is determined that POP Y is the closest POP to the Client. Note that the IP address of the Client is used here, whereas when using mod_supersparrow and Dents the IP address of the Client's DNS server is used. The Apache server then sends an HTTP redirect to a URL that resolves to the IP address of a web server or farm in POP Y, www.y.slarken.org.au.

    If POP X had been down then the client would most likely have stalled. A reload would probably have cased an HTTP request to be sent to the other IP address for www.slarken.org.au and POP Y would handle the request from there.

    If POP Y was down then the query to the route server in POP X would have shown that POP X was the closest POP to the Client and the Apache server would send an HTTP redirect to a URL that resolves to the IP address of a web server or farm in POP X, www.x.slarken.org.au.

    If both POPs were down then the connection would fail, as would a reload.

  3. The Client has www.y.slarken.org.au, the URL of a server in POP Y and makes an HTTP request to this server.

  4. The server responds to the Client's HTTP request.

mod_supersparrow and supersparrow Logging

Both mod_supersparrow and supersparrow utilise a consistent logging interface provided by libvanessa_logger. The logger used logs messages to syslog with the facility LOG_USER. In addition mod_supersparrow logs errors to the console.

All logs are preceded by an identifier, dents_mod_supersparrow or supersparrow and the process id. In addition syslog should prepend the date that the message was logged and the hostname of the host on which the message was produced. For example:

Oct  9 00:18:36 jasmine supersparrow[26355]: Warning: Self not set. See -s|--self option

When either application is in debugging mode, internal errors will be logged with a priority of LOG_DEBUG. Other messages are generally logged with a priority of LOG_WARNING. To ensure that these, and all other messages produced will be logged to disk by syslog add the following to /etc/syslog.conf.

user.debug                                              /var/log/messages

Once this is done restart syslogd. If syslog was installed as an RPM this may be done using the syslog init script.

$ /etc/rc.d/init.d/syslog restart
Shutting down kernel logger:                [  OK  ]
Shutting down system logger:                [  OK  ]
Starting system logger:                     [  OK  ]
Starting kernel logger:                     [  OK  ]

Otherwise this may be done by sending syslogd a SIGHUP to the running syslogd process.

killall -HUP syslogd

To test this run vanessa_logger_sample which is distributed as part of libvanessa_logger and check that the log with priority LOG_DEBUG appears in /var/log/messages. Note that vanessa_logger_sample must be run by a non-privileged user with UID and EUID greater than 100, as some of the tests write to a file without proper checking.

$ vanessa_logger_sample 
vanessa_logger_sample version 0.0.0pre0 Copyright Horms

Opening loggers
Logging message to stderr
vanessa_logger_sample[9498]: This should log to stderr: 7
Logging message to ./vanessa_logger_sample.log
Logging message to syslog facility LOG_USER, priority LOG_DEBUG
If the message is not logged to syslog then you may need to add
the following to /etc/syslog.conf and restart syslogd:
user.debug                                    /var/log/messages

Reopening loggers
Logging another message to stderr
vanessa_logger_sample[9498]: This should also log to stderr
Logging another message to ./vanessa_logger_sample.log
Logging another message to syslog facility LOG_USER, priority LOG_INFO

Testing that logs are filtered out by priority
No logs should appear after this line

Examining /var/log/messages.

Oct  9 01:00:50 jasmine vanessa_logger_sample[9498]: This should log to syslog facility LOG_USER, priority LOG_DEBUG: 7

CVS

Current development versions of the Super Sparrow code are available from CVS. Patches are always welcome. To browse the CVS repository over the web take a look at Super Sparrow's Source Forge page. To access the CVS repository do the following:

To set up the environment under a Bourne shell:

export CVSROOT=":pserver:anonymous@cvs.supersparrow.sourceforge.net:/cvsroot/supersparrow"

To set ip the environment under a C shell:

setenv CVSROOT ":pserver:anonymous@cvs.supersparrow.sourceforge.net:/cvsroot/supersparrow"

To login:

$ cvs login

When you see the following prompt just hit enter for an empty password:

(Logging in to anonymous@cvs.supersparrow.sourceforge.net)
CVS password:

libsupersparrow and supersparrow

To check out the run:

$ cvs -z3 co supersparrow

To update the code run:

$ cvs -z3 update -dP supersparrow

To compile run:

$ cd supersparrow
$ ./autogen-sh [--prefix=/usr]
$ make

To install run:

$ make install

RPMs can be built buy running:

$ make distcheck
$ rpm -ta <resulting_tar_ball.tar.gz>

mod_supersparrow

To check out the run:

$ cvs -z3 co dents_mod_supersparrow

To update the code run:

$ cvs -z3 update -dP dents_mod_supersparrow

Before compiling mod_supersparrow you will need to have libsupersparrow installed and will need the source to Dents. Details of which version of dents the module will compile against.

To compile Dents and mod_supersparrow run:

$ mv dents_mod_supersparrow <root_directory_for_dents_download>/modules/mod_supersparrow
$ cd <root_directory_for_dents_download>
$ ./autogen-sh --without-pthreads [--prefix=/usr]
$ make

To install Dents and mod_supersparrow run:

$ make install

An RPM of Dents and mod_supersparrow can be built buy running:

$ make distcheck
$ rpm -ta <resulting_tar_ball.tar.gz>

Notes on Commands

Commands shown in paragraphs of preformated text are prefixed by the shell prompt $ to avoid confusion between commands and their output. An instruction to run the command echo flim is formated as:
$ echo flim
flim



Copyright © 2000 Horms

Last Updated: Sun Nov 26 07:45:06 2000