Packaging the GATHER Appliance

21May08

Ease of use is on everyone’s short list of design goals for everything. Yet, project installation ranges from the elegance of Instiki’s “there is no step three” to the week-long exercise in mind-reading required for many “enterprise” applications. While we’d prefer to package GATHER into a simple executable jar file, having two separate web applications and a database makes that awkward.

Considering that our deployment scenarios always involve installing a physical server, we decided that our best option was to package GATHER into an appliance using the excellent tools available from rBuilder Online. With rBuilder, we’d have these features:

JeOS (just enough OS) using a Conary-based system
Web-based management for system updates
VMware image, Amazon EC2 AMI, or installable DVD from the same recipe

That last bullet is a killer, allowing us to create a release that is easily installed for testing (using a VMware image), for public access (running on Amazon EC2), or for deployment to a dedicated server.

I’ll walk through the steps of creating a new build of the GATHER appliance. To play along, you’ll need an account on rBuilder Online, and if you’re a GATHER team-member contact me to add your account to the project.

rPath tools

There are three command-line tools used for building and deploying an appliance image:

conary - the package-management tool used for system administration
- also note the Conary Commands wiki category
- this is a sysadmin tool for maintaining a conary-based system
cvc – the basic build tool, used mostly for source code management-like commands
- think of this as cvs or svn for conary recipe development
rmake - comprehensive build tool for developing appliances
- uses cvc for some operations
- always use an rmake command instead of the cvc equivalent

Get an Appliance Build Environment

Developing an appliance with rPath requires a conary-based system with the standard set of build tools. The easiest way to get that set up is to download and run the App2App image, which was custom built for the rPath Application to Appliance guide, but is perfect for general appliance development.

Download and install the free VMware Player (if you have Windows of Linux. Mac users will have to buy VMware Fusion or experiment with one of the open source virtualization options)
Download the App2App Image from rPath
Launch the App2App image
Log in using username/password = devuser/password

Now you are logged in to a conary-based system. If you look in the home directory you’ll notice that it is pre-populated with a development directory for an example project. For GATHER (or your own project), there are a few more steps.

Set up the developer Environment

Edit the .conaryrc file in the devuser home directory, editing the contact, name and user to match your user account on rbuilder, and adding the following section at the end of the file:

[gather-1-devel]
buildLabel gather.rpath.org@dimagi:gather-1-devel
installLabelPath gather.rpath.org@dimagi:gather-1-devel conary.rpath.com@rpl:1 ta.rpath.org@rpl:1 raa.rpath.org@rpath:rapa-3

That defines a context named “gather-1-devel” for working with package. The buildLabel indicates the branch of development. The installLabelPath indicates a search path for getting packages used by the branch.

Having defined a context for the branch, create a working directory and associate it with the context using ‘cvc context‘:

$ mkdir -p conary/gather/1-devel/
$ cd conary/gather/1-devel/
$ cvc context gather-1-devel

Modify any packages as needed

Now you can use cvc to check out any of the packages from the GATHER appliance project on rPath. The top-level package is “group-gather”, containing an appliance recipe which defines all the packages needed to build the image. You can checkout it, or any other GATHER package, with these steps:

$ cvc co group-gather
$ cd group-gather
$ ls
CONARY group-gather.recipe

If any of the packages have been changed, the group-gather package must be rebuilt and committed before a new image can be built using rBuilder online. The build will take a long time to complete as rmake fetches and assembles everything needed for a complete appliance, from the OS kernel to your packages. From within the group-gather directory, build the recipe using ‘rmake build‘:

$ rmake build group-gather.recipe

Note the job number, mentioned at the start of the build and appearing in square brackets throughout the build messages. In this example, the build is number 2. This is the number to use when committing the build. Commit the built binaries to the repository using ‘rmake commit‘:

$ rmake commit 2 -m "an example build of the group"

That can also take a long time to complete, as the binaries are uploaded to the online repository. Note the version reported at the end of the build.

Build a virtual machine image

Having committed a build that you’d like to distribute, switch to a web browser to use rBuilder Online.

Start at the GATHER project page
Click on “Manage Builds” in the left column
Click “Create a new build“
Select the kind of virtual machine you’d like to make under “Build Types”
Under “Build Contents” click through the following sequence
1. group-gather – the appliance package recipe
2. gather.rpath.org@dimagi:gather-1-devel – the branch to build
3. the version of group-gather you had committed – something like 0.1-17-1
4. the only flavor available
5. now the “Create Build” button should be active
Click “Create Build”

rBuilder Online will queue up the build request and present a page where you can monitor the progress. It’s ok to leave this page, the job will continue until it succeeds or fails. When it is done, the new image should be available on the build page of the project, under “Individual Builds”. These are only available to project members. A release must be built using the Manage Releases page to publish the build to the general community.