Tag: fabric


Deploying packages using Fabric

May 15th, 2011 — 1:58pm

I’ve started using fabric to help manage my servers. This is a great tool that allows me to build a series of commands in Python and then run them on one or more server, all from my own desk. It all boils down to some handy layering on top of an ssh library. This is early days, for me, as yet, and I don’t actually have a lot of servers (well, two at the last count), but I do have some things I do quite often that require a number of steps, so a tool like fabric is a potential time saver.

My main use is for deployment of updated software. This happens in bursts, as and when the focus falls on one package or another, and it is helpful to have a pre-packaged set of commands that I don’t have to remember. Roughly, the things I need to do are:

  1. Identify the latest version of the package and find the distribution file in the development environment.
  2. Copy the distribution file over to the target system.
  3. Install the distribution
  4. Stop and restart the server.

Step 1 is where fabric starts to be useful. I can identify the version, find the target file and do any checking I need to do all using Python. I’m not afraid of bash, but I am happier in Python.

Step 2 is dead easy. fabric has a copy command, so no further though is required.

Step 3 is where life gets more interesting, as this happens on the remote server. I use the fabric run command to put together a series of commands that can be executed from an ssh command. The trick here is to make sure that the command has all the environment that it needs. I need two things: one is to be in the correct directory, and the other is to activate the correct virtualenv environment.

My servers all run bash, so I can string commands together using ‘logical and’, &&, thus:

command_1 && command_2 && command_3

The ssh command is executed in a minimal environment, so your .bashrc is not going to be run. That means that useful stuff like virtualenvwrapper is not necessarily going to work. In my case I have to explicitly activate the required environment:

cd my_package_distribution_directory && \
source ~/.virtualenvs/my_target_environment/activate && \
python setup.py install

fabric can automatically add the cd some_directory if you want. I haven’t found a way of adding any general prefix. No matter, because I can build the prefix easily and just have it lying around in the code, so I don’t need more in my bash-centric world.

Step 4 introduces another learning point. When restarting the server I need the server to detach from the terminal and run in the background. Normally, I’d simply enter the command terminated with &:

lk_fcgi_server_up.py &

This doesn’t work too well. Doing this remotely using ssh seems to leave ssh just hanging. Using fabric is no better, as that tool appears to terminate forcibly, thus killing the background process. However, bash (version 4.0) comes to the rescue with the coproc command. This runs the target command as a coprocess, complete with connected pipes for I/O. I don’t need to look at standard output from the server, so the simplest invocation seems to do the job:

previous_commands && coproc lk_fcgi_server_up.py

And that’s it for my deploy tool. I can use standard Python for command line options, environment validation and whatever else, while fabric manages the remote connections. One other thing I have learnt, however. If the remote command sequence is at all complicated, it might be better to have a command script sitting on the remote server that handles the whole thing from a single parameterised remote invocation. Of course, the remote script has to be maintained, and deployed on all servers …

Update: Just for clarity:

  • Using c1 && c2 && c3 & is not that useful anyway because it puts the entire line into the background and not just the c3 part.
  • When using coproc you need to add pty=False to the fabric run call.

Comments Off on Deploying packages using Fabric | Development

Back to top