Tag: distutils2

Package distribution using Distutils2 concepts.

May 12th, 2011 — 11:04pm

I have been developing a product I want to publish as open source (It’s called Lokai; more of this later), and, being keen, I popped over to distutils2 (aka packaging) to see what’s going on, with the intention of using it if appropriate. I’m not claiming to be following the leading edge here, it just seemed that the original distutils is seen as needing replacement, and setuptools good but flawed in some way (package management, for instance, see PEP 376). Distutils2 is also going to be the official packaging tool for Python 3, so it has to be worth the look.

We are warned, pretty much at every turn, that distutils2 is not yet ready for production. Actually, there’s a lot there that is good, but it fell over (version 1.0a3) on package data. The developers already know about this; it happens to be the example that I tripped up over.

Let me explain. Lokai is an application, rather than a library, and it is web based. It is quite sizable (for me, anyway) and I have some things I need from a distribution tool:

  • I don’t want to have to spend time maintaining a package list. There are 32 packages at the last count, and, while this is not overwhelming, I prefer not to have to fiddle with this every time I add something or restructure.
  • I use Mercurial and I want to be able to link the distribution to a tag in the repository. If this can be done with some automation, so much the better.
  • Lokai is a web application, so there are CSS files, templates and other bits and pieces that are not Python but are essential. I need those to be carried through to the distribution, and the ultimate installation, without undue hassle.

None of these things is particularly difficult, but they emphasize a fact that distutils2 seems to recognize well: there is a real distinction between actions taken by the developer to create a distribution and actions taken by a user when installing from a distribution. This comes about simply because the developer has access to information about the package (from a repository, for example) that is not available to the user; the user relies solely on whatever information the developer has provided. For distutils2 this distinction seems to center on the setup.cfg file. This file drives the whole process and there is no need for a setup.py file. The developer can use all the information available to generate a setup.cfg file, and that file is then all that the user needs to build and install the package. If you want to get more complex than this, of course you can, but this seems to me to cover a wide range of cases.

So, thinking ahead, I decided to write a program to generate a setup2.cfg file. I’m quite happy to help things along by putting files in reasonable places, so the bulk of this program comes down to this:

  • I use distutils2.core.find_packages to find all my packages from the top level. It’s easy enough to write such a thing, or copy it from elsewhere, but this does the job.
  • I found hgtools, which provides a way of getting a version number from tags in the Mercurial repository. I had to copy and modify it, because the implied version number structure is based on StrictVersion from distutils. My version uses distutils2.version.NormalizedVersion, which gets me a step forward in future compatibility.
  • The program contains fixed, or slow moving, values, such as package metadata, embedded in the source.
  • The output is usable by distutils2. (Or, at least, the version I was playing with. Expect changes.)

Of course, I’m not using distutils2, so the next step is to take the output from my prepare.py program (setup2.cfg) and feed it into a setup.py. The setup.py is relatively simple – it reads the prepared configuration file and converts data and data names as appropriate for a call to distutils.core.setup. This setup.py, plus setup2.cfg are included in the distribution.

I have, I know, inserted an extra step into the process, but I can automate my way around that if I want to. What I do have, however, is a set of tools to build the distribution that stay with me. I don’t have to include all the package searching and version calculation stuff in the distribution. I get to keep these in my repository, of course, but the user of my distribution does not need them, and, with this process, never sees them. I think that is ultimately simpler for all concerned. I’m also a step towards using distutils2 when the time comes.

Comments Off on Package distribution using Distutils2 concepts. | Development

Back to top