Archive for June 2010


In celebration of filepath

June 26th, 2010 — 4:44pm

I was reminded the other day of some of the problems that we (the software industry, and the human race in general) have in thinking of software as some sort of engineering. The term Software Engineering has a nice ring to it, but the reality is disappointing. As Jeff Atwood so nicely pointed out, the advantage of, er, real world engineering is that it has immutable laws of physics. In software the programmer is free to invent his or her own laws as the project progresses. Of course, it is possible to relate physical laws to the software world, but, not, I suspect in the way that I, or Jeff Attwood, have in mind.

The particular issue I’m looking at here is software re-use. This is where project A and project B can both use some component, even though the two projects are very different. The mutable laws of physics thing comes in when the component in question has an API that is tuned to project A and therefore doesn’t cut it for project B. This is a known problem, with a set of causes, discussed by Douglas C. Schmidt in 1999, and one key aspect of software engineering is the disciplined approach that tries to address these causes. Interestingly, Schmidt picks up on two problems that are relevant here. One is that re-usable software has to be deliberately written that way. This means that the author has to understand a wide range of use cases, and have the funding or flexibility to be allowed to do it that way. The other is that re-usable components have to be attractive. Once the component has been written others must be attracted to it. For this, Schmidt uses the concept of a “re-use magnet” and he thought that the open source development process is a good and effective way of creating re-use magnets.

There’s a whole can of worms that opens up when software engineering is discussed, but that is not why I’m writing this. What reminded me of all of this is the recent release of filepath. This is a package that makes file handling that bit easier by hiding the details of the Python os.path module. In that way alone it is a potential re-use magnet. Why I like the fact of this release, however, is that filepath used to be part of Twisted. Now, Twisted is a good and useful web frame work, but I don’t happen to use it, and I would not want to install it just for the sake of using filepath. After all, I already have lots of code that uses os.path, so I’m not going to go out of my way to dig a new component out of a bunch of software that I’m not going to use. The advantages are not enough. On the other hand, with filepath as a down-loadable component on its own the re-use magnet is no longer shielded and I can feel the attraction. What is more, now that it is out there it can grow (or shrink) to fit the wider world (as a result of more use cases and more discussion).

So, thank you to the guys at Twisted. The software re-use paradigm lives on and Chaos and Old Night have been kept a bay in one small corner of the virtual world.

Comments Off on In celebration of filepath | Development

Parsing Configuration Files Bad for your Health

June 3rd, 2010 — 1:15pm

As part of my angst-driven search for stuff that other people might have done, and published, that would save me maintaining my own code, I took a look at things to handle INI files. Oh dear. They are everywhere, and they all do different things.

Python does provide a mechanism for reading INI files. ConfigParser is the basic in-built module, and it handles a reasonably simple form of config file, with some variable substitution. It is a starting point, but there are issues:

  • Command line programs would like to be able to override entries in the config file with optional command line settings.
  • The main problem here is that optparse has a different API and does not naturally recognise the kind of block structure that INI files allow.

  • Setting the configuration file name from the command line does not play well with the idea of overriding settings from an, as yet, unread config file.
  • When building an application made up of multiple packages, each with its own configuration, the config file should be able to integrate the configs for these packages, while keeping them appropriately separate.
  • It would be nice to be able to use different text formats. Conversely, there is no reason why the API should constrain the file format.

There are probably other issues, but these keep coming up. I am relieved, slightly, to find that many people have thought about this, and there has even been discussion in the Python wiki (ConfigParserShootout). I am also depressed. The discussion came to no great conclusion, and the issues are still open.

So I keep the old code, then.

Comments Off on Parsing Configuration Files Bad for your Health | Development

Back to top