Archive for July 2010


URL Dispatch in Quixote

July 5th, 2010 — 8:14am

I’ve spent some time recently thinking about implementation approaches for URL dispatch. In a recent post I spoke about some requirements for URL dispatch and promised myself that I would look at available packages. I began to wonder why it seemed that Quixote didn’t fit these requirements, and I began to follow a line of thought that had me writing my own URL dispatcher based on principles derived from Quixote. Of course, in the end it may be best to use an existing package, but along the way I’ve learnt some interesting things about Quixote, and the separation of URL from code.

The route I’m going to take starts with how Quixote separates the URL structure from the code structure. This is a key aspect of RESTfull web access, and is, perhaps, the main reason why we look for URL dispatch as a separate tool.

Quixote as URL interpreter

Quixote uses a Directory object to interpret a URL. The URL is passed to a starting Directory that strips off the first (leftmost) element of the URL and finds the matching attribute in the Directory. This attribute may reference a further Directory that looks at the next element in the URL, and so on. At some point in this sequence we reach a piece of code that generates HTML and this is the response to the URL.

For example, we might have a set of Directory type classes that looks roughly like this:

from Quixote.directory import Directory
import my_code_somewhere

class WorkStuff(Directory):
    ...

    def joblist(self):
        return my_code_somewhere.joblist_page()
    ...

class PlayStuff(self):
    ...

class MainEntry(Directory):
    ...

    work = WorkStuff()
    play = PlayStuff()

There are one or two things missing, but this is pretty much what we need. Given the URL /work/joblist, plus an indication in the publisher that MainEntry is the place to start, the code that underlies Directory identifies the work attribute of MainEntry and then finds joblist in WorkStuff. At the end of this process the function joblist_page provides the response.

There are a number of points to bring out here:

  • The code you see here represents the structure of the URLs that it will interpret. If you change the URL scheme then all you do is edit the set of Directory objects. You do not have to make any changes to joblist_page
  • The names WorkStuff and PlayStuff are arbitrary and completely insignificant. In many ways it would be fine if we could do anonymous nesting, but we have to provide names to make the link. The point is that the fact that ‘work’ and ‘WorkStuff’ appear to be related is down to convenience for humans. It does not constrain how we build a URL scheme.

The next thing to consider is the use of variables in a URL. What happens if we want to write /work/23/edit?. Well, Quixote provides a catch-all method, _q_lookup, that allows us to handle this. Take a look at:

from quixote.directory import Directory
import my_code_somewhere

class ProcessJobStuff(Directory):

    def edit(self):
        return my_code_somewhere.job_edit()

    def display(self):
        return my_code_somewhere.job_display()
        ...

class WorkStuff(Directory):
    ...

    def joblist(self):
        return my_code_somewhere.joblist_page()

    def _q_lookup(self, component):
        """ component contains the URL component that got us here."""
        set_environment('application_job_name', component)
        return ProcessJobStuff()

class PlayStuff(self):
    ...

class MainEntry(Directory):
    ...

    work = WorkStuff()
    play = PlayStuff()

In this case, the URL component 23 is not recognised as a method within WorkStuff, so the component is passed to _q_lookup. Here we simply store the component into the request environment and then proceed to ProcessJobStuff to interpret the edit component.

Once again, there is no connection between the URL interpretation structure and the responding code other than the need to provide a code entry point. It would be trivial to rearrange the structure above to interpret /work/edit/23, for example, and still link to the same job_edit responder, without changing the api for job_edit. For MVC fans, we might consider the module my_code_somewhere to be the controller, and joblist, job_edit, job_display (etc.) to be views within that.

Using indirection

In the examples above, the link between the URL endpoint and the code to be executed to build the response is done using import. This is all very Pythonic, but it might cause difficulties if you want to do something dynamic without reloading the application. Not a problem. We simply capture the appropriate information from the URL and pass it all to a ‘get this code’ function. This might be a bit more difficult to read, but it does the job:

from quixote.directory import Directory
from helpers import find_controller_and_view

class ProcessJobStuff(Directory):

    def _q_lookup(self, component):
        """ component contains the URL component that got us here."""
        set_environment('application_view', component)
        return find_controller_and_view()
        ...

class WorkStuff(Directory):
    ...

    def joblist(self):
        set_environment('application_controller', 'job_list')
        return find_controller_and_view()

    def _q_lookup(self, component):
        """ component contains the URL component that got us here."""
        set_environment('application_controller', 'job_list')
        set_environment('application_job_name', component)
        return ProcessJobStuff()

class PlayStuff(self):
    ...

class MainEntry(Directory):
    ...

    work = WorkStuff()
    play = PlayStuff()

The code in find_controller_and_view extracts the names of the controller and view from the environment and does whatever it needs to do to find the right stuff to execute, importing it if necessary.

Of course, Python allows us to modify objects on the fly, so we can also do dynamic extensions to the URL scheme by posting new attributes into the appropriate Directory objects. We may not always need many levels of indirection.

Backtracking

We have to be able to distinguish between URLs that contain variable components. We might have, for example, /work/{product}/{part} that needs to be distinguished from /work/{job}. For this we need to be able to backtrack.

Quixote’s Directory object already has a mechanism we can use: the Directory returns None if it cannot find a match. This None value is propagated back through the tree in just the same way as a valid HTML response, so we can test it at key points.

The general case looks something like this:

class SomeDictionary(Dictionary):
    ...

    def _q_lookup(component):
        """ This component can be one of a number of possibilities 
            represented by a list of possible sub-Directories
        """
        for variable_name, responder in list_of_options:
            response = responder()
            if response is not None:
                return response
        return None

We need something to manage the presence of the variable in the request environment, but I’m sure you get the picture.

Roundup

There are plenty of other things that can be done with a Directory, but I have tried your patience enough. I hope I have shown that the Directory object can be used to separate URL interpretation from the application code. Of course, it is still possible to write a URL schema that represents application structure. In my examples I use /work and /play. This appears to, and may actually, represent some split in the underlying set of applications, but that is a matter of design choice and not a constraint of the mechanism.

I think there are a number of challenges in using Directory objects.

  • The URL dispatch structure looks like (indeed, is) code. It is easy to mistake this for application code. This is something I did myself initially.
  • The process of writing the code to handle any particular dispatcher is clearly more challenging than writing map_url('/work/{job}', my_code_somewhere.job_display)
  • The result is not self documenting and can be difficult to interpret.

In fact, the Directory is really an implementation tool. The programmer is being asked to interpret, or compile, some imagined scheme into code. The set of objects we see is the result of this compilation process. The code, or something like it, would still exist even if we provided a mini language and a compiler to handle it. (How about /work/{job} = job_display if method == GET?) Does this matter? Quite possibly not. After all, URL schemes do not change much, so maybe all we need is a set of patterns to follow and some useful helpers and we are all set to go.

Of course, it still doesn’t satisfy my requirement for URL generation. That will have to be dealt with another time.

Comments Off on URL Dispatch in Quixote | Development

Back to top