defservice

The code for this system can now be found at http://github.com/marijnh/defservice.

I'll talk about a set of macros that we use in the HTTP code for AllegroGraph

some of you have no doubt defined similar frameworks
in fact, this is about the fourth system of this type that I wrote
i feel I've hit something of a sweet spot this time

no theoretically interesting advanced macrology, except maybe some remarks about redefinition handling

this is what code using the macros looks like

it defines, one by one, HTTP services, which are tied to a certain URL and method
each service definition is based on a context, which represents a start url in whose context it is defined
it then extends this url with a path containing zero or more elements

• strings represent fixed elements, variables represent 'any string', will be bound in the body

so /users/johnny and /users/marijn both match the example handler

a parameter minilanguage makes fetching and parsing HTTP parameters easy
(name type [default])
meaningful errors returned when required parameters are omitted, or wrong format is given
(:string :boolean :integer :float :list)

service def ends with a body, a piece of code that implements the service logic, optionally outputs the response

new contexts can be defined with defcontext, and (optionally) given a body of code that is executed when they are entered
contexts without bodies are simply shorthands
multiple context definitions of the same name are possible, creating multiple paths into that context and the services below it

the structure built up by the defservice/defcontext forms is a DAG
in the picture, gray ovals represent fixed-word edges, blue ones variable words
multiple edges to fixed words can come out of a node, only a single variable one
(they alternate here, and usually do, but don't have to. for example ...)

the dispatcher splits a url on slashes, url-unescapes the parts, and works its way through this DAG

each 'URL' (roughly, the nodes in the graph we just saw) has a set of handlers, one per method (GET/POST/PUT/DELETE)
during dispatch, when the whole url has been consumed, the right method is looked up in the final node
if no handler for the method, return a 405 (method not allowed) error

each node also has an outgoing edge

• either a variable edge
• or a set of edges labeled with literal strings, for a split

when, during dispatch, a node without outgoing edge is reached, or a split with no matching label, we have a 404 error

edges may also have a 'wrap' function, which originates in a defcontext body, and is executed when traversing that edge

the library stores a table mapping context labels to nodes
a 'start context' is simply an orphan node that is used as the start state in dispatch

i'd say that the defservice and defcontext forms are easier to think about than such a DAG
which isn't to say that there aren't some gotchas

• for example, there's a special flag, per handler, that can be used to turn off the nearest wrapper function which is needed when the wrappers check for 404s (very useful) and you define a PUT method under them

an important consideration with macros that maintain a global data structure is re-evaluation

on re-evaluation, the new definition should replace the old
redefining a context should not kill all nodes under it

lookup in a split edge is linear (beats hash table for typical split sizes)
the order of the edges is definition order, so new definitions append, not push
on redefinition, the old position of the edge is reused

error checking. it is possible to define 'impossible' edges, which are both variable and split
the macroexpansion just raises an error, you have to clear your start-context if you want to really change an edge
(this would actually be a great use of a restart)

we're using this system to great effect in a server implementing some 100 different services

the context wrappers can nicely 'centralize' some code, making individual services simpler

• user access, non-existant resources, ensuring the triple store that's being accessed is open

DAGs rather than trees are actually useful

• we saw catalogs before
• user permission management and role permission management (/users/x/... /roles/y/...) form a similar join

and of course, declarative parameters

trivial to add new services, even for people not very familiar with the code

other utilities that help make service definitions succinct

a hierarchy of condition types for HTTP responses

• not found, found, method not allowed, not modified, etc

a macro that wraps a body in a handler that

• returns the proper responses for such conditions
• returns an internal server error for other errors
• allows me to set a debug flag, in which case the errors are not handled, but land me in the slime debugger instead
  • with a restart to return the error code after all

content-negotiation system that allows the handlers to just specify a value and a type
will find the correct 'writer' for the value, given the Accept header that the request specifies
or return a not acceptable error if no writer available