RejectedSoftware Forums

Sign up

Improvements to the Router API

Hi all!

I've been thinking about routing lately and I think that the
system can be made more generic. There can be various way of
matching an URL like the current way, regexps etc. We should
start by providing the very basic functionality - API that
expects a predicate:

 enum HTTP {
     GET,
     POST,
     UPDATE,
     DELETE
 }

 class Router {
     ...
     Router match(HTTP reqType, bool delegate(string) 

predicate, HttpRequestHandler);

 }
 auto router = new Router;
 router.match(HTTP.GET, (url) => url == "/", &handleIndex);

Having the most generic API we can provide a more specific
implementations like the current or regex:

 class Router {
     ...
     Router match(HTTP reqType, string pattern, 

HttpRequestHandler);

 Router match(HTTP reqType, Regex!char pattern, 

HttpRequestHandler);

 }

Also, using enumeration for request type rather than dedicated
methods get(), post() etc is more flexible. Does it look good and
does anyone have any other thoughts? If yes, I will then go ahead
with it and submit a pull request.

Re: Improvements to the Router API

Am 12.09.2012 16:10, schrieb Eldar Insafutdinov:

Hi all!

I've been thinking about routing lately and I think that the system can
be made more generic. There can be various way of matching an URL like
the current way, regexps etc. We should start by providing the very
basic functionality - API that expects a predicate:

enum HTTP {
    GET,
    POST,
    UPDATE,
    DELETE
}

class Router {
    ...
    Router match(HTTP reqType, bool delegate(string) predicate,

HttpRequestHandler);

}
auto router = new Router;
router.match(HTTP.GET, (url) => url == "/", &handleIndex);

Having the most generic API we can provide a more specific
implementations like the current or regex:

class Router {
    ...
    Router match(HTTP reqType, string pattern, HttpRequestHandler);
    Router match(HTTP reqType, Regex!char pattern, HttpRequestHandler);
}

Also, using enumeration for request type rather than dedicated methods
get(), post() etc is more flexible. Does it look good and does anyone
have any other thoughts? If yes, I will then go ahead with it and submit
a pull request.

I was thinking about implementing a state machine based matching of
routes at some point to improve performance with many routes. That would
collide a bit with opaque callbacks. But maybe the performance will be
sufficient for almost any application anyway... I think it would be good
to make a quick benchmark to see how the number of routes affects the
total request time before committing to this approach.

Btw. router.match() is the same as router.addRoute() is now, but 'match'
seems like a nice naming alternative.

Re: Improvements to the Router API

On Wednesday, 12 September 2012 at 17:01:19 UTC, Sönke Ludwig
wrote:

Am 12.09.2012 16:10, schrieb Eldar Insafutdinov:

Hi all!

I've been thinking about routing lately and I think that the
system can
be made more generic. There can be various way of matching an
URL like
the current way, regexps etc. We should start by providing the
very
basic functionality - API that expects a predicate:

enum HTTP {
    GET,
    POST,
    UPDATE,
    DELETE
}

class Router {
    ...
    Router match(HTTP reqType, bool delegate(string) 

predicate,
HttpRequestHandler);

}
auto router = new Router;
router.match(HTTP.GET, (url) => url == "/", &handleIndex);

Having the most generic API we can provide a more specific
implementations like the current or regex:

class Router {
    ...
    Router match(HTTP reqType, string pattern, 

HttpRequestHandler);

Router match(HTTP reqType, Regex!char pattern, 

HttpRequestHandler);

}

Also, using enumeration for request type rather than dedicated
methods
get(), post() etc is more flexible. Does it look good and does
anyone
have any other thoughts? If yes, I will then go ahead with it
and submit
a pull request.

I was thinking about implementing a state machine based
matching of
routes at some point to improve performance with many routes.
That would
collide a bit with opaque callbacks. But maybe the performance
will be
sufficient for almost any application anyway... I think it
would be good
to make a quick benchmark to see how the number of routes
affects the
total request time before committing to this approach.

How many paths are you thinking here? tens or hundreds?

Btw. router.match() is the same as router.addRoute() is now,
but 'match'
seems like a nice naming alternative.

Yeah, I already noticed that that there is addRoute, so I'll
rename it.

Re: Improvements to the Router API

Am 12.09.2012 20:20, schrieb Eldar Insafutdinov:

How many paths are you thinking here? tens or hundreds?

I would say something in the range of one hundred is realistic for a
moderately complex web app (I have one half finished with about 50
routes). So if say 200 are not affecting performance, the state machine
approach is probably not needed.

Btw. router.match() is the same as router.addRoute() is now, but 'match'
seems like a nice naming alternative.

Yeah, I already noticed that that there is addRoute, so I'll rename it.

Ok

Re: Improvements to the Router API

On Wednesday, 12 September 2012 at 19:10:50 UTC, Sönke Ludwig
wrote:

Am 12.09.2012 20:20, schrieb Eldar Insafutdinov:

How many paths are you thinking here? tens or hundreds?

I would say something in the range of one hundred is realistic
for a
moderately complex web app (I have one half finished with about
50
routes). So if say 200 are not affecting performance, the state
machine
approach is probably not needed.

Btw. router.match() is the same as router.addRoute() is now,
but 'match'
seems like a nice naming alternative.

Yeah, I already noticed that that there is addRoute, so I'll
rename it.

Ok

I don't quite understand about how exactly to use state machines
here, but I just thought - if there were support for Regexps how
would that fit in?

Also as for 100-200 routs - is that realistic to use together
with the Router class? Because to me if you have this many routs,
you might as well write your own handler that implements
IHttpServerRequestHandler interface and does some custom logic?

Re: Improvements to the Router API

On Wed, 12 Sep 2012 21:36:52 +0200
"Eldar Insafutdinov" e.insafutdinov@gmail.com wrote:

I don't quite understand about how exactly to use state machines
here, but I just thought - if there were support for Regexps how
would that fit in?

AIUI:

It would essentially be a lexer.

Vibe.d would collect all the regexes. Then, instead of the usual regex
approach of building one NFA/DFA for each regex (with one "Accept"
state each), it would essentially combine them like this:

(regex1)|(regex2)|(regex3)|(etc.)

And each of those parts would have their own separate "accept" state.

Ie, a lexer.

Re: Improvements to the Router API

On Wednesday, 12 September 2012 at 19:50:22 UTC, Nick Sabalausky
wrote:

On Wed, 12 Sep 2012 21:36:52 +0200
"Eldar Insafutdinov" e.insafutdinov@gmail.com wrote:

I don't quite understand about how exactly to use state
machines here, but I just thought - if there were support for
Regexps how would that fit in?

AIUI:

It would essentially be a lexer.

Vibe.d would collect all the regexes. Then, instead of the
usual regex
approach of building one NFA/DFA for each regex (with one
"Accept"
state each), it would essentially combine them like this:

(regex1)|(regex2)|(regex3)|(etc.)

And each of those parts would have their own separate
"accept" state.

Ie, a lexer.

The main issue that got me started on this thing is that
currently wild-cards are not captured, so something like this
wouldn't work:

 router.get("/assets/*", serveStaticFiles("./public/"));

Currently when the matcher sees wildcard and it just returns a
match. That brings us to the question - do we really want to
support them properly? Sinatra has the following:

 get '/say/*/to/*' do
   # matches /say/hello/to/world
   params[:splat] # => ["hello", "world"]

To do that the parser has to be a lot more sofisticated than it
currently is. The easiest option would be of course just to match
and capture the first * and be done with it.

Re: Improvements to the Router API

On Wednesday, 12 September 2012 at 19:10:50 UTC, Sönke Ludwig
wrote:

Am 12.09.2012 20:20, schrieb Eldar Insafutdinov:

How many paths are you thinking here? tens or hundreds?

I would say something in the range of one hundred is realistic
for a
moderately complex web app (I have one half finished with about
50
routes). So if say 200 are not affecting performance, the state
machine
approach is probably not needed.

Ok, I just did some testing and 200 routes indeed degrade
performance by 10-20%. This needs to be thought through little
better then..

As for static files serving, I figured I can use
HttpFileServerSettings for that. So that particular problem is
solved. It'd be good to reflect that in the documentation.

Re: Improvements to the Router API

Am 12.09.2012 22:25, schrieb Eldar Insafutdinov:

On Wednesday, 12 September 2012 at 19:50:22 UTC, Nick Sabalausky wrote:

On Wed, 12 Sep 2012 21:36:52 +0200
"Eldar Insafutdinov" e.insafutdinov@gmail.com wrote:

I don't quite understand about how exactly to use state machines
here, but I just thought - if there were support for Regexps how
would that fit in?

AIUI:

It would essentially be a lexer.

Vibe.d would collect all the regexes. Then, instead of the usual regex
approach of building one NFA/DFA for each regex (with one "Accept"
state each), it would essentially combine them like this:

(regex1)|(regex2)|(regex3)|(etc.)

And each of those parts would have their own separate "accept" state.

Ie, a lexer.

The main issue that got me started on this thing is that currently
wild-cards are not captured, so something like this wouldn't work:

router.get("/assets/*", serveStaticFiles("./public/"));

(I know you figured this out already... just posting here to remember
adding it to the docs later)

The following ought to work:

auto settings = new HttpFileServerSettings;
settings.serverPathPrefix = "/assets/";
router.get("/assets/*", serveStaticFiles("./public/", settings));

Currently when the matcher sees wildcard and it just returns a match.
That brings us to the question - do we really want to support them
properly? Sinatra has the following:

get '/say/*/to/*' do
  # matches /say/hello/to/world
  params[:splat] # => ["hello", "world"]

To do that the parser has to be a lot more sofisticated than it
currently is. The easiest option would be of course just to match and
capture the first * and be done with it.

Personally, all I've ever needed was '*' at the end for matching routes
for serveStaticFiles() and :var-style routes for anything else. That's
obviously also a reason why the wildcard support was never improved
beyond that.

But even if it stays at its current limited functionality, I would
improve two things: throw an error if a route has a '' that is not at
the end of the string, and put the contents matched by the '
' into
something like params["wildcard"].

Re: Improvements to the Router API

Am 12.09.2012 21:36, schrieb Eldar Insafutdinov:

On Wednesday, 12 September 2012 at 19:10:50 UTC, Sönke Ludwig wrote:

Am 12.09.2012 20:20, schrieb Eldar Insafutdinov:

How many paths are you thinking here? tens or hundreds?

I would say something in the range of one hundred is realistic for a
moderately complex web app (I have one half finished with about 50
routes). So if say 200 are not affecting performance, the state machine
approach is probably not needed.

Btw. router.match() is the same as router.addRoute() is now, but
'match'
seems like a nice naming alternative.

Yeah, I already noticed that that there is addRoute, so I'll rename it.

Ok

I don't quite understand about how exactly to use state machines here,
but I just thought - if there were support for Regexps how would that
fit in?

I've thought about the state machine approach and it should actually mix
fine with arbitrary delegates, just that those are of coursed scanned
linearly.

The basic idea is that there is a decision graph where each node node
matches a single character or a range of characters. 'Children'
represent matches for the next character and any node contains a list of
matching routes. The input URL is then piped through the graph and every
matching route on the way is marked. At the end, the marked routes are
executed in sequence. Routes with arbitrary callbacks would just always
be executed in the last step in addition to the marked routes.

Maybe there will also need to be some additional node annotations for
capturing the wildcard/:variable contents but basically this is how it
should work.

All in all, since they don't really contradict the state machine
approach, arbitrary match delegates should be fine actually. Regexes
could also be inserted into the state machine graph, but that's
something I would rather not want to do because of it's complexity. So
unless someone is adventurous, they would probably just be implemented
as match delegates.

Also as for 100-200 routs - is that realistic to use together with the
Router class? Because to me if you have this many routs, you might as
well write your own handler that implements IHttpServerRequestHandler
interface and does some custom logic?

A possible pattern for pluggable modules is that the module registers
itself in a passed in UrlRouter (e.g. vibelog is doing this). If you
then have a few of these modules, it's easy to get to those numbers. Of
course it is also possible to nest multiple routers to get closer from
O(n) to O(log(n)):

UrlRouter blogroutes;
blogroutes.get("/blog/", ...);
blogroutes.get("/blog/:postid", ...);
// ...

UrlRouter mainroutes;
mainroutes.get("/", ...);
mainroutes.get("/blog/*", blogroutes);
// ...