Last active May 16, 2017

Sinatra 2.0

I have been contemplating a major Sinatra rewrite/refactoring for a few years now. In fact, I have started work on this a couple of times, but have thrown away the resulting code every single time.

I think Sinatra is great and there's nothing fundamentally wrong with the current code base, but I also think that a major replumbing could bring some neat advantages and make sure Sinatra stays on par with the developments in Ruby land.

This document is to outline some of my thoughts on Sinatra 2.0, everything is obviously up for debate, including the overall question if there should be a Sinatra 2.0.

Overview

General ideas:

The main API stays the same, most people shouldn't notice that they upgraded to Sinatra 2.0. I do not expect 100% backwards compatibility, but only for features people don't usually rely on.

Performance for common use cases should stay the same or be improved.

A cleaner code base. I expect the code base to stay small. Certain parts for the code are currently quite bloated (compile/compile!, distpatch/invoke, set, our template logic come to mind).

It will stay simple, it will stay easy.

Leverage Ruby 2.0 features, like keyword arguments.

Add a new extension and hooks API. Sinatra currently supports hooks, but only very inconsistently and with not enough meta data. This has lead to some ugly patching, both in Sinatra and in external tools like New Relic.

Things I would like to separate out into gems:

The logic for turning string patterns into regular expressions. This has already happened and the library is already used by otherprojects and in production systems (University of Copenhagen, Travis CI). The sinatra gem would depend on this gem.

Development mode logic. I think all the Sinatra logic for development mode should move to a separate gem. The main Sinatra gem would then try to load it in development mode and display a warning if this wasn't possible. Doing this would allow the development mode to have additional dependencies, like better_errors, which then don't bloat the production setup, and should make reading through and understanding the main code base easier. The sinatra gem would not depend on this gem.

Internals

Composition

Sinatra has a fairly large number of private and public methods in Sinatra::Base. All its features are implemented in the same class. This leads to clashes, especially when someone tries to name a setting the same as one of these methods (like routes). This also means the method cache for the class gets busted quite frequently on MRI. I propose using composition (ie, a router object, a settings object, a template renderer object) and only have the public API and the application's logic on the Sinatra::Base subclass.

No more Monkey Patches

A few monkey patches have been sneaking in: Sinatra is for instance patching Rack::CommonLogger and extending certain String objects (see render). I think it shouldn't.

Route arguments

Since Ruby 1.9, it is possible to reflect on the parameters of a block or method. I think we should use this to pick the arguments passed to a block rather than just passing captures.

get '/:category/:page?'do |page:1|
# page is "1" for "/blog/"# page is "2" for "/blog/2"end

This could also include query parameters:

get '/search'do |q:|
# q is "example" for "/search?q=example"# status is set to 400 if q is missingend

Extension and Hook API

I'm not 100% sure what this should look like. I few things that come to mind:

It should be possible to track metrics in production (via things like New Relic, Librato, Skylight, etc) without having to patch Sinatra::Base.

There should be a sane reflection API (list all routes, etc). People hack around with Sinatra's internal data structure in 1.x, which makes it hard to change it.

A good way to have a simple yet powerful extension API would probably be to turn large parts of Sinatra itself into extensions internally (that will then be always used), examples that come to mind would be templates and logging.

Patterns

This is more or less the only part that's already done.

All the Sinatra 1.x syntax will be supported (captures, splats, optional parts).

Performance for route matching will be equal or better compared to Sinatra 1.x.

set :pattern, capture: {
ext:%w[png jpg html txt], # have :ext caputre png, jpg, html or txt in any patternid:/\d+/# have :id capture only digits in any pattern
}
get '/:id'do# will match '/42', but not '/foo'end
get '/:slug(.:ext)?'do# slug will be 'foo' for '/foo.png'# slug will be 'foo.bar' for '/foo.bar'# slug will be 'foo.bar' for '/foo.bar.html'
params[:slug]
end

This feature is already in use at Travis CI.

Semi-greedy matching

In Sinatra up to 1.3.x, captures are greedy and splats are none greedy. In 1.4 some hacks were added to implement semi-greedy behavior for certain special cases, though it's really more of a hack and doesn't work for a lot of cases.

Imagine the pattern :name.?:ext? (or :name(.:ext)? with the new syntax) and the input string foo.bar.html.

With greedy matching, this would be parsed as {"name" => "foo.bar.html"}, with non-greedy matching this would be parsed as {"name" => "foo", "ext" => "bar.html"} and with semi-greedy matching, this will be matched to {"name" => "foo.bar", "ext" => "html"}.

Moreover, this behavior is configurable:

get '/:slug(.:ext)?', pattern: { greedy:false } do# slug will be 'foo' for '/foo.png'# slug will be 'foo' for '/foo.bar'# slug will be 'foo' for '/foo.bar.html'
params[:slug]
end

Development Tools

Sinatra should try loading the dev tools extension when started in development mode, and print a warning about it not being loaded if that fails (possibly with a link to docs).

Development tools should include (potentially using other dependencies):

A reloader, preferably an out of process reloader, like rerun/shotgun.

Ways to easily trace whats going on during a request (dev mode logger).

Things not clear to me yet

Logging

This seems to be something users tend to struggle with, maybe Sinatra should offer a better logging solution, not relying on Rack?

Exception handling

Again, something we get issues about from time to time. The logic is quite complex right now, yet it's still very easy to accidentally leak file handlers when capturing exceptions.

Rack

Rack 1.x is dying. It's also a terrible tool for anyone but people writing Rack apps directly. There has been some experimentation by Aaron Patterson for a successor, which might be used by Rails.

Rack's main strength is its support: It is the defacto standard in Ruby land.

It might be possible to keep all the Rack specific logic in one place, so it can be replaced with another implementation. However, this might also overcomplicate things.

Potential Features

Below are some features that I have considered, but I'm not 100% sure yet if they should be part of Sinatra (they could also become their own extension). I imagine this features to be quite simple to implement.

Decorator Style

This one is so easy to implement, that I implemented it once by accident.

Right now, we collect conditions in an array sitting there and waiting for a route to be defined:

get '/', agent:/Firefox/doend# equivilent to
agent /Firefox/
get '/'doend

We also generate a method from the given block (the see generate_method and compile). I experimented with treating the get part as a condition, and using a method_added hook, mostly to have a common logic for all the different types of routing conditions we have.

This came with the decoration style definition shown above.

Link generation

The pattern logic for Sinatra 2.0 actually parses the pattern string, instead of just gsubing it in hopes it becomes a valid regexp. This also allows expanding a pattern string with a given params hash to a full URL. Link generation would be the main use case for this. I'm not sure how to best expose this as a feature in Sinatra, or if we even want to, but it might allow for a few neat features:

get '/hello/:name'do |name|
# will redirect /hello/Frank to /hello/Sinatra
redirect to(name:'Sinatra') if name =="Frank""Hello #{name}!"end

On the other hand, I guess, link generation makes more sense with named routes.

Renderers

Right now, Sinatra has a hardcoded set of supported return values (ie, a String will become the response body, an Integer the HTTP status). It is possible to add support for additional response types by overriding the body method (ie, serialize a Hash as JSON). However, this breaks in a lot of edge cases and is not officially supported.

I could imagine having custom renderers, which could use the same logic we'd use for handling exceptions, routes, etc.

This comment has been minimized.

It's great to hear that Sinatra 2.0 may see the light :). Thanks for putting this together @rkh.
I love the modular approach you're looking at and I believe that we may have some good collaboration to be done with the Padrino team. Here are two related issues we are working on and that might be relevant at some point: Major routing syntax cleanup and Release 1.0.0.

This comment has been minimized.

Very exciting! There should definitely be a 2.0. A useful things to me would be named arguments. Named arguments would produce very clean application code. +1 on Link generators. Would love to help on this.