2012-08-23 22:10:09

In the most recent round of updates, Larz Conwell has done a shit-ton of amazing work on templates: making the templating system play nicely with various formats: EJS, Mustache, and Jade. It can now happily figure out what sort of template from the file-extension, and you can even mix and match different formats together in layouts/partials.

He's also implemented Rails-like view-helpers like urlFor and linkTo, making your views simpler and more DRY.

Last but not least, he's vastly improved the generators to support full scaffold-generation, not just minimal resources.

What else is changing?

There's a lot of work happening on Geddy -- mostly breaking the project apart to make it more modular.

Utilities

The large set of utility functions (most of which originated in the now-ancient Fleegix.js library) have been pulled out into their own library, Utilities.

This classic collection of JavaScript utilities is of course available on NPM. String-manipulation, datetime stuff, mixin, filesystem utilities -- there's a ton of useful shit in there.

Model

Even more importantly, the Geddy model code now resides in a separate project, simply called Model, which is a general-purpose, datastore-agnostic ORM library in JavaScript.

I built Postgres support first, but back-porting Geddy's existing support for MongoDB, and building a Riak adapter, are next in line. Model is also installable via NPM, but documentation hasn't moved over from the Geddy project, so it's still rather DIY.

When is this happening?

The libraries are usable as-is, but the Geddy work for this lives in a branch -- but as you see, it's solid enough to build real applications with, so it will be landing soon.
I'm pretty excited to have a solid framework for building Web apps like this in JavaScript.

It's finally far enough along I can replace my old blog application, which has been running on the Merb-based PmpknPi blog-engine for ages (anybody remember Merb?), with a very simple Geddy app. I've actually waited a really long time for this.

2012-02-03 13:24:00

At a recent meetup, a guy in the community (a guy whom I generally have a lot of respect for) said one of the stupidest things I've heard in a long time (and that's saying something, given the silly shit I hear coming out of my own mouth day to day).

We were talking about an open-source project I started working on quite a while back, and why development on it kind of stalled for a while. He was totally convinced that the reason it had stalled was because I'm an older guy with a family. I told him no, it was because my work on Yammer takes up most of my time. We're pretty busy these days at Yammer.

Now, this guy is at a startup, so I would assume he'd totally get this, but bizarrely, he acted as if he hadn't even heard me (we were at a noisy bar; I guess it's possible somehow he actually didn't hear me), and continued to hold forth, saying, with a big grin, "that's why I'd never hire an older guy with a family."

Yeah, sure, I'm rolling my eyes at this because I'm one of those older dudes with a family. But you can see this same silliness in other places: "I'd never hire a woman, because she might get pregnant and have to quit." "I'd never hire a person without a comp-sci degree from an elite univerity and a high GPA, because we want smart people." "I'd never hire a person from Uzbekistan, because fluctuations in their currency's exchange-rate might distract them from their work." "I'd never hire a person who's lactose-intolerant, because we'd have to keep that special milk in the office." None of these make any fucking sense either.

As wrong as it is, I guess I can understand where the idea comes from -- it seems obvious that younger folks have a lot more time to hack. That is, unless they're spending their free time rock-climbing, or snow-boarding, hanging out at bars, or even, say, recovering from hangovers (okay, I still do this last one from time to time, too).

A previous startup where I worked was populated overwhelmingly with younger dudes who prided themselves on their ability to code for days and days at a time. They produced prodigious amounts of code -- and it was some of the most appallingly bad code I've ever seen. Piles and piles of it.

It's possible that older guys with families work fewer hours (although that hasn't been my experience in startups so far), but it's also highly likely that with their experience they spend more time writing the right code. Just sayin'.

2010-07-18 01:28:00

Some of the work I'm doing at Yammer involves writing JavaScript code that runs in different environments. We're using geddy-model code in the browser, as well as with TheRubyRacer (which embeds the V8 JavaScript interpreter in a Ruby process).

This code is part of the Geddy web framework I have been building for Node.js -- so this means three different environments, and and four different JavaScript interpreters, that this code has to work within.

Multi-environment testing? Dual-sided? X-env?

I've been talking to people about ways to solve this -- in particular, Anders Conbere, who is working with similar issues at his job with Estately.

Testing multi-environment JavaScript code is an annoying problem. I looked around, I really didn't find any good solutions.

(I also didn't find anything nice and snappy to call JS code that has to run on both client and server. What's the "Ajax" of that? "Dual-sided"? "Multi-environment"? "JS-everywhere"?)

Meet Logan

I sat down to see what would be required to run the same tests in all these different environments, and it turned out to be pretty reasonable to build something that can run both brower- and server-side tests (thanks, Node.js!).

Despite having built Windmill's complex JavaScript API (or perhaps because of it), I've come around to Mikeal's way of thinking on testing: I want something minimal, without a lot of frameworky baggage -- something that just sets up some conditions and does some asserts.

What I've ended up with is Logan, a very small test runner that lets you run your tests in browsers, in Node.js, and in embedded V8 with TheRubyRacer.

The name is probably kind of obvious to anybody who's familiar with old sci-fi movies -- Logan is a runner), of course.

Logan requires Node.js, and of course if you want to test out tests in TheRubyRacer, you need to have it installed.

Sharing code between sides

For Logan's testing purposes, "server-side" means in Node.js, which uses CommonJS modules. "Client-side" includes the browsers, and TheRubyRacer, where there is no facility for loading code modules from within the runtime. Logan uses plain requires for the server-side, and eval/script-append in the clients.

I should take a second here to call out very specifically one of the requirements is synchronous loading of dependencies. (This is why I went the route of source-code transformation.)

Once you get past some initial hackery, it's not that bad loading code in these different enviroments. (After you've done enough client-side JavaScript, I suspect your threshold for hackery gets a little higher.)

To make this stuff work, you do have to follow some specific rules in both app-code, and testing code. (The constraints for the app code are needed if you plan to run your code on both client and server at all, not specifically for running Logan tests.)

1. Use namespaces in your modules

Use a namespace object inside the module file -- you won't have module scope when you load it in the browser. Name it the same name as the variable you'll be setting it to when you load it via CommonJS require.

3. Set require result to namespace name

Use the same name as the namespace you created in the module for the variable name of the require result.

4. Create top-level namespaces non-destructively.

In your tests, if you're creating top-level namespaces to hang your required objects on, use something to create them that checks for their existence before creating.

Logan includes a utility function, logan.namespace, to help with this. In your own app's code, you'll either have to use a utility function for doing this, or simply check with a typeof check for undefined.

Details

The reason for these special requirements is the difference between the way that code loads on the client and the way it loads via CommonJS.

In the client environment (including TheRubyRacer), there is no module-level scope, and the mechanism for loading code doesn't return any value, so namespace creation has to happen inside the module file, and has to be careful not to rewrite anything other modules might have done.

Logan makes this work with CommonJS require syntax in your tests by doing source-code transformation -- it uses the require statements to know what code to load, and rewrites the statements so they don't stomp on the results of the previous eval operation.

If you're at all interested in the icky rewriting tricks necessary, have a look at the source for the browser runner.

The process for running tests in TheRubyRacer is pretty weird too -- Node shells out to a Ruby script that then turns right back around and embeds V8. It's a crazy world we're living in.

What's next

Logan is very new. I'm sure there are better ways to deal with the problems of X-env testing. (How about that? Yeah, "X-env"? It has the letter "X" in it like Ajax, right?) At least I hope there are better ways. I'm very interested in getting feedback on how to make improvements.

2010-04-27 01:53:00

I've been settling into my new job at Yammer, getting to know the guys on the team, and enjoying the insanely delicious food. Yes, we have a chef that comes in; it's pretty ridiculous.

By happy accident, my team at Yammer is using the work I've done with Geddy on models and validations -- but in the browser.

It's kind of mind-bending to see code written for Node.js running in IE6, but that's a real-world example of the pure awesome we get with server-side JavaScript. No more writing the exact same input-validation in two different languages.

Geddy has an easy, intuitive way of defining models, with
a full-featured set of data validations. The syntax is very
similar to models in Ruby's ActiveRecord or DataMapper, or Python's SQLAlchemy.

As I mentioned, the model module works great both in Node, and in the browser, so it's very easy to share model and input validation code in your app between client and server.

Data-validation happens on the call to create, and any
validation errors show up inside an errors property on
the instance, keyed by field name. Instances have a valid
method that returns a Boolean indicating whether the instance
is valid.

I've been writing a bunch of tests so you can see some other examples.

Really cool seeing code that runs equally well on the server and in the browser. A credible server-side JavaScript plaform like Node really opens up a ton of new possibilities. Looking forward to seeing where it all goes.

2010-03-25 03:20:00

Geddy is a small Web-app development framework for Node.js. It uses a router syntax similar to Rails or Merb, and now has tools for quickly creating RESTful, resource-based routes as well.

Building and installing Geddy

Prerequisites: An up-to-date version of Node.js. I'm developing Geddy on Linux (and I'm lazy), so I'm not sure yet what adjustments might be needed for it to work smoothly on OS X. I hope one of you Mac guys can tell me what's needed.

Restart Geddy, and you'll see the new route working. Hit your new route -- for example, http://localhost:8000/snow_dogs.json, and you should see something like this:

{"method":"index","params":{"extension":"json"}}

Depending on how your browser is set up to deal with the JSON MIME type, it may want to open the response in a separate application.

Resource controllers

Take a look at the new file created, app/controllers/snow_dogs.js to see what actions a resource-based controller has. The actions are methods called on the controller, and get passed a copy of the parsed parameters from the URL and query string.

Okay, so REST isn't quite the same as CRUD, but Geddy's resource-based routes do take the expedient course of creating url/request-method mappings for easy CRUD operations like this:

GET /snow_dogs
(SnowDogs controller, index action)

GET /snow_dogs/add
(SnowDogs controller, add action, for any new-resource template -- "new" is not usable as a JavaScript action name)

POST /snow_dogs
(SnowDogs controller, create action)

GET /snow_dogs/:id
(SnowDogs controller, show action)

PUT /snow_dogs/:id
(SnowDogs controller, update action)

DELETE /snow_dogs/:id
(SnowDogs controller, remove action)

Content-negotiation

Geddy has some built-in ability to perform content-negotiation based on the abilities of the client, and the requested filename-extension. Right now, this feature is very bare-bones, supporting only JSON and plaintext, but can be easily extended to support XML, Atompub, or whatever.

If you have a JSON-serializable JavaScript object you want to return in JSON format, all you have to do is set up a resource with geddy-gen resource, and in the appropriate controller action, pass your JavaScript object to the respond method.

Geddy will automatically serialize item to JSON, and set the content-type appropriately -- assuming the client can accept JSON.

Eventually it should be possible to register handlers for new content-types on a per-controller, or per-model basis. (Doing this by way of defining a 'to_[format]' method on the objects to be returned isn't a workable solution in JavaScript, when you're likely to be returning built-in types.)

If you want to use a specific format, you can pass it as the second argument to the respond call -- even if it's not one of the listed formats in respondsWith.

2010-03-16 00:46:00

So I've posted previously about the embarrassment of riches that we have with Node.js modules, specifically Web frameworks.

I count no less than 14 frameworks and micro-frameworks for building Web-apps with Node right now on the list of Node modules.

I had actually hoped to use one of these to build a small Node app, but as it turns out, many of these are either half-finished, or are broken with current versions of Node. Pretty understandable, given how quickly Node is changing.

Express looks good, but I'm a little more interested in something that will let me build complex apps -- and the Sinatra-style routing, with the verb and everything kind of inline there, seems a little verbose. I prefer the simpler DSL of Rails-/Merb-style routes. To me it's just more expressive, flexible, and powerful.

So I started building a basic router, looking at a lot of the ideas in Bomber.js, and remembering back on how simple and hackable the Merb framework was to use.

The result is Geddy. It's a small Web-app development framework that's still very embryonic -- but it does routing and invoking controller actions, and serves static files. I'm having a blast hacking on it. It's crazy fun doing all this server-side code in JavaScript.

The controllers are all required and instantiated based on convention, but I'm not going all meshugah with the Railsy pluralization rules.

The controller files should be all lowercase names, with underscores separating words. The constructor function in the file should be a camel-cased, initial-capital-letter version of the same name.

Some examples:

File

Constructor

app/controllers/snow_dogs.js

SnowDogs

app/controllers/priests_of_syrinx.js

PriestsOfSyrinx

The action is a method then called on the controller instance, and passed a params object that contains any params pulled off by matching the pattern in the router -- and any query-string params.

So for example, in the case of the 'lerxstizer' example above, if you hit the following URL:

/users/alex/lerxstizer/2112?code=YYZ

The params object will look like this:

{
userid: 'alex',
lerxstid: '2112',
code: 'YYZ'
}

Static files

It's impossible to develop a Web application without serving the static assets needed. I took a look at how Paperboy and other frameworks did it, and added static-file support for Geddy.

In development-mode, if an URL matches no routes, Geddy will serve up matching filenames in the public/ directory rather than responding with a 404. Obviously this directory path needs to be configurable at some point.

Stuff to do

There's a long list of stuff to hack in:

Resource-based routes

Content negotiation

Templating support

Better error handling

Controller-generator script

Kiwi package

I also need to separate out the framework code from what will eventually be generated app code. Lots of stuff to work on, and hopefully lots of opportunity to get to know Node better.

About

This is the blog for Matthew Eernisse. I currently work at Yammer as a developer, working
mostly with JavaScript. All opinions expressed here are my own, not my
employer's.