Distributable Tasks and Workflows

I ran into trouble when I tried turning rake into something it is not. Rake is a build program designed for build-like tasks; rake is not a platform for general-purpose task libraries. Given it’s design goals, rake very sensibly does not facilitate extensive documentation (who needs it to compile something), inputs (although this has changed somewhat), configuration, testing, or distribution.

It’s also a dependency-based system; workflows constructed by rake are synthesized in reverse — it’ll be you, not your program, that gets forked when you try to make an imperative workflow. It’s simply the nature of rake! Rake is an excellent build program, but these types of things are in a different domain.

Tap was originally designed as a simple workflow engine, but it’s evolved into a general-purpose framework for creating configurable, distributable task libraries. Tap tasks can be defined in much the same way as a Rake task:

Configurations map to methods and can utilize a validation/transformation block. Tap defines a number of common blocks (ex c.integer, c.regexp, etc.) that may also imply metadata for the command line (ex c.flag):

% tap run -- hello world --reverse
I[20:04:33] olleh world

Distribution

Tap supports distribution of tasks as gems. To illustrate, say we installed the sample_tasks gem. Now our manifest looks like this:

Tap checks the installed gems for a ‘tap.yml’ configuration file or a ‘tapfile.rb’ task file; any gems with one (or both) of these files gets pulled into the execution environment. Now tasks can be specified either by a short name when there isn’t a name conflict (ex goodnight, print_tree), or by a full name that includes the environment (ex sample:goodnight, sample_tasks:print_tree).

Workflows and the Roadmap

Tap support simple workflows in the imperative style. Tasks can be assigned an on_complete block that executes when the task completes, allowing results to be examined and new tasks to be enqued as needed.

True support for workflows from the command line is lacking right now, but it will be coming soon. Here’s a short list of what else is planned:

Global Rake Tasks (hopefully!). I should be able to find and load rakefiles using the Tap execution environment. Tap already allows you to incorporate local rake tasks into a workflow using the ‘rake’ task.

Tap server. In a small-group environment where some people are computer savvy and others aren’t, it would be really useful to serve up your tasks using a web interface.

More test support. Tap provides several modules for testing tasks and supporting common types of tasks (ex file transformation tasks). Currently the modules are a bit incomplete, and they’re only geared towards Test::Unit. I’d like to add support for RSpec in the future.

3 comments

Sure, although I’m going to preface this by noting that I’ve never used thor (or sake) so this is all inferred from what I’ve read from the blog posts. Hopefully I’m not misrepresenting thor; apologies if I am.

Mapping to the command line

Thor: Maps methods from Thor classes to the command line.

Tap: Maps Task classes to the command line.

Classes, as you know, are easy to subclass, instantiate, use in other contexts, etc. Tap tasks were specifically designed to be useful off as well as on the command line.

Documentation

Thor: Uses rake-like declarations (desc, method_options) to build up information for the command line. As a consequence the file with the Thor class must be loaded into ruby to be recognized; most likely all Thor classes need to be loaded to get a manifest.

Tap: Builds manifests from documentation.

Task classes are not loaded to generate manifests — this helps avoid conflicts in ruby, and should be more scalable. Rake, which also needs to load declarations into ruby, sometimes suffers when you load a ton of tasks at once. In addition, task documentation is available for RDoc (you can see this by looking at the FileTask documentation… see, config documentation is there).

Although to be fair, I should mention Tap does load a task class to generate help, as when you use ‘tap run — task –help’ … but in this case it will only be that one class.

Distribution

Thor: Has a custom, neat-looking system (sake) for distributing/installing modules.

Tap: Allows distribution of libraries through gems.

Tap uses a nested-environment structure to make gem directories and their resources available in scripts. This allows tasks to be packaged and distributed as gems; in principle it also gives access to other resources (ex generators, rakefiles, etc… I’m trying to leverage this to make ‘global’ rake tasks, but I need to clean up the environment code a bit before that’s ready to go). Since tap task distribution is gem-based, versioning is naturally supported and it’s a system people know.

Other comments

It looks like thor and tap provide similar capabilities to the end-user: configurations and inputs. Thor also uses a syntax really similar to rake, so it should be easy for new users to pick up.

Tap almost certainly has more support for configurations; tap allows you to make and use static config files, for instance. Tap also supports workflows.