leave your migrations in your Rails engines

If you are using Rails engines to break up a single app into modular pieces, migrations (as they are currently implemented in Rails 3.2.13) become clumsy.

There are three options for migrations within an engine (spoiler: #3 is the best):

1) You can use the your_engine_name:install:migrations rake task, which copies the migrations out of the engine and into the wrapping Rails app where they can be run normally. This works fine if your migrations in your engine never change, but if you’re actively developing your engine you need to run this rake task each time you add a migration.

2) You can put all your migrations in your wrapping Rails app. This works if you’re using your engines as a way to break up your app, but it doesn’t feel right. If your models, views, and controllers all live within the engine (and depend on migrations), shouldn’t your migrations live within the engine as well? If your migrations live in the wrapper Rails app, you actually create a weird upward dependency where the engine is actually dependent on the wrapper app. This is bad.

3) You can monkey patch Rails so all of your engine’s migrations automatically get run in the wrapper Rails app. Everything just works, and migrations live where they should: in the engine. If you’re breaking up your large Rails app into engines, this is the way to go. Here’s how you do it….

Within your Rails engine, there should be a file called engine.rb here’s an example of it for an engine I called EngineWithMigrations:

All you need to do is tell Rails to add your engine’s migration directory to its list of places it looks for migrations (note: see the update at the bottom of the post if you are using Rails 4). Like so:

app.config is the config of your wrapper Rails app, config is the config of your engine. The above line adds the engine's migration directory to the wrapper Rails app's migration directory list. The unless wrapping it is to keep your migrations from running twice in your testing dummy app (which already runs migrations fine). Now when you run rake db:migrate from your wrapper app, your engine's migrations just work!

Rails 4 Update:

In order to get your migrations to work with Rails 4, the initializer needs to change slightly:

Update for rails 4 : paths are now stored in a special object instead of an array, therefore the intializer should look like :

initializer :append_migrations do |app|
unless app.root.to_s.match root.to_s
config.paths[“db/migrate”].expanded.each do |expanded_path|
app.config.paths[“db/migrate”] << expanded_path
end
end
end

July 19, 2013 at 7:36 am

Ben Smith says:

@Systho thanks for the Rails 4 tip! I just tried your initializer and it’s spot on. I’ll update the post.

July 21, 2013 at 3:45 pm

Jez says:

Is this technique appropriate if the engine is used in more than one application? Creating the engine’s schema in a new application from its migration history seems wrong in the same way it is not a good idea to recreate an application’s database from its migrations.

Engines released publicly often provide an “install” generator which generates a single installation migration for the application, but it seems in this case that would not work as the migrations would conflict.

Perhaps this isn’t a big deal if the engine is only to be used privately in a small number of applications, but I’m interested to hear if anyone has any experience of doing this.

August 2, 2013 at 6:38 am

Ben Smith says:

@Jez good question. My gut tells me that if you’re gemifying and distributing your engine, you should use the default rake task to copy the migrations from the engine into the wrapping Rails app. The main reason being the case where you remove the dependency on the engine at some point in the future… this would leave your database in a strange state.

However if you are using unbuilt engines as a technique to break your Rails app into smaller, more manageable pieces, then I think this technique is much better.

Even if you are developing an engine to be used by multiple Rails apps, you can start with the migrations within the engine (ie while you are actively developing the engine), then switch to doing the default copy-migrations-into-wrapping-app once your engine has solidified and is ready to be distributed and used in multiple Rails apps.

This is definitely a convenient technique, but it seems to fail when chaining rake db commands.

On it’s own “rake db:migrate” will pull in the changes just fine, however if you chain multiple commands like “rake db:drop db:create db:migrate” it does not include migrations from the engine.

I’m still trying to figure out why specifically it’s happening, but it seems to be that rake db:create doesn’t run any engine initializers and then the chained commands run in the same context so the ‘db/migrate’ paths are never added in.

October 8, 2013 at 2:17 pm

Mike C says:

@Joey Lorich,

I am seeing the same behavior you are for Rails 4 engines. Specifically I was using the technique to handle testing engines that depend on other engines. Does anybody have a reason that if I chain rake commands db:drop db:create db:migrate that only the local db:migrate gets called? I can see from puts statements that at some point in the rake process the append to function is being hit and the paths are correct. If I run db:migrate alone it all works as expected.

Other than this little snag, this is excellent test use case for teams with multiple engines for a given app.

On it’s own “rake db:migrate” will pull in the changes just fine, however if you chain multiple commands like “rake db:drop db:create db:migrate” it does not include migrations from the engine.

But you should never do “rake db:create db:migrate”. Migrations are only to change an existing database without losing data. If you’ve just created the database, then you should always do “rake db:schema:load” (or “rake db:structure:load”) instead. If you can’t do that, then your migrations are improperly written (and probably contain seed data that you should put in seeds.rb instead).

My gut tells me that if you’re gemifying and distributing your engine, you should use the default rake task to copy the migrations from the engine into the wrapping Rails app. The main reason being the case where you remove the dependency on the engine at some point in the future… this would leave your database in a strange state.

I don’t think you’re quite right here. Removing the engine would leave the *migrations* in a strange state, but wouldn’t have any issues with the DB.

Besides, if I’m using an engine, I think I probably don’t want to clutter up my app with its migration files. If I remove the engine, I can copy the files at that time. Till then, YAGNI.

I do agree with Jez’s concern about running lots of migrations to install an engine instead of loading a schema file, though. I’m not sure where that leaves us practically speaking.