Frustrated by Magento? Then you’ll love Commerce Bug, the must have debugging extension for anyone using Magento. Whether you’re just starting out or you’re a seasoned pro, Commerce Bug will save you and your team hours everyday. Grab a copy and start working with Magento instead of against it.

Updated for Magento 2! No Frills Magento Layout is the only Magento
front end book you'll ever need. Get your copy
today!

The short version: To assuage whatever guilt I have about selling commercial software and ebooks in an open source ecosystem, I’ve created and released a new n98-magerun command that allows Magento users to manually and incrementally run Magento’s setup resource migration scripts. Our long international nightmare of opaque Magento upgrades is over.

The next time you need to upgrade your Magento system (to, say, the latest 1.8.1 version), this new command will make the database portion of that upgrade much less scary. Also, if you’ve been looking for a place to get started developing with n98-magerun there’s a number of refinements this command could use, particularly around unit tests. See my comments at the end of the pull request and fork away on GitHub.

While useful, Magento’s implementation of this common pattern is not without its issues. To the surprise of many programmers coming from other frameworks, these setup resource scripts are not wrapped in database transactions. This means if there’s an error state half way though a script, or PHP halts for some other reason, the database could be left in an invalid state preventing future updates from running without direct programmer intervention.

Exacerbating that problem is Magento’s modular system and rich community of third party extensions. In applications worked on by one team, this sort of migration pattern creates a single line of upgrade scripts that are, more or less, easy to follow version to version. However, because each module in Magento has its own setup resource migration scripts, even a simple point release updating of Magento means running multiple scripts in one PHP request. Multiple long running scripts in a single PHP request increases the chances of the request timing out and causing a transactional integrity issue.

Finally, as hinted at above, these scripts don’t run at the prompting of the user. Instead, an uncached Magento request will examine the modules, determine if the scripts need to run, and then run then. The intent behind this is noble — automatic database updates for store owners — but in practice it makes the setup resource system and Magento upgrades opaque, and when things fails no one is quite sure why.

These are more than theoretical concerns — with every new Magento release there’s a spike in questions on the forums and Stack Exchanges about upgrades gone wrong and users unsure how to proceed.

The Solution

Unfortunately, this is a hard problem to completely solve. The weakness of MySQL’s transaction engine and the complicated nature of the problem (how to let thousands of developers safely modify a central database system) means both eBay/Magento Inc. and the community of developers around Magento have chosen to simply deal with the problems rather than solve them.

+--------------------------------------------------+
Run Structure Updates
+--------------------------------------------------+
All structure updates run before data updates.
The next structure update to run is core_setup
Press Enter to Run this update:
Structure update core_setup complete.
Ran in 653ms
(1 of 23)
The next structure update to run is eav_setup
Press Enter to Run this update:
Structure update eav_setup complete.
Ran in 11ms
(2 of 23)
The next structure update to run is admin_setup
Press Enter to Run this update:

you’ll have a much better idea of what’s going on. Failures may still occur, but you’ll quickly know which resource caused the problem. This means you can confidently move forward with a solution instead of desperately guessing at what’s wrong and playing whack-a-mole with the database layer until things work. Also, by running each script individually, the chances of a script failing due to resource consumption in PHP is greatly reduced.

Wrap Up

While work continues on Magento 2, it’s clear that Magento 1 is still the path forward for merchants who need production ready systems today. Also, if history is any indication, many organizations will stick with Magento 1 for a few years after Magento 2’s release.

With eBay’s focus split and Magento 1 clearly in maintenance mode, it’s up to the community of developers surrounding Magento to improve our own platform experience. Fortunately, despite the growing changes in how businesses approach open source software, we’re still free to alter, extend, and improve the system for ourselves.