Webinar: DB Maestro and Delphix

The simple truth is that databases can’t be given to every developer like source code.

Databases can be large, complex, and extremely difficult to copy. It is rare to see developers or even development groups that have a copy of the full production database to use for their tasks.

On the other hand, developers almost always have access to a full copy of the application code stack. Source control methods like git make it easy to clone, fork, and track code from the component level to the entire codebase.

For databases, the typical strategies are:

Share a full copy of the source database across a group

Provide small subsets of data from the source database to developers

Allocate schema only ‘databases’ per developer

Each of these strategies presents their own problems.

Sharing a database across a group of developers means that any change one developer makes can impact the other developers. There is no automated way to protect a group of developers from the changes of a single developer. To protect development problems that can be caused by new changes, the changes have to be reviewed. Reviews can take up to a couple of weeks, meaning the coder who wants to implement these changes has to wait. The first in a long line of delays as a result of inadequate database resources.

If developers are given subsets of the database (partial copies, only some data) or empty skeleton schemas, it will easily lead to code that has not been tested on the full range of data that is present in production. This of course opens the door to untested bugs born from outliers in the data and queries that are not optimized on the full set of data. Finding these bugs late in the development cycle is more costly by far than finding them immediately with the developer writing the code.

Delays and errors in database development

Let’s zoom out a bit and look at the whole development process. Doing so unfortunately presents additional challenges. These challenges increase due to multiple lines of development that have to be run in parallel and merged, run through QA and UAT, and then rolled out to production.

In most cases development is not a single player effort. Different teams are working in parallel, and code deployment is often practiced using structures called branches and trunks that create separate working environments for different teams to work uninterrupted. Prior to rolling out to production, these branches have to be merged into a single working environment that can be tested and rolled into production.

Doing this for source code is standard practice in the industry, as all that is required is for a new copy of the code files to be made in a new centralized location. This is often done automatically by a source control tool. But as we’ve seen, databases are more complex than source code and more difficult to copy.