Salesforce How to: Build your own Deployment Framework with ANT

1 – Introduction

Deploying in Salesforce is a non-negligible part of your project lifecycle, it can take quite a lot of time and effort.
Salesforce offers two main tools to support deployment activities. Change sets are very good for small one off deployments, and when you need a more robust solution for bigger and more frequent deployments, you’re usually better off using Force.com Migration Tool.
To reduce the time and effort required for each deployment, we started writing our own functionalities with ANT on top of the Force.com Migration Tool. Over time we ended up with a pretty useful framework that allows for easier and more flexible deployments.
In this article I’m going to present the deployment framework.
I assume that you are already familiar with Salesforce, the Force.com Migration Tool and ANT.

2 – Deployment Framework Overview

A typical deployment is composed of 3 main actions:
– Extract: extract from the source org the metadata to be deployed
– Tweak: tweak the extracted metadata if required
– Deploy: deploy the metadata in the destination org
The deployment framework is a set of batch files calling various ANT tasks that perform these 3 main actions.
The high level workflow for a typical deployment is as follow:

3 – Watch the Demo

You can watch the demo by clicking here: Coming Soon.

4 – Features Overview

4.1 – Extract

Extract is the first thing to do for a deployment. It consists of extracting from the source org (e.g. the DEV org) the metadata of the elements to deploy. This is pretty straightforward and all you need to do is specify which elements to include in the extract. For a list of all the metadata types you can extract/deploy with the Force.com Migration Tool, click here.

Once the extract is complete, you get a set of metadata files representing all the elements extracted.

4.2 – Specify Target Org

Deployments often require deploying the same thing to different target orgs (e.g. TEST, UAT, FULL, LIVE, etc). In order to simplify and automate deployments to multiple orgs, we added a functionality that lets you choose the target org you wish to deploy to.

All you need is to define all the available target orgs in the Deploy.build.properties file. You do this once, and then, every time you run a deployment, you’re asked to enter which target org you want to deploy to. It’s a much quicker and safer way to select amongst predefined target orgs than having to switch or modify the properties file every time. You can see how it works in the demo video.

4.3 – Tweak Metadata Files

It often happens that only partial elements from the source org should be deployed to a target org or that some values should be changed from one org to another. For instance, in DEV, you might have a few experimental fields in an object that shouldn’t be deployed anywhere else or some DEV values should be modified before being deployed to LIVE. The best solution for this is to extract the elements as is from the source org, and before deploying, to remove or change certain parts.

Removing or changing certain parts is done in what we call the tweak step. The tweak step is an ANT task that you can configure to perform the required changes on the metadata files before they’re deployed to the target org.

For instance, if in a specific object, you want to remove the field called TestField1, you would use an ANT RegEx to modify the contact metadata file. A lot of commonly used ANT tasks are already defined in the deployment framework to perform these type of changes. You can see how it works in the demo video.

4.4 – Backup Before Deploy

One very useful feature of the deployment framework is the ability to automatically backup elements from the target org before deploying new ones. This feature basically allows you to undo a deployment, something that is lacking in change sets and the Force.com Migration Tool.

Let’s say you’re deploying the Account object the Account layout. In your package.xml file, you have the list of elements to extract and deploy (i.e. Account and Account Layout). Just before deploying, the deployment framework will automatically perform an extract from the target org of all the elements referenced in the package.xml file. This will take a backup of the elements you’re about to deploy in the state they were before the deployment. You can see how it works in the demo video.

4.5 – Deploy

The deploy part is actually pretty straightforward. After all the extracted metadata files have been tweaked and a backup taken before deploying, all there is to do is to deploy the tweaked metadata files and optionally run some Apex tests.

This is done by either calling a generic ANT deploy task or creating a custom one if Apex tests are required.

4.6 – Backup Files with GIT

For additional safety/backup, the deployment framework can automatically push the metadata saved in the “Backup Before Deploy” step into a GIT repository. This is usually done only for deployments to a LIVE org.

4.7 – Deploy Data with Data Loader

For some deployments, you also need to deploy actual data along with the metadata. The deployment framework has the capability to deploy data using Salesforce data loader. This feature is discussed in another article.

4.8 – Modify Only Certain Components

One good thing about the deployment framework (or in this case the Force.com Migration Tool), is the flexibility you have to modify only certain elements in the metadata files (something you can’t do with change sets).

Let’s say you want to deploy the contact object and a custom visualforce page called contactTest.page. You extract the contact object and the contactTest page and deploy them to your target org. Now let’s say that later on, you want to deploy a modified version of the page but keep the current version of the contact object (maybe because someone is currently modifying it in DEV and hasn’t finalised the changes). You can simply do that by changing just the contactTest page in your extracted metadata files, and deploy it with the previous version of the contact object.

Of course you’d need to extract the contactTest page separately but the above scenario frequently occurs in real life so it’s good to know the feature is available.

5 – What’s in the Deployment Framework?

The deployment framework is composed of a set of batch files to launch processes (mostly ANT targets) and a set of config/xml files to configure the processes.

5.1 – Package.xml files

Package.xml files located in the 11-Extract\00-Extract_Package_XML folder and contain the list of elements to extract from the source org and deploy to a target org.

You need to create one package.xml file per deployment (e.g. extract.package.CRM_RELEASE_13.xml to extract and deploy filed for CRM Release 13).

5.2 – Extract batch files

For each deployment, you need to create an extract batch file. Extract batch files are very simple. All they do is call the common extract batch file with some parameters (the common extract batch file is already written).

Below is an example of an extract batch file for a deployment called CRM_RELEASE_13

The common 00.b-Extract.bat file that will do the extraction for us i.e.

Create a folder called Metadata_CRM_RELEASE_13 under the 11-Extract folder (see the tree structure below)

Extract all the metadata files for CRM Release 13 and place them in the folder created above. The components to be extracted are defined in the package.CRM_RELEASE_13.xml file located in the …\11-Extract\00-Extract_Package_XML folder.

5.3 – Deploy batch files

For each deployment, you need to create a deploy batch file. Just like extract batch files, deploy batch files call a common deploy batch file with some parameters (the common deploy batch file is already written).

Below is an example of a deploy batch file for a deployment called CRM_RELEASE_13.

Ask for target org: Ask for the target org in which to deploy to (available target orgs are set up separately in a specific config file)

Backup before deploy: If a backup before deploy is requested (e.g. for LIVE orgs), a backup of the deployed metadata files is taken from the target org.

Backup before deploy tweak: If a Backup Before Deploy Tweak Task is specified, run it to tweak the backup metadata files extracted from the target org.

Metadata files tweak: If a tweak ANT task is specified, run it to tweak the metadata files before deploying. Tweaks are specific to each deployment and/or target org. It can be things like removing certain fields or objects, changing certain values (e.g. emails), etc.

Deploy metadata: Deploy all the (tweaked) metadata files to the target org.

Commit backup metadata file to GIT: If requested, backup metadata files taken in the previous steps are committed to a GIT repository

5.4 – Build.xml files

There are 2 build.xml files, one for extracts and one for deploys. These files contain the definition of the Extract and Deploy ANT tasks.

The Extract build file (build.extract.xml) located in the 11-Extract folder is very simple and contains one ANT task called Extract. The Extract task is called from the common batch 00.b-Extract.bat and perform the extraction of metadata files by calling the sf:retrieve ANT task defined in the Force.com Migration Tool. You should never have to modify the extract build file.

The Deploy build file (build.deploy.xml) located in the 12-Deploy folder is a bit more complex and contains all the ANT tweak and deploy tasks. For each deployment, there is at least one deploy task and potentially several tweak tasks. All the deploy and tweak tasks are called from the common batch 00.c-Deploy.bat. You need to modify the deploy build file for each new deployment (e.g. CRM Release 13, CRM Release 14, etc).

The deploy tasks performs the deployment of (tweaked) metadata files by calling the sf:deploy ANT task defined in the Force.com Migration Tool. In each deploy task, you mainly have to define which parameters to use when calling sf:deploy (e.g. test level and which test you want to run, etc).

The tweak tasks are completely custom and dependent on each deployment requirements. However, they can use global ANT tasks for certain generic tasks (e.g. remove elements from metadata files, etc).

An example of the content of build.deploy.xml for CRM_RELEASE_13 is as follow:

5.5 – Build.properties files

There are 2 build.properties files, one for extracts and one for deploys. These files contain the definitions of global ANT variables used in the ANT processes (e.g. sf.username, sf.password, etc).

The extract properties file (11-Extract\Extract.build.properties) contains one set of global variables: sf.username, sf.password and sf.serverurl. They contain the connection credentials to the source org from where to extract the metadata (usually the main Development org).

The deploy (12-Deploy\Deploy.build.properties) contains one set of global variables per available target org. These variables contain mainly connection credentials to a target org. Their names are suffixed by the target org name for easy identification (e.g. sf.username.FULL, sf.username.LIVE, sf.password.FULL, sf.password.LIVE, ect). The deploy targets in build.deploy.xml use the variables matching the selected target org (i.e. sf.username.FULL and sf.password.FULL when deploying to FULL target org).

An extract of the content of Deploy.build.properties for the FULL org is as follow:

5.6 – Tree Structure

All the files composing the deployment framework are organised in the following tree structure.

Releases is the root folder that you can place anywhere on the hard disk. All the folders specified below are located within the Releases folder.

01-Additional Release Files
All additional files required for a deployment/release are located here. You can organise these files the way you want. One subfolder per release named with the deployment name / release name usually works well.

05-Release BatchesFolder where all the batch files are located.
It doesn’t directly contain any files, but only sub folders.

00-CommonContains all the common batch files used by all deployments (e.g. 00.b-Extract.bat and 00.c-Deploy.bat).

02-CRM ReleaseFolder for CRM release batch files. We separate CRM and Service Cloud releases but you don’t have to, as long as you create one subfolder per release.

13-Release 13Contains batches for Release 13.

03-SCI ReleaseFolder for Service Cloud release batch files.

11-ExtractFolder where all the extract files are located.

00-Extract_Package_XMLFolder where all package.xml files are located (they contain the elements to extract).
g. extract.package.CRM_RELEASE_13.xml is the file to extract CRM Release 13 files.

LogsFolder where log files are automatically generated

MetaData_CRM_RELEASE_13When an extract is run, a folder called with the release/deployment name is automatically created under the 11-ExtractMetaData_CRM_RELEASE_13 is the folder containing CRM Release 13 files.

12-DeployFolder where all the deploy files are located.

BackupBeforeDeployContains metadata files from a target org automatically backed up before a deployment.

Custom FilesContains custom files to be used for some deployments.

LogsFolder where log files are automatically generated

MetaData_CRM_RELEASE_13When a deploy is run is run, the folder called with the release/deployment name is automatically copied from the 11-Extract folder into the 12-DeployMetaData_CRM_RELEASE_13 is the folder containing CRM Release 13 files.

13-Data LoaderFolder where all the Data Loader files are located.
The data loader part of the deployment framework is covered in another article (to be published soon).

6 – Deployment Framework Source Code

7 – Improvements

7.1 – Deployment History

It’s a work in progress for the moment but a good thing would be to store in the target org, a history of all deployments (e.g. deployment name, version number, date, deployed files, etc).

7.2 – Removing Objects

Some deployments require removing some objects from a target org. This is handled by what’s called destructive package files (destructive pre or destructive post deployments). The framework currently supports destructive package files but the functionality requires improvement because running the same deployment several time cause the destructive step to fail. What’s required is a way to remove elements only if they are present in the target org.

7.3 – Automatically Handling New Target Orgs

In the current version of the deployment framework, when you want to add a new target org, you need to modify the framework files in 2 places (in the Deploy.build.properties file and in the build.deploy.xml file). An improvement would be to only have to add the new credentials into the Deploy.build.properties file.

8 – Conclusion

As we’ve seen in this article, with a little bit of upfront effort, you can build a deployment framework that will cover most of your deployment needs.