Manage Snapshots

Updated: September 13, 2015 16:36

Snapshots are an important part of a high availability strategy but keeping lots of old snapshots around can add up in usage charges. If you find that you have a number of snapshots that you no longer need, you can use the snapshot management / delete function to clear them out and reduce your monthly costs.

Snapshot management allows you to set the snapshot policy at the account level and at the environment level; you can also delete unneeded snapshots from the account or environment levels.

To manage snapshot retention policy at the environment level

Important: We recommend testing in a staging environment before applying changes in a production environment.

In Engine Yard, click Tools > Dashboard.

Select an environment by clicking the environment name.

On the Environment page, click Snapshots.

The Snapshot Management page appears, showing the snapshot retention period for the selected environment.

All snapshots specific to this environment appear in the snapshot list.

(Optional) Change the snapshot retention period if you need to decrease or increase the amount of time (in days) that snapshots are kept.

Click Update snapshot settings.

From this point forward, Engine Yard will delete snapshots after the specified number of days has passed for all snapshots related to any instances.

(Optional) If you need to delete snapshots, select the Destroy checkbox for those snapshots.

Click

Engine Yard immediately begins deleting the selected snapshots and displays [destroying] at the end of the snapshot description.

Configure snapshot retention policy

When you set up your environment, you specify the number of snapshots and the number of days to keep the snapshots.

Important: We recommend testing in a staging environment before applying changes in a production environment.

To configure snapshots in your environment

In Engine Yard, click Tools > Dashboard.

Select an environment by clicking the environment name.

On the Environment page, click Edit Environment.

Update the snapshot retention policy as needed:

Number of snapshots to keep, per instance.

Note: This applies to each instance in an environment. For example, if you have 5 instances and set this number of snapshots to 10, then your environment will accumulate 5x10 snapshots.

Snapshot retention period in days, for this environment. Engine Yard will delete snapshots after the specified number of days for all snapshots related to any instances associated with this environment.

Note: This setting takes precedence; once the retention period has passed, the snapshots get deleted, regardless of the number of snapshots.

Snapshot frequency is based on the frequency of backups.

Tip: You can set the frequency of backups under the Backups heading in the UI.

Click Update Environment to save the new snapshot settings.

Click Apply to push the policy changes to the instances.

Verify snapshot retention policy

Here are some ways that you can verify the snapshot retention policy for your environment.

To verify snapshot retention policy

You can view the account-wide setting for snapshot retention period (used for "orphaned" snapshots only) using Tools > Snapshots.

You can view the environment-specific setting for number of snapshots (per instance) and retention period (per environment) on the Edit Environment page. You can also view the retention period, in days, on the Snapshot Management page.

You can view the date and time a snapshot was created (and therefore, when you can expect it to be deleted), on the Snapshot Management page.

Once you have deleted snapshots, you can refresh the Snapshot Management page to see the remaining snapshots.

Deleting an instance does not make the related snapshots go away; Engine Yard deletes snapshots based on the set retention period for the environment (default of 90 days).

Deleting an environment does make the related snapshots go away; Engine Yard deletes snapshots immediately. In rare cases, some snapshots will not be caught during this delete process; those "orphaned" snapshots will be deleted based on the set retention period for the environment (default of 90 days).

FAQs

You might have these questions about the snapshot retention policy.

Why do I have WAY more snapshots than I expected (for example, I have 5 instances, yet have more than 500 (5x100) snapshots)?

This can happen if you cycle your instances often. Remember that the number of snapshots is per instance. So if you have 5 running instances and 10 deleted instances, you can have (5+10)x100 snapshots associated with that environment.

Why do I still have snapshots even though the related instance has been deleted?

Snapshots are kept even after an instance is stopped. This is a precaution, in case you need the snapshots to rebuild an instance. The default is to keep snapshots 90 days.

Why is there no name in the Environment column associated with snapshots in the list of snapshots for this environment?

Snapshots that appear in your snapshot management list without an associated environment name are probably "orphaned" snapshots from a deleted environment. If you no longer need them, it is OK to delete these snapshots. Or, if you do nothing, Engine Yard will delete them when the snapshot retention period (default 90 days) has passed.

Why am I being charged for snapshots even though the environment is long gone?

See above answer. The snapshot retention policy should be set based on your app's needs. If you have old snapshots that you no longer need, see Manage snapshots in an environment for information on deleting unused snapshots (even those no longer associated with a particular environment).

Can I set a retention period per instance? or per app?

No; retention period (in days) is set per environment. The account-level default is 90 days.

Can I keep more snapshots for my database (for example, 20 for database and 5 for app master)?

No; the number of snapshots setting is per instance, across the environment. Remember that snapshots are generally small and incremental and are relatively low-cost. Use backups to ensure data integrity.

Hi Andrew, the snapshot frequency is set to daily (at 1am) by default. However, the snapshot frequency mirrors the database backup frequency selected in the environment settings, so increasing this frequency will also result in snapshots being taken at the same frequency.

It will take a snapshot every hour? Keep the last 50 - meaning it will keep snapshots for the last 50 hours? And keep them for 90 days? So, will I end up with a lot more than 50 snapshots or not? If it's only keeping the last 50 hours of snapshots then it's not keeping them for 90 days. Sorry, I'm a bit confused.

The 'Keep the 50 most recent snapshots' option will ensure that the 50 most recent snapshots for each of the instances is kept. So in your case this will mean that 50 snapshots for each of the 4 instances in the environment are stored, the oldest being 50 hours old. As the snapshots are incremental, each snapshot is only the size of the changes since the previous snapshot, so you are not looking at hugely increased storage costs to increase the amount of snapshots stored, if this is required.

The 'Keep for 90 days' option is the retention period, meaning that any snapshots older than 90 days are deleted. Should any instance be deleted, or the environment shut down, the associated snapshots will be kept for 90 days, then automatically deleted, so you would not be able to reboot instances from these snapshots after this period had elapsed. (Deleting an environment would also delete all snapshots for that environment).

One other thing I want to bring to your attention here: snapshotting or backing up your database every hour is very likely to cause some disk I/O issues, so I strongly advise against it. Your performance could really suffer. Instead, if you need that kind of fast failover granularity or want to minimize data loss in terms of downtime, really the right way to go is adding a database replica, which is real easy with the dashboard (both MySQL and PostgreSQL are supported).

Hit us up with more questions if you have them. Hope this helps.

P.S. At this time the snapshots and backups are still running at 2 hour intervals on your environment, as you have not yet clicked 'Apply' to push this change out to the instances.

Keep the [dropdown] most recent snapshots and delete each one after either [dropdown] days or to make room for new snapshots

I presume that's what would happen? That if you're backing up every hour and keep 50 snapshots then they would never get a chance to be anywhere 90 days old (unless the instance was deleted)? So the "keep for x days" actually becomes redundant in a lot of cases (unless the instance is deleted)?

One other thing, it would be really useful if your system could retain backups for much longer periods. At the moment, if you back up every 2 hours, your oldest backup is only going to be 17 days old (2 x 200 covers a 400 hour period). So, you can never go back and look at a backup from 1, 2 or 3 months ago?