Manage Indexes with Variable Retention

With Multi-retention, you can create Index Partitions and Scheduled Views to store your data as needed, and set different retention periods for Partitions and Scheduled Views.

Your company log data comes in many types, and different types of data may require different retention periods. For example, some sensitive information may require a short life span, while other vital data, such as customer logins, may require a longer lifespan for daily use. Still other data may need to be available for historical analysis and to gather metrics to make business decisions, though it is not accessed every day.

With Variable Retention, you can create Index Partitions and Scheduled Views to store your data as needed, and set different retention periods for those Partitions and Scheduled Views. This way, you can keep the data you need for as long as you need it, while other less important data can be retained for a shorter period, reducing your overall cost of operation.

You set the retention period when you first create the Partition or Scheduled View, and you can also edit the retention period at any time in the future.

Retention is measured from the time that data was received, rather than the message time.

The minimum retention period is 1 day, and the maximum retention period is 5,000 days.

On the Manage Data > Settings page, there is a tab for Partitions and a tab for Scheduled Views.

On the Partitions tab, the first entry will always be the General Index, which is made up of:

The Default Index, which includes data that does not match any Partition’s routing expression.

The Sumo Logic Audit Index, which includes data stored in Source Categories such as sumologic_audit, sumologic_volume, and more. For details, see Enable and Manage the Audit Index.

Views populated by Scheduled Searches.

The default retention period of the General Index is 30 days.

Here you can see the total volume ingested in each Partition, including:

Edit the Retention Period

You may edit the retention period of Partitions and Scheduled Views. The minimum retention period is 1 day, and the maximum retention period is 5,000 days.

Go to Manage Data > Settings.

On the Partitions or Scheduled Views tab, hover over the item you would like to edit the retention period for.

Click the blue arrows on the right, and from the menu select Edit Retention Period. (You may also edit the retention period from the expanded information dialog. Next to Retention Period, click Edit.)

In the Edit Retention Period dialog, enter the new retention period in days. Or you may also select the check box Apply the retention period of the General Index.

If the new retention period is longer than the previous period, click Save.

If the new retention period is shorter than the previous period, a portion of your data will be deleted in 7 days or right away. Click Save and choose either:

Simulate this data deletion for a few days and then reduce it permanently by selecting Reduce retention period after 7 days. You will be billed for this data until the deletion is permanent, but it gives you a sense of how the deletion will impact you before it is final.
When you select Reduce Retention Period after 7 days, the retention period on your Partitions page will not change immediately. There will be a an icon indicating that a retention decrease is imminent in a number of days. When the counter expires, the data will be deleted, the warning icon will disappear, and the new retention policy will be in effect.

If you want to change your mind at any time during that period and revert to your previous retention period, click Click here to undo the reduction in retention period. The retention will stay at previous retention level, and the job to reduce the retention is canceled.

Or, if you are sure you want to proceed and end billing for this data immediately, enter DELETE in the text box, then click Reduce retention period immediately. The data will be deleted immediately as well.

The retention period is updated and displayed in the Retention Period column. If you used the reduce retention after 7 days option, you can undo your data retention reduction until the date specified by clicking Click here to undo the reduction in retention period. Otherwise, the data will be deleted after 7 days.

Recommended articles

Sumo Logic is the industry’s leading secure, cloud-native, machine data analytics service, delivering real-time, continuous intelligence across the entire application lifecycle and stack. More than 1,000 customers around the globe rely on Sumo Logic for the analytics and insights to build, run and secure their modern applications and cloud infrastructures.