Backup has Fallen – How Cyber Threats are Compromising Your Backup System

Most IT professionals consider the backup process the last line of defense in their efforts to protect the organization from cyberattacks like malware, ransomware, phishing, data extraction, and internal rogue employees. The problem is bad actors are breaching this last line of defense on a regular basis. Data protection software vendors have taken great strides to make sure they protect their repositories and configuration files from direct attacks, but they don’t protect against these new, more subtle attack vectors. Cyber incidents not only degrade backup quality but can take months to be identified and resolved. IT must move quickly to identify solutions that can verify the quality of the backups against morphing cyber threats as well as assist with remediation of these evolving and sophisticated attacks.

Cyber Incidents Go Well Beyond Ransomware

While ransomware has captured the headlines in the past few years, it is not the most pervasive cyberattack problem. In some regard, ransomware is the “best” way for an organization to be compromised because at least the attack is obvious. Of greater concern is malware that copies data out of the environment, without warning, to be used against the organization in the future. Malware can surreptitiously capture sensitive data like customer credit card information, employee social security numbers as well as employee or administrator passwords, which can cause even more damage. Consider also the ongoing risk from the rogue employee that deletes data prior to leaving the company or compromises critical processes like backup and replication.

Changing Nature of Malware Attacks

Basic malware triggers as soon as it breaches the environment. In the case of ransomware, it immediately starts encrypting data, attempting to alter as many files as fast as it possibly can. Other malware may attack and lock system files. In most cases, IT knows that its typical defenses were defeated soon after an attack is underway. In these cases, it starts the recovery process through its backup software and restores the most current restore point. Simple malware is successful largely because of weak backup practices. The solution to this problem is relatively simple ⎯ make sure the backup process is working and perform protection and testing events more frequently. Given the capabilities of modern backup solutions, responding to these basic malware attacks is pretty straightforward.

However, next-generation malware works differently and is much more invasive. Once it breaches the organization’s defenses, it sits idle, waiting for the data protection system to copy the malware files into the backup repository. Ironically, backup software, the very solution IT counts on for protection from a cyberattack, is now used as a carrier of the infection. These malware files will also discreetly copy themselves across the organization’s network, corrupting multiple systems over time.

Then, at a time designated by the bad actor, the malware triggers. Depending on the strain, the malware may attack all at once, or it may attack very slowly to further avoid detection. Since the infected data has propagated into the backup repository, restore points become worthless and even dangerous. Many malware detection solutions operate by monitoring the change rate of user and system files or the detection of higher than normal deletion rates in the production environment prior to the backup being taken. The slow attack rate of modern malware solutions enables them to operate while slipping past the typical and basic detection methods.

At some point, IT realizes it is under attack. The problem is that the attack could have been going on for weeks, if not months. Recovery is attempted using the same process as in a simple malware attack. However, backup copies now have the trigger files in them. As IT restores infected workloads, it is also restoring the trigger files, which immediately start attacking again. A blind restore of all the files from the last backup forces the organization into an endless attack loop as it restores the same corrupted files.

Why Current Verification Methods Fall Short

With the escalating frequency of cyber incidents, verifying the quality of the backups is now job one for data centers. The organization must have confidence in IT’s ability to recover from any cyberattack to conduct routine business. However, with limited resources and testing tools, most IT organizations don’t actually “verify” backups until needed in a restore situation. Unfortunately, everyone may be surprised and in a panic when they learn that these backups are compromised.

So, what are the options today? One method to verify backup quality is manual inspection. With this approach, the backup administrator needs to review all the backup logs for anomalies like an abnormal number of changed files or an unusual drop in files protected. It also requires searching each backup job for a list of known malware files to see if those are now infecting the backup repository. The problem is there are thousands of active malware files on threat intelligence lists, and the data is continuously being updated. The reality is that physical inspection of backup logs is beyond human capabilities. The chance for an error or mistake is incredibly high, and it is entirely too time-consuming.

Another verification option is to inspect files following a test restore. IT can, for example, restore workloads to a sandbox prior to moving them into production and then scan with some sort of malware detection or cyber analysis software. Given the time and resources needed, this approach could never be performed on every restore point or even on a daily basis. In an era where organizations are demanding nearly instant recoveries, a protracted resolution following a cyberattack is unacceptable ⎯ especially when management assumes all backup systems are being well tested in advance.

Next-Generation Backup Verification Solutions

Depending on how critical an application is, backup frequency might range from minutes to hours. However, in almost every cyber incident, there is a span of time when backups sit idle prior to IT needing them for a restore. For the reasons indicated, most backups are never tested, and organizations are left exposed following a cyberattack. The good news is that next-generation solutions will soon be available that provide automated verification of every restore point against a wide range of cyber issues by using backup repository data together with industry threat intelligence and remediation data.

Modern verification involves an automated version, and a more thorough one, of the manual process described above. Immediately after the backup completes, backup metadata can be structured, analyzed and correlated to threat intelligence data to reveal known cyber threats. Patterns can also be evaluated to identify abnormal or unexpected results in backups that have been taken. Both types of analysis can even be done retroactively in the event that threats are exposed in the future.

Key to next-generation backup cyber verification is transformation of the proprietary structure of backup repositories into formats that can be used for offline analysis, without impacting production or requiring recovery into a sandbox. In this way the verification solution can perform a much more detailed and thorough assessment of all restore points taken. That means the solution can scan for matches to known malware from threat intelligence sources as well as anomalous restore point patterns, detecting cybersecurity conditions quickly and in an automated, resource efficient way. More importantly, the solution can detect the first point of infection, so the organization knows exactly from which point to start a safe recovery.

Putting resting backup data to work provides the organization with a content-rich cybersecurity treasure trove that is critical to dealing with the escalating frequency of complex cyber threats. Keep in mind that two different opportunities exist here. First and foremost, the automated ability to analyze each restore point in the backup repository data at rest enables backup systems to provide the insurance policy needed and expected in the age of sophisticated cyber threats. Second, when backups are analyzed quickly, any anomalies are indicators of cyber problems that still exist in production. Thus, the prompt cybersecurity assessment of offline backup and disaster recovery systems becomes a “free” cyber tool for IT operations, since the data for analysis is already being captured for data protection purposes.

HotLink is offering a unique and compelling solution designed to comprehensively analyze backup data for cyber threats right after the backup completes and before it is needed for a restore. It uses a patent-pending fingerprint technology to transform proprietary backup metadata and enable security analysis on all restore points within the backup environment, by using a combination of threat intelligence correlation and advanced heuristics. Security dashboards highlight top issues it detects in the backup repository and provides a vulnerability score on specific assets and backups along with detailed remediation instructions for any cyber issue. Note that HotLink does NOT replace your current backup solution ⎯ it makes it cyber-secure. The technology is currently in beta for Veeam cloud service providers with general availability in the 1st quarter of 2019. Integration with other data protection solutions will quickly follow. Check out their cybersecurity technology by contacting info@hotlink.com.

The Bottom Line

Everyone needs to recognize that backups are not safe from cyberattacks today. IT has the responsibility to make sure data protection systems are secure and that corrupted data is not systematically propagating into backup repositories, rendering backups potentially useless following a compromise. For this to be possible, cost-effective, resource-efficient automation is needed to continuously verify restore points and identify what the last known good backup actually is. The backup repository is sitting idle most of the time, ripe for inspection. With next-generation solutions, data protection systems can not only provide a solution to continuous validation of backups as they are taken, but can also change the backup insurance policy into a valuable asset that can forewarn the organization of an impending cyber-attack nightmare.

Share this:

Like this:

Related

Twelve years ago George Crump founded Storage Switzerland with one simple goal; to educate IT professionals about all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought after public speaker. With over 25 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS and SAN, Virtualization, Cloud and Enterprise Flash. Prior to founding Storage Switzerland he was CTO at one of the nation's largest storage integrators where he was in charge of technology testing, integration and product selection.