Firms Increasingly Turn to Machine Learning for Security Solutions

Forty-seven percent of organizations have already deployed machine learning (ML) solutions, with another 23% engaged in pilot projects, to help detect increasingly sophisticated incursions and lower the cost of response.

A study (PDF) commissioned by Cylance and undertaken by Enterprise Strategy Group (ESG) surveyed 300 IT and security professionals from mid-market and large enterprises. The respondents are located in the United States (43%), Japan (21%), United Kingdom (13%), France (12%), and Germany (11%); and all are involved in the purchase process for endpoint security.

The study sought to identify the 'top of mind' security threats, and the impact those threats have on endpoint security purchasing decisions.

Phishing is the biggest concern for most respondents. In the last two years, 55% have experienced phishing with a malicious attachment, 54% have experienced phishing with a link to a malicious website, and 29% have experienced instant messaging phishing with a link to a malicious website.

Phishing -- particularly spear-phishing -- has proven resilient against technological solutions. Although there are current attempts to develop ML solutions, some researchers believe it will not be possible. “With such a small number of known spearphishing instances, standard machine learning approaches seem unlikely to succeed: the training set is too small and the class imbalance too extreme,” notes a recent research paper.

ESG suggests that training is the best solution to phishing. “Organizations can reduce phishing response rates and raise security awareness by training employees to recognize spoofed emails and texts, and to practice good hygiene, such as checking URLs before clicking links,” says the report. “Training repetition, phishing simulations, and red teams, where an independent group attempts to induce bad employee behavior, can reinforce initial cybersecurity training, and 35% of organizations that have suffered a security incident are instituting additional end-user training.”

However, it is not the phishing itself that is the problem, but what the phishing leads to -- which is usually, ultimately, malware. Asked which threats are the most difficult to detect, 43% of respondents replied, 'unknown malware'; 31% replied 'zero-day exploits of new and unknown vulnerabilities', and 29% replied 'fileless attacks which employ weaponized content'.

These are precisely the attacks that signature-based detection systems cannot recognize, and where machine-learning behavioral detection systems excel.

Ransomware, unsurprisingly, gets special consideration. “At 46%,” states the report, “nearly half of the respondents noted that their organization had been the victim of a ransomware attack in the last year and more than half of those (56%) reported more than 5% of their organization’s endpoints were infected. It is noteworthy that only 12% of affected organizations paid the ransom.”

But neither paying the ransom nor restoring impacted data from backup are necessarily the solution. “Nearly a quarter of research participants whose organizations have been recent ransomware victims stated that they experienced a recurrence of the same ransomware on the same endpoints and 38% experienced the same ransomware but on different endpoints.”

Somewhat surprisingly, financial loss (15%) and data loss (19%) are dwarfed by interruptions to standard business processes (32%) as the main effect of a security incident. Given the time and effort required to discover and remove malicious files with the risk of missing something, many organizations simply re-image the system from a golden master. This is still time-consuming.

“Twenty-nine percent of respondents said their organization reimages 100 or more systems every month, and 12% said their organization reimages more than 500 systems per month. It takes a dedicated team of IT professionals to reimage hundreds of endpoints each month, and the volume of infections hints at inadequacies in endpoint security controls,” says the report.

Just as cyber criminals have evolved their attacks to defeat first-generation signature based defenses with polymorphic malware and fileless attacks, it seems that security professionals are aware of the problems and are actively investigating or deploying second-generation ML-based behavioral detection systems in response.

Kevin Townsend is a Senior Contributor at SecurityWeek. He has been writing about high tech issues since before the birth of Microsoft. For the last 15 years he has specialized in information security; and has had many thousands of articles published in dozens of different magazines – from The Times and the Financial Times to current and long-gone computer magazines.