According to new research, 98% of leading companies across the U.S. and Europe are vulnerable to cybercriminals through their web applications. While this figure may seem high, it will surprise neither the companies themselves nor independent security experts.

Most large companies readily admit that they have shadow IT and legacy applications they do not know, and that this at least theoretically makes them vulnerable. It is generally considered to be an acceptable risk.

The purpose of this research from High-Tech Bridge (HTB) is designed to show that the problem is far bigger and less acceptable than most companies imagine. It was prompted, at least in part, by HTB's experience with one particular U.S. government agency client.

"They told us," HTB founder and CEO Ilia Kolochenko told SecurityWeek, "'We know we have shadow IT -- about 250 applications." HTB used its non-intrusive scanning tools and replied, "No, you have 8000 shadow IT applications." The implication is that this government agency has around 7,750 shadow IT applications that it doesn't know and isn't monitoring -- leaving it potentially vulnerable to an unquantifiable risk.

For its new research, HTB used its four free non-intrusive scanning products (Discovery, SSLScan, WebScan and Mobile App Scanner) to quantify the vulnerabilities and weaknesses of the FT U.S. 500 companies, and the FT Europe 500 companies. It is important to note that these non-intrusive scans do not detect all vulnerabilities -- only those that are exposed to the internet. But if HTB can see them via the internet, so can hackers.

The figures returned are quite staggering. First the basics. The 500 largest U.S. companies have 293,512 external systems accessible from the internet. 42,549 have a live web application with dynamic content and functionality. The figures for the 500 largest European companies are 112,750 and 22,162. Kolochenko points out that the figures are skewed somewhat by the sheer size of some of the American firms, with the likes of Apple, Google, Facebook and Microsoft each having many thousands of servers and many thousands of applications.

The results do not compare U.S. and European companies. Apart from the size differential there is a culture differential. Europe is conservative while the West Coast in particular is the home of innovation and experimentation. The U.S. and Europe are apples and pears; and the spread of firms chosen was simply to give a geographically dispersed view of the problem.

Nevertheless, these first figures show, according to the report, "a US company has an average of 86.5 applications that can be easily discovered externally and are not protected by 2FA, strong authentication or other security controls aimed to reduce application accessibility to untrusted parties. As for an EU company, there are 46 such applications per company."

HTB has its own method of grading installations based on a score out of 100 and ranging from A to F. The research found that 48.1% of U.S. web servers achieve an A grade for their SSL/TLS encryption -- but 32.21% have an F grade. In fact, 7.82% still have the vulnerable and deprecated SSL v3 protocol enabled. In Europe, the figures are 62.4% at A, 16.02% at F, and 5.15% with SSLv3 enabled.

The research also examined external indications of compliance with PCI DSS and GDPR to gauge the level of security for the internet-facing applications. For PCI, it shows that only 16.4% of the U.S. web servers have an SSL/TLS configuration compliant with PCI DSS 3.2.1 (and only 14.7% in Europe). The report notes, "a configuration non-compliant with PCI DSS does not necessarily mean poor encryption, but in many cases it does."

On indications of GDPR compliance, 16.2% of the US companies have at least two web applications that permit entry of personally identifiable information (PII) (e.g. via web forms) and run a vulnerable version of SSL/TLS, and/or outdated and vulnerable CMS or other web software. It is only slightly lower in Europe at 15.4%. "Numbers of non-compliant web applications may likely be much higher," comments the report, "but it is impossible to say how many of the outdated and vulnerable websites actually process or store PII without conducting intrusive tests."

You get the picture. The sheer quantity of weaknesses, concerns and vulnerabilities exposed by even the largest companies is far greater than most people would realize. But this is just the beginning. HTB's research also found:

• only 2.94% of U.S. companies achieve an A grade for properly implemented security hardening and configuration of web servers. Most, 76.9% score an F. The scores in Europe are almost identical at 2.98% and 76.9%.

• only 9.1% of U.S. companies have an enabled and properly configured content security policy (CSP) which is used to mitigate XSS and CSRF attacks on the server side. It is worse in Europe at just 4.39%.

• as many 8% of web applications in the U.S. (15.8% in Europe) use third-party software (CMS, JQuery, SharePoint) that is outdated and contains at least one publicly disclosed vulnerability

• 94% of all U.S. WordPress installations (99.5% in Europe) have a default admin location not protected by other means such as supplementary .htaccess authentication or IP whitelisting, making authentication attacks -- including via compromised plug-ins) much simpler

• 98.4% of U.S. web applications (98.1% in Europe) have no web application firewall (WAF) or have it in a too permissive mode

• 27% of U.S. companies (12% of European companies) have at least one external cloud storage (for example, an S3 bucket) accessible from the internet without any authentication. HTB's non-intrusive scanning does not know what the storage contains, but the report comments, "Some files in storages are expressly marked as ìinternalî pointing out that these cloud resources are probably not intended for public availability."

• 221 U.S. companies have a total of 1,232 vulnerability submissions on Open Bug Bounty -- of which 462 have not been patched. 162 European companies have 625 vulnerability submissions, of which 210 remain unpatched

• 62% of the U.S companies have at least one website access being sold on the Dark Web (78% of European companies)

However, knowing the size of the problem is no help to an overworked CISO. He or she is probably already aware that problems exist, although most likely not to this extent. The problem is knowing where to start.

AI Discovery

High-Tech Bridge has also launched a new product: Immuniweb AI Discovery. It can locate the problems listed above, but then uses machine learning techniques to relate the problems to HTB's own Big Data compilation of more than 853,783,291 known vulnerabilities and weaknesses in web applications. This data is compiled from all publicly available sources and added to HTB's own research. From this it can return a 'hackability score' and an 'attractive score'.

HTB first finds the problems, and then uses artificial intelligence to tell the company which issues are most easily exploited, and which issues are most likely to be exploited. In effect, it provides the CISO with a risk management-based roadmap for tackling the most critical vulnerabilities in his or her internet-facing infrastructure -- many of which may well have been unknown.

For the acid test, SecurityWeek asked Kolochenko if AI Discovery would have picked out and highlighted the Struts vulnerability exploited in the Equifax hack. Ever a stickler for accuracy and precision, Kolochenko replied, "It could have. It would not if the server concerned was disconnected from the internet at the time of the scan, or if an insider had taken other steps to hide it. Otherwise, it would have."

There are other products able to locate internet-facing security issues. What AI Discovery does is rank them in a 'fix-priority' order for CISOs. All the statistics used for this research came via HTB's free products. AI Discovery is a new paid-for product.

Kevin Townsend is a Senior Contributor at SecurityWeek. He has been writing about high tech issues since before the birth of Microsoft. For the last 15 years he has specialized in information security; and has had many thousands of articles published in dozens of different magazines – from The Times and the Financial Times to current and long-gone computer magazines.