Traditional firewalls track the domains that traffic is coming from and the ports it's going to. Nextgen firewalls go beyond that -- they also monitor the content of the messages for malware and data exfiltration and can react in real time to stop threats. The newest iterations do even more, adding behavioral analytics, application security, zero-day malware detection, support for cloud and hybrid environments, and even endpoint protection.

That's a lot of functionality in one place. The idea is that by putting everything in one place, the management task would be simplified. Some firewall vendors -- and third-party providers -- are beginning to tackle the management issue by offering intent-based security, allowing users to set consistent policies for management and configuration, as well as compliance-related policies.

According to Gartner, by 2020, nextgen firewalls will reach almost 100 percent of internet points of presence. Most organizations, however, will use only one or two of the nextgen features.

How the next-generation firewall market is changing

Next-generation firewalls have been around for ten years, but the market is still growing. According to NSS Labs, more than 80 percent of enterprises currently have nextgen firewalls in place. "It remains the number one security control for enterprises today," says Mike Spanbauer, vice president of strategy and research at NSS Labs.

However, none of the nextgen firewalls evaluated by NSS Labs in this summer's round of security tests demonstrated full resilience against attack variants, though six out ten scored more than 90 percent. That leaves substantial room for improvement.

NSS Labs recommendations for next-gen firewalls

According to the latest comparative firewall security test by NSS Labs, the following vendors received the top marks for the security effectiveness of their next-generation firewalls:

According to Markets & Markets, the nextgen firewall market is estimated to grow from $2.39 billion in 2017 to $4.27 billion by 2022 at a compound annual growth rate of 12.3 percent. The reason is that both the threat landscape and the corporate perimeter have changed dramatically over the past few years.

According to Gartner, the typical firewall lifecycle is three to five years. Back in 2011 and 2012, there was an uptick in purchases of next-generation firewalls, according to Gartner analyst Adam Hils -- a "significant number" of those firewalls should be replaced over the next 12 months, since they will no longer meet today's needs for throughput and decryption of outgoing Transport Layer Security (TLS) communications. Today's enterprises are also more likely to use cloud or hybrid infrastructure and have users who connect in via web applications and mobile devices.

Nextgen firewalls try to adapt to the cloud

So far, nextgen firewalls vendors haven't been able to fully translate their features to the needs of cloud environments, says NSS Labs' Spanbauer. "This is a significant engineering feat, and we're not quite there yet with a perfect replica, virtualized or physical."

However, they are taking advantage of other capabilities that cloud offers, including the real-time sharing of threat intelligence data. "If you're patient zero, then that's an incredibly difficult scenario to block against," he says. "However, if you give it a minute or two minutes, then patient 10 or 15 to 20, with real-time updates, can be protected by virtue of the cloud abilities of the firewall."

Will nextgen firewalls offer endpoint security?

There's also the possibility of nextgen firewalls expanding into the endpoint security space. "If they merged, that would be a lot easier for enterprises to manage," says Spanbauer. "But that's not going to happen."

Perimeter protection and endpoint protection will remain distinct for the foreseeable future, but the two sets of technologies could mutually benefit one another, he says. "So, the information that the endpoint sees helps the firewall work better."

According to firewall vendor Check Point Software Technologies, the next evolution of enterprise security, combining all the functionality of current nextgen firewalls with cloud, mobile, and endpoint protection, will no longer be a firewall, but a new category altogether.

Check Point's take on this is Check Point Infinity Architecture, says Darrell Burkey, the company's director of IPS products. "It is a new type of product," he says. "I don't see this as being a nextgen firewall. It's healthier to look at the entire infrastructure, and approach it from the perspective of all the different topologies as a unified, elastic system."

A firewall isn't enough to ensure full enterprise security he says. "It cannot provide full protection, so it's becoming a layer, or a component, of an advanced threat solution."

Advanced threat solutions, also referred to as advanced threat protection, can also include dedicated threat intelligence gateways that automatically score threats and block them at the perimeter, secure DNS services, micro-segmentation, and intelligent application controls, according to Enterprise Strategy Group analyst Jon Oltsik.

Growing complexity for nextgen firewall management and compliance

According to FireMon's state of the firewall report, the security professionals surveyed says that the complexity of firewall rules and policies were their biggest firewall challenge, with policy compliance and audit readiness in second place, and optimizing firewall rules a close third. In addition, the majority of companies surveyed had more than 10 firewalls in their environment, with 26 percent reporting that they had over 100 firewalls.

How to clean up the firewall rule base and minimize risk

Eliminate technical mistakes: Technical errors in firewall policies are rules best described as ineffective or incorrect, or those identified that do not serve a business purpose (e.g., hidden, shadowed, redundant and overlapping rules).

Remove unused access: There may be some rules within your rule base that are compliant and provide (or block) the right access, but they just aren’t being used. The best way to determine rule usage is to correlate active policy behavior against the network traffic pattern over a sustained period.

Refine overly permissive rules: Often the result of poorly defined business requirements coupled with tight deadlines, these are rules that provide greater access than is needed to meet the needs of the business -- for example, any rule that includes the use of the word "any."

Continuously monitor policies: It’s all about maintenance. Continuously monitor your policies to avoid recreating the firewall mess you just organized and to maintain a better security and compliance posture.

[Source: FireMon]

Any rules and policies that companies set up for their firewalls would typically be mirrored in other security products that they have in their environment. "The average network has 80 to 90 point solutions that's helping them secure at the desktop level, the server level, and the network level," says Tim Woods, VP for technology alliances at FireMon.

The newest iterations of firewalls can consolidate some of those point solutions, helping stem the tide, to some extent, but new threats continue to emerge, and enterprise environments continue to evolve, so it never stops. Meanwhile, some of the new at-risk areas, such as some cloud environments or SAAS applications, aren't even under the control of the security teams, but are managed by other departments.

In fact, the problem of complexity is getting more severe, says Woods. "As complexity continues to grow in the environment, the probability of error also continues to grow -- human error, configuration error, problems start to evolve and mature within the infrastructure as the complexity tends to grow."

Managing security by intent, where the specific firewall rules and security configurations are generated based on overarching principles, is supposed to address the problem, but the technology isn't there yet. "I haven't found a company yet that's confident to say that my security policy is an actual reflection of my security implementations," says Woods.

In the future, he says, companies will be able to define their security intent, and a policy compute engine will automatically create the required firewall rules. "It could be in the data center, in a traditional firewall, in a virtual firewall in the cloud, a native control, or a container," he says. "But we're going to look at what our security intent says, and how do we technically enforce our security policy automatically, using contextual intelligence, without human intervention. We want to remove these traditional processes that have created a gap between the speed of business and the speed of security enforcement."

Fortunately, security vendors are moving to open APIs that allow information to be exchanged and acted on from a central location. "All the major nextgen firewall vendors are providing that API fabric," Woods says. "For a centralized management product like FireMon, that's great news."

Individual firewall vendors are also getting into the policy and compliance management space, Woods says, but the vendors typically focus just on managing their own products, not those of the other vendors in the space.

The Check Point Compliance Blade product, for example, only works with Check Point products, the company says. "The vendors focus on their own products, not on the entirety of the products implemented in the enterprise infrastructure," says Firemon's Woods. "I don't think this is going to change. It's not a knock against them; they're trying to do the best job they can."