Having multiples sites with similar architecture, we would like to replicate setup for another NGFW appliances. A perfect automation case: change a few input parameters, produce rest of the typical configuration.

Ouch. CLI is not supported. Forget Python or Ansible.

Even worse: configuration templates or configuration cloning is not supported in centralized manager GUI. "Good idea, please create enhancements request", says $vendor sales managers. "And by the way we have REST API support in our centralized manager"

There’s ~30% feature parity between REST API and GUI configuration options. You still need to tweak majority of the configuration manually in manager GUI.

Even for configurable parameters not all REST API methods are supported. For example, to update an Object List to add a new host, you need to download the whole Object List, parse and modify its JSON structure, and upload the whole list with another REST API call. A single mistake and your list is gone.

Here’s the result of a long discussion with $vendor engineers:

Currently our REST API implementations is very raw. We don't recommend to use it in production. We’re accepting all your observations and will work toward improving the API. Stay tuned and check the next SW releases

Let’s not comment how our team is feeling at this moment…

Going further

So we manually (clicking madness) replicated configuration to other appliances. We’re making progress, everything is fine… until you have your first NGFW appliance hardware failure.

According to the RMA procedure, a new appliance is shipped and replaces the failed one. You set some initial settings to pair it with the centralized manager.

Configuration from the failed appliance is already in the centralized manager. You just need to sync/deploy it… but nothing happens. You get multiple error messages. Documentation is silent.

Then you get this explanation after lengthy investigation by $vendor TAC:

Currently it is not possible to Sync/Deploy config from centralized manager to replaced appliances due to some internal reasons, which we cannot disclose. As workaround please delete all configuration from manager and implement all from the scratch, as during new implementation. Please don't forget to make dozens of screen shots with existing configuration options

You could guess how our team is feeling after this revelation.

After long discussions with $vendor sales engineers they tell us: "We understand your frustration, we indeed registered some enhancements request. Based on priorities, they will be implemented"

I couldn’t resist adding…

Next step: CxO (who made the purchasing decision in the first place) will complain how expensive his internal security and networking teams are.

I also sent a draft of this blog post to my friends working in the security industry and one of them replied:

I don’t understand why some vendors actually believe that CLI’s will die. I think their vision of automation is a monkey trained to click certain sequences of GUI buttons to get to a banana…

As we all know it’s not just the vendors. So-called thought leaders with zero operational experience are no better.

But wait, that’s not all

Finally, the icing on the cake. The same reader sent me this gem a few days after the original email:

We asked the vendor: “Could you please officially share other customers’ experience with us?”

Their answer: “No, because that would affect our business”

Now you know how some players in this industry work.

Got more feedback from another unhappy customer (probably using the same gear). Here it is:

Allow me to add my experiences with $vendor and his $product (I had a look at all releases starting from 6.0 to the first 6.2 releases in the course of about 2015-2017):

Management access is utterly slow, needs an awful lot of resources to run just below an acceptable speed baseline,

The whole thing is a huge pile of bugs in runtime as well as management code,

Numerous TACs, often taking weeks to resolve one issue,

Essentially no help in $vendor forums (often discussions cease with “open a TAC”).

$vendor bought $product with the acquisition of $company some years ago and struggles since then to get the whole pile transformed to something I could recommend to customers with good faith.

On the other hand, I had contact with some techs at $vendor and they are competent and know how to diagnose problems.

The best part (for $vendor) is that $vendor charges customers money (via a so called "support contract") for allowing customers to help finding and (hopefully) fixing bugs the software shouldn't have in the first place. At least not in the given amount.

Looking at the base FW (without the shiny NG): it’s mature, has a lot less of features in comparison but is reasonably stable.

13 comments:

I understand why you might be reluctant to disclose the vendor name but it would be great if there was a way for these sort of vendor issues to be reported and shared openly. It would also be good to see the positive experiences that people have too.

Vendors constraining everyone with NDAs is not good for the industry (customers in particular).

Hah, judging from the end 6.0 and 6.2 comment and my familiarity with C --- O's NGFW product, I bet you're spot on. Although to be fair, CLI is supported from a troubleshooting/diagnosis perspective, just not config. I wish they'd take notes from Big Switch in their controller based architectures...

"My guess in this case would be that it starts with a C and ends with a t." That seems correct from our perspective, save for the last caveat described in the post: "delete all configuration from manager and implement all from the scratch". We actually could restore a configuration (didn't have to rebuild from stcatch).

Sounds like Checkpoint, in which case, maybe they can refer to this for some info on automating Checkpoint using Ansible; https://community.checkpoint.com/thread/5478-leveraging-the-r8010-api-to-automate-and-streamline-security-operations

and for the this reason we are still using the A*A firewalls for East-West data-center traffic as they are really mature and robust (with no performance penalty of DPI) and use the Fortigate for North-South traffic with too many security features like AVC , IPS , AV ....the FortiOS is really reach in CLI but lack some parity on the API side.

C***o trying to evolve from the A*A: First separate 3rd party deep inspection modules, then separate own modules, then bought big IPS vendor and mess everything up. Still separate module, half of the box has one os and cli, the other half doesn't and is managed only from external system, no one understands anything anymore. Dumped them and switched to FG.

FirePower is a nightmare.I just deleted like 5-6 lines of comment here pointing on all of the issues we had and we are having now with this "revolution".

We have enormous amount of issues using this product (and we are not using single standalone setup but enterprise customer solution) and we have asked the same:- are we the only one?

Noone will give you direct statement of course but i was pretty sure that blogs like this will come up eventually.

Fun fact: we have provided 3 A4-size pages to vendor (bugs/feature requests) like half year ago. Guess how many of these will be fixed in 6.2.3? :)

It's not even funny anymore.

My personal list of "features" - there is no logical backup!- if you have to re-register device to management, you will loose zone-to-interface bond and all L3 related configuration (even routing!)- try to ping unreachable routed host with for example 1000 repeats (you will see magic happening)- if management will accept something that device is not capable of, this results in config wipeout

The author

Ivan Pepelnjak (CCIE#1354 Emeritus), Independent Network Architect at ipSpace.net, has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced internetworking technologies since 1990.