Palo Alto Application: First Packets Will Pass!

I am using an almost hidden FTP server in my DMZ behind a Palo Alto Networks firewall. FTP is only allowed from a few static IP addresses, hence no brute-force attacks on my server. Furthermore, I have an “allow ping and traceroute from any to DMZ” policy since ping is no security flaw but really helpful while troubleshooting.

Now, here comes the point: My FTP server logfile showed dozens of connections from many different IP addresses from the Internet. WHAT? For the first moment I was really shocked. Have I accidentally exposed my FTP server to the Internet? Here is what happened:

Let’s have a look at the FTP server log first (FileZilla Server). It shows many incoming connections while almost all of them aborted with syntax error. However, a few sessions at least received the welcome message:

As I searched through my traffic log in the Palo Alto firewall I found that all of these connections were initially allowed by my “Ping from Untrust to DMZ allow” rule. Note that for all allowed connections the application was “incomplete” or “insufficient-data”. All connections in which the application was correctly identified as “ftp” or something else were denied, just as I expected it for *all* connections:

To have a complete view, this was my policy at the time of those events. I allowed the following four applications (due to application dependencies and the need for pinging over IPv6): ping, traceroute, icmp, ipv6-icmp. Service was set to “any” and NOT to application-default:

What Happened?

Well, the firewall can only recognize an “application” after some packets have traversed the device. Before this application data it only sees the TCP SYN (which would fit for a traditional layer 3/4 firewall). Though I knew that, I did not merge this behaviour to my harmless looking ping/traceroute policy. Since I allowed those four applications with service “any”, every single TCP attempt into my DMZ was allowed until the real application (such as telnet, ssh, web-browsing, snmp, …) was recognized and denied by the firewall.

“When the first TCP packet is received (SYN), the firewall must setup a session. Since the application can not be detected on a TCP session until at least one data packet traverses the device, the application will be incomplete. For the firewall to determine if it should even allow the SYN packet through it must do a security policy lookup. Because the application is not known when the SYN packet is received the application portion of the security policies can not be applied.”, ref: Tips & Tricks: What is Application Dependency?

Hm, is this good or bad? Well, in order to write policies based on applications rather than services/ports, the firewall must behave like that. That’s the core of those next-gen firewall with their application detection. The problem was my policy with service “any” and not “application-default”. Furthermore, the applications “icmp” and “ipv6-icmp” do not look that trustworthy at all. I am still unsure whether someone can send *any* kind of IPv6 ICMP message into my DMZ while I am only interested in echo-requests.

Lessons Learned

Just a few things to consider when writing security policies on a Palo Alto Networks firewall:

Have a look at your policies.

Have a second look.

As well as at the traffic log.

Test your policies by legitimate scans from the outside.

Take care of the “any” service.

Be aware when using the “icmp” or “ipv6-icmp” applications.

Don’t trust yourself. ;)

Note that all of this fancy application detection is not easy to understand at all. It is not that obvious as in those “source/destination IP address and port”policies in the (good?) old days.

Oh nice, thanks, jayvee, for the hint. Haven’t used it that way until now.

However, I *really* think that Palo Alto SHOULD have an application for ipv6-ping rather than simply ipv6-icmp. Ping is quite common while ICMPv6 at all is dangerous…

(By the way: There was a reason I used “traceroute” with port “any”, because I was testing Layer-4-Traceroutes, LFT, such as increasing the TTL while using an upper layer port such as 443 rather than ping. That’s why I opened the policy to “any”, not thinking about allowing the first packets of almost everything in.)

This sounds very similar to an RDP problem I’m seeing. The client sends a SYN and the server replies with a SYN-ACK then nothing apart from re-transmissions of SYN/SYN-ACK and finally a RST from the Server, which seems to be dropped by the Palo Alto. I ran the packet capture facility through the GUI for receive, transmit, firewall and drop with a filter of the source and destination IP’s and put the output through wireshark to see the above. Policy and routing all look good.
First thing Monday morning I’m going to check the service field isn’t set to ‘any’, but my Application is set to ms-rdp.
I’ve limited experience with Palo Alto’s, so any advice would be welcome. I took the Palo Alto Firewall 9.0 essentials course (EDU 210) at the end of April and this is my first deployment.

just for testing purposes, you can add a security rule allowing everything between this source and destination IP address. Everything means: ANY application *and* ANY service (=port). Then you can test your service and see, which application Palo Alto is detecting. Finally, you can add your desired security policy.

The application from PAN for simple RDP is called “ms-rdp”. The “application-default” service selection should fit as long as it runs on the standard port 3389.