Non intrusive sslbump for whitelisting (asked many times but..)

Non intrusive sslbump for whitelisting (asked many times but..)

Hi all,

## Intro

I read many blogs and emails on this list related to what I'm trying to
do, but most go into bumping or do things that are not as simple as I'm
trying to achieve.

I have an extremely slow line, with very high latency in a remote
location. About 14 people are sharing this line. Nowadays with all the
mobile apps trying to sync and such, the line stalls to unusable all the
time.

I tried doing filters with firewall or dns level, but those are not
effective. In the end I figured squid might be my best option.

## End intro

I have squid 3.5.27 running under LEDE (OpenWrt fork), ie its
cross-compiled for a MIPS based SoC (mediatek mt7621). I mention this
because you will see some options in the config file that won't make
sense otherwise.

It works great, here's what I'm trying to achieve: Allow access only to
a pre-defined list of websites (whitelist). http is straightforward, but
if the connection is https all I need to know is domain, if its allowed,
let it pass, otherwise terminate.

this setup is working as intended with the config attached below,
however the issue I'm facing is that some servers are "loadbalanced",
this would give me the forgery error, eg:

"SECURITY ALERT: Host header forgery detected on...."

Here's a specific example, there's a corporate domain for webmail
access, and some loadbalance config makes use of different IPs, I think
this is what triggers the error. My question is, can I just ignore this
error somehow and allow the connection? From what I gather this
connection is cut by squid before it reaches the client..

Re: Non intrusive sslbump for whitelisting (asked many times but..)

This post was updated on .

>I tried doing filters with firewall or dns level, but those are not
effective.<
(dnsmasq + ipset) + iptables should do it. You most likely need
(dnsmasq+ipset) to allow traffic to multi-IP sites like google, facebook
etc.
Will work on openwrt/LEDE, too. As I am using it.
In short words: Allow only destination IPs in ipset "allowed" using iptables.
Block all other destinations using iptables.
dnsmasq on your openwrt/LEDE is the DNS-server for the clients, configured to store resolved _AND_ _allowed_IPs of facebook etc. into ipset "allowed".

Re: Non intrusive sslbump for whitelisting (asked many times but..)

> Hi all,
>
>
> ## Intro
>
> I read many blogs and emails on this list related to what I'm trying to
> do, but most go into bumping or do things that are not as simple as I'm
> trying to achieve.
>
> I have an extremely slow line, with very high latency in a remote
> location. About 14 people are sharing this line. Nowadays with all the
> mobile apps trying to sync and such, the line stalls to unusable all the
> time.
>
> I tried doing filters with firewall or dns level, but those are not
> effective. In the end I figured squid might be my best option.
>
> ## End intro
>
>
> I have squid 3.5.27 running under LEDE (OpenWrt fork), ie its
> cross-compiled for a MIPS based SoC (mediatek mt7621). I mention this
> because you will see some options in the config file that won't make
> sense otherwise.
>

NP: That should not be making much difference to the squid.conf
settings. The worst limitations such devices impose are things that
should be solved by OS settings outside of squid.conf. eg the cache.log
going to a pipe for remote logging instead of a filename, and
system-level FD limits.

> It works great, here's what I'm trying to achieve: Allow access only to
> a pre-defined list of websites (whitelist). http is straightforward, but
> if the connection is https all I need to know is domain, if its allowed,
> let it pass, otherwise terminate.
>
> this setup is working as intended with the config attached below,
> however the issue I'm facing is that some servers are "loadbalanced",
> this would give me the forgery error, eg:
>
> "SECURITY ALERT: Host header forgery detected on...."
>

> Here's a specific example, there's a corporate domain for webmail
> access, and some loadbalance config makes use of different IPs, I think
> this is what triggers the error. My question is, can I just ignore this
> error somehow and allow the connection? From what I gather this
> connection is cut by squid before it reaches the client..

Squid default behaviour is to allow the connection only to the same
IP:port the client was connecting to. If that is not working your
network configuration is screwed up. Specifically your routing or NAT.

NAT of the dst-IP:port *MUST NOT* happen on any device between the
client machine and the proxy machine. Squid needs access directly to the
kernel NAT records of the device doing that NAT operation. So it can
only happen on the Squid device.
You must *route* the packets unchanged to the Squid device (possibly
over a tunnel if necessary).

Rather than allowing unlimited access to anyone on the Internet to use
your limited bandwidth outbound connection for access to port 443 you
should be using the localnet ACL that restricts use of the proxy to
people on your LAN - those 14 clients you mentioned sharing the line.

[NP: It is not possible in this setup to determine what remote users are
abusing your proxy. All traffic logs from your firewall etc will show
Squid as the client, not the remote [ab]user. Squid access.log records
you are sending to /dev/null is the *only* record of such activities.]

To make your whitelists have any effect replace the above "allow
ssl_ports" line with a "deny !localnet" line.

If that change causes issues then your whitelists are incorrect /
incomplete. You then need the (currently discarded) access.log and/or
cache.log data to solve the issue properly.

That seems fine. The problem is not part of this _config_. If you are
having any SSL-Bump issues please try a build of the latest Squid-4. It
may be related to bugs in Squid-3 SSL-Bump or modern TLS things Squid-3
cannot cope with - there is a growing list of those.

>
> cache deny all

In the latest Squid-3 use "store_miss deny all" instead of the above.

> access_log none

The above is fine if you are certain of the squid.conf working 100%
properly. But since you are debugging issues you may need those
transaction details.

NP: access.log can be logged to syslog or a TCP pipe by Squid. To
deliver the log content externally for normal audit purposes instead of
using space on the device.

> cache_log /dev/null

You *need* the information logged here. By default only the most
operationally critical errors are recorded.

NP: the cache.log can usually be a Unix-pipe delivering data to a remote
server if the local machine is constrained.

> cache_store_log stdio:/dev/null

Above line is *actively* harmful. The Squid-3 default is not to waste
cycles logging *unless* you enter something like the above in
squid.conf. The above makes Squid allocate device resources to logging
that data to /dev/null.

> logfile_rotate 0
>
> logfile_daemon /dev/null

/dev/null is not a valid application filename.

Build your Squid with --disable-logfile-daemon.

> coredump_dir /tmp/squid
> visible_hostname main_Firewall

The *visible* hostname is the domain delivered to clients and denied
parties in the URLs to fetch error message data and FTP icons from
Squid. It needs to be a valid FQDN.

My particular case is a single server only, a corporate email server.
This server is publicly accessible from internet (and has a valid signed
SSL cert), now, on the remote location, there's a VPN setup that
redirects access to the mail server to a private IP, eg 10.x.x.x (and
this differs depending on loadbalance decision).

Without squid, I can connect to webmail, but with squid I get the
forgery error. Does the EDNS0 fix this? See its almost working exactly
as I need now, except for access to this single domain.. so if there's a
workaround (even if it requires a recompile) to ignore this single
domain do let me know.

2.

> NAT of the dst-IP:port *MUST NOT* happen on any device between the
> client machine and the proxy machine. Squid needs access directly to the
> kernel NAT records of the device doing that NAT operation. So it can
> only happen on the Squid device.
> You must *route* the packets unchanged to the Squid device (possibly
> over a tunnel if necessary).

It happens on the same device (LEDE/OpenWrt router where squid is
running), so the router is configured to intercept http (80) and https
(443) traffic and redirect it to squid's ports:
80 ---> 3129
443 --> 3130

3.

> Rather than allowing unlimited access to anyone on the Internet to use
> your limited bandwidth outbound connection for access to port 443 you
> should be using the localnet ACL that restricts use of the proxy to
> people on your LAN - those 14 clients you mentioned sharing the line.
>
> [NP: It is not possible in this setup to determine what remote users are
> abusing your proxy. All traffic logs from your firewall etc will show
> Squid as the client, not the remote [ab]user. Squid access.log records
> you are sending to /dev/null is the *only* record of such activities.]
>
>

I think I didn't word my earlier email properly, apologies for not being
clear. No one from the internet has access to squid, the listening ports
are not open to public, only accessible from LAN.

With abuse I meant the 14 users.. you know nowadays with mobiles/tablets
and all the apps and syncing, I only allow ports 443 and 80 (and those
are intercepted and forwarded to squid). All other ports are blocked.
The bandwidth available is extremely scarce and hence why I'm setting
this up.

4.

> To make your whitelists have any effect replace the above "allow
> ssl_ports" line with a "deny !localnet" line.

When I do this, it doesn't work anymore. I get "Your connection is not
secure" from firefox, and since google has HSTS, I can't "ignore and
proceed". The squid access log shows (not .google.com is in whitelist.txt):

Once I switch back to "allow SSL_ports" I can connect (squid splices'
the connection, no complaints from firefox).

5.

I followed your comments about the config changes: changed acls to match
original config in upper case. Swapped the port numbers, but about
having my ssl-bump match on 3129, pls check my new one see if I did it
right.

> On 08/11/17 12:18, A. Benz wrote:
>> Hi all,
>>
>>
>> ## Intro
>>
>> I read many blogs and emails on this list related to what I'm trying
>> to do, but most go into bumping or do things that are not as simple as
>> I'm trying to achieve.
>>
>> I have an extremely slow line, with very high latency in a remote
>> location. About 14 people are sharing this line. Nowadays with all the
>> mobile apps trying to sync and such, the line stalls to unusable all
>> the time.
>>
>> I tried doing filters with firewall or dns level, but those are not
>> effective. In the end I figured squid might be my best option.
>>
>> ## End intro
>>
>>
>> I have squid 3.5.27 running under LEDE (OpenWrt fork), ie its
>> cross-compiled for a MIPS based SoC (mediatek mt7621). I mention this
>> because you will see some options in the config file that won't make
>> sense otherwise.
>>
>
> NP: That should not be making much difference to the squid.conf
> settings. The worst limitations such devices impose are things that
> should be solved by OS settings outside of squid.conf. eg the cache.log
> going to a pipe for remote logging instead of a filename, and
> system-level FD limits.
>
>
>> It works great, here's what I'm trying to achieve: Allow access only
>> to a pre-defined list of websites (whitelist). http is
>> straightforward, but if the connection is https all I need to know is
>> domain, if its allowed, let it pass, otherwise terminate.
>>
>> this setup is working as intended with the config attached below,
>> however the issue I'm facing is that some servers are "loadbalanced",
>> this would give me the forgery error, eg:
>>
>> "SECURITY ALERT: Host header forgery detected on...."
>>
>
> The workarounds and gotcha's listed at
> <https://wiki.squid-cache.org/KnowledgeBase/HostHeaderForgery> are the
> best you can hope for there. The most successful all-round solution is
> to increase EDNS0 capabilities.
>
>
>> Here's a specific example, there's a corporate domain for webmail
>> access, and some loadbalance config makes use of different IPs, I
>> think this is what triggers the error. My question is, can I just
>> ignore this error somehow and allow the connection? From what I gather
>> this connection is cut by squid before it reaches the client..
>
> Squid default behaviour is to allow the connection only to the same
> IP:port the client was connecting to. If that is not working your
> network configuration is screwed up. Specifically your routing or NAT.
>
> NAT of the dst-IP:port *MUST NOT* happen on any device between the
> client machine and the proxy machine. Squid needs access directly to the
> kernel NAT records of the device doing that NAT operation. So it can
> only happen on the Squid device.
> You must *route* the packets unchanged to the Squid device (possibly
> over a tunnel if necessary).
>
>
>>
>> Also if there's anything else obviously wrong with my setup please let
>> me know.
>>
>> Many thanks.
>>
>>
>> Here's my config:
>>
>>
>> ### squid.conf begin
>>
>> acl localnet src 10.0.0.0/8
>> acl localnet src 172.16.0.0/12
>> acl localnet src 192.168.0.0/16
>>
>> acl ssl_ports port 443
>>
>> acl safe_ports port 80
>> acl safe_ports port 443
>> acl connect method connect
>
> NP: the default above ACL names are case-sensitive and some of them
> involve built-in default values which you are preventing having any
> effect by using custom lower-case ACL names.
>
>
>> acl http_whitelist dstdomain "/etc/squid/whitelist.txt"
>> acl https_whitelist ssl::server_name "/etc/squid/whitelist.txt"
>> acl ips_whitelist dst "/etc/squid/ips.txt"
>>
>> http_port 3128 intercept
>> http_port 3129
>
> Port 3128 is registered for forward-proxy traffic. Ideally you would
> have those lines reversed like so:
>
> http_port 3128
> http_port 3129 intercept
>
> ... with the corresponding NAT change for the intercept port.
>
> Also, to have your SSL-Bump whitelists applied to forward-proxy CONNECT
> traffic you should have ssl-bump settings on that 3128 forward-proxy
> port matching those on the port 3130 line.
>
>
>>
>> http_access deny !safe_ports
>> http_access deny connect !ssl_ports
>
>> http_access allow ssl_ports
>
> Rather than allowing unlimited access to anyone on the Internet to use
> your limited bandwidth outbound connection for access to port 443 you
> should be using the localnet ACL that restricts use of the proxy to
> people on your LAN - those 14 clients you mentioned sharing the line.
>
> [NP: It is not possible in this setup to determine what remote users are
> abusing your proxy. All traffic logs from your firewall etc will show
> Squid as the client, not the remote [ab]user. Squid access.log records
> you are sending to /dev/null is the *only* record of such activities.]
>
>
> To make your whitelists have any effect replace the above "allow
> ssl_ports" line with a "deny !localnet" line.
>
> If that change causes issues then your whitelists are incorrect /
> incomplete. You then need the (currently discarded) access.log and/or
> cache.log data to solve the issue properly.
>
>
>> http_access allow http_whitelist
>> http_access allow ips_whitelist
>> http_access deny all
>>
>> https_port 3130 intercept ssl-bump \
>> cert=/etc/squid/myCA.pem \
>> generate-host-certificates=off dynamic_cert_mem_cache_size=4MB
>>
>> acl step1 at_step SslBump1
>> acl step2 at_step SslBump2
>> acl step3 at_step SslBump3
>>
>> ssl_bump peek step1 all
>> ssl_bump splice https_whitelist
>> ssl_bump splice ips_whitelist
>> ssl_bump terminate all
>>
>
> That seems fine. The problem is not part of this _config_. If you are
> having any SSL-Bump issues please try a build of the latest Squid-4. It
> may be related to bugs in Squid-3 SSL-Bump or modern TLS things Squid-3
> cannot cope with - there is a growing list of those.
>
>>
>> cache deny all
>
> In the latest Squid-3 use "store_miss deny all" instead of the above.
>
>> access_log none
>
> The above is fine if you are certain of the squid.conf working 100%
> properly. But since you are debugging issues you may need those
> transaction details.
>
> NP: access.log can be logged to syslog or a TCP pipe by Squid. To
> deliver the log content externally for normal audit purposes instead of
> using space on the device.
>
>> cache_log /dev/null
>
> You *need* the information logged here. By default only the most
> operationally critical errors are recorded.
>
> NP: the cache.log can usually be a Unix-pipe delivering data to a remote
> server if the local machine is constrained.
>
>> cache_store_log stdio:/dev/null
>
> Above line is *actively* harmful. The Squid-3 default is not to waste
> cycles logging *unless* you enter something like the above in
> squid.conf. The above makes Squid allocate device resources to logging
> that data to /dev/null.
>
>> logfile_rotate 0
>>
>> logfile_daemon /dev/null
>
> /dev/null is not a valid application filename.
>
> Build your Squid with --disable-logfile-daemon.
>
>> coredump_dir /tmp/squid
>> visible_hostname main_Firewall
>
> The *visible* hostname is the domain delivered to clients and denied
> parties in the URLs to fetch error message data and FTP icons from
> Squid. It needs to be a valid FQDN.
>
> Amos
> _______________________________________________
> squid-users mailing list
> [hidden email]> http://lists.squid-cache.org/listinfo/squid-users

Re: Non intrusive sslbump for whitelisting (asked many times but..)

Administrator

On 09/11/17 11:34, A. Benz wrote:

> Hi Amos,
>
> Many thanks for your detailed reply.
>
> I have modified the config following your comments, before you see the
> new config (attached below), pls let me know your thoughts on the
> following:
>
> 1.
>
> > The workarounds and gotcha's listed at
> > <https://wiki.squid-cache.org/KnowledgeBase/HostHeaderForgery> are the
> > best you can hope for there. The most successful all-round solution is
> > to increase EDNS0 capabilities.
>
> My particular case is a single server only, a corporate email server.
> This server is publicly accessible from internet (and has a valid signed
> SSL cert), now, on the remote location, there's a VPN setup that
> redirects access to the mail server to a private IP, eg 10.x.x.x (and
> this differs depending on loadbalance decision).

Do you mean the VPN exit point has that 10/8 IP address? or that the
traffic from the client is altered to be going to that IP before it
reaches Squid?

The latter is broken because it destroys the original dst-IP values on
the TCP connection. Which Squid needs to setup the server connection.

>
> Without squid, I can connect to webmail, but with squid I get the
> forgery error. Does the EDNS0 fix this? See its almost working exactly
> as I need now, except for access to this single domain.. so if there's a
> workaround (even if it requires a recompile) to ignore this single
> domain do let me know.
>

EDNS0 fixes problems with services that load balance by rotating the IP
addresses delivered in response to A/AAAA queries, possibly omitting
some records if the final few don't fit. That results in Squid getting
different IPs occasionally than the one the client is using. EDNS0
extends the available DNS response packet size to fit all the records so
Squid can see them all even when a large set is rotating.

There are a few major hosting providers that have that behaviour in
their DNS. If you have not hit it yet you are very lucky.

>
> 2.
>
> > NAT of the dst-IP:port *MUST NOT* happen on any device between the
> > client machine and the proxy machine. Squid needs access directly to the
> > kernel NAT records of the device doing that NAT operation. So it can
> > only happen on the Squid device.
> > You must *route* the packets unchanged to the Squid device (possibly
> > over a tunnel if necessary).
>
> It happens on the same device (LEDE/OpenWrt router where squid is
> running), so the router is configured to intercept http (80) and https
> (443) traffic and redirect it to squid's ports:
> 80 ---> 3129
> 443 --> 3130
>
>
> 3.
>
> > Rather than allowing unlimited access to anyone on the Internet to use
> > your limited bandwidth outbound connection for access to port 443 you
> > should be using the localnet ACL that restricts use of the proxy to
> > people on your LAN - those 14 clients you mentioned sharing the line.
> >
> > [NP: It is not possible in this setup to determine what remote users are
> > abusing your proxy. All traffic logs from your firewall etc will show
> > Squid as the client, not the remote [ab]user. Squid access.log records
> > you are sending to /dev/null is the *only* record of such activities.]
> >
> >
>
> I think I didn't word my earlier email properly, apologies for not being
> clear. No one from the internet has access to squid, the listening ports
> are not open to public, only accessible from LAN.

If for any reason those firewall rules change in unexpected ways or
don't block something you expect to be blocked this may leave a security
hole open. It does not seem to be necessary really, so best to close.

>
> With abuse I meant the 14 users.. you know nowadays with mobiles/tablets
> and all the apps and syncing, I only allow ports 443 and 80 (and those
> are intercepted and forwarded to squid). All other ports are blocked.
> The bandwidth available is extremely scarce and hence why I'm setting
> this up.

The point I was trying to emphasize is that your Squid is accepting
*anything* in those port 443 connections.

>
> 4.
>
> > To make your whitelists have any effect replace the above "allow
> > ssl_ports" line with a "deny !localnet" line.
>
> When I do this, it doesn't work anymore. I get "Your connection is not
> secure" from firefox, and since google has HSTS, I can't "ignore and
> proceed". The squid access log shows (not .google.com is in whitelist.txt):

HSTS requires a header "Strict-Transport-Security" to be delivered from
servers before it takes effect. You can erase that header from replies
going through Squid with the reply_header_access directive. Current
Squid should be doing it automatically.
There may be issues with HSTS if the header is received before the
device connects to your network, or if it arrives over an uncontrolled
CONNECT tunnel. But there is not much to be done about those cases.

From my place: mail.amosprivateserver.org > 10.x.x.x
(corporate network, accessible only from within the place).

Anyways no worries about this! I decided to make an exception in
the redirect rule, so that if the outgoing traffic matches the IP
10.x.x.x then the firewall will not redirect the traffic to squid
and instead establish a connection directly.

This is not ideal, but it works.

2.

If for any reason those firewall rules
change in unexpected ways or don't block something you expect to
be blocked this may leave a security hole open. It does not seem
to be necessary really, so best to close.

Then I can't get any https work per my whitelist (google for
example complains of HSTS..etc) There must be another thing messed
up in my config then?

3.

The point I was trying to emphasize is
that your Squid is accepting *anything* in those port 443
connections.

HSTS requires a header "Strict-Transport-Security" to be
delivered from servers before it takes effect. You can erase
that header from replies going through Squid with the
reply_header_access directive. Current Squid should be doing it
automatically.
There may be issues with HSTS if the header is received before
the device connects to your network, or if it arrives over an
uncontrolled CONNECT tunnel. But there is not much to be done
about those cases.

I'm not sure I follow (please bare with me!), see I'm simply
trying to prevent abuse by only allowing connections to whats in
whitelist, is my approach in my current config doing that? If
there's a better way please tell me. This is going over a
satellite connection with extremely poor latency and bandwidth, so
Squid's job is to stop the connections from reaching out, and only
allowing whats in whitelist.

4.

That means the server that HTTPS
connection is attempting to reach is not on your whitelist. It
is therefore one of the things you wanted to be blocked
according to your stated policy.

- as I mentioned earlier, if that causes any issues your
whitelist is incomplete.

But it is, eg whitelist.txt has .google.com, shouldn't google.com
be accessible then (without complaining about HSTS, since I'm
splicing?). Note if I reverse the commented out lines, as below,
then it works:

As I mentioned no worries about the forgery error as I'm simply
adding an exception for that to circumvent squid, now I simply
need to polish up my squid.conf to only allow whats in whitelist
(while not consuming any unnecessary bandwidth).

My particular case is a single server only, a corporate email
server. This server is publicly accessible from internet (and
has a valid signed SSL cert), now, on the remote location,
there's a VPN setup that redirects access to the mail server to
a private IP, eg 10.x.x.x (and this differs depending on
loadbalance decision).

Do you mean the VPN exit point has that 10/8 IP address? or that
the traffic from the client is altered to be going to that IP
before it reaches Squid?

The latter is broken because it destroys the original dst-IP
values on the TCP connection. Which Squid needs to setup the
server connection.

Without squid, I can connect to webmail, but with squid I get
the forgery error. Does the EDNS0 fix this? See its almost
working exactly as I need now, except for access to this single
domain.. so if there's a workaround (even if it requires a
recompile) to ignore this single domain do let me know.

EDNS0 fixes problems with services that load balance by rotating
the IP addresses delivered in response to A/AAAA queries, possibly
omitting some records if the final few don't fit. That results in
Squid getting different IPs occasionally than the one the client
is using. EDNS0 extends the available DNS response packet size to
fit all the records so Squid can see them all even when a large
set is rotating.

There are a few major hosting providers that have that behaviour
in their DNS. If you have not hit it yet you are very lucky.

2.

> NAT of the dst-IP:port *MUST NOT* happen on any device
between the
> client machine and the proxy machine. Squid needs access
directly to the
> kernel NAT records of the device doing that NAT operation.
So it can
> only happen on the Squid device.
> You must *route* the packets unchanged to the Squid
device (possibly
> over a tunnel if necessary).

It happens on the same device (LEDE/OpenWrt router where squid
is running), so the router is configured to intercept http (80)
and https (443) traffic and redirect it to squid's ports:
80 ---> 3129
443 --> 3130

3.

> Rather than allowing unlimited access to anyone on the
Internet to use
> your limited bandwidth outbound connection for access to
port 443 you
> should be using the localnet ACL that restricts use of the
proxy to
> people on your LAN - those 14 clients you mentioned
sharing the line.
>
> [NP: It is not possible in this setup to determine what
remote users are
> abusing your proxy. All traffic logs from your firewall
etc will show
> Squid as the client, not the remote [ab]user. Squid
access.log records
> you are sending to /dev/null is the *only* record of such
activities.]
>
>

I think I didn't word my earlier email properly, apologies for
not being clear. No one from the internet has access to squid,
the listening ports are not open to public, only accessible from
LAN.

If for any reason those firewall rules change in unexpected ways
or don't block something you expect to be blocked this may leave a
security hole open. It does not seem to be necessary really, so
best to close.

With abuse I meant the 14 users.. you know nowadays with
mobiles/tablets and all the apps and syncing, I only allow ports
443 and 80 (and those are intercepted and forwarded to squid).
All other ports are blocked.
The bandwidth available is extremely scarce and hence why I'm
setting this up.

The point I was trying to emphasize is that your Squid is
accepting *anything* in those port 443 connections.

4.

> To make your whitelists have any effect replace the above
"allow
> ssl_ports" line with a "deny !localnet" line.

When I do this, it doesn't work anymore. I get "Your connection
is not secure" from firefox, and since google has HSTS, I can't
"ignore and proceed". The squid access log shows (not
.google.com is in whitelist.txt):

HSTS requires a header "Strict-Transport-Security" to be delivered
from servers before it takes effect. You can erase that header
from replies going through Squid with the reply_header_access
directive. Current Squid should be doing it automatically.
There may be issues with HSTS if the header is received before the
device connects to your network, or if it arrives over an
uncontrolled CONNECT tunnel. But there is not much to be done
about those cases.

Once I switch back to "allow SSL_ports" I can connect (squid
splices' the connection, no complaints from firefox).

That means the server that HTTPS connection is attempting to reach
is not on your whitelist. It is therefore one of the things you
wanted to be blocked according to your stated policy.

- as I mentioned earlier, if that causes any issues your
whitelist is incomplete.

5.

I followed your comments about the config changes: changed acls
to match original config in upper case. Swapped the port
numbers, but about having my ssl-bump match on 3129, pls check
my new one see if I did it right.

I read many blogs and emails on this list related to what
I'm trying to do, but most go into bumping or do things that
are not as simple as I'm trying to achieve.

I have an extremely slow line, with very high latency in a
remote location. About 14 people are sharing this line.
Nowadays with all the mobile apps trying to sync and such,
the line stalls to unusable all the time.

I tried doing filters with firewall or dns level, but those
are not effective. In the end I figured squid might be my
best option.

## End intro

I have squid 3.5.27 running under LEDE (OpenWrt fork), ie
its cross-compiled for a MIPS based SoC (mediatek mt7621). I
mention this because you will see some options in the config
file that won't make sense otherwise.

NP: That should not be making much difference to the
squid.conf settings. The worst limitations such devices impose
are things that should be solved by OS settings outside of
squid.conf. eg the cache.log going to a pipe for remote
logging instead of a filename, and system-level FD limits.

It works great, here's what I'm trying
to achieve: Allow access only to a pre-defined list of
websites (whitelist). http is straightforward, but if the
connection is https all I need to know is domain, if its
allowed, let it pass, otherwise terminate.

this setup is working as intended with the config attached
below, however the issue I'm facing is that some servers are
"loadbalanced", this would give me the forgery error, eg:

Here's a specific example, there's a
corporate domain for webmail access, and some loadbalance
config makes use of different IPs, I think this is what
triggers the error. My question is, can I just ignore this
error somehow and allow the connection? From what I gather
this connection is cut by squid before it reaches the
client..

Squid default behaviour is to allow the connection only to the
same IP:port the client was connecting to. If that is not
working your network configuration is screwed up. Specifically
your routing or NAT.

NAT of the dst-IP:port *MUST NOT* happen on any device between
the client machine and the proxy machine. Squid needs access
directly to the kernel NAT records of the device doing that
NAT operation. So it can only happen on the Squid device.
You must *route* the packets unchanged to the Squid device
(possibly over a tunnel if necessary).

Port 3128 is registered for forward-proxy traffic. Ideally you
would have those lines reversed like so:

http_port 3128
http_port 3129 intercept

... with the corresponding NAT change for the intercept port.

Also, to have your SSL-Bump whitelists applied to
forward-proxy CONNECT traffic you should have ssl-bump
settings on that 3128 forward-proxy port matching those on the
port 3130 line.

http_access deny !safe_ports
http_access deny connect !ssl_ports

http_access allow ssl_ports

Rather than allowing unlimited access to anyone on the
Internet to use your limited bandwidth outbound connection for
access to port 443 you should be using the localnet ACL that
restricts use of the proxy to people on your LAN - those 14
clients you mentioned sharing the line.

[NP: It is not possible in this setup to determine what remote
users are abusing your proxy. All traffic logs from your
firewall etc will show Squid as the client, not the remote
[ab]user. Squid access.log records you are sending to
/dev/null is the *only* record of such activities.]

To make your whitelists have any effect replace the above
"allow ssl_ports" line with a "deny !localnet" line.

If that change causes issues then your whitelists are
incorrect / incomplete. You then need the (currently
discarded) access.log and/or cache.log data to solve the issue
properly.

That seems fine. The problem is not part of this _config_. If
you are having any SSL-Bump issues please try a build of the
latest Squid-4. It may be related to bugs in Squid-3 SSL-Bump
or modern TLS things Squid-3 cannot cope with - there is a
growing list of those.

cache deny all

In the latest Squid-3 use "store_miss deny all" instead of the
above.

access_log none

The above is fine if you are certain of the squid.conf working
100% properly. But since you are debugging issues you may need
those transaction details.

NP: access.log can be logged to syslog or a TCP pipe by Squid.
To deliver the log content externally for normal audit
purposes instead of using space on the device.

cache_log /dev/null

You *need* the information logged here. By default only the
most operationally critical errors are recorded.

NP: the cache.log can usually be a Unix-pipe delivering data
to a remote server if the local machine is constrained.

cache_store_log stdio:/dev/null

Above line is *actively* harmful. The Squid-3 default is not
to waste cycles logging *unless* you enter something like the
above in squid.conf. The above makes Squid allocate device
resources to logging that data to /dev/null.

logfile_rotate 0

logfile_daemon /dev/null

/dev/null is not a valid application filename.

Build your Squid with --disable-logfile-daemon.

coredump_dir /tmp/squid
visible_hostname main_Firewall

The *visible* hostname is the domain delivered to clients and
denied parties in the URLs to fetch error message data and FTP
icons from Squid. It needs to be a valid FQDN.

Re: Non intrusive sslbump for whitelisting (asked many times but..)

Administrator

On 11/11/17 01:05, A. Benz wrote:

> Hi Amos,
>
> Thanks for your continued support.
>
> 1.
>
>> Do you mean the VPN exit point has that 10/8 IP address? or that the
>> traffic from the client is altered to be going to that IP before it
>> reaches Squid?
>>
>> The latter is broken because it destroys the original dst-IP values on
>> the TCP connection. Which Squid needs to setup the server connection.
>
> Let me put it as an example:
>
> From the normal internet: mail.amosprivateserver.org > publicly
> accessible IP.
>
> From my place: mail.amosprivateserver.org > 10.x.x.x (corporate
> network, accessible only from within the place).
>
> Anyways no worries about this! I decided to make an exception in the
> redirect rule, so that if the outgoing traffic matches the IP 10.x.x.x
> then the firewall will not redirect the traffic to squid and instead
> establish a connection directly.
>
> This is not ideal, but it works.
>

Or have Squid relay everything through the same server(s) and
the server do the distinguishing between traffic and just relay
everythign to the same

>
> 2.
>
>> If for any reason those firewall rules change in unexpected ways or
>> don't block something you expect to be blocked this may leave a
>> security hole open. It does not seem to be necessary really, so best
>> to close.
>
> But if I put the lines you told me:
>
> # http_access allow SSL_ports (commented out as you advised)
> http_access deny !localnet
>
> Then I can't get any https work per my whitelist (google for example
> complains of HSTS..etc) There must be another thing messed up in my
> config then?
>

Seems so.

>
> 3.
>
>> The point I was trying to emphasize is that your Squid is accepting
>> *anything* in those port 443 connections.
>>
>> HSTS requires a header "Strict-Transport-Security" to be delivered
>> from servers before it takes effect. You can erase that header from
>> replies going through Squid with the reply_header_access directive.
>> Current Squid should be doing it automatically.
>> There may be issues with HSTS if the header is received before the
>> device connects to your network, or if it arrives over an uncontrolled
>> CONNECT tunnel. But there is not much to be done about those cases.
> I'm not sure I follow (please bare with me!), see I'm simply trying to
> prevent abuse by only allowing connections to whats in whitelist, is my
> approach in my current config doing that? If there's a better way please
> tell me. This is going over a satellite connection with extremely poor
> latency and bandwidth, so Squid's job is to stop the connections from
> reaching out, and only allowing whats in whitelist.
>

HSTS is enabled on if the client receives a header called
"Strict-Transport-Security". And only for a time indicated by the header
itself.

So removing that header prevents HSTS being turned on in the client.

However the client can receive that header via any one of several
different protocols (eg. HTTP/2, SPDY, QUIC, and HTTPS if you are
splicing any of it). So preventing HSTS breaking things randomly
requires they be blocked and only the filtered traffic allowed.

>
> 4.
>
>> That means the server that HTTPS connection is attempting to reach is
>> not on your whitelist. It is therefore one of the things you wanted to
>> be blocked according to your stated policy.
>>
>> - as I mentioned earlier, if that causes any issues your whitelist is
>> incomplete.
> But it is, eg whitelist.txt has .google.com, shouldn't google.com be
> accessible then (without complaining about HSTS, since I'm splicing?).

Splicing allows traffic to go through without the HSTS header removal.
So the above (3) issues from HSTS maybe still happening.

As a general rule of thumb, if you are splicing a domain never bump it
and if you are wanting to bump it never splice it nor allow any other
ways to reach it except through the bumping proxy.

It is technically possible to transition a domain from being
spliced/relayed to bumped, but it can be a painful process involving a
lot of unavoidable TLS related "errors" for clients during the
transition timeout dictated by the HSTS header and/or similar things.

As to the whitelist 'not working'. Earlier you posted a access.log entry
that contained:

Note how that IP address is *not* a domain in *.google.com.
Even its reverse-DNS does not match *.google.com.

Googles rDNS server names are in *.1e100.net domain, not *.google.com.
They also have a lot of other service name crossovers like "YouTube"
actually being *.googlevideos.com, and gmail actually being
docs.google.com sometimes (heh!).

To access any service with SSL-Bump your whitelist needs to allow its
public domain name(s), reverse-DNS server name(s), and maybe also raw-IP
address(s). That goes for any service, not just Google's.

And ... that rule of thumb I mention above applies to the entire bunch
as a collective service. That is where major corps like Google cause
headaches simply by having so many inter-related pieces with crossover
like gmail and youtube domains vs the 1e100.net reverse-DNS.

Re: Non intrusive sslbump for whitelisting (asked many times but..)

Administrator

On 11/11/17 14:03, Amos Jeffries wrote:

> On 11/11/17 01:05, A. Benz wrote:
>> Hi Amos,
>>
>> Thanks for your continued support.
>>
>> 1.
>>
>>> Do you mean the VPN exit point has that 10/8 IP address? or that the
>>> traffic from the client is altered to be going to that IP before it
>>> reaches Squid?
>>>
>>> The latter is broken because it destroys the original dst-IP values
>>> on the TCP connection. Which Squid needs to setup the server connection.
>>
>> Let me put it as an example:
>>
>> From the normal internet: mail.amosprivateserver.org > publicly
>> accessible IP.
>>
>> From my place: mail.amosprivateserver.org > 10.x.x.x (corporate
>> network, accessible only from within the place).
>>
>> Anyways no worries about this! I decided to make an exception in the
>> redirect rule, so that if the outgoing traffic matches the IP 10.x.x.x
>> then the firewall will not redirect the traffic to squid and instead
>> establish a connection directly.
>>
>> This is not ideal, but it works.
>>
>
> Or have Squid relay everything through the same server(s) and
> the server do the distinguishing between traffic and just relay
> everythign to the same
>

Damn that sounds daft.

What I meant to write was:

Or have Squid relay everything through the same server(s) and
the server do the distinguishing between traffic .

Or setup a cache_peer and have the traffic with src IP of the internal
clients going to that domain sent there.

Re: Non intrusive sslbump for whitelisting (asked many times but..)

Hi Amos,

Just wanted to follow up on this saying thanks for taking the time to reply.

Cheers.

Regards,
A. Benz

On 11/11/17 09:54, Amos Jeffries wrote:

> On 11/11/17 14:03, Amos Jeffries wrote:
>> On 11/11/17 01:05, A. Benz wrote:
>>> Hi Amos,
>>>
>>> Thanks for your continued support.
>>>
>>> 1.
>>>
>>>> Do you mean the VPN exit point has that 10/8 IP address? or that
>>>> the traffic from the client is altered to be going to that IP
>>>> before it reaches Squid?
>>>>
>>>> The latter is broken because it destroys the original dst-IP values
>>>> on the TCP connection. Which Squid needs to setup the server
>>>> connection.
>>>
>>> Let me put it as an example:
>>>
>>> From the normal internet: mail.amosprivateserver.org > publicly
>>> accessible IP.
>>>
>>> From my place: mail.amosprivateserver.org > 10.x.x.x (corporate
>>> network, accessible only from within the place).
>>>
>>> Anyways no worries about this! I decided to make an exception in the
>>> redirect rule, so that if the outgoing traffic matches the IP
>>> 10.x.x.x then the firewall will not redirect the traffic to squid
>>> and instead establish a connection directly.
>>>
>>> This is not ideal, but it works.
>>>
>>
>> Or have Squid relay everything through the same server(s) and
>> the server do the distinguishing between traffic and just relay
>> everythign to the same
>>
>
> Damn that sounds daft.
>
> What I meant to write was:
>
> Or have Squid relay everything through the same server(s) and
> the server do the distinguishing between traffic .
>
> Or setup a cache_peer and have the traffic with src IP of the internal
> clients going to that domain sent there.
>
> Amos
> _______________________________________________
> squid-users mailing list
> [hidden email]> http://lists.squid-cache.org/listinfo/squid-users