Squid 3.4.8 release notes

Squid Developers

This document contains the release notes for version 3.4 of Squid.
Squid is a WWW Cache application developed by the National Laboratory
for Applied Network Research and members of the Web Caching community.

The helper consulted after the internal OpenSSL validation, regardless of the
validation results. The helper will receive:

the origin server certificate (chain),

the intended domain name, and

a list of OpenSSL validation errors (if any).

If the helper decides to honor an OpenSSL error or report another validation
error(s), the helper will return:

A list of certificates.

A list of items consists the the validation error name (see %err_name
error page macro and %err_details code for logformat), error reason
(%ssl_lib_error macro), and the offending certificate.

The returned information mimics what the internal OpenSSL-based validation code
collects now. Returned errors, if any, are fed to sslproxy_cert_error,
triggering the existing SSL error processing code.

The helper invocation controlled by the sslcrtvalidator_program and
sslcrtvalidator_children configurations options which are similar to the
ssl_crtd related options.

This feature is a redesigned equivalent to the Squid-2.7 feature known as StoreURL-rewrite.

Notice that this is not a direct portage of the Squid-2.7 feature so behaviour
differences do exist. Although the new feature works in similar enough ways that the old
helper scripts used for Squid-2.7 are expected to work in this and later versions of Squid.

Squid traditionally uses the requested URL as an index key ID to locate objects in cache.
It is not the only key possible and the Store-ID feature exposes an API for external
helpers to provide Squid with an alternative key name for any URL.

When any client request is received which requires a cache lookup the URL is passed to
a helper specified with the store_id_program directive to check for an alternative
Store ID. This allows the helper to identify URLs which refer to duplicate resources and
de-duplicate the cache content. store_id_access is provided to allow ACL-based
tuning of which traffic gets sent to the helper and reduce overheads.

One subtle and noteworthy difference between Squid-2 and Squid-3 which is highlighted by
this feature is that refresh_pattern applies its regex argument against the Store
ID key and not the transaction URL. So using the Store-ID feature to alter the value
affects which refresh_pattern directive will be matched.

Store-ID helpers bundled with Squid can be built with the --enable-storeid-rewrite-helpers
option which is added in this version. Currently there is a file helper
provided.

The Packet Filter (PF) firewall in OpenBSD 4.4 and later offers traffic interception
using several very simple methods. One of which is the divert-to rule type
which acts as a simple routing diversion instead of performing NAT packet alterations.

The IP Firewall (IPFW) on FreeBSD 9+ contains a port of the Linux Netfilter TPROXY feature.

This version of Squid adds support for these features through the ./configure
options --enable-pf-transparent and --enable-ipfw-transparent when Squid is built on
systems with the required support. No special extras are required to enable
http_port ... tproxy configuration to work.

NOTE: To resolve NAT lookup issues on recent PF firewall versions the code behind
./configure --enable-pf-transparent has been altered and is expected to
break on the version of PF firewall shipped with BSD systems such as NetBSD and FreeBSD
which do not yet support the getsockname() API.
These systems require --with-nat-devpf to enable /dev/pf support when using PF firewall.

Previously the only annotation methods available were ICAP/eCAP HTTP header insertions
or external ACL tag= result code. Each of which had only limited possibilities
for use and little or no correlation.

It is now possible to add annotations to a client transaction from several sources:

Directly from squid.conf using the note directive with
ACL-based selection of which annotation is linked to any
particular transaction.

By configured helper processes returning a key=value pair.
The key name becomes the annotation name.

Annotations on the transaction can be passed to ICAP services or eCAP modules using the
adaptation_meta directive to send them as headers.
They can also be logged using the %note log format code in custom logs. With
the new helper response syntax changes this means all helper response key=value details
such as URL-rewrite or store-id changes, external ACL tag etc. are now able to be logged.

Annotations which are already assigned to a transaction can be checked using an ACL test
of the new note ACL type. This can match a particular note by name and value,
of for any notes with a given name.

NOTE: not all helper interfaces are yet enabled to convert key=value into annotations
and the external ACL interface does not yet send annotations to the helper.

The dns_multicast_local directive must be set to on to enable this
feature.

The multicast DNS group IP addresses for IPv4 and IPv6 resolving are added to the set
of available DNS resolvers and used automatically for domain names ending in .local
and reverse-DNS lookups before attempting a secondary resolution on the configured
resolvers. Domains without .local are resolved using only the configured resolvers.

Statistics for multicast DNS resolution can be found on the idns cache manager
report.

NOTE that the external DNS helper interface is now deprecated and has been
removed from future Squid versions. Any installations still using it for local hostname
resolution need to upgrade to mDNS resolution with this Squid version.

Whether Squid supports directive parameters with spaces, quotes, and other
special characters. Surround such parameters with "double quotes" and
also set this directive on/off around the relevant squid.conf line(s)
making use of such quoting.

dns_multicast_local

Use multicast DNS for .local domains and reverse-DNS resolution.

note

Use ACLs to annotate a transaction with customized annotations
which can be logged in access.log

spoof_client_ip

Access control to determine whether to disable the TPROXY spoofing on upstream traffic.

sslcrtvalidator_children

Specifies the settings for how many SSL server certificate
validator helpers are run and when they are started.

sslcrtvalidator_program

Specifies the location of a SSL server certificate validator helper.

store_id_access

Whether the URL for a given request is passed to the Store-ID helper process.
Used to improve StoreID performance by quickly eliminating helper delays using ACL tests.

Ported equivalent to storeurl_access from 2.7

store_id_bypass

Whether the StoreID helper may be bypassed when overloaded.

store_id_children

Controls the number of StoreID helper processes.

Options startup=N, idle=N, concurrency=N

startup=N allow finer tuning of how many helpers are started initially.

idle=N allow fine tuning of how many helper to retain as buffer against sudden traffic loads.

concurrency=N was previously called url_rewrite_concurrency as a distinct directive.

store_id_rewrite_program

A helper program to provide cache storage internal key ID value for a request.

New option buffer-size= to specify how large the log buffer
for this log is to be when buffered_logs is enabled.

New option on-error= to specify what handling is to be done
if the logging module encounters a non-recoverable error writing logs.
With the value die (the default) Squid halts operation.
With the value drop Squid drops log lines and continue running.

Support IPv6 for intercept mode. Requires ip6tables support on Linux,
PF support on OpenBSD and IPFW support on FreeBSD. Squid will no longer complain
about misconfiguration if IPv6 support is missing, we now rely on the firewall
tools reporting misconfiguration when the NAT rules are created.

Support tproxy mode traffic on BSD systems with BINDANY support
(OpenBSD 5+, FreeBSD 9+ so far).

New option to control which Store-ID helpers are built. As with other
helper options use --disable-* to prevent any helpers building and
omit to get all helper auto-detected.

Currenly only a helper using file for backend is provided.

--disable-arch-native

New option to disable use of -march=native compiler flag.

The new flag auto-enables CPU-specific optimizations in GCC and is
required by Clang++ v3.2 for correct 64-bit environment detection.
It does not always work well however, so this build option is provided
to remove it when necessary.

--with-nat-devpf

New option to alter the behaviour of http_port ... intercept option
in squid.conf.

When this option is used Squid performs the /dev/pf lookups required to
support PF rdr-to rules. Otherwise Squid will perform perform the
getsockname() API calls to support PF divert-to rules.

NOTE: systems such as NetBSD and FreeBSD which do not yet support
the getsockname() API in recent PF versions require this option.

NAT table support updated to use the getsockname() API provided by the
latest PF versions divert-to. This allows http_port
in squid.conf to support both intercept and tproxy traffic
and to silence NAT lookup failure messages on recent BSD.

NOTE: systems such as NetBSD and FreeBSD which do not yet support
the getsockname() API in recent PF versions require --with-nat-devpf
to re-enable /dev/pf support when using PF firewall.

--disable-translation

Default changed to prevent translating error page templates during build.
Use --enable-translation to explicitly build and install the templates.