Squid 3.1.0.4 release notes

Squid Developers

This document contains the release notes for version 3.1 of Squid.
Squid is a WWW Cache application developed by the National Laboratory
for Applied Network Research and members of the Web Caching community.

A large number of the show-stopper bugs have been fixed along with general improvements to the ICAP support.
While this release is not deemed ready for production use, we believe it is ready for wider testing by the community.

Begining with 3.1 the Squid Developers are trialling a new release numbering system.

We have decided, based on input from interested users to drop the Squid-2 terminology of
(DEVEL, PRE, RC, and STABLE) from the release package names.
These are replaced with a simpler 3-tier system based around the natural code development cycle.

Daily generated snapshots of all current versions are provided as testing (old DEVEL) and bug-fix releases.
These are numbered from their last release with a date appended.
Snapshots generated from 3.HEAD continue to be highly volatile.

Regular feature releases from Squid-3 will be branched out as sub-versions. Such as this Squid-3.1.

All this is previous policy you should be accustomed to. Now we get to the new numbering change.

Initial branch packages will be generated with a 3.X.0.Z version as testing packages.
Packages and Snapshots generated with these 3-dot numbers are expected to be relatively stable regarding feature behaviors.
Suitable for testing, but without any guarantees under production loads. This replaces both the old PRE and RC packages.

If a large number of bugs are found several *.0.Z packages may be attempted before any is considered production-ready.

When one of these Squid-3.X.0.Z packages passes our bug-free standards a 3.X.Y numbered release will be made.

We can only hope enough testing has been done to consider these ready for production use.
As always we are fully dependent on people testing the previous packages and reporting all bugs.

In support of all this are several squid-dev process changes which have been worked out over the last year.

We no longer accept new features into branches.
Those are reserved for the next feature release.
The cycle for major releases is hoped to be fast enough to suit some peoples needs for new features
and others need for stability in the branched releases.

We now audit and vote on all feature and major code additions.
Requiring at least two sets of developer eyes on any new features before they are committed to 3.HEAD.
Vastly reducing the number of bugs in all code.

Don't worry, few operational changes have been made.
Older configs from are still expected to run in 3.1 with only the usual minor
changes seen between major release. Details on those are listed below.

New users will be relieved to see a short 32-line or less squid.conf on clean installs.
Many of the options have reasonable defaults but had previously needed them explicitly configured!
These are now proper built-in defaults and no longer need to be in squid.conf unless changed.

All of the option documentation has been offloaded to another file squid.conf.documented which
contains a fully documented set of options previously cluttering up squid.conf itself.

Package maintainers are provided with a second file squid.conf.default which as always contains the default
config options provided on a clean install.

New Features for IPv6

Squid handles localhost values seperately. For the purpose of ACLs and also external
connections ::1 is considered a seperate IP from 127.0.0.1. This means all ACL which
define behaviour for localhost may need ::1/128 included.

Pinger has been upgraded to perform both ICMP and ICMPv6 as required.
As a result of this and due to a change in the binary protocol format between them,
new builds of squid are no longer backwards-compatible with old pinger binaries.
You will need to perform "make install-pinger" again after installing squid.

Peer and Client SNMP tables have been altered to handle IPv6 addresses.
As a side effect of this the long-missing fix to show seperate named peers on one IP
has been integrated. Making the SNMP peer table now produce correct output.
The table structure change is identical for both IPv4-only and Dual modes but with
IPv4-only simply not including any IPv6 entries. This means any third-party SNMP
software which hard coded the MIB paths needs to be upgraded for this Squid release.

Limitations of IPv6 Support

Specify a specific tcp_outgoing_address and the clients who match its ACL are limited
to the IPv4 or IPv6 network that address belongs to. They are not permitted over the
IPv4-IPv6 boundary. Some ACL voodoo can however be applied to explicitly route the
IPv6/IPv4 bound traffic (DIRECT access) out an appropriate interface.

WCCP is not available (neither version 1 or 2). It remains built into squid for use with IPv4 traffic but IPv6 cannot use it.

Transparent Interception is done via NAT at the OS level and is not available in IPv6.
Squid will ensure that any port set with transparent, intercept, or tproxy options be an IPv4-only
listening address. Wildcard can still be used but will not open as an IPv6.
To ensure that squid can accept IPv6 traffic on its default port, an alternative should
be chosen to handle transparently intercepted traffic.

http_port 3128
http_port 8080 intercept

The bundled NTLM Auth helper is IPv4-native between itself and the NTLM server.
A new one will be needed for IPv6 traffic between the helper and server.

The bundled RADIUS Auth helper is IPv4-native, both in traffic between and data storage
with the RADIUS server. A new helper will be needed for IPv6 RADIUS protocol.

Zero Penalty Hit created a patch to set QoS markers on outgoing traffic.

Allows you to select a TOS/Diffserv value to mark local hits.

Allows you to select a TOS/Diffserv value to mark peer hits.

Allows you to selectively mark only sibling or parent requests

Allows any HTTP response towards clients to have the TOS value of the response coming from
the remote server preserved.
For this to work correctly, you will need to patch your linux kernel with the TOS preserving ZPH patch.
The kernel patch can be downloaded from
http://zph.bratcheda.org

Allows you to mask certain bits in the TOS received from the remote server,
before copying the value to the TOS send towards clients.

Squid Configuration

Squid 3.1 needs to be configured with --enable-zph-qos for the ZPH QoS controls to be available.

The configuration options for 2.7 and 3.1 are based on different ZPH patches.
The two releases configuration differs and only the TOS mode settings are directly translatable.

qos_flows local-hit=0xff Responses found as a HIT in the local cache

qos_flows sibling-hit=0xff Responses found as a HIT in a sibling peer

qos_flows parent-hit=0xff Responses found as a HIT in a parent peer

The lines above are spearated for documentation. qos_flows may be configured with all options on one line, or separated as shown.
Also options may be repeated as many times as desired. Only the final configured value for any option will be used.

The legacy Option and Priority modes available in Squid-2.7 are no longer supported.

Squid-in-the-middle decryption and encryption of straight CONNECT and transparently redirected SSL traffic,
using configurable client- and server-side certificates.
While decrypted, the traffic can be inspected using ICAP.

This Squid version can run on Windows as a system service using the Cygwin emulation environment,
or can be compiled in Windows native mode using the MinGW + MSYS development environment. Windows NT 4 SP4 and later are supported.
On Windows 2000 and later the service is configured to use the Windows Service Recovery option
restarting automatically after 60 seconds.

Some new command line options were added for the Windows service support:

The service installation is made with -i command line switch, it's possible to use -f switch at
the same time for specify a different config-file settings for the Squid Service that will be
stored on the Windows Registry.

A new -n switch specify the Windows Service Name, so multiple Squid instance are allowed.
"Squid" is the default when the switch is not used.

So, to install the service, the syntax is:

squid -i [-f file] [-n name]

Service uninstallation is made with -r command line switch with the appropriate -n switch.

The -k switch family must be used with the appropriate -f and -n switches, so the syntax is:

squid -k command [-f file] -n service-name

where service-name is the name specified with -n options at service install time.

To use the Squid original command line, the new -O switch must be used ONCE, the syntax is:

squid -O cmdline [-n service-name]

If multiple service command line options must be specified, use quote. The -n switch is
needed only when a non default service name is in use.

Don't use the "Start parameters" in the Windows 2000/XP/2003 Service applet: they are
specific to Windows services functionality and Squid is not designed for understand they.

In the following example the command line of the "squidsvc" Squid service is set to "-D -u 3130":

The process status helper functions make it easier for you to obtain information about
processes and device drivers running on Microsoft® Windows NT®/Windows® 2000. These
functions are available in PSAPI.DLL, which is distributed in the Microsoft® Platform
Software Development Kit (SDK). The same information is generally available through the
performance data in the registry, but it is more difficult to get to it. PSAPI.DLL is
freely redistributable.

PSAPI.DLL is available only on Windows NT, 2000, XP and 2003. The implementation in Squid is
aware of this, and try to use it only on the right platform.

On Windows platforms, if no value is specified in the dns_nameservers option on
squid.conf or in the /etc/resolv.conf file, the list of DNS name servers are
taken from the Windows registry, both static and dynamic DHCP configurations
are supported.

A reasonably recent release of
Cygwin or
MinGW is needed.
The usage of the Cygwin environment is very similar to other Unix/Linux environments, and -devel version of libraries must be installed.
For the MinGW environment, the packages MSYS, MinGW and msysDTK must be installed. Some additional libraries and tools must be downloaded separately:

Whether to use any result found by follow_x_forwarded_for in further ACL processing.
Default: ON

Controls whether the indirect client address
(see follow_x_forwarded_for) is used instead of the
direct client address in acl matching.

adaptation_access

Sends an HTTP transaction to an ICAP or eCAP adaptation service.

adaptation_access service_name allow|deny [!]aclname...
adaptation_access set_name allow|deny [!]aclname...
At each supported vectoring point, the adaptation_access
statements are processed in the order they appear in this
configuration file. Statements pointing to the following services
are ignored (i.e., skipped without checking their ACL):
- services serving different vectoring points
- "broken-but-bypassable" services
- "up" services configured to ignore such transactions
(e.g., based on the ICAP Transfer-Ignore header).
When a set_name is used, all services in the set are checked
using the same rules, to find the first applicable one. See
adaptation_service_set for details.
If an access list is checked and there is a match, the
processing stops: For an "allow" rule, the corresponding
adaptation service is used for the transaction. For a "deny"
rule, no adaptation service is activated.
It is currently not possible to apply more than one adaptation
service at the same vectoring point to the same HTTP transaction.
See also: icap_service and ecap_service

adaptation_service_set

Defines a named adaptation service set. The set is populated in
the order of adaptation_service_set directives in this file.
When adaptation ACLs are processed, the first and only the first
applicable adaptation service from the set will be used. Thus,
the set should group similar, redundant services, rather than a
chain of complementary services.
If you have a single adaptation service, you do not need to
define a set containing it because adaptation_access accepts
service names.
Example:
adaptation_service_set svcBlocker urlFilterPrimary urlFilterBackup
adaptation service_set svcLogger loggerLocal loggerRemote

delay_pool_uses_indirect_client

Whether to use any result found by follow_x_forwarded_for in delay_pool assignment.
Default: ON

Controls whether the indirect client address
(see follow_x_forwarded_for) is used instead of the
direct client address in delay pools.

dns_v4_fallback

New option to prevent squid from always looking up IPv4 regardless of whether IPv6 addresses are found.
Squid will follow a policy of prefering IPv6 links, keeping the IPv4 only as a safety net behind IPv6.

Standard practice with DNS is to lookup either A or AAAA records
and use the results if it succeeds. Only looking up the other if
the first attempt fails or otherwise produces no results.
That policy however will cause squid to produce error pages for some
servers that advertise AAAA but are unreachable over IPv6.
If this is ON squid will always lookup both AAAA and A, using both.
If this is OFF squid will lookup AAAA and only try A if none found.
WARNING: There are some possibly unwanted side-effects with this on:
*) Doubles the load placed by squid on the DNS network.
*) May negatively impact connection delay times.

ecap_enable

Controls whether eCAP support is enabled. Default: OFF

ecap_service

Defines a single eCAP service

ecap_service servicename vectoring_point bypass service_url
vectoring_point = reqmod_precache|reqmod_postcache|respmod_precache|respmod_postcache
This specifies at which point of transaction processing the
eCAP service should be activated. *_postcache vectoring points
are not yet supported.
bypass = 1|0
If set to 1, the eCAP service is treated as optional. If the
service cannot be reached or malfunctions, Squid will try to
ignore any errors and process the message as if the service
was not enabled. No all eCAP errors can be bypassed.
If set to 0, the eCAP service is treated as essential and all
eCAP errors will result in an error page returned to the
HTTP client.
service_url = ecap://vendor/service_name?custom&cgi=style&parameters=optional
Example:
ecap_service service_1 reqmod_precache 0 ecap://filters-R-us/leakDetector?on_error=block
ecap_service service_2 respmod_precache 1 icap://filters-R-us/virusFilter?config=/etc/vf.cfg

New option to replace the old configure option --enable-default-err-language
New translations can be downloaded from http://www.squid-cache.org/Versions/langpack/

Set the default language which squid will send error pages in
if no existing translation matches the clients language
preferences.
If unset (default) generic English will be used.

error_log_languages

Log to cache.log what languages users are attempting to
auto-negotiate for translations.
Successful negotiations are not logged. Only failures
have meaning to indicate that Squid may need an upgrade
of its error page translations.

follow_x_forwarded_for

Enable processing of the X-Forwarded-for header for various administration tasks.

Allowing or Denying the X-Forwarded-For header to be followed to
find the original source of a request.
Requests may pass through a chain of several other proxies
before reaching us. The X-Forwarded-For header will contain a
comma-separated list of the IP addresses in the chain, with the
rightmost address being the most recent.
If a request reaches us from a source that is allowed by this
configuration item, then we consult the X-Forwarded-For header
to see where that host received the request from. If the
X-Forwarded-For header contains multiple addresses, and if
acl_uses_indirect_client is on, then we continue backtracking
until we reach an address for which we are not allowed to
follow the X-Forwarded-For header, or until we reach the first
address in the list. (If acl_uses_indirect_client is off, then
it's impossible to backtrack through more than one level of
X-Forwarded-For addresses.)
The end result of this process is an IP address that we will
refer to as the indirect client address. This address may
be treated as the client address for access control, delay
pools and logging, depending on the acl_uses_indirect_client,
delay_pool_uses_indirect_client and log_uses_indirect_client
options.
SECURITY CONSIDERATIONS:
Any host for which we follow the X-Forwarded-For header
can place incorrect information in the header, and Squid
will use the incorrect information as if it were the
source address of the request. This may enable remote
hosts to bypass any access control restrictions that are
based on the client's source addresses.
For example:
acl localhost src 127.0.0.1
acl my_other_proxy srcdomain .proxy.example.com
follow_x_forwarded_for allow localhost
follow_x_forwarded_for allow my_other_proxy

ftp_epsv_all

FTP Protocol extensions permit the use of a special "EPSV ALL" command.
NATs may be able to put the connection on a "fast path" through the
translator, as the EPRT command will never be used and therefore,
translation of the data portion of the segments will never be needed.
When a client only expects to do two-way FTP transfers this may be useful.
If squid finds that it must do a three-way FTP transfer after issuing
an EPSV ALL command, the FTP session will fail.
If you have any doubts about this option do not use it.
Squid will nicely attempt all other connection methods.
Requires ftp_passive to be ON (default)

include

New option to import entire secondary configuration files into squid.conf.

Squid will follow the files immediately and insert all their content
as if it was at that position in squid.conf. As per squid.conf some
options are order-specific within the config as a whole.
A few layers of include are allowed, but too many are confusing and
squid will enforce an include depth of 16 files.
Syntax:
include /path/to/file1 /path/to/file2

Whether to use any result found by follow_x_forwarded_for in access.log.
Default: ON

Controls whether the indirect client address
(see follow_x_forwarded_for) is used instead of the
direct client address in the access log.

netdb_filename

A filename where Squid stores it's netdb state between restarts.
To disable, enter "none".

pinger_enable

New option to enable/disable the ICMP pinger helper with a reconfigure instead of a full rebuild.

Control whether the pinger is active at run-time.
Enables turning ICMP pinger on and off with a simple squid -k reconfigure.
default is on when --enable-icmp is compiled in.

ssl_bump

New Access control for which CONNECT requests to an http_port
marked with an sslBump flag are actually "bumped". Please
see the sslBump flag of an http_port option for more details
about decoding proxied SSL connections.
DEFAULT: No requests are bumped.

For example, the following lines will bypass all validation errors
when talking to servers located at 172.16.0.0/16. All other
validation errors will result in ERR_SECURE_CONNECT_FAIL error.
acl BrokenServersAtTrustedIP dst 172.16.0.0/16
sslproxy_cert_error allow BrokenServersAtTrustedIP
sslproxy_cert_error deny all
This option must use fast ACL expressions only. Expressions that use
external lookups or communication result in unpredictable behavior or
crashes.
Without this option, all server certificate validation errors
terminate the transaction. Bypassing validation errors is dangerous
because an error usually implies that the server cannot be trusted and
the connection may be insecure.
See also: sslproxy_flags and DONT_VERIFY_PEER.

qos_flows local-hit= sibling-hit= parent-hit=

Allows you to select a TOS/DSCP value to mark outgoing
connections with, based on where the reply was sourced.
TOS values really only have local significance - so you should
know what you're specifying. For more information, see RFC2474,
RFC2475, and RFC3260.
The TOS/DSCP byte must be exactly that - octet value 0x00-0xFF.
Note that in practice often only values up to 0x3F are usable
as the two highest bits have been redefined for use by ECN
(RFC3168).
This setting is configured by setting the source TOS values:
local-hit=0xFF Value to mark local cache hits.
sibling-hit=0xFF Value to mark hits from sibling peers.
parent-hit=0xFF Value to mark hits from parent peers.
NOTE: 'miss' preserve feature is only possible on Linux at this time.
For the following to work correctly, you will need to patch your
linux kernel with the TOS preserving ZPH patch.
The kernel patch can be downloaded from http://zph.bratcheda.org
disable-preserve-miss
If set, any HTTP response towards clients will
have the TOS value of the response comming from the
remote server masked with the value of miss-mask.
miss-mask=0xFF
Allows you to mask certain bits in the TOS received from the
remote server, before copying the value to the TOS sent
towards clients.
Default: 0xFF (TOS from server is not changed).

The helper binary bundled with Squid under the name ntlm_auth has been renamed to accurately reflect
its real behavior and to prevent confusion with the more useful Samba helper using the same name.

Despite being used for NTLM, the helper does not in fact provide true NTLM function. What it does provide is
SMB LanManager authentication through the NTLM interface without the need for a domain controller. Thus the
new name is ntlm_smb_lm_auth.

WARNING: due to the name clash with Samba helper, admin should be careful to only update their squid.conf if the
squid bundled binary is used and needed. If the Samba helper is in use, the squid.conf should not be altered.

balance_on_multiple_ip

The previous default behavour (rotate per-request) of this setting causes failover clashes with IPv6 built-in mechanisms.
It has thus been turned off by default. Making the 'best choice' IP continue in use for any hostname until it encounters a connection failure and failover drops to the next known IP.

Modern IP resolvers in squid sort lookup results by preferred access.
By default squid will use these IP in order and only rotates to
the next listed when the most preffered fails.
Some load balancing servers based on round robin DNS have been
found not to preserve user session state across requests
to different IP addresses.
Enabling this directive Squid rotates IP's per request.

use 'htcp-no-clr' to send HTCP to the neighbor but without
sending any CLR requests. This cannot be used with
htcp-only-clr.
use 'htcp-no-purge-clr' to send HTCP to the neighbor
including CLRs but only when they do not result from
PURGE requests.
use 'htcp-only-clr' to send HTCP to the neighbor but ONLY
CLR requests. This cannot be used with htcp-no-clr.
use 'htcp-forward-clr' to forward any HTCP CLR requests
this proxy receives to the peer.
use 'connection-auth=off' to tell Squid that this peer does
not support Microsoft connection oriented authentication,
and any such challenges received from there should be
ignored. Default is 'auto' to automatically determine the
status of the peer.

Now an optional entry in squid.conf. If present it will force all visitors to receive the error pages
contained in the directory it points at. If absent, error page localization will be given a chance.

If you wish to create your own versions of the default
error files to customize them to suit your company COPY
the error/template files to another directory and point
this tag at them.
WARNING: This option will disable multi-language support
on error pages if used.

external_acl_type

New options 'ipv4' and 'ipv6' are added to set the IPv4/v6 protocol between squid and its helpers.
Please be aware of some limits to these options. These options only affet the transport protocol used
to send data to and from the helpers. Squid in IPv6-mode may still send %SRC addresses in IPv4 or IPv6
format, so all helpers will need to be checked and converted to cope with such information cleanly.

ipv4 / ipv6 IP-mode used to communicate to this helper.
For compatability with older configurations and helpers
'ipv4' is the default unless --with-localhost-ipv6 is used.
--with-localhost-ipv6 changes the default to 'ipv6'.
SPECIAL NOTE: explicit use of these options override --with-localhost-ipv6

New header input format specifiers. To seperate Request and Reply headers when both passed back.

If set to "transparent", Squid will not alter the
X-Forwarded-For header in any way.
If set to "delete", Squid will delete the entire
X-Forwarded-For header.
If set to "truncate", Squid will remove all existing
X-Forwarded-For entries, and place itself as the sole entry.

http_port transparent intercept sslbump connection-auth[=on|off]

Option 'transparent' is being deprecated in favour of 'intercept' which more clearly identifies what the option does.
For now option 'tproxy' remains with old behaviour meaning fully-invisible proxy using TPROXY support.

New port options

intercept Rename of old 'transparent' option to indicate proper functionality.
connection-auth[=on|off]
use connection-auth=off to tell Squid to prevent
forwarding Microsoft connection oriented authentication
(NTLM, Negotiate and Kerberos)
keepalive[=idle,interval,timeout]
Enable TCP keepalive probes of idle connections
idle is the initial time before TCP starts probing
the connection, interval how often to probe, and
timeout the time before giving up.
sslBump Intercept each CONNECT request matching ssl_bump ACL,
establish secure connection with the client and with
the server, decrypt HTTP messages as they pass through
Squid, and treat them as unencrypted HTTP messages,
becoming the man-in-the-middle.
When this option is enabled, additional options become
available to specify SSL-related properties of the
client-side connection: cert, key, version, cipher,
options, clientca, cafile, capath, crlfile, dhparams,
sslflags, and sslcontext. See the https_port directive
for more information on these options.
The ssl_bump option is required to fully enable
the SslBump feature.

https_port intercept sslbump connection-auth[=on|off]

New port options. see http_port.

maximum_object_size_in_memory

Default size limit increased to 512KB.

negative_ttl

New default of 0 seconds. To prevent negative-caching of failure messages unless explicitly
permitted by the message generating web server.

Changing this is an RFC 2616 violation and now requires --enable-http-violations

refresh_pattern

New set of basic patterns. These should always be listed after any custom ptterns.
They ensure RFC compliance with certain protocol and request handling in the absence
of accurate Cache-Control: and Expires: information.

This option causes some problems when bridging IPv4 and IPv6. A workaround has been provided.

Squid is built with a capability of bridging the IPv4 and IPv6 internets.
tcp_outgoing_address as previously used breaks this bridging by forcing
all outbound traffic through a certain IPv4 which may be on the wrong
side of the IPv4/IPv6 boundary.
To operate with tcp_outgoing_address and keep the bridging benefits
an additional ACL needs to be used which ensures the IPv6-bound traffic
is never forced or permitted out the IPv4 interface.
acl to_ipv6 dst ipv6
tcp_outgoing_address 2002::c001 good_service_net to_ipv6
tcp_outgoing_address 10.0.0.2 good_service_net !to_ipv6
tcp_outgoing_address 2002::beef normal_service_net to_ipv6
tcp_outgoing_address 10.0.0.1 normal_service_net !to_ipv6
tcp_outgoing_address 2002::1 to_ipv6
tcp_outgoing_address 10.0.0.3 !to_ipv6

Build with support for loadable content adaptation modules.
Cannot be used with --disable-loadable-modules.

--enable-follow-x-forwarded-for

Support following the X-Forwarded-For HTTP header for determining the
original or indirect client when a request has been forwarded through other
proxies.

--enable-zph-qos

Build with support for ZPH Quality of Service controls

--disable-auto-locale

Disable error page localization for visitors.

error_directory option is required if this option is used.

--disable-ipv6

Build without IPv6 support. The default is to auto-detect system capabilities
and build with IPv6 when possible.

--disable-loadable-modules

Build without support for loadable modules.

--with-dns-cname

Enable CNAME recursion within the Internal DNS resolver stub squid uses.
This has no effect on the external DNS helper.

Please note this extension is still experimental and may encounter problems.
To see if it is actually needed you can run squid without it for a period and
check the CNAME-Only Requests statistics squid maintains.

If it produces ongoing serious problems the external helper may be needed
but please report the bugs anyway.

--with-localhost-ipv6

Build support for squid to map all 127.0.0.1 traffic onto ::1.
The default is to build with 127.0.0.1 and ::1 being considered seperate IP.
see the IPv6 details above for a better description.

WARNING: This is an RFC violation. Use is discouraged.

--with-logdir=PATH

Allow build-time configuration of Default location for squid logs.

--with-ipv6-split-stack

Enable special additions for IPv6 support in Windows XP.
see the IPv6 details above for a better description.

--with-po2html=PATH

Absolute path to po2html executable.
Default is to automatically detect the binary.