Distro updating issues behind pfsense

for the last few weeks I have been constantly having problems to run package updates on various boxes sitting behind my pfsense router. The issues I am encountering are different from time to time but generally are regarding problems fetching files or other index files, retrieving repo info, etc…

I have 8 machines running behind pfsense, 5 Centos machines, and 2 Ubuntu machines, and a Windows XP machine. No problem to run windows upate on the wxp machine but I have issues on all other linux based machines.

pfSense is currently configured with Snort, Squid, Squidguard, HAVP and pfBlocker.

I have tried these to proceed by elimination and find the root cause, to no avail:

-Disable snort completely: No hosts were blocked by snort but nevertheless I did not take any chance and disabled it.
-Uninstall Snort completely and reboot router.
-Disable squidguard
-Look in the firewall logs for hosts being blovked, none found so far.
-Disable pfblocker temporarily
-Uninstall pfblocker completely then rebooting router

E: Some index files failed to download. They have been ignored, or old ones used instead.

I have tried flushing the package manager's cache (on centos yum clean all, manually deleting the cache files on ubuntu machines, following all I could find on the web) but nothing helps. Everything points to either ISP transparent proxy cache corruption, or router issues.

The repos failing to properly update change from time to time, and sometimes the update succeeds but most of the time it fails with similar errors as reported above..

I'm currently trying to investigate some trouble I have with squid/squidGuard. Simply disabling squidGuard does not immediate solve the issue; it seems that squid needs a restart to get on its feet. Switching squid from transparent to normal helps immediately. I currently have very little data, as the issue pops up at some time, then everything's fine again for a few days, the logs show nothing…a typical Heisenbug.

As a follow up to this thread, and for future reference, I ended up reinstalling pfsense completely without squid and its associated packages (SG, etc). Now everything works fine and firewall performance is as expected. Its been a month since I restarted fresh and so far so good.

But before I did so, I did a test and reinstalled pfsense with squid, then uninstalled squid. I ended up with the same performance issues..

I strongly assume that somehow, installing squid and squidguard alters pfsense in such a way that when uninstalled, pfsense remains altered and that causes the performance degradation I had.

Same issue again. This time, I need squid to perform web filtering and caching, but of course the issue with package managers on LAN clients resurfaced and I am getting the very same issues as before..

I posted a bug report on pfsense bug tracker but the ticket was rejected saying "this is almost certainly a problem within squid itself, or a problem on the servers in question"

AFAIK all of my servers are configured for standard connection to the web and yum (or synaptics, apt-get, windows update, etc) are all configured standard out of the box (no special proxy settings).. As a matter of fact, squid is configured to be a transparent proxy on my pfsense tbox so LAN clients shouldnt "see it". Also, why all platforms would have issues? I mean even a standard windows XP box has issues with Windows Update..

Nevertheless, it doesnt work and I am really disappointed that so far I havent been able to find a solution to this other than manually clearing squid's cache. If I do this (with the commands of post #4 here) all is well until the cache is filled again a few days/weeks later and needs a flush again..

Who maintains squid? Perhaps talking to this (these) guys would be a good start..

Apparently the problem is a combination of package manager assuming HTTP/1,1 protocol mechanisms (revalidation particularly) while Squid-2.7 is only HTTP/1.0 compliant. Anyways this is what I understood from the reply I got on Squid-cache's bugzilla.

Apparently, the APT problem has been confirmed fixed years ago in Squid-3.1

IMO Squid2 should be marked deprecated in the package repo or strong warning!!!