You could probably run Squid in conjunction with this (or I don't see why you couldn't use netcat as an http proxy).
–
gravyfaceMar 3 '11 at 0:42

I was thinking you could use tshark instead of tcpdump to eliminate the need for grep, but there doesn't seem to be a [display filter][1] for the body of an http response. [1]: wireshark.org/docs/dfref/h/http.html
–
sciurusMar 3 '11 at 5:28

I've used tcpdump before, and think it's a great tool, but I was hoping for something HTTP specific. For example, an HTTP-specific tool could uncompress gzip/deflate'd content before sending it to STDOUT. The packet capture tools like tcpdump and ngrep are showing me compressed content.
–
richardkmillerMar 4 '11 at 17:47

@richardkmiller, I understand your desire to get the uncompressed content, but I don't think an HTTP proxy would do what you want either. I don't believe they typically will do anything about uncompressing the HTML body. A google search for http capture gzip decompression revealed this page which looks close to your requirements.
–
ZoredacheMar 4 '11 at 18:36

You may be right. In the past, I've used Apache as a proxy and used ExtFilterDefine to define a script through which to pass the content, but I had to turn off accept-encoding in my browser, as mentioned by sciurus below. I was hoping there was an easier way, especially from the command-line, but perhaps there aren't any easy alternatives.
–
richardkmillerMar 23 '11 at 15:47

Thanks for introducing me to ngrep. I played around with it a bit and found it easier to use than tcpdump. While using it, it occurred to me that neither ngrep nor tcpdump handles gzipped or deflated content, which is how my browser (Chrome) is requesting it. Do you know if it's possible to ungzip/inflate the HTTP bodies?
–
richardkmillerMar 4 '11 at 17:44

Sorry if this seems like a shameless plug, but awhile ago I wrote a small http proxy that outputs all of the data that passes through it. I just posted it on my website in case it would be useful for somebody. Hope it helps: OutputProxy

Thanks for this pointer. I searched a bit for a command-line option for invoking a proxy server in SimpleHTTPServer but didn't find anything. It may be a matter of my not knowing enough Python.
–
richardkmillerMar 4 '11 at 17:53