I have a file (git blob) that is compressed using DEFLATE, and I want to uncompress it. The gzip command does not seem to have an option to directly use the DEFLATE algorithm, rather than the gzip format.

Ideally I'm looking for a standard Unix/Linux tool that can do this.

edit: This is the output I get when trying to use gzip for my problem:

Yeah, I looked at that. But I would definitely prefer a commonly packaged tool.
–
felixgeJul 5 '10 at 10:10

@fekixge: Well, thanks. I tried it yesterday on an gzipped file (stripping the gzip header away) but it crashed, unluckily. I'm debugging now as this is quite interesting to me.
–
mkluweJul 6 '10 at 9:48

Note that the zlib sub-command (and the -z option to the enc sub-command) is not available if your build of openssl was configured with the default options, which include --no-zlib and --no-zlib-dynamic. So this answer only works if your openssl was compiled with the no- prefix removed from one of those configure options. You can tell by looking for -DZLIB in the output from openssl version -f
–
HercyniumMay 13 '14 at 16:02

As Jack points out above, the output of git cat-file -p <SHA1> is not the complete contents of the zlib decompression of .git/objects/<SHA1>. The difference is key if you're trying to implement a Git commit hash calculator ...
–
ntc2Jan 23 '14 at 4:15

See the bottom of this page: progit.org/book/ch9-2.html Gzip does implement DEFLATE, but it doesn't seem like you can directly apply the algorithm. Gzip expects the data to be in gzip format (which adds a bunch of headers & stuff around the DEFLATE'ed data). (I just edited my post to include the output from gunzip)
–
felixgeJul 5 '10 at 10:07

2

Ah ok, so the data is compressed using the zlib library, then it stands to reason you can uncompress using zlib too! You could try a ruby, perl or other binding to wip up a simple deflate script. Or if you're not afraid to try your hands at compiling a c program, try this: zlib.net/zlib_how.html
–
Marc van KempenJul 5 '10 at 10:20

I'm preparing for a little git-workshop I'm going to give soon. One of the examples involves showing what 'git add' does by hand. De-compressing the blob using git itself doesn't make sense since I want to show the underlaying functionality. I will probably end up using ruby or perl, but I was hoping I could stick with a simple bash oneliner.
–
felixgeJul 5 '10 at 10:58

I found this question looking for a work-around with a bug with the -text utility in the new version of the hadoop dfs client I just installed. The -text utility works like cat, except if the file being read is compressed, it transparently decompresses and outputs the plain-text (hence the name).

The answers already posted were definitely helpful, but some of them have one problem when dealing with Hadoop-sized amounts of data - they read everything into memory before decompressing.

So, here are my variations on the Perl and Python answers above that do not have that limitation: