Andrew Stewart <boss at airbladesoftware.com> wrote:
> On 28 Nov 2012, at 22:10, Eric Wong wrote:
> > Yes. Using the Unicorn::PrereadInput middleware (before Rails or
> > any other middleware/framework touches env["rack.input"]) should do
> > everything you want.
>> Excellent.
>> This isn't enabled by default so I wonder whether there any
> disadvantages or side-effects one should be aware of?
It'll be slower for fast clients doing large uploads (as it leads to
flushing of the page cache to stable storage). For small uploads/POST
requests (normal web browser stuff), it likely won't matter.
Keep in mind nginx basically does what PrereadInput does on its end,
too.
Now, for Rack apps that deal with raw PUT requests (from "curl -T" on a
LAN) several gigabytes large:
Buffering a large request up front means the data could be out of the
kernel page cache by the time an application sees it. As a result, the
application is will trigger additional disk seeks to reread the data it
buffered.
With the default streaming/lazy-buffering (TeeInput), unicorn reads data
off the socket as your application needs it. This means the data your
app sees is still "hot" in memory, and won't have to wait for disk (or
even slower memory accesses). All the writes to the (rewindable)
temporary file are still buffered by the OS to $TMPDIR, but those disk
writes are usually asynchronous and minimally affects application
processing time.
unicorn also supports "rewindable_input false" in its config. This
violates the Rack 1.x spec, but it allows applications which process a
data stream to avoid generating any disk I/O at all. No filesystem/disk
limitations to worry about at all here, one could stream petabytes of
data through unicorn this way.