panic: file desc: malloc limit exceeded

I have a crontab entry set to run pkgsrc-update and src-update. Besides that,
some WSGI process running with Gunicorn (http://gunicorn.org) proxied by nginx
with access to pgsql. These are not very demanded, mainly testing webapps. It
also hosts a SnapLogic (http://snaplogic.org/) server.

History

This is the start of a patch to handle overflows of this malloc zone. It is not
enough, internal to fdcopy() there is a loop around the allocation of fd arrays;
the loop allocations have not yet been corrected.

Commit 2994659f1e6c1ef260241491bceca91c9d2553b3 is a partial fix to the problem;
it does not handle overflows in the spinlock loop path in fdcopy and it is still
possible to make the system unusable with the sample program posted below.

Perhaps we should also raise the malloc zone limit to maxproc * MAX_FDS_PER_PROC?

:Venkatesh Srinivas <vsrinivas@dragonflybsd.org> added the comment:
:
:Commit 2994659f1e6c1ef260241491bceca91c9d2553b3 is a partial fix to the problem;
:it does not handle overflows in the spinlock loop path in fdcopy and it is still
:possible to make the system unusable with the sample program posted below.
:
:Perhaps we should also raise the malloc zone limit to maxproc * MAX_FDS_PER_PROC?

No, won't work, the maximum will baloon well past any reasonable limit
when you try to do that.

We have a kern.maxfilesperuser that's supposed to handle that sort of
attack, is it not working? It might not be applicable to root though.

Hmm. Clearly kern.maxfilesperuser isn't going to help for the
sparse file descriptor table attack. The defaults on an i386
box seem to be on the order of 6000 processes x 25000 descriptors
per process, which winds up being significant greater than a gigabyte
of ram (let alone kvm)... so it goes boom.

I think we do have to apply the maxfilesperuser limit to this situation
counted based on the size of the fd table instead of based on the number
of actual descriptors. That would handle the situation.