After the change I ran a test Perl program that opens/creates 256 files and closes 256 file handles at the end of script. When it creates 253 files the program dies saying too many open files. I don't understand why I'm getting this error.

4 Answers
4

According to this article this is a default limitation of 32-bit Solaris. A program is normally limited to using the first 256 file numbers. STDIN, STDOUT and STDERR take 0, 1 and 2 which leaves you with 253. It's not a simple process to work around it, ulimit won't do it, and I don't know if Perl will honor it.

Thanks a lot for the info. FileCache worked. How does filecache work and how it is able to overcome OS Limitation? Does this happen for JAVA and C programs have separate module like FileCache to use many file handles.
–
AravNov 16 '12 at 4:02

1

Yes, it effects every 32 bit program on Solaris 10 unless they've been specially written and compiled as explained in the article. Here's a Java person with the same problem and another. Though seeing as how Oracle owns both Solaris and Java I suspect they've worked around it for Java... but I wouldn't be too surprised if they haven't. I don't know what Java and C programmers do to work around it.
–
SchwernNov 16 '12 at 4:08

1

C programmers work around it by either compiling 64-bit or using the extended FILE api either via source changes or LD_PRELOAD options.
–
alancNov 17 '12 at 3:53

FileCache automatically closes and re-opens files if you exceed your system's maximum number of file descriptors, or the suggested maximum maxopen (NOFILE defined in sys/param.h).

In my case, on a linux box, it is 256:

#david@:~/Test$ grep -B 3 NOFILE /usr/include/sys/param.h
/* The following are not really correct but it is a value
we used for a long time and which seems to be usable.
People should not use NOFILE and NCARGS anyway. */
#define NOFILE 256

Using the lsof (list open files) command, the modified version of your script opened at most 260 of the 100334 files:

Thanks a lot for the info. FileCache worked. How does filecache work and how it is able to overcome OS Limitation? Does this happen for JAVA and C programs have separate module like FileCache to use many file handles?
–
AravNov 16 '12 at 4:04

Note that it fails when you try to open the process's 257th handle (STDIN + STDOUT + STDERR + 253 = 256), which leads me to believe the number of open file handles a process can have must fit in 8 bits on your system. You could try verifying this by writing an equivalent C program and running it on the same machine.