Re: ZFS kmem_map too small. - FreeBSD

This is a discussion on Re: ZFS kmem_map too small. - FreeBSD ; On Nov 6, 2007, at 11:00 AM, Pawel Jakub Dawidek wrote:
> If you use vm_kern.c.2.patch, can you show loader.conf and exact
> command
> that can provke the panic?
I can elaborate a bit. I repeated the test while ...

Re: ZFS kmem_map too small.

On Nov 6, 2007, at 11:00 AM, Pawel Jakub Dawidek wrote:
> If you use vm_kern.c.2.patch, can you show loader.conf and exact
> command
> that can provke the panic?

I can elaborate a bit. I repeated the test while I kept some
connections open,
watching zpool iostat, the Bonnie++ command output, the make
buildworld, and
pinging the machine continuously.

Wired memory reached 1696 MB according to top, and the programs seemed
to be stopped. I
didn't see more activity from top, zpool iostat, or make buildworld.
But the kernel
was still answering to pings, although with a very long delay, an
average of 300 - 500 ms.
I'm connected to the same local network, and the typical delay I get
with the machine is
around 0.3 ms. It was until things started to go wrong.

The machine kept answering pings (very late, but with no packet loss),
until I pressed
ctrl-C at the terminal session where I was running Bonnie++. That
seemed to trigger the
panic, kmem_map to small.

And I've found something. I was using SCHED_ULE instead of SCHED_4BSD.
I've switched back
to SCHED_4BSD (I apologise I didn't mention it, I thought it shouldn't
make a difference) and
the kmem usage climbs much slower, sustaning a 30 MB/s write
throughput during the Bonnie's "writing intelligently" test.

Using ULE, wired memory climbs and tops 1.6 GB in less than a minute,
while using 4BSD it's
climbing much slower. I've seen it reach 1.2 GB, go back to 800 MB...
and suddenly go down to
256 MB, while it keeps sustaining a write throughput of about 25 - 30
MB/s (according to
zpool iostat at 10 second intervals).

I've compiled the kernel with debugger support and this is what the
backtrace reads:

----------------
"The thing he realised about the windows was this: because they had
been converted into openable windows after they had first been
designed to be impregnable, they were, in fact, much less secure than
if they had been designed as openable windows in the first place."
Douglas Adams, "Mostly Harmless"