sbcl-help

Apologies if this is a thick question, but what is the maximum memory
usage allowed?
On OS X 10.3.3 in SBCL 0.8.8 and 0.8.9 I am trying:
(let ((f (make-array '(4096 4096) :element-type 'double-float)))
(= (aref f 10 11) 99d0)) ;; this line is to access the array & return bool
This should allocate 2^12*2^12*2^3=2^27 bytes, which is about 135 MB,
which I thought would be within the addressable space.
The machine has 2 GB of RAM, and system limits are given at bottom.
But when I do this on 0.8.8 I get the error
set_auto_gc_trigger: tried to set gc trigger too high! (0x853bc5c)
fatal error encountered in SBCL pid 2373:
lost
There's no LDB in this build; exiting.
and in 0.8.9 I find that the next (gc :full t) fails with
fatal error encountered in SBCL pid 2399:
GC invariant lost, file "gc-common.c", line 612
The system is too badly corrupted or confused to continue at the Lisp
Many thanks in advance.
JK
==================================
usage limits as given in tcsh
sbcl-0.8.9-ppc-darwin> limit
cputime unlimited
filesize unlimited
datasize 750000 kbytes
stacksize 40960 kbytes
coredumpsize 20000000 kbytes
memoryuse unlimited
descriptors 256
memorylocked unlimited
maxproc 532
__________________________________
Do you Yahoo!?
Yahoo! Photos: High-quality 4x6 digital prints for 25¢
http://photos.yahoo.com/ph/print_splash

John Klein <jk271828@...> writes:
> Apologies if this is a thick question, but what is the maximum memory
> usage allowed?
>
> On OS X 10.3.3 in SBCL 0.8.8 and 0.8.9 I am trying:
I think the VM gives you ~2GB. You'll have to check that.
> (let ((f (make-array '(4096 4096) :element-type 'double-float)))
> (= (aref f 10 11) 99d0)) ;; this line is to access the array & return bool
I can reproduce your bug. (Is that really an SBCL bug? OpenMCL also
blows up if you try the above.) OpenMCL doesn't give you as big an
array size limit as SBCL. Here is a work around for you:
$ sbcl
This is SBCL 0.8.9.35, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/&gt;.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* (describe 'ARRAY-TOTAL-SIZE-LIMIT)
ARRAY-TOTAL-SIZE-LIMIT is an external symbol in #<PACKAGE "COMMON-LISP">.
It is a constant; its value is 536870911.
Constant documentation:
the exclusive upper bound on the total number of elements in an array
* (let ((f (make-array '(4096 4096) :initial-element 1d0)))
(= (aref f 10 11) 99d0))
NIL
This works in just a couple seconds on my machine. Tell you what.
I'll do an exact time:
* (time (let ((f (make-array '(4096 4096) :initial-element 1d0)))
(= (aref f 10 11) 99d0)))
Illegal instruction
Heh. Oops! Let's try again. Restart sbcl...
* (time (let ((f (make-array '(4096 4096) :initial-element 1d0)))
(= (aref f 10 11) 99d0)))
Evaluation took:
5.929 seconds of real time
4.62 seconds of user run time
0.55 seconds of system run time
0 page faults and
67109088 bytes consed.
NIL
If I specify :element-type 'double-float, I find myself spending all
my CPU time in system and I get too impatient to let it finish,
leading me to force quit sbcl.
BTW, I tried this in OpenMCL and this is what happened:
$ openmcl
Welcome to OpenMCL Version (Beta: Darwin) 0.14.1-p1!
? (time (let ((f (make-array '(4096 4096) :initial-element 1d0)))
(= (aref f 10 11) 99d0)))
> Error in process listener(1): value 16777216 is not of the expected type (UNSIGNED-BYTE 24).
> While executing: CCL::MAKE-UARRAY-1
> Type :POP to abort.
Type :? for other options.
1 > :pop
? (describe 'ARRAY-TOTAL-SIZE-LIMIT)
Symbol: ARRAY-TOTAL-SIZE-LIMIT
Constant
EXTERNAL in package: #<Package "COMMON-LISP">
Print name: "ARRAY-TOTAL-SIZE-LIMIT"
Value: 16777216
Function: #<Unbound>
Plist: NIL
HTH
--
I wouldn't mind the rat race so much if it wasn't for all the damn cats.

Just to see if I understand where things stand...
--- David Steuber <david@...> wrote:
>
> I can reproduce your bug. (Is that really an SBCL bug? OpenMCL also
> blows up if you try the above.) OpenMCL doesn't give you as big an
> array size limit as SBCL. Here is a work around for you:
>
> ARRAY-TOTAL-SIZE-LIMIT is an external symbol in #<PACKAGE "COMMON-LISP">.
> It is a constant; its value is 536870911.
so ARRAY-TOTAL-SIZE-LIMIT is much smaller than the 10^24 elements in my
array. And SBCL shouldn't die when allocating an excessively big
array, I think.
>
> * (let ((f (make-array '(4096 4096) :initial-element 1d0)))
> (= (aref f 10 11) 99d0))
>
> This works in just a couple seconds on my machine. Tell you what.
> I'll do an exact time:
OK, but I _think_ this makes an array of 32 bit pointers pointing
to the same boxed double float. So I think it is half as big as the
'double-float array.
> BTW, I tried this in OpenMCL and this is what happened:
>
> $ openmcl
> Welcome to OpenMCL Version (Beta: Darwin) 0.14.1-p1!
.....
> ? (describe 'ARRAY-TOTAL-SIZE-LIMIT)
> Symbol: ARRAY-TOTAL-SIZE-LIMIT
> Value: 16777216
With OpenMCL I understand the problem - the array is simply too big. But with
SBCL it looks like a bug - in 0.8.9 the memory gets trashed for the next
GC, not immediately. Also, it seems to be triggered if
I try to allocate two smaller arrays - this is how I originally hit the
problem.
__________________________________
Do you Yahoo!?
Yahoo! Photos: High-quality 4x6 digital prints for 25¢
http://photos.yahoo.com/ph/print_splash

John Klein <jk271828@...> writes:
> Just to see if I understand where things stand...
>
> so ARRAY-TOTAL-SIZE-LIMIT is much smaller than the 10^24 elements in my
> array. And SBCL shouldn't die when allocating an excessively big
> array, I think.
$ sbcl
This is SBCL 0.8.9.58, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/&gt;.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* (> array-total-size-limit (* 4096 4096))
T
So SBCL should be OK with the array you are trying to make.
> > * (let ((f (make-array '(4096 4096) :initial-element 1d0)))
> > (= (aref f 10 11) 99d0))
>
> OK, but I _think_ this makes an array of 32 bit pointers pointing
> to the same boxed double float. So I think it is half as big as the
> 'double-float array.
I think you are correct. I'm going to try what you were doing with a
MUCH smaller 2d array. I don't want to thrash my machine to death in
system calls ;-)
* (let ((f (make-array '(10 10) :element-type 'double-float
:initial-element 1D0)))
(= (aref f 7 8) 99d0))
NIL
> With OpenMCL I understand the problem - the array is simply too big. But with
> SBCL it looks like a bug - in 0.8.9 the memory gets trashed for the next
> GC, not immediately. Also, it seems to be triggered if
> I try to allocate two smaller arrays - this is how I originally hit the
> problem.
I see. Does this work for you?
* (loop for i from 1 to 100000 do
(let ((f (make-array 4096 :element-type 'double-float)))
(= (aref f 10) 1d0)))
NIL
I'm going to try your original again.
* (let ((f (make-array '(4096 4096) :element-type 'double-float)))
(= (aref f 10 11) 99d0))
set_auto_gc_trigger: tried to set gc trigger too high! (0x807e37c)
fatal error encountered in SBCL pid 1299:
lost
The system is too badly corrupted or confused to continue at the Lisp
level. If the system had been compiled with the SB-LDB feature, we'd drop
into the LDB low-level debugger now. But there's no LDB in this build, so
we can't really do anything but just exit, sorry.
OK, it's not fixed in the latest version yet.
This is interesting:
* (progn (gc-off)
(let ((f (make-array '(4096 4096) :element-type 'double-float)))
(= (aref f 10 11) 99d0))
(gc-on)
(gc))
fatal error encountered in SBCL pid 1304:
GC invariant lost, file "gc-common.c", line 616
The system is too badly corrupted or confused to continue at the Lisp
level. If the system had been compiled with the SB-LDB feature, we'd drop
into the LDB low-level debugger now. But there's no LDB in this build, so
we can't really do anything but just exit, sorry.
We get a line number even without LDB enabled. I find this function
there:
static int
scav_other_pointer(lispobj *where, lispobj object)
{
lispobj first, *first_pointer;
gc_assert(is_lisp_pointer(object));
/* Object is a pointer into from space - not FP. */
first_pointer = (lispobj *) native_pointer(object);
first = (transother[widetag_of(*first_pointer)])(object);
if (first != object) {
set_forwarding_pointer(first_pointer, first);
#ifdef LISP_FEATURE_GENCGC
*where = first;
#endif
}
#ifndef LISP_FEATURE_GENCGC
*where = first;
#endif
gc_assert(is_lisp_pointer(first));
gc_assert(!from_space_p(first));
return 1;
}
I think it is blowing up on that last gc_assert.
For my next build, I'm going to go ahead and enable the LDB. This is
an interesting problem. While the code asks a lot from the system,
it certainly should be managable.
--
I wouldn't mind the rat race so much if it wasn't for all the damn cats.

David Steuber <david@...> writes:
> For my next build, I'm going to go ahead and enable the LDB. This is
> an interesting problem. While the code asks a lot from the system,
> it certainly should be managable.
Then again, perhaps there is a hard limit. I'm not sure what this
means, but this looks like it might be a clue:
$ sbcl
This is SBCL 0.8.9.58, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/&gt;.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* (progn (gc-off)
(let ((f (make-array '(4096 4096) :element-type 'double-float)))
(= (aref f 10 11) 99d0))
(gc-on)
(gc))
set_auto_gc_trigger: tried to set gc trigger too high! (0x8054b54)
fatal error encountered in SBCL pid 1948:
lost
LDB monitor
ldb> p 0x8054b54
0x08054b54: odd fixnum: 33641173
ldb>
I don't really know how to do anything useful with ldb :-(
--
I wouldn't mind the rat race so much if it wasn't for all the damn cats.

John Klein <jk271828@...> writes:
> --- David Steuber <david@...> wrote:
>
> > I'm going to try your original again.
> >
> > * (let ((f (make-array '(4096 4096) :element-type 'double-float)))
> > (= (aref f 10 11) 99d0))
> > set_auto_gc_trigger: tried to set gc trigger too high! (0x807e37c)
> > fatal error encountered in SBCL pid 1299:
>
> Yes, this is what I get.
>
> On Linux with 0.8.5 this slightly more rigorous test (with much less physical
> memory) works:
I just tested on an old Linux box (384MB RAM) that is in theory far
less powerful than my PowerBook G4:
$ sbcl
This is SBCL 0.8.9.58, an implementation of ANSI Common Lisp.
...
* (time (let ((f (make-array '(4096 4096) :element-type 'double-float
:initial-element 99d0)))
(= (aref f 4000 4000) 99d0)))
Evaluation took:
8.791 seconds of real time
6.97394 seconds of user run time
1.427783 seconds of system run time
0 page faults and
134217736 bytes consed.
T
* (gc :full t)
NIL
*
I've also got sb-thread turned on on this one.
It would be nice to get the PPC/Darwin version of SBCL caught up. I
think the PPC is a better architecture and should run much faster.
I'm still learning Lisp, so I can only contribute so much at the
moment.
--
I wouldn't mind the rat race so much if it wasn't for all the damn cats.