Re: [Sbcl-help] Re: km + clib

On Sun, Nov 27, 2005 at 01:47:52AM +0200, Juho Snellman wrote:
> That said... Do you perhaps declaim (OPTIMIZE (DEBUG 3)) in your
> .sbclrc?
I tried that but it appears that the sbcl available from debian unstable
doesn't include debugging support:
; The function DEBUG is undefined, and its name is reserved by ANSI CL
so
that
; even if it were defined later, the code doing so would not be
portable.
Shall I recompile sbcl with some debugging options enabled?

On Thu, Nov 24, 2005 at 06:47:06PM +0530, Joshua N Pritikin wrote:
> Hi, I tried using sbcl 0.9.5.50 to load KM and clib
>
> http://www.cs.utexas.edu/users/mfkb/RKF/km.html
> &
> http://www.cs.utexas.edu/users/mfkb/RKF/clib.html
>
> sbcl blows up with an out-of-memory error. My machine has 700M of RAM.
> Is there some configuration option for sbcl which might help?
This report is a bit low on details. It'd help to know at least:
* What platform (os, architecture) is this?
* What exact out-of-memory error are you getting? "Argh!
gc_find_freeish_pages failed"?
* What exact steps are you doing to trigger the error? Preferably
with a level of detail sufficient to allow someone with no
knowledge of KM to repeat the problem. I tried the following,
but it only used modest amounts of memory:
(load "km-2-0-19.lisp")
(km)
(load-kb "tmp/rkf-clib-one.km" :verbose t)
That said... Do you perhaps declaim (OPTIMIZE (DEBUG 3)) in your
.sbclrc?
--
Juho Snellman

On Sun, Nov 27, 2005 at 01:47:52AM +0200, Juho Snellman wrote:
> On Thu, Nov 24, 2005 at 06:47:06PM +0530, Joshua N Pritikin wrote:
> > Hi, I tried using sbcl 0.9.5.50 to load KM and clib
> >=20
> > http://www.cs.utexas.edu/users/mfkb/RKF/km.html
> > &
> > http://www.cs.utexas.edu/users/mfkb/RKF/clib.html
> >=20
> > sbcl blows up with an out-of-memory error. My machine has 700M of RAM.
> > Is there some configuration option for sbcl which might help?
>=20
> This report is a bit low on details. It'd help to know at least:
>=20
> * What platform (os, architecture) is this?
Debian GNU/Linux 2.6.13.4 i686
> * What exact out-of-memory error are you getting? "Argh!=20
> gc_find_freeish_pages failed"?
Yes, "gc_find_free_space failed" to be exact.
> * What exact steps are you doing to trigger the error? Preferably
> with a level of detail sufficient to allow someone with no
> knowledge of KM to repeat the problem. I tried the following,
> but it only used modest amounts of memory:
Here is a session log:
joshua@...:~$ sbcl
This is SBCL 0.9.6, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/&gt;.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* (load "km.fasl")
STYLE-WARNING: defining *KM-PACKAGE* as a constant, even though the name
follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *VAR-MARKER-CHAR* as a constant, even though the name
follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *CLASSIFY-IN-LOCAL-SITUATIONS* as a constant, even though
the name follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *GLOBAL-SITUATION* as a constant, even though the name
follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *DEFAULT-FAIL-MODE* as a constant, even though the name
follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *BUILT-IN-SEQ-AGGREGATION-SLOTS* as a constant, even though
the name follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *BUILT-IN-SUBSLOTS* as a constant, even though the name
follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *DEFAULT-DEFAULT-FLUENT-STATUS* as a constant, even though
the name follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *COMMENT-MARKER-CHAR* as a constant, even though the name
follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *INDENT-INCREMENT* as a constant, even though the name
follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *PRUNE-POINTS* as a constant, even though the name follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *IGNORE-ITEMS* as a constant, even though the name follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING: defining *MAXDEPTH* as a constant, even though the name
follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING: defining *CHECKPOINT* as a constant, even though the name
follows
the usual naming convention (names like *FOO*) for special variables
STYLE-WARNING:
defining *MAX-CONCAT-LENGTH* as a constant, even though the name
follows
the usual naming convention (names like *FOO*) for special variables
Resetting KM...
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D
KM - THE KNOWLEDGE MACHINE - INFERENCE ENGINE v2.0.19
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D
Copyright (C) 1994-2005 Peter Clark and Bruce Porter. KM comes with
ABSOLUTELY
NO WARRANTY. This is free software, and you are welcome to redistribute
it
under certain conditions. Type (license) for details.
Documentation at http://www.cs.utexas.edu/users/mfkb/km/
Type (km) for the KM interpreter prompt!
T
* (defun load-lib (&optional (comp-path "./components/"))
(let (result)
(dolist (component (traverse-directory comp-path) result)
(cond
((equal (pathname-type component) "km")
(load-kb component)
(setf result (cons (pathname-name component) result)))
((equal (pathname-type component) "lisp")
(load component))))))
(defun traverse-directory (root-dir-path)
(let ((dir-list (directory root-dir-path)) sub-dir-path result)
(dolist (element dir-list result)
(if (directory (setf sub-dir-path (concatenate 'string
root-dir-path (pathname-name element) "/")))
(setf result (append (traverse-directory sub-dir-path)
result))
(setf result (cons element result))))))
; in: LAMBDA NIL
; (TRAVERSE-DIRECTORY COMP-PATH)
;=20
; caught STYLE-WARNING:
; undefined function: TRAVERSE-DIRECTORY
;=20
; caught STYLE-WARNING:
; This function is undefined:
; TRAVERSE-DIRECTORY
;=20
; compilation unit finished
; caught 2 STYLE-WARNING conditions
LOAD-LIB
*=20
TRAVERSE-DIRECTORY
* (load-lib "clib/components/")
Argh! gc_find_free_space failed (first_page), nbytes=3D8.
Gen Boxed Unboxed LB LUB !move Alloc Waste Trig WP GCs
Mem-age
0: 0 0 0 0 0 0 0 2000000 0 0
0.0000
1: 0 0 0 0 0 0 0 2000000 0 0
0.0000
2: 0 0 0 0 0 0 0 2000000 0 0
0.0000
3: 51878 679 21 24 0 215401208 56584 2000000 0 0
2.5204
4: 0 0 0 0 0 0 0 2000000 0 0
0.0000
5: 0 0 0 0 0 0 0 2000000 0 0
0.0000
6: 1 0 0 0 0 4096 0 2000000 1 0
0.0000
Total bytes allocated=3D520273944
fatal error encountered in SBCL pid 3134(tid 3085338304)
The system is too badly corrupted or confused to continue at the Lisp
level. If the system had been compiled with the SB-LDB feature, we'd
drop
into the LDB low-level debugger now. But there's no LDB in this build,
so
we can't really do anything but just exit, sorry.

On Sun, Nov 27, 2005 at 04:04:59PM +0530, Joshua N Pritikin wrote:
> (defun traverse-directory (root-dir-path)
> (let ((dir-list (directory root-dir-path)) sub-dir-path result)
> (dolist (element dir-list result)
> (if (directory (setf sub-dir-path (concatenate 'string
> root-dir-path (pathname-name element) "/")))
> (setf result (append (traverse-directory sub-dir-path)
> result))
> (setf result (cons element result))))))
You aren't running out of memory while loading the software, but
running out of memory while recursing infinitely in this function. I
don't think this function can work correctly in an ANSI-conforming
Lisp. (DIRECTORY on non-wild pathnames has to return a list of 0 or 1
items, this code seems to assume that DIRECTORY on a non-wild pathname
that names a directory returns the contents of that directory).
Something like the following should work better in SBCL:
(defun traverse-directory (root-dir-path)
(directory
(merge-pathnames (make-pathname :directory (list :relative :wild-inferiors)
:name :wild
:type :wild)
root-dir-path)))
This won't work on an implementation that doesn't support wild
inferiors in pathnames. A completely portable version left as an
exercise.
--
Juho Snellman

On Sun, Nov 27, 2005 at 02:36:40PM +0200, Juho Snellman wrote:
> You aren't running out of memory while loading the software, but
> running out of memory while recursing infinitely in this function. I
>=20
> (defun traverse-directory (root-dir-path)
Yup, that work for me. Thanks, I'll pass the info upstream.

On Sun, Nov 27, 2005 at 01:47:52AM +0200, Juho Snellman wrote:
> That said... Do you perhaps declaim (OPTIMIZE (DEBUG 3)) in your
> .sbclrc?
I tried that but it appears that the sbcl available from debian unstable
doesn't include debugging support:
; The function DEBUG is undefined, and its name is reserved by ANSI CL
so
that
; even if it were defined later, the code doing so would not be
portable.
Shall I recompile sbcl with some debugging options enabled?

On Sun, 27 Nov 2005, Joshua N Pritikin wrote:
> On Sun, Nov 27, 2005 at 01:47:52AM +0200, Juho Snellman wrote:
>> That said... Do you perhaps declaim (OPTIMIZE (DEBUG 3)) in your
>> .sbclrc?
>
> I tried that but it appears that the sbcl available from debian unstable
> doesn't include debugging support:
>
> ; The function DEBUG is undefined, and its name is reserved by ANSI CL
> so
> that
> ; even if it were defined later, the code doing so would not be
> portable.
Whatever you did was not
(DECLAIM (OPTIMIZE (DEBUG 3)))
...or something is deeply wrong. In either case doing this is a bad
idea. DEBUG 3 is currently something you should reserve for local
declares with SBCL. For ease of debugging something like
(DECLAIM (OPTIMIZE (DEBUG 2) (SPEED 0) (SPACE 0)))
should do the job fairly well without killing the performance quite
as throughly as DEBUG 3.
Cheers,
-- Nikodemus Schemer: "Buddha is small, clean, and serious."
Lispnik: "Buddha is big, has hairy armpits, and laughs."

On Sun, Nov 27, 2005 at 04:07:46PM +0530, Joshua N Pritikin wrote:
> On Sun, Nov 27, 2005 at 01:47:52AM +0200, Juho Snellman wrote:
> > That said... Do you perhaps declaim (OPTIMIZE (DEBUG 3)) in your
> > .sbclrc?
>
> I tried that but it appears that the sbcl available from debian unstable
> doesn't include debugging support:
Just to clarify, I wasn't asking you to do that, but whether you
already had done it :-) Compiling code with a high debug setting can
cause very high memory consumption in some circumstances, so I
initially thought it might be one likely reason for your problems.
--
Juho Snellman