Ok, Thiemo Seufer has committed a change to gc-common.c that gets rid of an extra instruction cache flush on non GENCGC platforms (that is, CHENEYGC). Turns out this is a no-op on x86 and x86-64 due to their cache-coherency, but might be required for a generational collector on other platforms should one ever get ported to, say, PPC or MIPS.

As for the additional sync call, this may be necessary on multi-processor systems as the instructions seen in the icache by one processor may be seen as data in another and the data-cache syncing is necessary. So this should stay as is.

Here are some cl-bench results from recent SBCL builds on PPC with a couple minor caching changes. The first is the vanilla SBCL, the second is with an osicacheflush removed from the cheneygc and the second is with this change and a sync removed from ppcflushcache_line. Here's the patch:

(I've posted this to the cclan list, but I thought I'd put a copy of this here for posterity's sake.) So I'm trying to package up some C code to use as a test for gcc-xml-ffi. I have some C code that I'd like to use to generate a shared library, which will then be loaded by SBCL at runtime. To do this I can 1) hand-code the Makefile rules with the approrpriate platform-specific compiler/linker incantations to get the shared library, 2) use autotools, or 3) try to rig up some sort of asdf-based system to do this. I've done 1 for darwin and got 2 to work, but I wasn't super happy with distributing this giant mess of m4 files just to compile some shared libraries, so I decided to explore 3. Actually, as I was considering this, I decided that I also wanted to use asdf to invoke gccxml to generate the xml for gcc-xml-ffi.

So far so good. SBCL has examples of how to do this in sb-posix (and somewhere else, IIRC this is a violation of OAOO, but let's put that aside for the moment.

But I've run into a couple issues: </p

asdf perform methods specialize on operation and component, but not on the system. This means that if you define a method to compile C files for compile-op and c-source-file it will be applied to all c-source-files. This might not be desired. Clearly one can work around this by subclassing c-source-file, but that seems a bit silly. having a way to provide per-module or per-system operations seems like a good thing. Perhaps there is a way, but I'm missing on obviously good way to do this.

cross-module dependencies. I'd like to declare multiple module with subcomponents like module foo with some c-header-files and c-source-files. Then I'd like for components in module bar to depend on, say, one of the c-header-files in module foo. I can't seem to figure out how to do this. The :depends-on arg as it currently exists seems 1) very lisp specific and 2) limited to dependencies within a particular component. I can't, for instance, say that a particular file depends on another system. I can only do system level dependencies for a system. Or at least I can't figure out how to do them. This isn't actually what I want to do, but I think it illustrates an example of the problem. In my case, what I want is for a source file in module bar to depend on a header file in module foo. This works as is in the sense that if a header file in module foo is changed, foo itself is recompiled, but in this case I want to trigger a recmopile of bar as well.

Clearly, asdf is an extensible system and through the right extensions I can make this work, but if anyone has any suggestions or comments on how to address these issues, I'd appreciate it.

It seems the good people of #lisp aren't a big fan of i) autotools, ii) m4, and iii) needlessly large distributions. Addressing these three complaints in one fell swoop, I present new versions of chutil and meta-ffi.

Ok, I'm finally getting my lazy act together and trying to package some of the software I've been working on for release. The first two packages are rather trivial, chutil and meta-ffi . They don't do much. chutil is some relatively trivial utilities and meta-ffi is a MOP metaclass for doing FFI to foreign structs. meta-ffi allows for making a :metaclass uffi-foreign-struct-class and then define slots with :foreign-field class. It's pretty trivial but a nice way for me to learn how to make a metaclass. There's no examples or documentation, but I hope to fix that in the near future.

will cause SBCL to rather unceremoniously exit. Apparpently, there are some compiler settings that determine the runtime memory layout and the gc sections only get 128MB each. I need to do some homework to figure out how to address this. I'd like to be able to use SBCL for image processing but, images from the d2x are 75MB each and I'd like to have more than one in memory at a time (or at least more than one representation of an image - especially a floating point representation and 16-bit per channel per pixel representation.

Ok, so it looks like the kewl kids on #lisp don't want Xach to syndicate my blog on planet.sbcl.org as that's reserved for output from CVS commit logs, chandler/chavatar's sbcl bulid-tester (very cool, BTW), and comments from ranking SBCL developers. I still don't want all of my drivel going to planet.lisp.org, which means it's relegated to the SBCL category of my blog, which means nobody will ever see it, but that's fine with me, at least for the moment.

Ok, on to my next SBCL problem. Profiling fails on PPC. I sent a bug to sbcl-devel, but so far the interest in learning more about the problem or fixing it seems rather low. I guess nobody else is trying to profile SBCL code on PPC at the moment. Changing timer-overhead-iterations from 500000 to 100000 fixes sb-profile:report, at least in the trivial case. This is helpful, but I'm not quite sure what the 500000 was for in the first place, or, more to the point, why bumping this down to 100000 fixes the problem.

I still don't understand how or why we get a large negative number instead of an unsigned-byte. Clearly more investigative work is required here.

Now that Xach has set up planet.sbcl.org , I'll move my gripes, helfpful (yeah, right...) comments, opinions, etc... about SBCL to the new SBCL category of my blog and maybe Xach will be kind enough ot pick up the feed.

As a first entry, let me continue to try to convince the SBCL gurus on #lisp that changing the size of compact-info-entries-index is a good thing.

As I have mentioned before, I've been trying to generate FFI definitions for the MacOS Carbon APIs and have run into a problem attempting to run purify, which gets called automatically by save-lisp-and-die. (I'm ignoring the fact that without threads and callbacks on the PPC, the carbon integration is going to be lacking but Brian Mastenbrook has been working on callbacks and hopefully I, or better yet, somebody who knows what they're doing, will have a chance to take a look at threads and the required gencgc stuff one of these days. Now back to the original thread).

The call to purify fails, giving an error saying that 65536 not being an (integer 0 65535) or something similar. Purify (and perhaps other things?) attempts to create a compact environment that contains the same information as an existing environment in a more compact representation. There is a variable compact-info-env-entries-bits that is used to determine the size (in bits) of the compact-info-env-entries-index type, which, in turn, is used as the type of the compact-info-inv/cache-index and as the type of the simple-array compact-info-env/index.

As an environment gets more than 65535 entries, it becomes impossible to compact it do tue the size of the index into the underlying data structures. So the obvious thing to try is to boost the size of the index. I made compact-info-env-entries-bits 32 and have been using this for a few weeks (on sbcl 0.8.20.xx on PPC) with no problems.

Enough background. Now for some specifics:

Why is this necessary? Without this patch one can only have 65535 functions in an environment.

The value 65536 is not of type (UNSIGNED-BYTE 16).
[Condition of type TYPE-ERROR]

What are the impacts of the change? The compact-info-entries-index type itself is only used by the index and cache-index fields of the compact-info-env struct. These data structures hold indices into a table of the particular things being stored in this compact-info-env. Note that this change doesn't change the size of the things in the table, just the size of the index and the cached index into the table, AFAICT.

There is one potential problem. At some point a table-size gets calculated based on the number of names in the environment. We call primify to get a prime number size for the table. primify declares its argument x as an (unsigned-byte x) and then iterates over the odd numbers >= x until it finds a prime number. But positive-prime declares its argument to be a fixnum, so it's possible that we could have a table size that is > most-positive-fixnum which cause positive-primep to fail. I haven't run into this in practice, but we might want to watch it for this and/or come up with a better approach for getting the size of the table. There's a comment about using almost-primify from hash-table.lisp, but this must be an out-of-date comment, as I can't find this function in hash-table.lisp. But failing when we get larger than most-positive-fixnum is much better than failing when we hit 65536 entries in a compact info env.

I'm getting tired of flogging this horse, but I think it's important that this get fixed. If anyone disagrees with what I've proposed, I'd love to hear it.