[clisp-list] How can I override readtable capital letters and avoid
|barring-all-symbols| ?

Can anyone suggest help/insight involving clisp's readtable when capital letters are overridden?
I'm implementing some extensions to the Common Lisp reader that, I hope, is more readable. My implementation works in clisp, but there's an oddity that I hope someone here can answer.
Basically, when in clisp I override the readtable's *capital* letters, all the produced symbols are later written with vertical bars. I don't want that, and sbcl doesn't do this.
Is there a way to prevent this behavior? Or at least, can someone who's more familiar with clisp's implementation explain what's going on and why? I presume this involves the "modern" mechanisms, which I'm happy to support, but no matter what happens the vertical-bars seem to be forced.
One reason I'm doing this is because I want "f(a b c)" to be interpreted as "(f a b c)" in my reader. If you're curious about the details, please check us out at readable.sourceforge.net.
--- David A. Wheeler
P.S. Here's sample code that demonstrates this:
=====================================
(defvar *original-readtable* (copy-readtable)
"Use this table when about to read a neoteric expression")
(defun ugly (stream char)
(unread-char char stream)
(let ((*readtable* *original-readtable*))
(read stream)))
(dolist (c
'(#\! #\$ #\% #\& #\* #\+ #\- #\. #\/
#\A))
(set-macro-character c #'ugly nil))
(write (read))
abc
; Writes "|ABC|", not "abc" or "ABC".
--- David A. Wheeler

Thread view

Can anyone suggest help/insight involving clisp's readtable when capital letters are overridden?
I'm implementing some extensions to the Common Lisp reader that, I hope, is more readable. My implementation works in clisp, but there's an oddity that I hope someone here can answer.
Basically, when in clisp I override the readtable's *capital* letters, all the produced symbols are later written with vertical bars. I don't want that, and sbcl doesn't do this.
Is there a way to prevent this behavior? Or at least, can someone who's more familiar with clisp's implementation explain what's going on and why? I presume this involves the "modern" mechanisms, which I'm happy to support, but no matter what happens the vertical-bars seem to be forced.
One reason I'm doing this is because I want "f(a b c)" to be interpreted as "(f a b c)" in my reader. If you're curious about the details, please check us out at readable.sourceforge.net.
--- David A. Wheeler
P.S. Here's sample code that demonstrates this:
=====================================
(defvar *original-readtable* (copy-readtable)
"Use this table when about to read a neoteric expression")
(defun ugly (stream char)
(unread-char char stream)
(let ((*readtable* *original-readtable*))
(read stream)))
(dolist (c
'(#\! #\$ #\% #\& #\* #\+ #\- #\. #\/
#\A))
(set-macro-character c #'ugly nil))
(write (read))
abc
; Writes "|ABC|", not "abc" or "ABC".
--- David A. Wheeler

"David A. Wheeler" <dwheeler@...> writes:
> Can anyone suggest help/insight involving clisp's readtable when
> capital letters are overridden?
>
> I'm implementing some extensions to the Common Lisp reader that, I
> hope, is more readable.
Bouahahahaha, bouahahaha, bouahahahaha!
Well, at least you'll learn about CL reader macros.
> My implementation works in clisp, but there's
> an oddity that I hope someone here can answer.
>
> Basically, when in clisp I override the readtable's *capital* letters,
> all the produced symbols are later written with vertical bars. I
> don't want that, and sbcl doesn't do this.
That's because you have *print-escape* set to true.
> Is there a way to prevent this behavior?
Try:
(setf *print-escape* nil)
and of course:
(setf *print-readably* nil)
but if that wasn't the case, you'd get:
|COMMON-LISP-USER|::|ABC|
> Or at least, can someone
> who's more familiar with clisp's implementation explain what's going
> on and why?
Well, we could, but perhaps it'd be better if you just read the printer
chapter of CLHS (along with the syntax chapter and the reader chapter).
> I presume this involves the "modern" mechanisms, which I'm happy to
> support, but no matter what happens the vertical-bars seem to be
> forced.
No, it's totally unrelated to the modern mode. It's just one
interpretation of the CLHS.
> One reason I'm doing this is because I want "f(a b c)" to be
> interpreted as "(f a b c)" in my reader. If you're curious about the
> details, please check us out at readable.sourceforge.net.
While it's possible to hack the lisp reader by making a lot of reader
macros, for your purpose it's probably much simplier to have a single
reader macro, and to write a normal parser. Since you seem to like
braces, you could write a #\{ reader macro that would read up to the
next balanced #\}, and parse the content with a normal scanner and
parser, returning the sexp you want.
See for example:
http://groups.google.com/group/comp.lang.lisp/browse_frm/thread/245342429c61800/f7966bf3df9f716c?hl=en&q=objective-cl+group:comp.lang.lisp+author:pascal+author:bourguignon#f7966bf3df9f716c
(I like brackets).
So you'd write:
'{
define factorial(n)
if {n <= 1}
1
{n * factorial{n - 1}}
factoria(42)
}
--> (progn (defun factorial (n)
(if (<= n 1)
1
(* n (factorial (- n 1)))))
(factorial 42))
> P.S. Here's sample code that demonstrates this:
>
> =====================================
>
>
> (defvar *original-readtable* (copy-readtable)
> "Use this table when about to read a neoteric expression")
How is the "original" readtable, ie. the read time at the time this
variable is initialized, interesting at all?
This read table can be far from the standard read table.
In general, I prefer to use the standard read table, since it is
precisely specified. You can get it with (copy-readtable nil).
Check clhs copy-readtable.
> (defun ugly (stream char)
> (unread-char char stream)
> (let ((*readtable* *original-readtable*))
> (read stream)))
>
> (dolist (c
> '(#\! #\$ #\% #\& #\* #\+ #\- #\. #\/
> #\A))
> (set-macro-character c #'ugly nil))
You should have set your macro characters non-terminating. When they're
terminating, you cannot evaluate your code again, you have to escape
special characters:
(defvar |*ORIGINAL-READTABLE*| (|COPY-READTABLE|)
"Use this table when about to read a neoteric expression")
(defun ugly (stream char)
(|UNREAD-CHAR| char stream)
(let ((*readtable* |*ORIGINAL-READTABLE*|))
(read stream)))
(dolist (c
'(#\! #\$ #\% #\& #\* #\+ #\- #\. #\/
#\A))
(|SET-MACRO-CHARACTER| c #'ugly t)) ; <-- after than, you can
; again drop the escapes.
> (write (read))
> abc
>
> ; Writes "|ABC|", not "abc" or "ABC".
WRITE respects all the *print-…* variables.
--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Pascal J. Bourguignon:
> Try:
>
> (setf *print-escape* nil)
Thanks very much for the response!
Sadly, that's not what I want. If I then (without "-modern") read |abc|, the printer will print abc *WITHOUT* the vertical bars. The problem is that then, when I read it back in later, it will no longer really be |abc| but be ABC... not the same.
I *want* clisp to print vertical bars when they are needed... but only when they are needed. Sbcl does this, and clisp normally does this. Some special mode or flag setting seems to happen when clisp reads through a capital later constituent if that letter has had its readtable set-macro-character.
> Well, we could, but perhaps it'd be better if you just read the printer
> chapter of CLHS (along with the syntax chapter and the reader chapter).
Already have; this seems to be clisp-unique behavior.
...
I appreciate the note that it's easier to just catch {...},
but I really want to catch all text, from the outset, even if it's trickier.
Also, a clarification: My goal isn't to translate Scheme to Common Lisp,
it's to create a variant of s-expression notation where (for example)
indentation implies parentheses.
If someone wants "defun", they'll still need to say "defun" :-).
Thus, the factorial example in Common Lisp would look like:
> defun factorial (n)
> if {n <= 1}
> 1
> {n * factorial{n - 1}}
>
> factorial 42
To continue...
> How is the "original" readtable, ie. the read time at the time this
> variable is initialized, interesting at all?
> This read table can be far from the standard read table.
> In general, I prefer to use the standard read table, since it is
> precisely specified. You can get it with (copy-readtable nil).
> Check clhs copy-readtable.
Sure, but it's possible that someone has modified the readtable. I'd like to include those changes, if possible, so I didn't use the optional parameter for copy-readtable. If someone wants to use the original readtable, they can reset and then enable.
> > (defun ugly (stream char)
> > (unread-char char stream)
> > (let ((*readtable* *original-readtable*))
> > (read stream)))
> >
> > (dolist (c
> > '(#\! #\$ #\% #\& #\* #\+ #\- #\. #\/
> > #\A))
> > (set-macro-character c #'ugly nil))
>
> You should have set your macro characters non-terminating. When they're
> terminating, you cannot evaluate your code again, you have to escape
> special characters:
If I do that, "read" may gobble up whitespace afterwards, which I specifically do *not* want it to do. Unless I've misunderstood something, which is always possible.
Anyway, any other thoughts would be great.
Thanks.
--- David A. Wheeler

"David A. Wheeler" <dwheeler@...> writes:
> Pascal J. Bourguignon:
>
>> Try:
>>
>> (setf *print-escape* nil)
>
> Thanks very much for the response!
>
> Sadly, that's not what I want. If I then (without "-modern") read
> |abc|, the printer will print abc *WITHOUT* the vertical bars. The
> problem is that then, when I read it back in later, it will no longer
> really be |abc| but be ABC... not the same.
>
> I *want* clisp to print vertical bars when they are needed... but only
> when they are needed.
But they are needed! Since you put a reader macro on #\A, the printer
cannot assume that ABC will be read as the symbol ABC. The printer has
to print a symbol named "ABC" so that it is read back as the symbol
named "ABC". With an overriden reader macro, it has to escape it.
> Sbcl does this,
I won't comment on sbcl here…
> and clisp normally does this.
> Some special mode or flag setting seems to happen when clisp reads
> through a capital later constituent if that letter has had its
> readtable set-macro-character.
You could revert to a natural *readtable* when printing.
(defvar *original-readtable* (copy-readtable))
(defvar *ugly-readtable* (copy-readtable))
(defun ugly (stream char)
(unread-char char stream)
(let ((*readtable* *original-readtable*))
(read-preserving-whitespace stream)))
(dolist (c '(#\! #\$ #\% #\& #\* #\+ #\- #\. #\/
#\A))
(set-macro-character c #'ugly t *ugly-readtable*))
;; reading using the ugly readtable by default:
[5]> (setf *readtable* *ugly-readtable*)
#<READTABLE #x0003343FD4F8>
[6]> (read-from-string "abc")
|ABC| ;
3
;; reading by explicitely specifying the ugly readtable, using the
;; standard or original readtable by default:
[7]> (setf *readtable* *original-readtable*)
#<READTABLE #x0003343FB2D8>
[8]> (let ((*readtable* *ugly-readtable*)) (read-from-string "abc"))
ABC ;
3
;; explicitely specifying the readtable while reading and while printing:
[9]> (let ((*readtable* *original-readtable*))
(print (let ((*readtable* *ugly-readtable*)) (read-from-string "abc"))))
ABC
ABC
;; works whatever the default readtable is:
[10]> (setf *readtable* *ugly-readtable*)
#<READTABLE #x0003343FD4F8>
[11]> (let ((*readtable* *original-readtable*))
(print (let ((*readtable* *ugly-readtable*)) (read-from-string "abc"))))
ABC
|ABC|
[12]>
>> Well, we could, but perhaps it'd be better if you just read the printer
>> chapter of CLHS (along with the syntax chapter and the reader chapter).
>
> Already have; this seems to be clisp-unique behavior.
The point is not whether it's unique, but whether it's conforming.
AFAIK, it is conforming.
>> How is the "original" readtable, ie. the read time at the time this
>> variable is initialized, interesting at all?
>> This read table can be far from the standard read table.
>> In general, I prefer to use the standard read table, since it is
>> precisely specified. You can get it with (copy-readtable nil).
>> Check clhs copy-readtable.
>
> Sure, but it's possible that someone has modified the readtable. I'd
> like to include those changes, if possible, so I didn't use the
> optional parameter for copy-readtable. If someone wants to use the
> original readtable, they can reset and then enable.
Good.
>> > (defun ugly (stream char)
>> > (unread-char char stream)
>> > (let ((*readtable* *original-readtable*))
>> > (read stream)))
>> >
>> > (dolist (c
>> > '(#\! #\$ #\% #\& #\* #\+ #\- #\. #\/
>> > #\A))
>> > (set-macro-character c #'ugly nil))
>>
>> You should have set your macro characters non-terminating. When they're
>> terminating, you cannot evaluate your code again, you have to escape
>> special characters:
>
> If I do that, "read" may gobble up whitespace afterwards, which I
> specifically do *not* want it to do. Unless I've misunderstood
> something, which is always possible.
Then use READ-PRESERVING-WHITESPACE
--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Pascal J. Bourguignon" <pjb@...> wrote:
> WRITE respects all the *print-…* variables.
Your previous email gave me a useful hint, thanks. It appears that the behavior of clisp's WRITE is affected by the currently-active readtable. If I read in stuff, then switch to the traditional readtable, the extra vertical bars are no longer displayed by clisp.
So... is there a way to disable that clisp feature of adding |...| when the readtable is different?
--- David A. Wheeler

"David A. Wheeler" <dwheeler@...> writes:
> Pascal J. Bourguignon" <pjb@...> wrote:
>> WRITE respects all the *print-…* variables.
>
> Your previous email gave me a useful hint, thanks. It appears that
> the behavior of clisp's WRITE is affected by the currently-active
> readtable. If I read in stuff, then switch to the traditional
> readtable, the extra vertical bars are no longer displayed by clisp.
Yes. That is the suggestion in my new answer.
> So... is there a way to disable that clisp feature of adding |...|
> when the readtable is different?
I don't know. One would have to have a look at the sources.
--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.

I asked:
> > So... is there a way to disable that clisp feature of adding |...|
> > when the readtable is different?
"Pascal J. Bourguignon" <pjb@...> wrote:
> I don't know. One would have to have a look at the sources.
Okay, I've looked through the clisp code and I found the problem.
The issue is in file "src/io.d" function "pr_symbol_part"
(which starts line 7004 on 2013-05-06), with this code.
It forces |...| around symbols based on what happens to be in the current readtable:
======================================================
var object syntax_table; /* syntaxcode-table, with char_code_limit elements */
var uintW rtcase; /* readtable-case */
{
var object readtable;
get_readtable(readtable = ); /* current Readtable */
syntax_table = TheReadtable(readtable)->readtable_syntax_table;
rtcase = RTCase(readtable);
}
/* traverse string: */
SstringDispatch(string,X, {
...
if (!(syntax_table_get(syntax_table,c) == syntax_constituent))
goto surround; /* no -> must use |...| */
======================================================
Would the clisp folks be willing to entertain a tweak so that
users could better control this auto-escaping, if they desire?
For example, perhaps there could be a *pr-symbol-readtable* variable that defaults
to nil. Then, in the code above, use the readtable referenced in *pr-symbol-readtable*
if it's non-nil, and if it's nil (the default), use the current readtable.
That would maintain the current behavior by default, but be more flexible.
--- David A. Wheeler

"David A. Wheeler" <dwheeler@...> writes:
> I asked:
>> > So... is there a way to disable that clisp feature of adding |...|
>> > when the readtable is different?
>
> "Pascal J. Bourguignon" <pjb@...> wrote:
>> I don't know. One would have to have a look at the sources.
>
> Okay, I've looked through the clisp code and I found the problem.
> The issue is in file "src/io.d" function "pr_symbol_part"
> (which starts line 7004 on 2013-05-06), with this code.
> It forces |...| around symbols based on what happens to be in the current readtable:
>
> ======================================================
> var object syntax_table; /* syntaxcode-table, with char_code_limit elements */
> var uintW rtcase; /* readtable-case */
> {
> var object readtable;
> get_readtable(readtable = ); /* current Readtable */
> syntax_table = TheReadtable(readtable)->readtable_syntax_table;
> rtcase = RTCase(readtable);
> }
> /* traverse string: */
> SstringDispatch(string,X, {
> ...
> if (!(syntax_table_get(syntax_table,c) == syntax_constituent))
> goto surround; /* no -> must use |...| */
> ======================================================
>
>
>
> Would the clisp folks be willing to entertain a tweak so that
> users could better control this auto-escaping, if they desire?
No. As I demonstrated, you can already control it, binding *readtable*
when it matters to you.
--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.
You can take the lisper out of the lisp job, but you can't take the lisp out
of the lisper (; -- antifuchs

Pascal J. Bourguignon:
> No. As I demonstrated, you can already control it, binding *readtable*
> when it matters to you.
That's not sufficient control for my purposes.
My goal is to bind the readtable and have it *STAY* that way. Even during the REPL, even after it returns a value, and so on. (Another approach would be to have a way to replace "read", though that would not be portable.)
--- David A. Wheeler

Community

Help

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

I agree to receive quotes, newsletters and other information from sourceforge.net and its partners regarding IT services and products. I understand that I can withdraw my consent at any time. Please refer to our Privacy Policy or Contact Us for more details