Hi all,
In reading the CMUCL source code for stream I found this simple
implementation for peeking...
;; lisp-stream
(generalized-peeking-mechanism
peek-type eof-value char
(read-char stream eof-errorp eof-value)
(unread-char char stream))
...and instantly realised that peeking a character invalidates the use of
unread character. As my code contains this algorithm in one place...
;;use up #\.
(read-char stream t)
(let ((nextchar (peek-char nil stream)))
(cond ...
(t ;;replace #\. if peeked char was not a delimiter
(unread-char firstchar stream)))))
...I was essentially using unread-char twice in a row.
This implementation of PEEK-CHAR is permitted by the Hyperspec: "The
consequences of invoking unread-char on any character preceding that which
is returned by peek-char (including those passed over by peek-char that
has a non-nil peek-type) are unspecified. In particular, the consequences
of invoking unread-char after peek-char are unspecified."
This means peek-char may be implemented as syntactic sugar that merely
obscures the dangerous use of unread-char. My implementation has to be
rewritten and it's going to be messy as the only way to know if a . makes
up the start of a symbol or number is by reading the next character. But
after the next character has been read it's no longer possible to replace
the . before calling read. Which means (a wrapper for) read now has to be
implemented as well.
I can now appreciate the design decisions behind CMUCL's reader
implementation!
Regards,
Adam

> But after the next character has been read it's no longer possible to
> replace the . before calling read. Which means (a wrapper for) read now
> has to be implemented as well.
Bless the specification writers and Common Lisp implementors.
I asked myself, "Wouldn't it be great if I could reconstruct the start of
the stream and join it to the existing stream so that I could pass a newly
constructed stream to read."
After consulting the Hyperspec it turns out I had described CONCATENATED
input streams! You just construct another stream which access the first
stream and seamlessly switches to the second stream after the first stream
runs out.
The essence of the fix is a rewrite of READ-MAYBE-NOTHING to support an
additional prependchar argument (nil for no character to prepend or a
character that is prepended to the stream):
(cond (prependchar
(list (read (make-concatenated-stream
(make-string-input-stream (string prependchar))
stream) t)))
(t (case peekedchar
(#\; (read-comment stream nil)
nil)
(otherwise (list (read stream t))))))
Regards,
Adam