2011/07/10

Quasiquoting

It may be arguable a bug, or an unspecified behavior in R5RS.
The outer-level quasiquote should expand unquote-splicing,
and that should produce (unquote 1 2); however, quasiquote
syntax in R5RS does not allow such pattern, so the resulting
form wouldn't be a valid quasiquotation. It doesn't explain
the above result, but if it's an error in R5RS term,
anything can happen.
(In fact, the result wasn't intended at all;
it was just an outcome of failing recognition of the pattern.)

Still, it is allowed to have (unquote 1 2) as a pure datum,
it may be more reasonable that ``(,,@(list 1 2))
expands into `((unquote 1 2)).

R6RS solved this ambiguity by allowing unquote to have
multiple arguments. Since it is upper compatible to R5RS,
I decided to implement R6RS behavior in Gauche. Now you get this:

gosh> ``(,,@(list 1 2))
`((unquote 1 2))

And this:

gosh> `((unquote 1 2))
(1 2)

★ ★ ★

While thinking about this, I found a difference between
Scheme and CL about quasiquoting that I was not aware before.

In Scheme, `a, ,a, ,@a are merely a reader-level
abbreviation
of (quasiquote a), (unquote a) and (unquote-splicing a),
respectively. The real semantics of quasiquoting is defined in terms
of the latter S-expressions.

In CL, quasiquoting (bakcquotes) are defined in reader level.
That is, comma-datum doesn't need to be an abbreviation of something
like (unquote datum) or (comma datum). It doesn't
need to have list equivalents at all. The comma and comma-atmark
can be directly parsed by the reader, and transformed
to whatever piece of code that produces the desired result.

It is not obvious in a simple case, but see the followings:

In Allegro:

cl-user> `'`(,,(cons 1 (cons 2 nil)))
'(excl::bq-list (1 2))

In CLisp:

[6]> `'`(,,(cons 1 (cons 2 nil)))
'(LIST (1 2))

This means the inner backquote form are interpreted rather
liberally.

On the other hand, I believe in Scheme this is the only
possible result (modulo printing `a or (quasiquote a) etc.)

gosh> `'`(,,(cons 1 (cons 2 '())))
'`(,(1 2))

A practical outcome of this difference is that, in CL,
you cannot write an S-expression with a comma without corresponding backquote.

Kaz Kylheku (2011/10/29 01:58:25):

In CL, it is not specified where backquotes are expanded. It could be done at read time, or the syntax can produce Scheme-like quasiquote macro which does the work at macro-expansion time.

What makes ``(,,@form) work in CL is the specification in the HyperSpec which establishes a correspondence between the quasiquote syntax (using the read syntax since that's the only representation) and the equivalent list construction code.

See CLHS 2.4.6.

Thus, analyzing its inner backquote, ``(,,@(list a b)) can be understood as the intermediate fictitious form `(append [,,@(list a b)]) which matches the pattern [,form] -> (list form), thus giving us `(append (list ,@(list a b)). The list of values produced by (LIST A B) is spliced into the outer (list ...) becoming its arguments. LIST, of course, handles multiple arguments!
A macro-based quasiquote in Common Lisp doesn't have to have an UNQUOTE operator which handles multiple arguments. If the semantics described in 2.4.6 are obeyed while translating the macro into list construction, the right behavior will pop out.

Cheers ... kaz <at> kylheku.com

shiro (2011/10/29 11:20:02):

Thanks for clear explanation, Kaz.

It caught me interesting that you showed how to analyze from "inner" backquote / "outer" unquote, while in Scheme I tend to think from "outer" backquote / "inner" unquote. Certainly if you unfold unquotes outside-in, you don't need to deal with multiple argument unquote.

Bill Schottstaedt (2012/05/31 22:24:54):

I noticed another consequence of Scheme's choice to treat
,x as (unquote x) at the reader level. (let (, 'a) unquote) is 'a,
but I think in CL it's an error. Similarly,
(let (, (lambda (x) (+ x 1))) ,,,,'3) is 7 if I counted right.
The same trick works with unquote-splicing. Now
to try quasiquote...

shiro (2012/06/01 04:07:49):

Yes, that's a technique pervasive in the code golfing community, those who try to write the shortest program to solve the given problem.

I don't know whether this "feature" is good or bad... certainly it smells bad, and I don't want to read such code. OTOH, it is at least consistent, and I see no reason to prohibit it ("Possibility for people to abuse" isn't a good reason to prohibit a feature, I think).