As I currently understand them, fully-expandable macros are analogous to pure/effect-free functions in functional programming. Correspondingly, things that are not expandable, such as \def, are effectful computations -- they can't be "evaluated" inside an \edef, for example. Effect-free functions are generally beneficial (think immutable objects in OOP), and I would expect this to carry over to fully-expandable TeX macros.

There are very rare occasion when it may be useful to create functions using a
fully-expandable argument grabber. [...] This facility should only be used
when absolutely necessary; if you do not understand when this might be, do not
use these functions!

So, when should I use fully-expandable macros? What are the advantages and disadvantages of fully-expandable macros?

3 Answers
3

I think it is best not to compare the expandable/not expandable
distinction to concepts from other languages. The main issues relating
to expansion are really particular (some would say peculiar) to the
execution model of TeX.

TeX has two main modes of operation.All assignments and boxing
operations happen (in "the stomach" in TeXBook terminology) as
non-expandable operations. Macro expansion happens before that but
unlike (say) the macro expansion of the C pre-processor, macro
expansion and non-expandable operations are necessarily intertwined.

It is probably worth noting that the question as posed is not well defined.

TeX tokens are either expandable or non-expandable but "fully expandable"
is a grey area full of traps into which the unwary may fall.

Any token defined by \def (or \newcommand etc) is by definition expandable.

then each of these is expandable, with expansion <nothing>a\def\zze{}\ifmmode a \else b\fi respectively.

However which of these is fully expandable ?

Clearly \zza is. But if the definition of "fully expandable" means may be expanded repeatedly leaving no unexpandable tokens then the only fully expandable tokens will all expand to nothing.

So most preople would class \zzb as fully expandable, even though it expands to a which is not expandable.

So a better (or at least more accurate) term than "fully expandable" is "safe in an expansion-only context". Inside \edef and \write and when TeX is looking for a number or dimension, and a few other places, TeX only does expansion and does not do any assignment or other non-expandable operations.

\edef\yyb{\zzb}

is of course safe, it is the same as def\yyb{a}. So \yyb is safe in an expansion-only context.

\edef\yyc{\zzc}

is not safe, it is the same as

\edef\yyc{\def\zze{}}

Now \def doesn't expand but in an expansion-only context the token
just stays inert it does not make a definition so TeX then tries to
expand \zze which typically is not yet defined so this leads to an
error, or if \zze has a definition then this will be expanded which
is almost always unwanted behaviour. This is the basic cause of the
infamous "fragile command in a moving argument" errors in LaTeX.

So \zzc is not safe in an expansion-only context. If it had been defined by the e-TeX construct

\protected\def\zzc{\def\zze{}}

Then in an expansion-only construct protected tokens are made non-expandable so

\edef\yyc{\zzc}

would then be safe, and the sane as \def\yyc{\zzc} So a protected
command is safe in an expansion-only context but since this safety
comes by making the token temporarily non-expandable it probably isn't
accurate to say it is "fully expandable".

\edef\yyd{\zzd}

is

\edef\yyd{\ifmmode a \else b\fi}

which is

\def\yyd{b}

or \def\yyd{a} if the definition is happening inside $...$ (or an equation display). Similarly it will
expand to b at the start of an array cell as the expansion will
happen while Tex is expanding looking for \omit (\multicolumn) and
so before it has inserted the $ to put the array cell in to math
mode. Again a protected definition to limit expansion is what is
required here.

So sometimes it is good to make things expandable as it keeps more
options open.

both \testa{n}{yes}{no} and \testb{n}{yes}{no} will exectute yes
if n is 0 and no otherwise but \testb works by expansion and
so is safe in an expansion-only context (if its arguments are
safe). The \testa version relies on the internal non-expandable
operation of \def\next. (Plain TeX and LaTeX2.09 use many tests
using \def\next LaTeX2e changed them to the expandable form where
possible.)

For a numeric test it is easy to use the expandable form, but if you
want to test if two "strings" are equal by far the easiest way is to go

but now even though we have used the \expandafter\firstoftwo
construct, the test relies on two non-expandable definitions. If you
really need to test in an expandable way you can find some questions
on this site but any answer is typically full of special conditions
and cases where it doesn't work and relies on some kind of slow token
by token loop through the two arguments testing if they are equal. In
99% of the cases this complication is just not needed and the
non-expandable test is sufficient. If you are trying to define a
consistent set of tests (as in the ifthen package \ifthenelse for
example, then if you resign yourself to the fact that some tests are
necessarily non-expandable then you may choose to make them all
non-expandable so they behave in a consistent way.

On the e-TeX protected approach, in expl3 we've been very clear that everything is either expandable or uses the protected mechanism, to avoid people being caught out by this.
–
Joseph Wright♦Aug 7 '12 at 10:01

Thanks for the excellent answer! One side question I have is: "Why is \expandafter needed in front of \firstoftwo and \secondoftwo in the \testb and \testc examples?" I also noticed the same in Martin Scharrer's answer about \@firstoftwo etc.. Let me know if I should ask this as a full-fledged question instead of in a comment.
–
Henry DeYoungAug 9 '12 at 18:36

3

if you remove the expandafter and the test is true the \firstoftwo will be expanded which will take two arguments and use the first, but its arguments would be the next two tokens which would be \else and \secondoftwo which isn't what you want. The \expandafter expandnds the \else before teh \firstoftwo and the expansion of \else eats everything up to the \fi so then \firstoftwo sees the two supplied arguments as #1 and #2
–
David CarlisleAug 9 '12 at 18:40

Nice answer! A small error though: \ifmmode can be true or false within an \edef: you might be confusing with \write (and \immediate\write) where indeed the mode is set to no mode. Test: $\edef\foo{\ifmmode Math\fi}\show\foo$\bye gives Math.
–
Bruno Le FlochJul 24 '14 at 23:51

While TeX is Turing complete, this says nothing about whether a particular operation can be carried out expandably. Thus the first issue with aiming when considering if something can be coded expandably is whether it is even possible. As discussed in Why isn't everything expandable?, some operations are not expandable as the underlying primitives are not. Obvious cases include assignments, grouping and any form of typesetting in boxes. This means that for example it is not possible to measure the typeset width of material in an expandable manner, and so you can't do what you might aim for with something like

\newdim\mydim
\mydim\wd\hbox{foo}%

even though this seems 'logical', and instead have to take an approach equivalent to

\newdim\mydim
\newbox\mybox
\setbox\mybox=\hbox{foo}%
\mydim\wd\mybox

(This shows up more clearly when you start wanting to do more complex expressions.)

The second issue, alluded to in the xparse manual, is that there are often limitations to expandable methods which do not show up in non-expandable ones. A case that is particularly noticeable is looking ahead for optional arguments. In a non-expandable manner, this is done using \futurelet (an assignment, and so not expandable). To do the same expandably, you have to grab an argument. That method is not as robust, as if we have

\foo{[} ...

then \futurelet will detect {, while argument grabbing will see [. Assuming we are looking for a LaTeX-like optional argument, the expandable test will be 'wrong'. So expandable code here is more limited in terms of what it can deal with.

Writing expandable code is often much more tricky than the non-expandable case. You cannot save anything other than by leaving it in the input stream in an known way, which can be very hard to keep a track of. This can also have performance implications, as for a complex operation you may need to keep track of an awful lot of stuff.

A related issue here is that there are two types of expandability. Within a \edef, TeX will keep expanding tokens until only non-expandable ones remain. In this case, you can add 'output' tokens to the front of the input stream and have TeX continue to process the rest of the operation. On the other hand, you can do an expandable expansion using \romannumeral-`\q, However, this stops on the first non-expandable token. So if you want to write a function which works inside this 'full' expansion you have to watch the output tokens very carefully. (The LaTeX3 expl3 language differentiates between these two forms of expansion as x- and f-type, respectively, and the documentation indicates whether expandable functions will work properly inside f-type situations.)

This might make you wonder why you'd write expandable functions. As the question notes, there are advantages. The most obvious is that they can be used inside assignments, such as setting dimensions. Bruno Le Floch has written an expandable FPU for LaTeX3, which would be a great example of this. TeX also expands tokens in some other contexts, most obviously at the start of a table (\halign) cell to look for \omit or \noalign, and there you need to use expandable methods if you want to deal (easily) with escaping from alignment. (There are other approaches to this!)

When TeX writes to files, it carries out full expansion. This may or may not be desirable with a given macro. For example, a lot of LaTeX macros are writing without expansion to the .aux file as it's the 'wrong' time to expand.

In general, with the exception of programming layers (where you do want expandability if possible), creating expandable macros is not usually a priority, hence the statement in xparse. Broadly, I would look to create expandable functions when I know that one is needed.

Your first sentence made me wonder. For a suitable/sensible definition of "expandable", do we know the complexity/expressibility of TeX expansion? (if it's not Turing-complete, what is it?)
–
Juan A. NavarroAug 8 '12 at 15:02

Thanks for the answer! I did not realize that there are two forms of "full" expansion. One other question: if there were some way to grab a single token argument (as opposed to the standard "either single token or group"), would the problem that you mention with expandably grabbing optional arguments disappear?
–
Henry DeYoungAug 9 '12 at 18:41

@HenryDeYoung That's not really a question with an answer :-) The way you test reliably for an optional argument grabs using \futurelet, which provided an implicit token. That can then be tested with \ifx or similar. However, if you (hypothetically) grabbed an explicit { then you'd still have the issue of doing the test. The problem is that this is not how TeX works, so it's not really possible to give an answer.
–
Joseph Wright♦Aug 9 '12 at 21:38