The TeX primitive \output behaves almost like a toks register, except in the fact that it is always surrounded by braces. For instance, after \output={\plainoutput}, the result of \showthe\output is {\plainoutput}, instead of what we could naively expect (no braces).

The status of the surrounded braces is a little odd. My specific problem happens when replacing the closing brace with an explicit or implicit end-group character which does not directly come from TeX's internal mechanisms.

Specifically, compiling the following example in plain TeX triggers the ! Unbalanced output routine. error at line 6: the first case works, but not the second.

In both cases, I redefine the output routine, then force TeX to call it (with \penalty-10000). I keep the normal \plainoutput routine, and afterwards grab the closing brace in \next, and put it back, either directly (\next), or indirectly (\Next). In the first, direct, case, TeX recognizes the closing brace as being from the \output routine, while in the second case, TeX doesn't, and gets into a state where nothing I type has any effect.

Why are the two situations different?

Assuming that I have typed the code above in the command line, is there any way to recover?

but typing } doesn't do any good, as we fall in the black hole when TeX doesn't interpret any more token.

It's interesting that TeX adds braces when the assignment to \output is of the form

\output=<general text>

but not when it's like \output=<token register>.

The relevant code for the addition of braces is in module 1226, where Knuth comments "For safety’s sake, we place an enclosing pair of braces around an \output list.", and module 1227.

An interesting module to examine is 1100, which ends with output_group: followed by what is in module 1026.

I realize this is not a full answer: it's only to show that it's better not to monkey with the closing brace. TeX is in a very particular type of group, when it performs the output routine and disturbing it during this task reveals to be quite dangerous.

As with any other case where we can have the left brace as an implicit {, the right brace has to be either } itself or an implicit } not hidden within a macro. So hiding } inside \Next will fail in exactly the same way as for any other situation. (You can test that the } is not 'special' in any other way by using \def\Next{\egroup}, which works fine.)

You'll also notice that the output group is actually started not by the { but is explicitly coded, as TeX knows that there has to be a group here. scan_left_brace finds an (implicit) left brace, but throws it away.

@JosephWright Does \Next, by having being defined end up in the output as a box? If this is the case then this is the cause and why the first one works and not the second example.
–
Yiannis LazaridesDec 25 '11 at 12:43

EDIT: my bad, I was wrong. But I think this is the only case where not hiding the implicit } in a macro matters. Normally, an implicit left brace can be hidden in as many macro expansions as we like (TeX uses get_x_token), and right braces are almost always explicit.
–
Bruno Le FlochDec 25 '11 at 12:46

Actually, you mention "any other situation", but is there really any similar situation?
–
Bruno Le FlochDec 25 '11 at 13:11

begin if (loc<>null) or
((token_type<>output_text)and(token_type<>backed_up)) then
@<Recover from an unbalanced output routine@>;

I don't understand when the loc<>null is true/false. But the rest is easier to understand: the token_type tells us where the token comes from: is it inserted by TeX, is it <to be read again>, etc. Here, output_text is for tokens coming from the \output token parameter, which includes the closing brace, and backed_up is for tokens which are skipped over by \expandafter. I don't know why Knuth chose to allow those two types of tokens (well, output_text is obvious): any other type of token makes the test true, which calls the code for an unbalanced output routine. Thus, to close the output routine ourselves, we can simply use \expandafter\egroup\empty, as below: