Introspection

Today's lesson is about introspection - interactive exploration of the
Forth system itself. Many Forth systems have some degree of introspection;
the facilities in Open Firmware are fairly extensive.

Decompiler

The decompiler "see" displays a pretty-printed representation
of the source code for the target word.

ok see here
: here
dp @
;

In the example above, the target word was a system-defined colon definition.
In addition to the colon definitions, which we have already learned about,
Forth has some other kinds of words; we will learn about them later.
The decompiler knows about those other kinds of words:

The example above shows a word "+" which is implemented in assembly
language. A few words at the core of the system are like this, but
most of the several thousand words are implemented as colon definitions.

You can also use the disassembler for assembly language code that
is not in the form of Forth code words. For example, if you have loaded
a Linux kernel at the address 0x100000, you could inspect it with:

ok 100000 dis

You can decompile the entire Forth system, down to the very roots. So
in some sense, Open Firmware has always been "open source", since the
very first Open Firmware system included the full decompiler. (Some
will no doubt quibble about this claim; in practice, for all the
"free speech" talk, most people really want "free beer".)

Look-ahead Words

Most Forth words take their arguments from the stack, but there are
a few that get one or more string arguments by parsing ahead in
the input stream. We have already seen some examples of this, notably
comments and ":". "see" is another example. The amount of "look ahead"
in such cases is typically fixed, not context-dependent. (You could
write a context-dependent look-ahead word, but such usage is exceedingly
rare in common Forth usage.)

For most "look-ahead" words, there is a variant version with a slightly
different name that takes its arguments from the stack instead of from
the input stream. The look-ahead versions are convenient for interactive
use, whereas the stack-based versions are convenient for use within other
words.

Wordfinder

The "sifting" command parses a whitespace-delimited string from the
input stream and searches for words whose name have it as a substring.

You may see several lines like "In vocabulary forth", with other names
in place of "forth". The system contains several "vocabularies" (lists
of words). The "forth" vocabulary is by far the largest, containing
all of the core Forth words. There are other vocabularies with words
that implement special-purpose functions. There is a search order that
controls which vocabularies the interpreter searches at any given time.
Forth Lesson 21 describes vocabularies and search orders in more detail.

Open Firmware "device tree nodes" are implemented as vocabularies with
a special structure. "sifting" looks only in the ordinary vocabularies,
but there is a similar word for looking in device tree nodes:

ok sift-devs show-b
In device /pci/nandflash@c
(ff88db78) show-bbt

Callfinder

To find where a word is used,

ok ' save-image .calls
Called from save-image at b7dca444
Called from save-forth at b7dca4e8

The first item in the list of usages is often the word itself. That does
not necessarily mean that the word is recursive. Rather, it is an artifact
of the algorithm that the callfinder uses. So don't believe the list
completely. The callfinder will find every compiled-in use of the word,
but it will sometimes get spurious "hits" too. The callfinder will *not*
find transient usages, i.e. references to the word that are stored in
variables.

Execution Tokens and "'"

The stack diagram of ".calls" is "( xt -- )". "xt" is the notation for
"execution token", which is a numeric identifier that uniquely specifies
a particular instance of a word. Recall that you can reuse the same
word name several times, with the older instances remaining present in the
system (possibly still called within other words). Each such instance
has a distinct execution token. In many implementations, the execution
token is the memory address of a data structure representing the word,
but portable code must treat execution tokens as opaque identifiers.

"'" (a single quote) is a standard word that parses a whitespace-delimited name from the
input stream and pushes the execution token of the most recent definition
of that name. Normally, when the Forth interpreter encounters a word,
it executes it, but there are times when you want to refer to the word
instead of executing it. That is what "'" is for. You can think of it
as "quoting" the word. Note that "'" applies to already-defined Forth
words, not to arbitrary text strings. If you try to apply "'" to a string
that is not a defined word, you will get an error message.

Referring back to the "sifting" example above, the number in parentheses
before each word is its execution token. You can use this to get a
handle on words whose names have been redefined. "(see)" is a variant
of "see" whose argument is an execution token from the stack instead of
a string from the input stream:

ok b7dce2d4 (see)

The number above comes from the "sifting valid" example; it happens to
be the execution token of "state-valid" in the system I'm using right now.
In general, that number probably won't work on your system. But you could
say:

ok ' here (see)

"sifting <something>" then "<xt> (see)" (where <xt> is a number you type
based on looking at the output of "sifting") is a good way to look at
the definitions of superseded versions of words.

'' inside definitions

Inside a colon definition, ' won't do what you probably expect.

ok : myword ' save-image .calls ;

You might expect your new word "myword" to be equivalent to interactively
executing:

ok ' save-image .calls

but that is not what happens. What happens is that ' parses the
input stream when "myword" is executed, not when "myword" is compiled.

Here is how to write what you probably meant:

ok : myword ['] save-image .calls ;

['] is a variant of ' that does the look-ahead parsing during compilation.

Yes, I realize this is confusing. It's an historical artifact of the Forth language. Just use ' outside colon definitions and ['] inside,
and the right thing will happen.