Navigation

TableGen’s purpose is to help a human develop and maintain records of
domain-specific information. Because there may be a large number of these
records, it is specifically designed to allow writing flexible descriptions and
for common features of these records to be factored out. This reduces the
amount of duplication in the description, reduces the chance of error, and makes
it easier to structure domain specific information.

The core part of TableGen parses a file, instantiates the declarations, and
hands the result off to a domain-specific backend for processing.

Note that if you work on TableGen much, and use emacs or vim, that you can find
an emacs “TableGen mode” and a vim language file in the llvm/utils/emacs and
llvm/utils/vim directories of your LLVM distribution, respectively.

TableGen files are interpreted by the TableGen program: llvm-tblgen available
on your build directory under bin. It is not installed in the system (or where
your sysroot is set to), since it has no use beyond LLVM’s build process.

TableGen runs just like any other LLVM tool. The first (optional) argument
specifies the file to read. If a filename is not specified, llvm-tblgen
reads from standard input.

To be useful, one of the backends must be used. These backends are
selectable on the command line (type ‘llvm-tblgen-help’ for a list). For
example, to get a list of all of the definitions that subclass a particular type
(which can be useful for building up an enum list of these records), use the
-print-enums option:

The default backend prints out all of the records. There is also a general
backend which outputs all the records as a JSON data structure, enabled using
the -dump-json option.

If you plan to use TableGen, you will most likely have to write a backend
that extracts the information specific to what you need and formats it in the
appropriate way. You can do this by extending TableGen itself in C++, or by
writing a script in any language that can consume the JSON output.

With no other arguments, llvm-tblgen parses the specified file and prints out all
of the classes, then all of the definitions. This is a good way to see what the
various definitions expand to fully. Running this on the X86.td file prints
this (at the time of this writing):

This definition corresponds to the 32-bit register-register add instruction
of the x86 architecture. defADD32rr defines a record named
ADD32rr, and the comment at the end of the line indicates the superclasses
of the definition. The body of the record contains all of the data that
TableGen assembled for the record, indicating that the instruction is part of
the “X86” namespace, the pattern indicating how the instruction is selected by
the code generator, that it is a two-address instruction, has a particular
encoding, etc. The contents and semantics of the information in the record are
specific to the needs of the X86 backend, and are only shown as an example.

As you can see, a lot of information is needed for every instruction supported
by the code generator, and specifying it all manually would be unmaintainable,
prone to bugs, and tiring to do in the first place. Because we are using
TableGen, all of the information was derived from the following definition:

This definition makes use of the custom class I (extended from the custom
class X86Inst), which is defined in the X86-specific TableGen file, to
factor out the common features that instructions of its class share. A key
feature of TableGen is that it allows the end-user to define the abstractions
they prefer to use when describing their information.

TableGen has a syntax that is loosely based on C++ templates, with built-in
types and specification. In addition, TableGen’s syntax introduces some
automation concepts like multiclass, foreach, let, etc.

TableGen files consist of two key parts: ‘classes’ and ‘definitions’, both of
which are considered ‘records’.

TableGen records have a unique name, a list of values, and a list of
superclasses. The list of values is the main data that TableGen builds for each
record; it is this that holds the domain specific information for the
application. The interpretation of this data is left to a specific backend,
but the structure and format rules are taken care of and are fixed by
TableGen.

TableGen definitions are the concrete form of ‘records’. These generally do
not have any undefined values, and are marked with the ‘def’ keyword.

In this example, FeatureFPARMv8 is SubtargetFeature record initialised
with some values. The names of the classes are defined via the
keyword class either on the same file or some other included. Most target
TableGen files include the generic ones in include/llvm/Target.

TableGen classes are abstract records that are used to build and describe
other records. These classes allow the end-user to build abstractions for
either the domain they are targeting (such as “Register”, “RegisterClass”, and
“Instruction” in the LLVM code generator) or for the implementor to help factor
out common properties of records (such as “FPInst”, which is used to represent
floating point instructions in the X86 backend). TableGen keeps track of all of
the classes that are used to build up a definition, so the backend can find all
definitions of a particular class, such as “Instruction”.

Here, the class ProcNoItin, receiving parameters Name of type string and
a list of target features is specializing the class Processor by passing the
arguments down as well as hard-coding NoItineraries.

TableGen multiclasses are groups of abstract records that are instantiated
all at once. Each instantiation can result in multiple TableGen definitions.
If a multiclass inherits from another multiclass, the definitions in the
sub-multiclass become part of the current multiclass, as if they were declared
in the current multiclass.

TableGen files have no real meaning without a back-end. The default operation
of running llvm-tblgen is to print the information in a textual format, but
that’s only useful for debugging of the TableGen files themselves. The power
in TableGen is, however, to interpret the source files into an internal
representation that can be generated into anything you want.

Current usage of TableGen is to create huge include files with tables that you
can either include directly (if the output is in the language you’re coding),
or be used in pre-processing via macros surrounding the include of the file.

Direct output can be used if the back-end already prints a table in C format
or if the output is just a list of strings (for error and warning messages).
Pre-processed output should be used if the same information needs to be used
in different contexts (like Instruction names), so your back-end should print
a meta-information list that can be shaped into different compile-time formats.

Despite being very generic, TableGen has some deficiencies that have been
pointed out numerous times. The common theme is that, while TableGen allows
you to build Domain-Specific-Languages, the final languages that you create
lack the power of other DSLs, which in turn increase considerably the size
and complexity of TableGen files.

At the same time, TableGen allows you to create virtually any meaning of
the basic concepts via custom-made back-ends, which can pervert the original
design and make it very hard for newcomers to understand the evil TableGen
file.

There are some in favour of extending the semantics even more, but making sure
back-ends adhere to strict rules. Others are suggesting we should move to less,
more powerful DSLs designed with specific purposes, or even re-using existing
DSLs.

Either way, this is a discussion that will likely span across several years,
if not decades. You can read more in the TableGen Deficiencies
document.