Part I. Introduction

Pandocomatic is a tool to automate the use of pandoc. With pandocomatic you can express common
patterns of using pandoc for generating
your documents. Applied to a directory, pandocomatic can act as a static
site generator. For example, this manual is generated with
pandocomatic!

Chapter 1. Acknowledgements

I would like to thank Ian for
his contributions of patches, bug reports, fixes, and suggestions. With
your help pandocomatic is growing beyond a simple tool for personal use
into a useful addition to the pandoc ecosystem.

Chapter 2. Installation

Pandocomatic is a Ruby
program and can be installed through RubyGems as follows:

This will install pandocomatic and paru, a Ruby wrapper around pandoc. To use pandocomatic, you also need a
working pandoc installation. See pandoc’s installation guide for
more information about installing pandoc.

Sure, when I write about history, the CSL file and bibliography change. And I
do not need the --mathjax option like I do when I am writing
about mathematics education. Still, all these invocations are quite
similar.

I already wrote the program do-pandoc.rb as part of a
Ruby wrapper around pandoc,
paru. Using
do-pandoc.rb I can specify the options to pandoc as pandoc
metadata in the source file itself. The above pandoc invocation then
becomes:

It saves me from typing out the whole pandoc invocation each time I
run pandoc on a source file. However, I have still to setup the same
options to use in each document that I am writing, even though these
options do not differ that much from document to document.

Pandocomatic is a tool to re-use these common configurations
by specifying a so-called pandocomatic template in a YAML configuration file. For example, by placing
the following file, pandocomatic.yaml in pandoc’s data
directory:

With just two lines of pandoc metadata, I can tell pandocomatic what
template to use when converting a file. You can also use multiple
templates in a document, for example to convert a markdown file to both
HTML and PDF. Adding file-specific pandoc options to the conversion
process is as easy as adding a pandoc property with those
options to the pandocomatic_ metadata property in the source
file.

Once I had written a number of related documents this way, it was a
small step to enable pandocomatic to convert directories as well as
files. Just like that, pandocomatic can be used as a static site
generator!

Part II. Using pandocomatic: Quick start and
overview

Chapter 4. Converting a single
document

Pandocomatic allows you to put pandoc command line options in the
document to be converted itself. Instead of a complex pandoc command line
invocation, pandocomatic allows you to convert your markdown document
with just:

Pandocomatic starts by mining the YAML
metadata in hello_world.md for a pandocomatic_
property. If such a property exists, it is treated as an internal
pandocomatic template and the file is converted according to
that pandocomatic template. For more information about
pandocomatic templates, see the chapter about templates in this manual.

For example, if hello_world.md contains the following
pandoc markdown text:

pandocomatic is instructed by the pandoc section to
convert the document to the HTML file
hello_world.html. With the command-line option
--output goodday_world.html, you can instruct pandocomatic
to convert hello_world.md to goodday_world.html
instead. For more information about pandocomatic’s command-line options,
see the chapter about command-line
options in this manual.

You can instruct pandocomatic to apply any pandoc command-line option
in the pandoc section. For example, to use a custom pandoc
template and add a CSS file to the
generated HTML, extend the pandoc section as follows:

Besides the pandoc section to configure the pandoc
conversion, pandocomatic templates can also contain a list of
preprocessors and postprocessors.
Preprocessors are run before the document is converted with pandoc and
postprocessors are run afterwards:

Figure 4.1 How pandocomatic works: a simple conversion

For example, you can use the following script to clean up the HTML
generated by pandoc:

The path ./tidy.sh tells pandocomatic to look for the
tidy.sh script in the same directory as the file to convert.
You can also specify an absolute path (starting with a slash
/) or a path relative to the pandocomatic data
directory such as for the pandoc template. See the
Section about specifying paths in
pandocomatic for more information. If you use a path relative to the
pandocomatic data directory, you have to use the
--data-dir option to tell pandocomatic where to find its
data directory.

Thus, to convert the above example, use the following pandocomatic
invocation:

Chapter 5. Converting a series
of documents

5.1 Using external templates

Adding an internal pandocomatic template to a markdown file
helps a lot by simplifying converting that file via pandoc. Once you
start using pandocomatic more and more to convert your documents, you
will discover that most of these internal pandocomatic templates
are a lot alike. You can re-use these internal pandocomatic
templates by moving the common parts to an external
pandocomatic template.

External pandocomatic templates are defined in a
pandocomatic configuration file. A pandocomatic
configuration file is a YAML file. Templates are specified in the
templates section as named sub properties. For example, the
internal pandocomatic template specified in the
hello_world.md file (see previous chapter) can be specified
as the external pandocomatic templatehello in the
pandocomatic configuration filemy-config.yaml as
follows:

To convert external_hello_world.md you need to tell
pandocomatic where to find the external pandocomatic template
via the --config command-line option. For example, to
convert external_hello_world.md to out.html,
you use the following pandocomatic invocation:

5.2
Customizing external templates with an internal template

Because you can use an external pandocomatic templates in
many files, these external templates tend to setup more general options
of a conversion process. You can customize a conversion process in a
particular document by extending the internal pandocomatic
template. For example, if you want to apply a different CSS style
sheet and adding a table of contents, customize the hello
template with the following internal pandocomatic template:

hello’s pandoc section if extended with the
--toc option, the style.css is removed, and
goodbye-style.css is added. If you want to add the
goodbye-style.css rather than have it replace
style.css, you would specify:

Lists and properties in internal pandocomatic templates are
merged with external pandocomatic templates; simple values, such
as strings, numbers, or Booleans, are replaced. Besides the
pandoc section of a template you can also customize other
template sections.

5.3 Extending templates

In a similar way that an internal pandocomatic template
extends an external pandocomatic template you can also
extend an external pandocomatic template
directly in the pandocomaitc configuration file. For example,
instead of customizing the hello template, you could also
have extended hello as follows:

This author template specifies the metadata
section of a template. This metadata will be mixed into each document
that uses this template. If you want the goodbye template to
also set the author automatically, you can change its
extends section to:

Setting up templates by extending other smaller templates makes for a
modular setup. If you share your templates with someone else, she only
has to change the author template to her own name in one
place to automatically put her name on all her documents while using your
templates.

Chapter 6. Converting a
directory tree of documents

Once you have created a number of documents that can be converted by
pandocomatic, and you change something significant in one of the
external pandocomatic templates, you have to run pandocomatic on
all of the documents again to propagate the changes. That is fine for a
document or two, but more than that and it becomes a chore.

For example, if you change the pandoc template
hello-template.html in the pandocomatic data
directory, or switch to another template, you need to regenerate all
documents you have already converted with the old pandoc template. If you
run pandocomatic on an input directory rather than one input file, it
will convert all files in that directory, recursively.

7.1 General arguments: help
and version

Show the version. If this option is used, all other options are
ignored.

-h, --help

Show a short help message. If this option is used, all other
options except --version or -v are
ignored.

7.2 Input/output arguments

-i PATH, --input PATH

Convert PATH. If this option is not given,
INPUT is converted. INPUT and
--input or -i cannot be used together.

-o PATH, --output PATH

Create converted files and directories in PATH.

You can specify the output file in the metadata of a pandoc
markdown input file. In that case, you can omit the output argument.
Furthermore, if no output file is specified whatsoever, pandocomatic
defaults to output to HTML by replacing the extension of the input
file with html.

The input and output should both be files or both be directories.
Pandocomatic will complain if the input and output types do not
match.

Configure pandocomatic to use FILE as its
configuration file during the conversion process. Default is
DATA_DIR/pandocomatic.yaml.

7.4 Arguments to
change how pandocomatic operates

-m, --modified-only

Only convert files that have been modified since the last time
pandocomatic has been run. Or, more precisely, only those source
files that have been updated at a later time than the corresponding
destination files will be converted, copied, or linked. Default is
false.

-q, --quiet

By default pandocomatic is verbose when you convert a directory.
It tells you about the number of commands to execute. When executing
these commands, pandocomatic tells you what it is doing, and how many
commands still have to be executed. Finally, when pandocomatic is
finished, it tells you how long it took to perform the
conversion.

If you do not like this verbose behavior, use the
--quiet or -q argument to run pandocomatic
quietly. Default is false.

-y, --dry-run

Inspect the files and directories to convert, but do not actually
run the conversion. Default is false.

-b, --debug

Run pandocomatic in debug mode. At the moment this means that all
pandoc invocations are printed as well.

7.5 Status codes

When pandocomatic runs into a problem, it will return with status
codes 1266 or 1267. The former is returned if
something goes wrong before any conversion is started and the latter when
something goes wrong during the conversion process.

Chapter 8. Pandocomatic
configuration

Pandocomatic can be configured by means of a pandocomatic
configuration file, which is a YAML file. For example, the following
YAML code is a valid pandocomatic configuration file:

By default, pandocomatic looks for the configuration file in the
pandocomatic data directory; by convention this file is named
pandocomatic.yaml.

You can tell pandocomatic to use a different configuration file via
the command-line option --config. For example, if you want
to use a configuration file my-config.yaml, invoke
pandocomatic as follows:

These two sections are discussed after presenting an example of a
configuration file. For more in-depth information about pandocomatic
templates, please see the Chapter on
pandocomatic templates.

8.1 Settings

You can configure five optional global settings:

data-dir

match-files

skip

recursive

follow-links

The latter three are used only when running pandocomatic to convert a
directory tree. These are discussed in the next sub section.

The first setting, data-dir (String), tells pandocomatic
where its data directory is. You can also specify the
pandocomatic data directory via the command-line option
--data-dir. For example, if you want to use
~/my-data-dir as the pandocomatic data directory,
invoke pandocomatic as follows:

Any directory can be used as a pandocomatic data directory,
there are no conventions or requirements for this directory other than
being a directory. However, it is recommended to create a meaningful sub
directory structure rather than to put everything together. For example,
a sub directory for processors, filters, CSL files, and pandoc templates
makes it easier to point to these assets.

The setting match-files controls how pandocomatic chooses
a template to use to convert a file. Possible values for
match-files are first and all.
These options have the following effect: Pandocomatic matches a file to a
template as follows:

If the file has one or more use-template statements
in the pandocomatic metadata, it will use these specified
templates.

However, if no such templates are specified in the file,
pandocomatic tries to find global templates as follows:

If the setting match-files has value
all, all templates with glob pattern that matches
the input file are used to convert the file. For example, you can
have specified a template www to convert
*.md files to HTML and a template pdf
to convert *.md to PDF. In this case, a markdown
file will be converted to both output formats. You can use this
to generate a website with a print PDF page for each web
page.

If the setting match-files has value
first, the first template with a glob pattern that
matches the input file is used to convert the file.

This is the default.

Configuring converting a directory tree

You can convert a directory tree with pandocomatic by invoking
pandocomatic with a directory as input rather than a file. Of course,
once you start converting directories, more fine-grained control over
what files to convert than “convert all files” is required. There are
four settings you can use to control which files to convert. Three of
them are global settings, the other one is the glob property
of an external pandocomatic template. The glob
property is discussed later.

The three global settings to control which files to convert are:

recursive (Boolean), which tells pandocomatic to
convert sub directories or not. This setting defaults to
true.

follow-links (Boolean), which tells pandocomatic to
treat symbolic links as files and directories to convert or not. This
setting defaults to false.

skip (Array of glob patterns), which tells
pandocomatic which files not to convert at all. This setting defaults
to ['.*', 'pandocomatic.yaml']: ignore all hidden files
(starting with a period) and also ignore default pandocomatic
configuration files.

Default configuration

no configuration is specified via the command-line option
--config, and

no default configuration file (pandocomatic.yaml) can
be found in the pandocomatic data directory.

When converting a directory tree, each time pandocomatic enters a
directory, it also looks for a default configuration file to
update the current settings. In other words, you can have
pandocomatic behave differently in a sub directory than the current
directory by putting a pandocomatic.yaml file in that sub
directory that changes the global settings or external pandocomatic
templates.

8.2 Templates

Besides the global settings section, a pandocomatic
configuration file can also contain a templates
section. In the templates section you define the
external pandocomatic templates you want to use when converting
files with pandocomatic. Pandocomatic templates are discussed in detail
in the Chapter on pandocomatic
templates. The glob property of external
pandocomatic templates is related to configuring pandocomatic when
converting a directory. It tells pandocomatic which files in a directory
are to be converted with a template.

If you look at the example pandocomatic configuration file at
the start of this chapter, you see that the webpage template
is configured with property glob: ['*.md']. This tells
pandocomatic to apply the template webpage to all markdown
files with extension .md. In other words, given a directory
with the following files:

directory/
+ sub directory/
| + index.md
- index.md
- image.png

Running pandocomatic with the example pandocomatic configuration
file will result in the following result"

directory/
+ sub directory/
| + index.html
- index.html
- image.png

That is, all .md files are converted to HTML and all
other files are copied, recursively.

Chapter 9. Pandocomatic templates

Pandocomatic automates the use of pandoc by extracting common usage
patterns of pandoc into so called pandocomatic templates, which
you can then apply to your documents. As described in Part II, there are internal
and externalpandocomatic templates. The
difference between these two types of templates is their scope:
internal pandocomatic templates only affect the document they
are defined in, whereas external pandocomatic templates, which
are defined in a pandocomatic configuration file, affect all
documents that use that template.

Although you can create an one-off internal pandocomatic
template for a document—sometimes you just have an odd writing
project that differs too much from your regular writings—, most often you
use an external pandocomatic template and customize it in the
internal pandocomatic template.

In this Chapter the definition, extension, customization, and use of
templates are discussed in detail.

9.1 Defining a template

An external pandocomatic template is defined in the
templates section of a pandocomatic configuration
file. For example, in the following YAML code, the template
webpage is defined:

Each template is a YAML property in the templates
section. The property name is the template name. The property value is
the template’s definition. A template definition can contain the
following properties:

extends

glob

setup

preprocessors

metadata

pandoc

postprocessors

cleanup

Before discussing these properties in detail, the way pandocomatic
resolves paths used in these sections is described first. For paths can
be used in most of these properties.

Specifying paths

Because templates can be used in any document, specifying paths
pointing to assets to use in the conversion process is not
straightforward. Using global paths works, but has the disadvantage that
the templates are no longer shareable with others. Using local paths
works if the assets and the document to convert are located in the same
directory, but that does not hold for more general external
pandocomatic templates. As a third alternative, pandocomatic also
supports paths that are relative to the pandocomatic data
directory.

You can specify these types of paths as follows:

All local paths start with a ./. These paths
are local to the document being converted. When converting a
directory tree, the current directory is being prepended to the path
minus the ./.

On the Windows operating system, a local path starts with
.\. Note that backslashes might need escaping, like
.\\.

Global paths start with a /. These paths are
resolved as is. On the Windows operating system, a global
path starts with a letter followed by a colon and a backslash, for
example C:\. Note that backslashes might need escaping,
like .\\.

Paths relative to the pandocomatic data
directory do not start with a ./ nor a
/. These paths are resolved by prepending the path to
the pandocomatic data directory. These come in handy for
defining general usable external pandocomatic templates.

Note. For filters, the path is first checked against the
PATH. If pandocomatic finds an executable matching the
path, it will resolve that executable instead.

Template properties

extends

A template can extend zero or more templates by supplying a list of
template names to extend. The extension builds from left to right.

glob

When a template is used for converting files in a directory tree, you
can specify which files in the directory should be converted by a
template. The glob section expects a list of glob
patterns. All files that match any of these glob patterns are
converted using this template.

When there are more templates that have matching glob patterns, the
first one is used.

If there is also a skip configured (see the Section on global settings, the skip
setting has precedence of the glob setting. Thus, if
skip is ['*.md'] and glob is
['*.md'], the template will not be applied.

Examples

Apply this template to all files with extension .md
(i.e. all markdown files):

setup

For more involved conversion patterns, some setup of the environment
might be needed. Think of setting Bash environment variables, creating
temporary directories, or even installing third party tools needed in the
conversion. Startup scripts can be any executable script.

preprocessors

After setup, pandocomatic executes all preprocessors in order of
specification in the preprocessor property, which is a list.
A preprocessor is any executable script that takes as input the document
to convert and outputs that document after “preparing” it somehow. You
can use a preprocessor to add metadata, include other files, replace
strings, and so on.

metadata

Metadata is used in pandoc’s templates as well as a means of
communicating with a filter. Some metadata is common to many documents,
such as language, author, keywords, and so on. In the
metadata property of a template you can specify this global
metadata. The metadata property is a map.

For convenience, the virtual output format pdf is added
by pandocomatic. It allows you to specify PDF output without needing to
use the output option. This allows for general pandoc
configurations for generating PDF files. You specify the PDF output
format by to: pdf. Pandocomatic will determine the actual
output format based on the value of pdf-engine. If that
option is not set, pandocomatic defaults to latex.

To give the use more control over what filename extension will be
used, the virtual pandoc option use-extension has been
added. If set, and the output option is not being used, the
value of the use-extension option is used as the extension
of the output file. For example, to generate a PDF presentation using the
beamer output format, you can specify the following pandoc options:

Finally, the virtual pandoc option rename has been added
to allow you to rename the output file via a script. This script will
receive the destination path on STDIN and is supposed to
write the renamed output path to STDOUT. It allows you to
perform quite complex behavior with regards to the output directory and
name of output files.

I use this virtual pandoc option when I am generating my static sites
with both HTML and PDF output and my input file is named
index.md. For the HTML format I want index.html
as the output file name, but for the PDF output I do not want
index.pdf as output filename. Instead, I prefer to use the
name of the input directory with extenstion .pdf. To that
end I setup pandocomatic as follows:

postprocessors

Similar to the preprocessors property, the
postprocessors property is a list of scripts to run after
the pandoc conversion. Each post processor takes as input the converted
document and outputs that document with the changes made by the post
processor. Post processors come in handy for cleaning up output, checking
for dead links, do string replacing, and so on.

cleanup

The counterpart of the setup property. The
cleanup property is a list of scripts to run after the
conversion of the document. It can be used to clean up temporary files,
resetting the environment, uploading the resulting document, and so
on.

If the parent value is a list, two different extension
mechanisms can take effect depending on the type of the child’s
value:

If the child is a list as well, the resulting value will
be the child’s list merged with the parent’s list. Duplicate
values will be removed. Lists in pandocomatic templates are
treated as sets. Examples:

If the child is a map, it is assumed to have properties
remove and add, both lists. The
resulting value will be the parent’s value with the items
from the remove list removed and items from the
add list added. Examples:

To remove a property in a child template, that child’s value should be
nil. You can create a nil value in YAML by
having a key without a value.

9.3
Customizing an external template in an internal template

To use an external pandocomatic template you have to use it
in a document by creating an internal pandocomatic template
which has the use-template property set to the name of the
external pandocomatic template you want to use. After that, you
can customize the template to you liking, for example adding extra pandoc
command-line options or adding another preprocessor.

You create an internal pandocomatic template by adding a
pandocomatic_ section to the document’s YAML metadata. The
pandocomatic_ section can have the same properties as an
external pandocomatic template except for the glob
and extends properties. Actually, you can add these two
properties as well, but they are ignored.

For example, if you use the my-webpage template, but you
would like to use a different bibliography and check all links in the
converted document, your document would look like:

Multiple conversions

The use-template property can also be a list of
external pandocomatic template names. In that case, the document
is converted once for each of these templates. For example, this allows
you to generate both a HTML and a PDF version of a document at the same
time:

Do note, however, that an internal pandocomatic template will
apply to all used external pandocomatic templates. It is not
possible to customize one used template differently than another. This
means that you have to move the customization to the used external
pandocomatic templates or you have customize the internal
pandocomatic template such that it is applicable to all used
external pandocomatic templates (as in the example above).

Part IV. Appendix

Chapter 10. Frequently asked questions (FAQ)

10.1 How do I use pandoc2
with pandocomatic?

Pandocomatic uses paru to run pandoc.
Paru itself uses the pandoc executable in your
PATH. If that already is pandoc2, you do not need to do
anything.

If you have pandoc1 installed by default, however, and you want to run
a nightly
version of pandoc2, you have to set the PARU_PANDOC_PATH
to point to the pandoc2 executable. For example: