Text Formatting

Even with proliferation of graphical and voice user interfaces, text remains one
of the main ways for humans to interact with computer programs and programming
languages provide a variety of methods to perform text formatting.
The first thing we do when learning a new programming language is often writing
a "Hello, World!" program that performs simple formatted output.

C++ has not one but two standard APIs for doing formatted output, the
printf family of functions inherited from C and the I/O streams
library (iostreams).
While iostreams are usually the recommended way of doing formatted
output in C++ for safety and extensibility reasons, printf offers
some advantages, such as arguably more natural function call API, separation of
formatted message and arguments, possibly with argument reordering as a POSIX
extension, and often more compact code, both source and binary.

This paper proposes a new text formatting library that can be used as a
safe and extensible alternative to the printf family of functions.
It is intended to complement the existing C++ I/O streams library and reuse
some of its infrastructure such as overloaded insertion operators for
user-defined types.

Variations of the printf format string syntax are arguably the most popular
among the programming languages and C++ itself inherits printf
from C [1]. The advantage of the printf syntax is that many
programmers are familiar with it. However, in its current form it has a number
of issues:

Many format specifiers like hh, h, l,
j, etc. are used only to convey type information.
They are redundant in type-safe formatting and would unnecessarily
complicate specification and parsing.

There is no standard way to extend the syntax for user-defined types.

There are subtle differences between different implementations. For example,
POSIX positional arguments [2] are not supported on
some systems [6].

Using '%' in a custom format specifier, e.g. for
put_time-like time formatting, poses difficulties.

Although it is possible to address these issues while maintaining resemblance
to the original printf format, this will still break compatibility and can
potentially be more confusing to users than introducing a different syntax.

Therefore we propose a new syntax based on the ones used in Python
[3], the .NET family of languages [4],
and Rust [5]. This syntax employs '{' and
'}' as replacement field delimiters instead of '%'
and it is described in details in the syntax reference.
Here are some of the advantages:

Consistent and easy to parse mini-language focused on formatting rather
than conveying type information

Extensibility and support for custom format strings for user-defined
types

Positional arguments

Support for both locale-specific and locale-independent formatting (see
Locale support)

The syntax is expressive enough to enable translation, possibly automated,
of most printf format strings. The correspondence between printf
and the new syntax is given in the following table.

printf

new

-

<

+

+

space

space

#

#

0

0

hh

unused

h

unused

l

unused

ll

unused

j

unused

z

unused

t

unused

L

unused

c

c (optional)

s

s (optional)

d

d (optional)

i

d (optional)

o

o

x

x

X

X

u

d (optional)

f

f

F

F

e

e

E

E

a

a

A

A

g

g (optional)

G

G

n

unused

p

p (optional)

Width and precision are represented similarly in printf and the
proposed syntax with the only difference that runtime value is specified by
* in the former and {} in the latter, possibly with
the index of the argument inside the braces.

As can be seen from the table above, most of the specifiers remain the same
which simplifies migration from printf. Notable difference is
in the alignment specification. The proposed syntax allows left, center,
and right alignment represented by '<', '^',
and '>' respectively which is more expressive than the
corresponding printf syntax. The latter only supports left and
right (the default) alignment.

The following example uses center alignment and '*' as a fill
character:

fmt::format("{:*^30}", "centered");

resulting in "***********centered***********".
The same formatting cannot be easily achieved with printf.

Both the format string syntax and the API are designed with extensibility in
mind. The mini-language can be extended for user-defined types and users can
provide functions that do parsing and formatting for such types.

The general syntax of a replacement field in a format string is

replacement-field ::= '{' [arg-id] [':' format-spec] '}'

where format-spec is predefined for built-in types, but can be
customized for user-defined types. For example, the syntax can be extended
for put_time-like date and time formatting

Formatting functions rely on variadic templates instead of the mechanism
provided by <cstdarg>. The type information is captured
automatically and passed to formatters guaranteeing type safety and making
many of the printf specifiers redundant (see
Format String Syntax). Buffer management is automatic to prevent
buffer overflow errors common to printf.

As pointed out in
P0067R1: Elementary string conversions there is a number of use
cases that do not require internationalization support, but do require high
throughput when produced by a server. These include various text-based
interchange formats such as JSON or XML. The need for locale-independent
functions for conversions between integers and strings and between
floating-point numbers and strings has also been highlighted in
N4412: Shortcomings of iostreams. Therefore a user should be able to
easily control whether to use locales or not during formatting.

We follow Python's approach [3] and designate a separate format
specifier 'n' for locale-aware numeric formatting. It applies to
all integral and floating-point types. All other specifiers produce output
unaffected by locale settings. This can also have positive effect on performance
because locale-independent formatting can be implemented more efficiently.

An important feature for localization is the ability to rearrange formatting
arguments because the word order may vary in different languages
[7]. For example:

printf("String `%s' has %d characters\n", string, length(string)));

A possible German translation of the format string might be:

"%2$d Zeichen lang ist die Zeichenkette `%1$s'\n"

using POSIX positional arguments [2]. Unfortunately these
positional specifiers are not portable [6]. The C++ I/O
streams don't support such rearranging of arguments by design because they
are interleaved with the portions of the literal string:

The formatting library has been designed with performance in mind. It tries to
minimize the number of virtual function calls and dynamic memory allocations
done per a formatting operation. In particular, if formatting output can fit
into a fixed-size buffer allocated on stack, it should be possible to avoid
them altogether by using a suitable API.

To this end, a buffer abstraction represented by the
fmt::basic_buffer template is introduced. A buffer is a contiguous
block of memory that can be accessed directly and can optionally grow. Only one
virtual function, grow, needs to be called during formatting and
only when the buffer is not large enough.

The locale-independent formatting can also be implemented more efficiently than
the locale-aware one. However, the main goal for the former is to support
specific use cases (see Locale support) rather than to
improve performance.

In order to minimize binary code size, each formatting function that uses
variadic templates is a small inline wrapper around its non-variadic
counterpart. This wrapper creates an object representing an array of argument
references with fmt::make_args and calls the non-variadic function
to do the actual work. For example, the format variadic function
calls vformat.

Multiple argument type codes can be combined and passed into a
formatting function as a single integer if the number of arguments is small.
Since argument types are known at compile time this can be an integer
constant and there will be no code generated to compute it, only to store
according to calling conventions.

Given a reasonable optimizing compiler, this will result in a compact
per-call binary code, effectively consisting of placing argument pointers
(or, possibly, copies for primitive types) and packed argument type codes on stack
and calling a formatting function.

The formatting library uses a null-terminated string view
basic_cstring_view instead of basic_string_view.
This results in somewhat smaller and faster code because the string size,
which is not used, doesn't have to be computed and passed. Also having
a termination character makes parsing easier.

Format strings contain replacement fields surrounded by curly braces
{}. Anything that is not contained in braces is considered literal
text, which is copied unchanged to the output. A brace character can be
included in the literal text by doubling: {{ and }}.

In less formal terms, the replacement field can start with an
arg-id that specifies the argument whose value is to be formatted
and inserted into the output instead of the replacement field. The
arg-id is optionally followed by a format-spec,
which is preceded by a colon ':'. These specify a non-default
format for the replacement value.

If the numerical arg-ids in a format string are 0, 1, 2, ... in
sequence, they can all be omitted (not just some) and the numbers 0, 1, 2, ...
will be automatically inserted in that order.

Some simple format string examples:

"First, thou shalt count to {0}" // References the first argument
"Bring me a {}" // Implicitly references the first argument
"From {} to {}" // Same as "From {0} to {1}"

The format-spec field contains a specification of how the value
should be presented, including such details as field width, alignment, padding,
decimal precision and so on. Each value type can define its own formatting
mini-language or interpretation of the format-spec.

Most built-in types support a common formatting mini-language, which is
described in the next section.

A format-spec field can also include nested replacement fields
in certain position within it. These nested replacement fields can contain only
an argument index; format specifications are not allowed. This allows the
formatting of a value to be dynamically specified.

Format specifications are used within replacement fields contained
within a format string to define how individual values are presented (see
Format string syntax). Each formattable type may define
how the format specification is to be interpreted.

Most built-in types implement the following options for format specifications,
although some of the formatting options are only supported by the numeric types.

The fill character can be any character other than '{'
or '}'. The presence of a fill character is signaled by the
character following it, which must be one of the alignment options. If the
second character of format-spec is not a valid alignment option,
then it is assumed that both the fill character and the alignment option are
absent.

The meaning of the various alignment options is as follows:

Option

Meaning

'<'

Forces the field to be left-aligned within the available space (this is
the default for most objects).

'>'

Forces the field to be right-aligned within the available space (this is
the default for numbers).

'='

Forces the padding to be placed after the sign (if any) but before the
digits. This is used for printing fields in the form
+000000120. This alignment option is only valid for numeric
types.

'^'

Forces the field to be centered within the available space.

Note that unless a minimum field width is defined, the field width will always
be the same size as the data to fill it, so that the alignment option has no
meaning in this case.

The sign option is only valid for number types, and can be one of
the following:

Option

Meaning

'+'

Indicates that a sign should be used for both positive as well as negative
numbers.

'-'

Indicates that a sign should be used only for negative numbers (this is
the default behavior).

space

Indicates that a leading space should be used on positive numbers, and a
minus sign on negative numbers.

The '#' option causes the alternate form to be used for
the conversion. The alternate form is defined differently for different types.
This option is only valid for integer and floating-point types. For integers,
when binary, octal, or hexadecimal output is used, this option adds the prefix
respective "0b" ("0B"), "0", or
"0x" ("0X") to the output value. Whether the prefix
is lower-case or upper-case is determined by the case of the type specifier,
for example, the prefix "0x" is used for the type 'x'
and "0X" is used for 'X'. For floating-point numbers
the alternate form causes the result of the conversion to always contain a
decimal-point character, even if no digits follow it. Normally, a decimal-point
character appears in the result of these conversions only if a digit follows it.
In addition, for 'g' and 'G' conversions, trailing
zeros are not removed from the result.

width is a decimal integer defining the minimum field width. If
not specified, then the field width will be determined by the content.

Preceding the width field by a zero ('0') character
enables sign-aware zero-padding for numeric types. This is equivalent to a
fill character of '0' with an alignment
type of '='.

The precision is a decimal number indicating how many digits should
be displayed after the decimal point for a floating-point value formatted with
'f' and 'F', or before and after the decimal point
for a floating-point value formatted with 'g' or 'G'.
For non-number types the field indicates the maximum field size - in other
words, how many characters will be used from the field content. The
precision is not allowed for integer, character, Boolean, and
pointer values.

Finally, the type determines how the data should be presented.

The available string presentation types are:

Type

Meaning

's'

String format. This is the default type for strings and may be omitted.

none

The same as 's'.

The available character presentation types are:

Type

Meaning

'c'

Character format. This is the default type for characters and may be
omitted.

none

The same as 'c'.

The available integer presentation types are:

Type

Meaning

'b'

Binary format. Outputs the number in base 2. Using the '#'
option with this type adds the prefix "0b" to the output
value.

'B'

Binary format. Outputs the number in base 2. Using the '#'
option with this type adds the prefix "0B" to the output
value.

'd'

Decimal integer. Outputs the number in base 10.

'o'

Octal format. Outputs the number in base 8.

'x'

Hex format. Outputs the number in base 16, using lower-case letters for the
digits above 9. Using the '#' option with this type adds the
prefix "0x" to the output value.

'X'

Hex format. Outputs the number in base 16, using upper-case letters for the
digits above 9. Using the '#' option with this type adds the
prefix "0X" to the output value.

'n'

Number. This is the same as 'd', except that it uses the
buffer's locale to insert the appropriate number separator characters.

none

The same as 'd'.

Integer presentation types can also be used with character and Boolean values.
Boolean values are formatted using textual representation, either true or false,
if the presentation type is not specified.

The available presentation types for floating-point values are:

Type

Meaning

'a'

Hexadecimal floating point format. Prints the number in base 16 with prefix
"0x" and lower-case letters for digits above 9. Uses
'p' to indicate the exponent.

'A'

Same as 'a' except it uses upper-case letters for the prefix,
digits above 9 and to indicate the exponent.

'e'

Exponent notation. Prints the number in scientific notation using the
letter 'e' to indicate the exponent.

'E'

Exponent notation. Same as 'e' except it uses an upper-case
'E' as the separator character.

'f'

Fixed point. Displays the number as a fixed-point number.

'F'

Fixed point. Same as 'f', but converts nan to
NAN and inf to INF.

'g'

General format. For a given precision p >= 1, this rounds the
number to p significant digits and then formats the result in
either fixed-point format or in scientific notation, depending on its
magnitude.
A precision of 0 is treated as equivalent to a precision of
1.

'n'

Number. This is the same as 'g', except that it uses the
buffer's locale to insert the appropriate number separator characters.

none

The same as 'g'.

The available presentation types for pointers are:

Type

Meaning

'p'

Pointer format. This is the default type for pointers and may be
omitted.

none

The same as 'p'.

Formatting functions

Effects: The function returns a string object
constructed from the format string argument format_str with each
replacement field substituted with the character representation of the
argument it refers to, formatted according to the specification given in the
field.

Returns: The formatted string.

Throws: format_error if format_str is not a
valid format string.

string vformat(cstring_view format_str, fmt::args args);

Effects: The function returns a string object
constructed from the format string argument format_str with each
replacement field substituted with the character representation of the
argument it refers to, formatted according to the specification given in the
field.

Effects: The function appends to buf the format string
format_str with each replacement field substituted with the character
representation of the argument it refers to, formatted according to the
specification given in the field.

Effects: The function appends to buf the format string
format_str with each replacement field substituted with the
character representation of the argument it refers to, formatted according to
the specification given in the field.

Remarks:
The return type is the common type of all possible INVOKE
expressions in the Effects section. Since exact value types are
implementation-defined, visitors should use type traits to handle multiple
types.

Complexity: The invocation of the callable object does not depend on
the number of possible values types of a formatting argument.

The class basic_buffer<T> represents a contiguous
memory buffer with an optional growing ability. An instance of
basic_buffer<T> stores elements of type T.
The elements of a basic_buffer are stored contiguously, meaning
that if b is a basic_buffer<T> then it obeys
the identity &b.data()[n] == &b.data()[0] + n for
all 0 <= n < d.size().

basic_buffer() noexcept;

Effects: Constructs an empty buffer.

Postcondition: size() == 0.

set(Char* s, size_type n) noexcept;

Effects: Sets data and capacity.

Postcondition: data() == s and
capacity() == n.

size_type size() const noexcept;

Returns: The buffer size.

size_type capacity() const noexcept;

Returns: The total number of elements that the buffer can hold
without requiring reallocation.

Effects: A directive that informs a buffer of a planned change in
size, so that it can manage the storage allocation accordingly. After
reserve(), capacity() is greater or equal to the
argument of reserve if reallocation happens; and equal to the
previous value of capacity() otherwise. Reallocation happens
at this point if and only if the current capacity is less than the argument of
reserve(), and it is performed by calling grow(n).
If an exception is thrown other than by the move constructor of a
non-CopyInsertable type, there are no effects.

void grow(size_type n);

Requires: n > capacity().

Effects: Reallocates the buffer to increase its capacity to at least
n in a way that is defined separately for each class derived
from basic_buffer.

Throws: length_error if n exceeds a limit
defined by a derived class, in particular, if the latter has a fixed capacity.

Char* data() noexcept;
const Char* data() const noexcept;

Returns: A pointer such that [data(), data() + size())
is a valid range.

Effects: The function replaces the buffer controlled by
*this with a buffer of length size() + n whose first
size() elements are a copy of the original buffer controlled by
*this and whose remaining elements are a copy of the elements
in the range.

locale locale() const;

Returns: The locale to be used for locale-specific formatting.
The default implementation returns a copy of the global C++ locale but derived
classes may return different locales.

User-defined types

If a format string refers to an object of a user-defined type as in

X x;
string s = format("{}", x);

the formatting function will call format_value(buf, x, ctx),
where buf is a reference to the formatting buffer, x
is a const reference to the argument and ctx is a reference to
the formatting context. ctx.ptr() will point to one of the
following positions in the format string being parsed:

':' preceding format-spec for the current
argument,

'}' if there is no format-spec.

The format_value function should parse format-spec,
format the argument and advance ctx.ptr() to point to
'}' that ends replacement-field for the current
argument.

The default implementation of format_value calls ostream insertion
operator<< to format the value.

The Boost Format library [8] is an established formatting library that uses
printf-like format string syntax with extensions. The main differences between
this library and the current proposal are:

Syntax: for the reasons descibed in section
Format String Syntax this proposal
uses a new syntax instead of extending the printf one. This allows much
simpler and easier to parse grammar, not burdened by legacy specifiers used to
convey type information. For example, Boost Format has two ways
to refer to an argument by index and allows but ignores some format specifiers.

Performance: The implementation of this proposal is several times faster
that the implementation of Boost Format on tinyformat benchmarks [9],
generates smaller binary code and is faster to compile.

A printf-like Interface for the Streams Library [10] is similar to the Boost
Format library but uses variadic templates instead of operator%.
Unfortunately it hasn't been updated since 2013 and the same arguments about
format string syntax apply to it.

The FastFormat library [11] is another well-known formatting library.
Similarly to this proposal, FastFormat uses brace-delimited format specifiers,
but otherwise the format string syntax is different and the library has
significant limitations [12]:

Three features that have no hope of being accommodated within the current
design are:

Leading zeros (or any other non-space padding)

Octal/hexadecimal encoding

Runtime width/alignment specification

Formatting facilities of the Folly library [13] are the closest to the current
proposal. Folly also uses Python-like format string syntax nearly identical
to the one described here. However, the API details are quite different. The current
proposal tries to address performance and code bloat issues that are largely
ignored by Folly Format. For instance formatting functions in Folly Format
are parameterized on all argument types while in this proposal, only the inlined
wrapper functions are, which results in much smaller binary code and better compile
times.