A blog about music engraving and LilyPond notation software.

Reveries of a Solitary Music Typographer: about November 2.0

Unlike designers of (digital) text fonts, music font designers historically were not restricted by standards and had great freedom, which has been both a blessing and a curse. While there was some common sense about what the kernel of music symbols should be (clefs, noteheads, accidentals, etc.), the actual position of characters in the font, their naming (though there was generally none provided), and the addition of rarer symbols beyond the basic set was left up to the designer’s imagination and to some specific requirements of the target music notation software.

Things are changing drastically with SMuFL, a font standardisation effort initiated by Daniel Spreadbury from Steinberg and now hosted at W3C in a welcome “joint venture” with MusicXML. This addresses issues of symbol position, naming and repertoire in a universal way, the idea being a world where fonts can be used in different scoring applications – just as everybody expects text fonts to be usable in any text processor or layout program. SMuFL is a great source of inspiration for the designer – surely one of its benefits – but it also imposes new constraints and requirements, and leads to a more demanding design workflow.

November 1.0 within the evolution of music fonts

In 1998 I designed November, an alternative music font, specifically for Finale. It had a distinctive “vivid” design and its repertoire of 330 characters, spread over two font files, ranging through historical periods spanning the Renaissance to the 20th century avant-garde, was considered large at that time (about twice as big as Sonata, for instance). Before SMuFL, the extension of November’s repertoire had often been considered, but, because Finale or Sibelius were using some ASCII-based map for music symbols (i.e. not Unicode, as the Unicode Music Symbols, a technological failure, had never been adopted), it would have most likely led to the multiplication of font files and more “character map anarchy”, as had occurred with, for example, Opus (Sibelius) or Maestro (Finale) ; consequently only small updates had been made over the years.

Then I regarded the emergence of SMuFL in 2013 as a great opportunity for November to make a bigger jump: one single font file with a greatly extended range of characters, wrapped in OpenType, and complying with a new standard.

That said, by switching to SMuFL, the font designer, who generally is a single individual, must be ready to face the temptation of adding more and more symbols, making the development process potentially much longer. And not only must the designer deal with thousands of vectors and points, but also to some extent he or she must turn into a programmer – but this won’t come as a surprise to the readers of this blog, who, as LilyPond users, have code lines in their blood, intimately mingled with the music itself. Python scripting, for instance, can be a great ally for generating the required metadata automatically; this was used extensively for the November 2.0 project. For SMuFL-scaled font projects, it is impractical to create those metadata manually, and, to make the design workflow even better, one can invent sophisticated tools, for instance to compare the font being crafted with the reference font, Bravura.

All of these considerations change the font development workflow deeply and, for fonts with over 1000 characters, you most likely want more automation in order to keep your hand and mind as free as possible.

November 2.0: a SMuFL-compliant font project

November 2.0, a multi-target commercial product released in February 2015, has now over 1200 characters, with 80% of them coming from the SMuFL specifications, and is the first commercially-released font to comply with SMuFL. Although it includes a much broader symbol range and many new features, the 2.0 version is still “in tune” with the design spirit of the font’s 1998 inception.

While the SMuFL standard itself has been quickly emerging (with Steinberg’s work-in-progress on a new scoring application in the background), existing notation software such as Finale, Sibelius or LilyPond follow at a somewhat slower pace.
At the present time, though this is likely forthcoming, no currently available notation software officially supports SMuFL in an open fashion (MuseScore seems to be SMuFL compliant, but sadly the choice of available font is by-design hard-wired). Therefore, in the short- to medium-term, a SMuFL-compliant font like November 2.0 must still be packaged specifically for each notation program. The SMuFL metadata, for instance, is currently not consumed at all by any of the major existing applications (including Finale, Sibelius, and LilyPond), and idiosyncratic component files must be supplied along with the font in order to ensure a smooth user experience.

Ravel – Shérérazade, engraved in LilyPond using November 2.0

The same example engraved with Emmentaler

November 2.0 is distributed as a single downloadable archive containing installers for Mac, Windows & Linux, targeting Finale, Sibelius, and of course LilyPond.
The default installation detects any versions of the aforementioned notation programs, and installs the right files at the right locations. November 2.0 also includes a comprehensive 200-page pdf documentation in English and German (French is in progress…)

Using November 2.0 in LilyPond

So how specifically does November 2.0 comply with LilyPond? In LilyPond we know that the font paradigm is similar to LaTeX: the Emmentaler font (and all its style variations: Feta, Parmesan, etc.) is constructed at compile time from meta font data, resulting in a single OpenType font family, mapped onto a specific character table. Comparing with Maestro or Opus, Emmentaler has had – for years – the advantage of being Unicode-based, addressing the problem of a unified font its own way. In order to use November 2.0 in LilyPond, I have followed the steps of Nathan Ho and Joram Berger. And, sharing with me the obsession for font variety and glyph “vivid” details, Abraham Lee has been of a great help, too.

So once November 2.0 is installed, switching is fairly easy as it only requires

\include"november2.ly"

at the beginning of the music file.

For a local switch to November 2.0, you may also use:

\novemberOn% ... music ...\novemberOff

within the music code.

Under the hood, we have some special ‘snippet’ component files making the use of November 2.0 within the LilyPond environment as smooth as possible. You generally don’t want to bother with these files, but, as an example and just for the sake of curiosity, here is how the stem tremolo is finely taken care of:

That said, while I could cover most of the notational aspects, there are still rare areas that are bit of a mystery to me, such as trill wiggles (wavy lines). Having looked into LilyPond’s code, they seem to be hard wired and I would advocate for LilyPond to provide easier (or, simply, possible) ways to overload those, through Scheme.

Meanwhile, the current compatibility of November 2.0 with LilyPond is fairly good. And more generally I hope that the claim of SMuFL-compliance for a rather popular music font, ranging across the major notation programs, can potentially help serve as an impetus for the developers of music notation software to support SMuFL more quickly and more comprehensively.
And I also believe that such a multi-target and SMuFL-compatible project may have the virtue of making the “vs.” reflex somehow irrelevant (Finale vs. Sibelius vs. LilyPond, free- vs. pay-ware, text- or GUI-based and so on).
In fact, whether we use music notation tools for our own purpose or under specific professional constraints, the access to style variety – as in text processing – should be left wide open, and graphic beauty should never be compromised.

I have to disagree. I understand and can to some extent accept that it’s part of the GNU guidelines to not endorse non-free software or content in any way, including even mentioning. But we are not LilyPond, and the only persons who should take issue with what is appropriate for publication on this platform are the three maintainers. And while I can’t speak for the other two none of them has ever (this post was dangling for quite some time for several reasons) expressed the slightest concerns about it.

As for the content, I support the idea of approaching SMuFL, and creating a SMuFL compliant font and making that available for LilyPond as well is a good idea in my eyes. Apart from that the post does not only promote buying (or even using) non-free resources but also gives very interesting insights that are perfectly in line with what we have on this blog in general.

Alternative perspectives on score layout and visualization are always welcome imho. For instance, I really like the (4 different) staff terminators in the “Le Chant des Oyseaulx” sample score.
Thank you for the insightful article!

> what the kernel of music symbols should be

In my opinion, mensual-noteheads and mensual-flags should be dropped from the notation character set, which already contains modern-noteheads and modern-flags.
Then, mensual notation font can simply be provided in a separate font notation file.
Besides this example, there are more opportunities for making the notation font more light-weight…

> the actual position of characters in the font

Yes, obviously this information is very important for inter-font-compatibility, but neither Lilypond nor SMuFL provide this data in their human-readable-documentation (last time I checked). Hopefully this will be included one day.

> Unicode 8.0.0 Music Symbols

To be fair, the table was only introduced rather “late”. For instance, Java does not support 3 byte chars natively, but only 16-bit unicode.

> [Lilypond’s Emmentaler has] the advantage of being Unicode-based

Lilypond’s “char to glyph mapping” may change with each version. What (typically) remains invariant are the “string identifier to glyph mapping”.

> Things are changing drastically with SMuFL

I somewhat disagree: At this moment, SMuFL is not a superset of Emmentaler (e.g. mensural 3-flags are missing and more).

> design spirit of the font’s 1998 inception

Please feel free to elaborate more on what motivated the particular glyph design of November?

In my opinion, mensual-noteheads and mensual-flags should be dropped from the notation character set, which already contains modern-noteheads and modern-flags.

Well, the aim of SMuFL is precisely to have a range as wide as possible, and to include also style variations. I think it simply always better to have a greater choice.

Then, mensual notation font can simply be provided in a separate font notation file. Besides this example, there are more opportunities for making the notation font more light-weight…

The font file size is really not a factor, and November 2.0 at this day is about 897ko, so that shouldn’t be a problem!

> the actual position of characters in the font
Yes, obviously this information is very important for inter-font-compatibility, but neither Lilypond nor SMuFL provide this data in their human-readable-documentation (last time I checked).

Sorry if that was not probably clear enough: within the Unicode context, the codepoint is what I mean by position, and this is already well documented in both SMuFL and November 2.0 docs.

> Things are changing drastically with SMuFL
I somewhat disagree: At this moment, SMuFL is not a superset of Emmentaler (e.g. mensural 3-flags are missing and more).

My understanding is that SMuFL is a community work in progress that is being enriched and fine-tuned over time. As many others I have myself contributed to add new symbols to the map, so feel free to contribute! (https://www.w3.org/community/music-notation/)

> design spirit of the font’s 1998 inception
Please feel free to elaborate more on what motivated the particular glyph design of November?

When I first designed November I was obsessed (and already am) by the principle of making almost infinitesimal irregularities part of the essence of the font. At a macroscopic level, upon printing for example (especially with hires printers), those microscopic details makes the font clearly more vivid, more “human”… I gives an impression (so to speak!) similar to the inked metal plate in traditional music engraving (Abraham Lee shares with me this idea – see for instance https://fonts.openlilylib.org/cadence/). I think November 1.0 was the first font having this principle in its veins (apart from fonts like Jazz Font etc. that mimic the hand writing, which category November clearly is not in).
Standard fonts like Maestro or Opus look so sad and “cold” on printed sheets! Fortunately, more recent fonts like Emmentaler or Bravura have, to some extent, also used this concept.

I have attached a example of a glyph from November 2.0 with an giant zoom factor that illustrates what I mean.

>> datahaki: I somewhat disagree: At this moment, SMuFL is not a superset of Emmentaler (e.g. mensural 3-flags are missing and more).

Robert: My understanding is that SMuFL is a community work in progress that is being enriched and fine-tuned over time. As many others I have myself contributed to add new symbols to the map, so feel free to contribute! (https://www.w3.org/community/music-notation/)

When SMuFL approached its initial release Daniel Spreadbury asked for input from everywhere. Specifically he also asked for assistance with some parts of Emmentaler where he lacked the knowledge of proper use cases he’d need to incorporate them in SMuFL. Our community failed to provide the necessary details so it’s not SMuFL’s fault that Emmentaler (and therefore LilyPond) is not completely represented in the new standard.

I am not worried about size of the font file in kB. Instead, my suggestion aims to identify opportunities to reduce redundancy, and to reduce the amount of characters. In my survey, the mensural noteheads have not been altered from their default appearance throughout all the fonts that I visualized. So why not have modern fonts (where noteheads are modern), and antique fonts (where noteheads are mensural) etc.?

2) “the codepoint is what I mean by position, and this is already well documented in both SMuFL and November 2.0 docs.”

I agree: the address/number in the unicode table is documented.

What I didn’t find in any documentation – on the other hand – is where the character appears when I draw the glyph at coordinates (0,0). Some characters appear right from that position, some on top, some centered… Answering that question, required some trial and error, and fine-tuning. In my graphics, I have drawn blue horizontal+vertical lines to indicate the “orgin/(0,0)” of each character.

What I didn’t find in any documentation – on the other hand – is where the character appears when I draw the glyph at coordinates (0,0). Some characters appear right from that position, some on top, some centered… Answering that question, required some trial and error, and fine-tuning. In my graphics, I have drawn blue horizontal+vertical lines to indicate the “orgin/(0,0)” of each character.

Ah! yes I see. That’s a good idea, and I will keep that in mind for the upcoming (free) update of the font (2.1)
That said I have already used some visual helper for several characters (see the presentation PDF, for instance U+E220, U+E618, U+E619, U+E623, or U+F604) but I can in fact make it more general.

I’m happy to see this article on Scores of Beauty. I may purchase this font myself.

I’m also excited that individuals designing commercial music fonts like November are keeping LilyPond in mind. As a reader, I have no problem with commercial projects being highlighted on this blog and hope that more such projects are highlighted in the future.