Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Dear list

On Fedora 31 the pango library has recently updated to version >= 1.44
and in doing so has switched to using the HarfBuzz library (from
FreeType) and dropped Adobe Type 1 font support. This causes problems
with plotmath as all bar one of the glyphs doesn't render (see
attached PNG image if it makes it through the list filters - if not I
have shared a copy via my google drive:
https://drive.google.com/file/d/1llFqKHD7LFKzQbVuq6sibY1UizRn7xxS/view?usp=sharing)

PS font users need to switch to OpenType fonts or work with their
prefered font upstream to convert in modern well supported formats
(font format wars have endend last millenium, even before the browser
wars ended, it’s long past time to deprecate the losers).

That’s normal IT format obsolescence.

That being said, that’s not what is happening here.

R brought this all on itself by hardcoding a Windows-only “Symbol” font
family name in its default conf. Linux systems are UTF-8 by default for
~20 years now, they don’t need the forcing of magic font families to
handle symbols not present in the 8-bit legacy Windows encodings.

The actual effect of this conf is not the selection of font files with
special and unusual symbols. It is to priorize fonts that match the
"Symbol" magic name. And those fonts are few and crumbling on Linux
systems, because no one has needed to bother with them since Linux
switched to UTF-8 last millenium.

Just stop using “Symbol” in R and things will work a lot better.
Alternatively, prepare to maintain the “Symbol” aliasing stack in
fontconfig (and fight with wine for it), because *no* *one* *else*
*cares* about this legacy Windows-specific stuff.

Fontconfig upstream already told this to R users in its own issue
tracker.

>
> <snip>
>
> R brought this all on itself by hardcoding a Windows-only “Symbol” font
> family name in its default conf. Linux systems are UTF-8 by default for
> ~20 years now, they don’t need the forcing of magic font families to
> handle symbols not present in the 8-bit legacy Windows encodings.
>
> The actual effect of this conf is not the selection of font files with
> special and unusual symbols. It is to priorize fonts that match the
> "Symbol" magic name. And those fonts are few and crumbling on Linux
> systems, because no one has needed to bother with them since Linux
> switched to UTF-8 last millenium.
>
> Just stop using “Symbol” in R and things will work a lot better.
> Alternatively, prepare to maintain the “Symbol” aliasing stack in
> fontconfig (and fight with wine for it), because *no* *one* *else*
> *cares* about this legacy Windows-specific stuff.

So, in the light of Nicolas' input (thanks!), I think that font
selection should be fixed upstream in R. I'd be happy to put all this
together in R's bugzilla, but I don't have an account. Could someone
please invite me?

As suggested, the plan is to allow the R user to specify a font family
other than "symbol" for plotmath output (or, more generally, in R
parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.

Paul

On 27/03/20 11:30 pm, Iñaki Ucar wrote:

> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> <[hidden email]> wrote:
>>
>> <snip>
>>
>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
>> family name in its default conf. Linux systems are UTF-8 by default for
>> ~20 years now, they don’t need the forcing of magic font families to
>> handle symbols not present in the 8-bit legacy Windows encodings.
>>
>> The actual effect of this conf is not the selection of font files with
>> special and unusual symbols. It is to priorize fonts that match the
>> "Symbol" magic name. And those fonts are few and crumbling on Linux
>> systems, because no one has needed to bother with them since Linux
>> switched to UTF-8 last millenium.
>>
>> Just stop using “Symbol” in R and things will work a lot better.
>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
>> fontconfig (and fight with wine for it), because *no* *one* *else*
>> *cares* about this legacy Windows-specific stuff.
>
> So, in the light of Nicolas' input (thanks!), I think that font
> selection should be fixed upstream in R. I'd be happy to put all this
> together in R's bugzilla, but I don't have an account. Could someone
> please invite me?
>
> Iñaki
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel>

>
> Hi
>
> Thanks for your input on this Iñaki and Nicolas.
>
> I am starting testing an R fix for this problem today.
>
> As suggested, the plan is to allow the R user to specify a font family
> other than "symbol" for plotmath output (or, more generally, in R
> parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
>
> Paul
>
>
> On 27/03/20 11:30 pm, Iñaki Ucar wrote:
> > On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> > <[hidden email]> wrote:
> >>
> >> <snip>
> >>
> >> R brought this all on itself by hardcoding a Windows-only “Symbol” font
> >> family name in its default conf. Linux systems are UTF-8 by default for
> >> ~20 years now, they don’t need the forcing of magic font families to
> >> handle symbols not present in the 8-bit legacy Windows encodings.
> >>
> >> The actual effect of this conf is not the selection of font files with
> >> special and unusual symbols. It is to priorize fonts that match the
> >> "Symbol" magic name. And those fonts are few and crumbling on Linux
> >> systems, because no one has needed to bother with them since Linux
> >> switched to UTF-8 last millenium.
> >>
> >> Just stop using “Symbol” in R and things will work a lot better.
> >> Alternatively, prepare to maintain the “Symbol” aliasing stack in
> >> fontconfig (and fight with wine for it), because *no* *one* *else*
> >> *cares* about this legacy Windows-specific stuff.
> >
> > So, in the light of Nicolas' input (thanks!), I think that font
> > selection should be fixed upstream in R. I'd be happy to put all this
> > together in R's bugzilla, but I don't have an account. Could someone
> > please invite me?
> >
> > Iñaki
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel> >

... to specify that the OpenSymbol family should be used as the "symbol"
font (e.g., for "plotmath") in R.

This is just a separate branch for now because, while I have tested it
under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
(right now) or Mac (ever) and I do not want to drop a bomb on R-devel at
this stage of the release process for R 4.0.0.

The attached file contains at least an outline of steps required to do a
minimal test if anyone wants to try the fix on Linux.

cc'ing Simon and Jeroen in case they are able to help with checking that
this builds and works on Mac and/or Windows.

NOTEs:
- 'symbolfamily' can only be specified when a graphics device is opened,
and it is then fixed for that device.
- on Windows, for cairo-based devices, the "symbol" font is still
hard-coded as "Standard Symbols L"

Paul

On 30/03/20 8:15 am, Paul Murrell wrote:

> Hi
>
> Thanks for your input on this Iñaki and Nicolas.
>
> I am starting testing an R fix for this problem today.
>
> As suggested, the plan is to allow the R user to specify a font family
> other than "symbol" for plotmath output (or, more generally, in R
> parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
>
> Paul
>
>
> On 27/03/20 11:30 pm, Iñaki Ucar wrote:
>> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
>> <[hidden email]> wrote:
>>>
>>> <snip>
>>>
>>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
>>> family name in its default conf. Linux systems are UTF-8 by default for
>>> ~20 years now, they don’t need the forcing of magic font families to
>>> handle symbols not present in the 8-bit legacy Windows encodings.
>>>
>>> The actual effect of this conf is not the selection of font files with
>>> special and unusual symbols. It is to priorize fonts that match the
>>> "Symbol" magic name. And those fonts are few and crumbling on Linux
>>> systems, because no one has needed to bother with them since Linux
>>> switched to UTF-8 last millenium.
>>>
>>> Just stop using “Symbol” in R and things will work a lot better.
>>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
>>> fontconfig (and fight with wine for it), because *no* *one* *else*
>>> *cares* about this legacy Windows-specific stuff.
>>
>> So, in the light of Nicolas' input (thanks!), I think that font
>> selection should be fixed upstream in R. I'd be happy to put all this
>> together in R's bugzilla, but I don't have an account. Could someone
>> please invite me?
>>
>> Iñaki
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel>>

> Hi
>
> I have created an R branch that contains a potential fix ...
>
> https://svn.r-project.org/R/branches/R-symfam/>
> This allows, for example, ...
>
> cairo_pdf(symbolfamily="OpenSymbol")
>
> ... to specify that the OpenSymbol family should be used as the "symbol"
> font (e.g., for "plotmath") in R.
>
> This is just a separate branch for now because, while I have tested it
> under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
> (right now) or Mac (ever) and I do not want to drop a bomb on R-devel at
> this stage of the release process for R 4.0.0.
>
> The attached file contains at least an outline of steps required to do a
> minimal test if anyone wants to try the fix on Linux.
>
> cc'ing Simon and Jeroen in case they are able to help with checking that
> this builds and works on Mac and/or Windows.
>
> NOTEs:
> - 'symbolfamily' can only be specified when a graphics device is opened,
> and it is then fixed for that device.
> - on Windows, for cairo-based devices, the "symbol" font is still
> hard-coded as "Standard Symbols L"
>
> Paul
>
> On 30/03/20 8:15 am, Paul Murrell wrote:
> > Hi
> >
> > Thanks for your input on this Iñaki and Nicolas.
> >
> > I am starting testing an R fix for this problem today.
> >
> > As suggested, the plan is to allow the R user to specify a font family
> > other than "symbol" for plotmath output (or, more generally, in R
> > parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
> >
> > Paul
> >
> >
> > On 27/03/20 11:30 pm, Iñaki Ucar wrote:
> >> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> >> <[hidden email]> wrote:
> >>>
> >>> <snip>
> >>>
> >>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
> >>> family name in its default conf. Linux systems are UTF-8 by default for
> >>> ~20 years now, they don’t need the forcing of magic font families to
> >>> handle symbols not present in the 8-bit legacy Windows encodings.
> >>>
> >>> The actual effect of this conf is not the selection of font files with
> >>> special and unusual symbols. It is to priorize fonts that match the
> >>> "Symbol" magic name. And those fonts are few and crumbling on Linux
> >>> systems, because no one has needed to bother with them since Linux
> >>> switched to UTF-8 last millenium.
> >>>
> >>> Just stop using “Symbol” in R and things will work a lot better.
> >>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
> >>> fontconfig (and fight with wine for it), because *no* *one* *else*
> >>> *cares* about this legacy Windows-specific stuff.
> >>
> >> So, in the light of Nicolas' input (thanks!), I think that font
> >> selection should be fixed upstream in R. I'd be happy to put all this
> >> together in R's bugzilla, but I don't have an account. Could someone
> >> please invite me?
> >>
> >> Iñaki
> >>
> >> ______________________________________________
> >> [hidden email] mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel> >>
>
> --
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> [hidden email]> http://www.stat.auckland.ac.nz/~paul/> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel>

>
> Hi
>
> I have created an R branch that contains a potential fix ...
>
> https://svn.r-project.org/R/branches/R-symfam/>
> This allows, for example, ...
>
> cairo_pdf(symbolfamily="OpenSymbol")
>
> ... to specify that the OpenSymbol family should be used as the "symbol"
> font (e.g., for "plotmath") in R.

Will this be a default on Linux? Or are you planning any mechanism
(env variable, option...) to make it the default? Because, otherwise,
as pango is updated across distributions, R graphics will be "broken"
by default unless the user explicitly calls the graphics device in
that way to set that option, which I would say is uncommon.

Iñaki

> This is just a separate branch for now because, while I have tested it
> under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
> (right now) or Mac (ever) and I do not want to drop a bomb on R-devel at
> this stage of the release process for R 4.0.0.
>
> The attached file contains at least an outline of steps required to do a
> minimal test if anyone wants to try the fix on Linux.
>
> cc'ing Simon and Jeroen in case they are able to help with checking that
> this builds and works on Mac and/or Windows.
>
> NOTEs:
> - 'symbolfamily' can only be specified when a graphics device is opened,
> and it is then fixed for that device.
> - on Windows, for cairo-based devices, the "symbol" font is still
> hard-coded as "Standard Symbols L"
>
>
> Paul
>
> On 30/03/20 8:15 am, Paul Murrell wrote:
> > Hi
> >
> > Thanks for your input on this Iñaki and Nicolas.
> >
> > I am starting testing an R fix for this problem today.
> >
> > As suggested, the plan is to allow the R user to specify a font family
> > other than "symbol" for plotmath output (or, more generally, in R
> > parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
> >
> > Paul
> >
> >
> > On 27/03/20 11:30 pm, Iñaki Ucar wrote:
> >> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> >> <[hidden email]> wrote:
> >>>
> >>> <snip>
> >>>
> >>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
> >>> family name in its default conf. Linux systems are UTF-8 by default for
> >>> ~20 years now, they don’t need the forcing of magic font families to
> >>> handle symbols not present in the 8-bit legacy Windows encodings.
> >>>
> >>> The actual effect of this conf is not the selection of font files with
> >>> special and unusual symbols. It is to priorize fonts that match the
> >>> "Symbol" magic name. And those fonts are few and crumbling on Linux
> >>> systems, because no one has needed to bother with them since Linux
> >>> switched to UTF-8 last millenium.
> >>>
> >>> Just stop using “Symbol” in R and things will work a lot better.
> >>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
> >>> fontconfig (and fight with wine for it), because *no* *one* *else*
> >>> *cares* about this legacy Windows-specific stuff.
> >>
> >> So, in the light of Nicolas' input (thanks!), I think that font
> >> selection should be fixed upstream in R. I'd be happy to put all this
> >> together in R's bugzilla, but I don't have an account. Could someone
> >> please invite me?
> >>
> >> Iñaki
> >>
> >> ______________________________________________
> >> [hidden email] mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel> >>
>
> --
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> [hidden email]> http://www.stat.auckland.ac.nz/~paul/

>
> Hi
>
> I have created an R branch that contains a potential fix ...
>
> https://svn.r-project.org/R/branches/R-symfam/>
> This allows, for example, ...
>
> cairo_pdf(symbolfamily="OpenSymbol")
>
> ... to specify that the OpenSymbol family should be used as the "symbol"
> font (e.g., for "plotmath") in R.
>
> This is just a separate branch for now because, while I have tested it
> under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
> (right now) or Mac (ever) and I do not want to drop a bomb on R-devel at
> this stage of the release process for R 4.0.0.
>
> The attached file contains at least an outline of steps required to do a
> minimal test if anyone wants to try the fix on Linux.
>
> cc'ing Simon and Jeroen in case they are able to help with checking that
> this builds and works on Mac and/or Windows.
>
> NOTEs:
> - 'symbolfamily' can only be specified when a graphics device is opened,
> and it is then fixed for that device.
> - on Windows, for cairo-based devices, the "symbol" font is still
> hard-coded as "Standard Symbols L"
>
> Paul
>
> On 30/03/20 8:15 am, Paul Murrell wrote:
> > Hi
> >
> > Thanks for your input on this Iñaki and Nicolas.
> >
> > I am starting testing an R fix for this problem today.
> >
> > As suggested, the plan is to allow the R user to specify a font family
> > other than "symbol" for plotmath output (or, more generally, in R
> > parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
> >
> > Paul
> >
> >
> > On 27/03/20 11:30 pm, Iñaki Ucar wrote:
> >> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> >> <[hidden email]> wrote:
> >>>
> >>> <snip>
> >>>
> >>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
> >>> family name in its default conf. Linux systems are UTF-8 by default for
> >>> ~20 years now, they don’t need the forcing of magic font families to
> >>> handle symbols not present in the 8-bit legacy Windows encodings.
> >>>
> >>> The actual effect of this conf is not the selection of font files with
> >>> special and unusual symbols. It is to priorize fonts that match the
> >>> "Symbol" magic name. And those fonts are few and crumbling on Linux
> >>> systems, because no one has needed to bother with them since Linux
> >>> switched to UTF-8 last millenium.
> >>>
> >>> Just stop using “Symbol” in R and things will work a lot better.
> >>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
> >>> fontconfig (and fight with wine for it), because *no* *one* *else*
> >>> *cares* about this legacy Windows-specific stuff.
> >>
> >> So, in the light of Nicolas' input (thanks!), I think that font
> >> selection should be fixed upstream in R. I'd be happy to put all this
> >> together in R's bugzilla, but I don't have an account. Could someone
> >> please invite me?
> >>
> >> Iñaki
> >>
> >> ______________________________________________
> >> [hidden email] mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel> >>
>
> --
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> [hidden email]> http://www.stat.auckland.ac.nz/~paul/

> Hi
>
> I have created an R branch that contains a potential fix ...
>
> https://svn.r-project.org/R/branches/R-symfam/>
> This allows, for example, ...
>
> cairo_pdf(symbolfamily="OpenSymbol")
>
> ... to specify that the OpenSymbol family should be used as the
> "symbol" font (e.g., for "plotmath") in R.

Thanks for looking at it!

But, really, there is no such thing as a Symbol font on Linux anymore.
Symbol is pre-unicode thinking. Most modern general-purpose unicode
fonts will include every codepoint Symbol ever shipped, and fontconfig
will fallback gracefully when that’s not the case (unless your
fontconfig integration is broken).

Just use the sans-serif or monospace fontconfig defaults. You don’t
need Symbol, or OpenSymbol, or any special font setup.

Symbol’s codepoint coverage is laughable by 2020’s UTF-8 standards.

Symbol << Normal Unicode font (DejaVu*) << Special math fonts (STIX2)

I you do advanced math stuff, you may need a special math font like
STIX, but that’s lights years more advanced than Symbol, and a general
purpose font like DejaVu has been shipping a MATH block for several
years now

Thanks Gabriel. Sounds like both you and Brian can build the branch on
Mac. Just need to check that Windows builds before I commit to r-devel.

Paul

On 30/03/20 7:43 pm, Gabriel Becker wrote:

> I do my devel/patch work on Mac so I can take a shot at testing your
> branch in the next couple days.
>
> ~G
>
> On Sun, Mar 29, 2020 at 7:24 PM Paul Murrell <[hidden email]> <mailto:[hidden email]>> wrote:
>
> Hi
>
> I have created an R branch that contains a potential fix ...
>
> https://svn.r-project.org/R/branches/R-symfam/>
> This allows, for example, ...
>
> cairo_pdf(symbolfamily="OpenSymbol")
>
> ... to specify that the OpenSymbol family should be used as the
> "symbol"
> font (e.g., for "plotmath") in R.
>
> This is just a separate branch for now because, while I have tested it
> under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
> (right now) or Mac (ever) and I do not want to drop a bomb on
> R-devel at
> this stage of the release process for R 4.0.0.
>
> The attached file contains at least an outline of steps required to
> do a
> minimal test if anyone wants to try the fix on Linux.
>
> cc'ing Simon and Jeroen in case they are able to help with checking
> that
> this builds and works on Mac and/or Windows.
>
> NOTEs:
> - 'symbolfamily' can only be specified when a graphics device is
> opened,
> and it is then fixed for that device.
> - on Windows, for cairo-based devices, the "symbol" font is still
> hard-coded as "Standard Symbols L"
>
> Paul
>
> On 30/03/20 8:15 am, Paul Murrell wrote:
> > Hi
> >
> > Thanks for your input on this Iñaki and Nicolas.
> >
> > I am starting testing an R fix for this problem today.
> >
> > As suggested, the plan is to allow the R user to specify a font
> family
> > other than "symbol" for plotmath output (or, more generally, in R
> > parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics
> device.
> >
> > Paul
> >
> >
> > On 27/03/20 11:30 pm, Iñaki Ucar wrote:
> >> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> >> <[hidden email]> <mailto:[hidden email]>> wrote:
> >>>
> >>> <snip>
> >>>
> >>> R brought this all on itself by hardcoding a Windows-only
> “Symbol” font
> >>> family name in its default conf. Linux systems are UTF-8 by
> default for
> >>> ~20 years now, they don’t need the forcing of magic font
> families to
> >>> handle symbols not present in the 8-bit legacy Windows encodings.
> >>>
> >>> The actual effect of this conf is not the selection of font
> files with
> >>> special and unusual symbols. It is to priorize fonts that match the
> >>> "Symbol" magic name. And those fonts are few and crumbling on Linux
> >>> systems, because no one has needed to bother with them since Linux
> >>> switched to UTF-8 last millenium.
> >>>
> >>> Just stop using “Symbol” in R and things will work a lot better.
> >>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
> >>> fontconfig (and fight with wine for it), because *no* *one* *else*
> >>> *cares* about this legacy Windows-specific stuff.
> >>
> >> So, in the light of Nicolas' input (thanks!), I think that font
> >> selection should be fixed upstream in R. I'd be happy to put all
> this
> >> together in R's bugzilla, but I don't have an account. Could someone
> >> please invite me?
> >>
> >> Iñaki
> >>
> >> ______________________________________________
> >> [hidden email] <mailto:[hidden email]> mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel> >>
>
> --
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> [hidden email] <mailto:[hidden email]>
> http://www.stat.auckland.ac.nz/~paul/> ______________________________________________
> [hidden email] <mailto:[hidden email]> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel>

> On Mon, 30 Mar 2020 at 04:24, Paul Murrell <[hidden email]> wrote:
>>
>> Hi
>>
>> I have created an R branch that contains a potential fix ...
>>
>> https://svn.r-project.org/R/branches/R-symfam/>>
>> This allows, for example, ...
>>
>> cairo_pdf(symbolfamily="OpenSymbol")
>>
>> ... to specify that the OpenSymbol family should be used as the "symbol"
>> font (e.g., for "plotmath") in R.
>
> Will this be a default on Linux? Or are you planning any mechanism
> (env variable, option...) to make it the default? Because, otherwise,
> as pango is updated across distributions, R graphics will be "broken"
> by default unless the user explicitly calls the graphics device in
> that way to set that option, which I would say is uncommon.

Good question. Currently, for x11() (and png() etc) the default is
taken from X11.options(). So it is possible to set this default for a
session, or even for an installation via one of the ?Startup mechanisms
(e.g., an R_HOME/etc/Rprofile.site file).

For svg(), cairo_pdf(), and cairo_ps(), the default is hard-coded in the
function arguments, but I *think* they are used less as default graphics
devices.

Another option would be to try to detect Fedora and set the default
X11.options() differently there. Two problems: I am not sure there is
a reliable R code chunk for detecting Fedora (sessionInfo()$running?)
let alone Fedora >= 30; what to set the default to? (just has to be a
font with a good Unicode coverage that is pretty much guaranteed to be
in a default Fedora install).

Paul

> Iñaki
>
>> This is just a separate branch for now because, while I have tested it
>> under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
>> (right now) or Mac (ever) and I do not want to drop a bomb on R-devel at
>> this stage of the release process for R 4.0.0.
>>
>> The attached file contains at least an outline of steps required to do a
>> minimal test if anyone wants to try the fix on Linux.
>>
>> cc'ing Simon and Jeroen in case they are able to help with checking that
>> this builds and works on Mac and/or Windows.
>>
>> NOTEs:
>> - 'symbolfamily' can only be specified when a graphics device is opened,
>> and it is then fixed for that device.
>> - on Windows, for cairo-based devices, the "symbol" font is still
>> hard-coded as "Standard Symbols L"
>>
>>
>> Paul
>>
>> On 30/03/20 8:15 am, Paul Murrell wrote:
>>> Hi
>>>
>>> Thanks for your input on this Iñaki and Nicolas.
>>>
>>> I am starting testing an R fix for this problem today.
>>>
>>> As suggested, the plan is to allow the R user to specify a font family
>>> other than "symbol" for plotmath output (or, more generally, in R
>>> parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
>>>
>>> Paul
>>>
>>>
>>> On 27/03/20 11:30 pm, Iñaki Ucar wrote:
>>>> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
>>>> <[hidden email]> wrote:
>>>>>
>>>>> <snip>
>>>>>
>>>>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
>>>>> family name in its default conf. Linux systems are UTF-8 by default for
>>>>> ~20 years now, they don’t need the forcing of magic font families to
>>>>> handle symbols not present in the 8-bit legacy Windows encodings.
>>>>>
>>>>> The actual effect of this conf is not the selection of font files with
>>>>> special and unusual symbols. It is to priorize fonts that match the
>>>>> "Symbol" magic name. And those fonts are few and crumbling on Linux
>>>>> systems, because no one has needed to bother with them since Linux
>>>>> switched to UTF-8 last millenium.
>>>>>
>>>>> Just stop using “Symbol” in R and things will work a lot better.
>>>>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
>>>>> fontconfig (and fight with wine for it), because *no* *one* *else*
>>>>> *cares* about this legacy Windows-specific stuff.
>>>>
>>>> So, in the light of Nicolas' input (thanks!), I think that font
>>>> selection should be fixed upstream in R. I'd be happy to put all this
>>>> together in R's bugzilla, but I don't have an account. Could someone
>>>> please invite me?
>>>>
>>>> Iñaki
>>>>
>>>> ______________________________________________
>>>> [hidden email] mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel>>>>
>>
>> --
>> Dr Paul Murrell
>> Department of Statistics
>> The University of Auckland
>> Private Bag 92019
>> Auckland
>> New Zealand
>> 64 9 3737599 x85392
>> [hidden email]>> http://www.stat.auckland.ac.nz/~paul/>
>
>

>
> Hi
>
> On 30/03/20 10:43 pm, Iñaki Ucar wrote:
> > On Mon, 30 Mar 2020 at 04:24, Paul Murrell <[hidden email]> wrote:
> >>
> >> Hi
> >>
> >> I have created an R branch that contains a potential fix ...
> >>
> >> https://svn.r-project.org/R/branches/R-symfam/> >>
> >> This allows, for example, ...
> >>
> >> cairo_pdf(symbolfamily="OpenSymbol")
> >>
> >> ... to specify that the OpenSymbol family should be used as the "symbol"
> >> font (e.g., for "plotmath") in R.
> >
> > Will this be a default on Linux? Or are you planning any mechanism
> > (env variable, option...) to make it the default? Because, otherwise,
> > as pango is updated across distributions, R graphics will be "broken"
> > by default unless the user explicitly calls the graphics device in
> > that way to set that option, which I would say is uncommon.
>
> Good question. Currently, for x11() (and png() etc) the default is
> taken from X11.options(). So it is possible to set this default for a
> session, or even for an installation via one of the ?Startup mechanisms
> (e.g., an R_HOME/etc/Rprofile.site file).
>
> For svg(), cairo_pdf(), and cairo_ps(), the default is hard-coded in the
> function arguments, but I *think* they are used less as default graphics
> devices.
>
> Another option would be to try to detect Fedora and set the default
> X11.options() differently there. Two problems: I am not sure there is
> a reliable R code chunk for detecting Fedora (sessionInfo()$running?)
> let alone Fedora >= 30; what to set the default to? (just has to be a
> font with a good Unicode coverage that is pretty much guaranteed to be
> in a default Fedora install).

As per Nicolas' comment (I failed to include him in CC in my last
email, and he's not in this list, sorry for that) any font installed
by default would have good symbol coverage, so there's really no need
to set a different font for symbols. According again to Nicolas (he's
one of the font experts in Fedora), the "sans-serif" or "monospace"
fontconfig defaults would work out of the box, and if a symbol is not
available, fontconfig should fallback gracefully to another font.

So maybe instead of a new "symbolfamily" argument, maybe it's better
to just use the "family" for all characters, including symbols, on
Linux, and fontconfig should take care of everything (if I understood
correctly your explanation, Nicolas; please correct me if I'm wrong).

> Le lundi 30 mars 2020 à 15:24 +1300, Paul Murrell a écrit :
>> Hi
>>
>> I have created an R branch that contains a potential fix ...
>>
>> https://svn.r-project.org/R/branches/R-symfam/>>
>> This allows, for example, ...
>>
>> cairo_pdf(symbolfamily="OpenSymbol")
>>
>> ... to specify that the OpenSymbol family should be used as the
>> "symbol" font (e.g., for "plotmath") in R.
>
> Thanks for looking at it!
>
> But, really, there is no such thing as a Symbol font on Linux anymore.
> Symbol is pre-unicode thinking. Most modern general-purpose unicode
> fonts will include every codepoint Symbol ever shipped, and fontconfig
> will fallback gracefully when that’s not the case (unless your
> fontconfig integration is broken).

Yep, the "symbol" font is an (outdated) R "plotmath" concept, but one
that would take a fair bit of surgery to remove. R plotmath converts
certain R expressions (in certain contexts) to code points in the Adobe
Symbol Encoding (ASM), but for cairo-based devices, those are converted
to UTF8 code points.

> Just use the sans-serif or monospace fontconfig defaults. You don’t
> need Symbol, or OpenSymbol, or any special font setup.

Agreed. I got reasonable coverage from DejaVu Sans and FreeSerif.
There are still a number of ASM code points that are not covered though,
for example, ...

Right, but there are still code points that are apparently not common in
fonts with a very broad Unicode coverage.

I did find a TrueType font that I could add a Unicode cmap to that gave
complete coverage pretty much all by itself, but it is not distributable.

> Symbol << Normal Unicode font (DejaVu*) << Special math fonts (STIX2)
>
> I you do advanced math stuff, you may need a special math font like
> STIX, but that’s lights years more advanced than Symbol, and a general
> purpose font like DejaVu has been shipping a MATH block for several
> years now

Sure. One way to look at the new 'symbolfamily' argument is that it
allows us to tell R that the "symbol" font is just a normal font (for
cairo-based graphics devices).

and 23AF/23D0 for arrow extensions (though arrow font support seems
messy, probably because it sees little use; it’s a pity R comes so late
to the party, those are just lines, it would have been trivial to get
them into DejaVu before the project gone dormant). GFS NeoHellenic
(Math block) seems complete but it’s not a common font family.

> On Mon, 30 Mar 2020 at 22:41, Paul Murrell <[hidden email]> wrote:
>>
>> Hi
>>
>> On 30/03/20 10:43 pm, Iñaki Ucar wrote:
>>> On Mon, 30 Mar 2020 at 04:24, Paul Murrell <[hidden email]> wrote:
>>>>
>>>> Hi
>>>>
>>>> I have created an R branch that contains a potential fix ...
>>>>
>>>> https://svn.r-project.org/R/branches/R-symfam/>>>>
>>>> This allows, for example, ...
>>>>
>>>> cairo_pdf(symbolfamily="OpenSymbol")
>>>>
>>>> ... to specify that the OpenSymbol family should be used as the "symbol"
>>>> font (e.g., for "plotmath") in R.
>>>
>>> Will this be a default on Linux? Or are you planning any mechanism
>>> (env variable, option...) to make it the default? Because, otherwise,
>>> as pango is updated across distributions, R graphics will be "broken"
>>> by default unless the user explicitly calls the graphics device in
>>> that way to set that option, which I would say is uncommon.
>>
>> Good question. Currently, for x11() (and png() etc) the default is
>> taken from X11.options(). So it is possible to set this default for a
>> session, or even for an installation via one of the ?Startup mechanisms
>> (e.g., an R_HOME/etc/Rprofile.site file).
>>
>> For svg(), cairo_pdf(), and cairo_ps(), the default is hard-coded in the
>> function arguments, but I *think* they are used less as default graphics
>> devices.
>>
>> Another option would be to try to detect Fedora and set the default
>> X11.options() differently there. Two problems: I am not sure there is
>> a reliable R code chunk for detecting Fedora (sessionInfo()$running?)
>> let alone Fedora >= 30; what to set the default to? (just has to be a
>> font with a good Unicode coverage that is pretty much guaranteed to be
>> in a default Fedora install).
>
> As per Nicolas' comment (I failed to include him in CC in my last
> email, and he's not in this list, sorry for that) any font installed
> by default would have good symbol coverage, so there's really no need
> to set a different font for symbols. According again to Nicolas (he's
> one of the font experts in Fedora), the "sans-serif" or "monospace"
> fontconfig defaults would work out of the box, and if a symbol is not
> available, fontconfig should fallback gracefully to another font.
>
> So maybe instead of a new "symbolfamily" argument, maybe it's better
> to just use the "family" for all characters, including symbols, on
> Linux, and fontconfig should take care of everything (if I understood
> correctly your explanation, Nicolas; please correct me if I'm wrong).

I think R will retain the idea of a separate symbol font in at least the
short term because of backward compatibility and cross-platform support
and support for a range of graphics devices. So this fix is just for
cairo-based devices on Linux at most (probably only Fedora).

So this becomes just a decision about user interface and default settings.

I did consider the option of allowing the existing "family" parameter to
be length-two (with the second one being an optional symbol font
specification), but because of the overlaps of X11/cairo and different
cairo-based device interfaces, this became awkward. Hence the separate
"symbolfamily" interface. And in any case, this still means a separate
"symbol" font specification (for the reasons above).

Regarding changing to a default symbolfamily=family on Linux generally
(rather than just on Fedora), I have at least one counter-example (my
Ubuntu 18.04) that shows that this would degrade output significantly.
For one, the symbols are a LOT uglier, plus there are some incorrect
glyphs. So I think we have to stay with treating Fedora as a special
case for now.

Thanks for your point about just using symbolfamily=family as the Fedora
default. That seems reasonable (and definitely better than it just
being completely broken!).

That does still leave the problem of how to set the default value for
"symbolfamily" JUST on Fedora. I am not convinced we can use R code to
detect Fedora >= 30 reliably (but happy to learn otherwise). Is it a
possibility for the Fedora distribution to include a .Rprofile.site file
that sets the X11.options() ?

However, the situation is still not completely straightforward. The
style of the symbols is also an issue and the DejaVu symbols are not as
elegant as, say, the OpenSymbol symbols. What makes things tricky is
that, AFAICS, DejaVu has (TTX Unicode cmap output) ...

... but neither has the other. So we could not simply switch to
standard Unicode code points because, while that would work with the
"ugly" DejaVu glyphs, that would mean that we could not access the
"pretty" OpenSymbol glyphs.

OpenSymbol is incorrect (it suffers from the same pre-unicode bias as
R). However, it is, to my knowledge, actively maintained. You can ask
its upstream (LibreOffice) for Unicode conformance fixes if you find
problems. Especially when it’s just fixing the map of an existing
glyph, that should not be hard for them to fix. Anything PUA-related
won’t interoperate well in an unicode world.

(you can ask DejaVu too, maybe a request from a project like R will
wake up its maintainers. But, that’s a long shot. DejaVu suffers from
an almost done state without enough remaining work to interest
designers).

>
> I think R will retain the idea of a separate symbol font in at least the
> short term because of backward compatibility and cross-platform support
> and support for a range of graphics devices. So this fix is just for
> cairo-based devices on Linux at most (probably only Fedora).
>
> So this becomes just a decision about user interface and default settings.
>
> I did consider the option of allowing the existing "family" parameter to
> be length-two (with the second one being an optional symbol font
> specification), but because of the overlaps of X11/cairo and different
> cairo-based device interfaces, this became awkward. Hence the separate
> "symbolfamily" interface. And in any case, this still means a separate
> "symbol" font specification (for the reasons above).
>
> Regarding changing to a default symbolfamily=family on Linux generally
> (rather than just on Fedora), I have at least one counter-example (my
> Ubuntu 18.04) that shows that this would degrade output significantly.
> For one, the symbols are a LOT uglier, plus there are some incorrect
> glyphs. So I think we have to stay with treating Fedora as a special
> case for now.

You can try Noto Sans Symbols (google-noto-sans-symbols-fonts) or
Symbola (gdouros-symbola-fonts). We could make the R package depend on
any of these fonts included in Fedora.

> Thanks for your point about just using symbolfamily=family as the Fedora
> default. That seems reasonable (and definitely better than it just
> being completely broken!).
>
> That does still leave the problem of how to set the default value for
> "symbolfamily" JUST on Fedora. I am not convinced we can use R code to
> detect Fedora >= 30 reliably (but happy to learn otherwise). Is it a
> possibility for the Fedora distribution to include a .Rprofile.site file
> that sets the X11.options() ?

2. Yes, we can include any custom configuration files or patches. In
fact we will need to patch R 3.6.3 for Fedora 31 at least, because
Fedora 32 is about to be released, and thus R 4.0.0 won't be included
in Fedora 31. The problem with the .Rprofile.site is that any
user-specific .Rprofile will prevent the default from being loaded,
right? And I'd say ~/.Rprofile is pretty common out there, and even
project-specific .Rprofile.