Inappropriate scriptsize/scriptscriptsize range calculation

[RuixiZhang42]

Description

The ranges for applying different ssty features are IMHO calculated inappropriately. More specifically, the calculation relies on class declaration \DeclareMathSizes but this declaration will be overwritten later. This leads to inconsistency and sometimes incorrect results.

Check/indicate

• Relevant for XeTeX
• Relevant for LuaTeX
• Issue tracker has been searched for similar issues?

Minimal example demonstrating the issue

% !TeX program = XeLaTeX or LuaLaTeX
\documentclass{ctexart}
\usepackage{unicode-math}
\setmathfont{Fira Math}
\begin{document}
$E_{E_E}$
\the\fontdimen6\scriptscriptfont2
\end{document}

Further details

I choose the ctexart class and the Fira Math font specifically for this issue. The default \normalsize of ctexart is 10.5bp = 10.539375pt, and there is a class declaration of the following form:

\DeclareMathSizes{ 10.539375pt }{ 10.539375pt }{ 7pt }{ 5pt }

According to unicode-math:

• +ssty=1 is applied to sizes -6, since (5 + 7)/2;
• +ssty=0 is applied to sizes 6-8.769685, since (7 + 10.539375)/2;
• No additional feature is applied to sizes 8.769685-.

But the problem is that Fira Math has ScriptScriptPercentScaleDown = 60. So unicode-math will eventually load the second-level subscript at 10.539375pt * 0.6 = 6.323625pt, to which first-level subscript optical size is applied.

Perhaps instead of

\fp_gset:Nn \g_@@_size_tfsf_fp   { (\f@size +\sf@size )/2 }
\fp_gset:Nn \g_@@_size_sfssf_fp  { (\sf@size+\ssf@size)/2 }

We should consider

\fp_gset:Nn \g_@@_size_tfsf_fp   { (\f@size +\@@_fontdimen_to_scale:nN {10} \g_@@_trial_font )/2 }
\fp_gset:Nn \g_@@_size_sfssf_fp  { (\@@_fontdimen_to_scale:nN {10} \g_@@_trial_font +\@@_fontdimen_to_scale:nN {11} \g_@@_trial_font)/2 }

P.S. Fira Math does not contain optical sizes yet (firamath/firamath#23) but the current issue stands.

[RuixiZhang42]

Somewhere near Section 4.2 Script and scriptscript fonts/features, I think the following additions in the documentation would be great:

1. In contrast to Minion Math, most other OpenType math fonts do not use multiple files for optical sizes. Instead, these fonts use the ssty feature tag to access alternative glyphs for first- and second-level subscripts and superscripts. These alternative glyphs constitute an (optical) size feature rather than a style feature. See Application of ssty feature at different text sizes #275.
2. Furthermore, the legacy \DeclareMathSizes declarations are the results of the limitation of bit-map math font. Most notably, 11pt and 12pt share the same first-level script size 8pt and the same second-level script size 6pt. However, OpenType math fonts can be scaled up or down arbitrarily and contain parameters ScriptPercentScaleDown and ScriptScriptPercentScaleDown, so it is reasonable to use OpenType parameters rather than the legacy math sizes. See Discrepancies in font size selection #478.
3. Therefore, unicode-math will respect the OpenType parameters — which are set by the font designers — and overwrite the legacy math sizes, but only for \normalsize (i.e., body text). Hence, the size for first-level scripts will be ScriptPercentScaleDown % of the text size, while the size for second-level scripts will be ScriptScriptPercentScaleDown % of the text size.
4. (Raised in this issue, better wording desired) In addition, unicode-math will apply +ssty=1 to any sizes that are less than ( \sf@size + \ssf@size )/2 and +ssty=0 to any sizes that are between ( \sf@size + \ssf@size )/2 and ( \f@size + \sf@size )/2, with the newly calculated \sf@size and \ssf@size. This sets up optical sizes appropriately (I hope).

I hope the above additions will address #275, #478 and this issue at once.

[RuixiZhang42]

Here is a more “visual” example:

\documentclass{ctexart}
\usepackage{unicode-math}
\setmathfont{Fira Math}[
  script-font  = TeX Gyre Pagella Math,
  sscript-font = TeX Gyre Termes Math
]
\begin{document}
$E_{E_E} 2_{2_2}$
\end{document}

I was expecting TeX Gyre Termes Math in the second-level subscript but it was TeX Gyre Pagella Math.

[wspr]

Thanks, that’s a good example. I hadn’t overlooked this, I just don’t have much time at the moment… I really appreciate the work you’ve done to work through the issue!

[stone-zeng]

It seems that this issue shows the reason of #503. I tried to change Script(Script)PercentScaleDown (of Fira Math) and then ssty2 can be loaded.

[stone-zeng]

If we use font size command as \Huge etc, the position and style of super/sub-scripts will become abnormal:

\documentclass{article}
\usepackage{amsmath,unicode-math}
\setmathfont{XITS Math}

\def\TAG#1{\tag*{\scriptsize\ttfamily\textbackslash#1}}

\begin{document}

{\tiny         \[ ab^{ab^{ab}} + f'(x) + g_{x_y}(x) \TAG{tiny}         \]}
{\scriptsize   \[ ab^{ab^{ab}} + f'(x) + g_{x_y}(x) \TAG{scriptsize}   \]}
{\footnotesize \[ ab^{ab^{ab}} + f'(x) + g_{x_y}(x) \TAG{footnotesize} \]}
{\small        \[ ab^{ab^{ab}} + f'(x) + g_{x_y}(x) \TAG{small}        \]}
{\normalsize   \[ ab^{ab^{ab}} + f'(x) + g_{x_y}(x) \TAG{normalsize}   \]}
{\large        \[ ab^{ab^{ab}} + f'(x) + g_{x_y}(x) \TAG{large}        \]}
{\Large        \[ ab^{ab^{ab}} + f'(x) + g_{x_y}(x) \TAG{Large}        \]}
{\LARGE        \[ ab^{ab^{ab}} + f'(x) + g_{x_y}(x) \TAG{LARGE}        \]}
{\huge         \[ ab^{ab^{ab}} + f'(x) + g_{x_y}(x) \TAG{huge}         \]}
{\Huge         \[ ab^{ab^{ab}} + f'(x) + g_{x_y}(x) \TAG{Huge}         \]}

\end{document}

IMHO, script/scriptscript font should not based on the absolute font size.

[RuixiZhang42]

@stone-zeng

IMHO, script/scriptscript font should not based on the absolute font size.

This is indeed a controversial problem. In #275 Will and I had a discussion on whether the ssty tag should be “style”-linked or “size”-linked, see, #275 (comment) and onward. I still stand by a “size”-linked approach for the reasons I have explained in that issue.

But I also acknowledge that by doing so (i.e., “size”-linked) we are breaking \prime (which is a TeX to Unicode mess to begin with).

---

The current issue points out that ssty is to be applied based on old \DeclareMathSizes, but unicode-math later re-declares math sizes based on Script(Script)PercentScaleDown.

It seems that this issue shows the reason of #503. I tried to change Script(Script)PercentScaleDown (of Fira Math) and then ssty2 can be loaded.

An easy fix for now, without changing font information, is to issue

\DeclareMathSizes{10}{10}{8}{6}% article
...
\DeclareMathSizes{10.5bp}{10.5bp}{8.4bp}{6.3bp}% ctex article
...

before \usepackage{unicode-math}.

If 80(60) are intended for Script(Script)PercentScaleDown, there is no need to change them to 70-ish and 50-ish. From a design point of view, I think a sans-serif math font for presentation should have slightly larger PercentScaleDown values (for comparison, the TeX Gyre Math families have PercentScaleDown values of 74 and 55).

[stone-zeng]

I have modified the Script(Script)PercentScaleDown values to 72 and 52 58. This is more likely a design ratther than a hack. 80% actually looks a little bit large, even for a sans-serif font.

Note that "Unicode subscripts and superscripts" are smaller than the "TeX" sub/superscripts, so decreasing PercentScaleDown is a way to make them looks consistent (I am not sure whether it's reasonable).

(The fonts shown above are Latin Modern, Fira, XITS and Cambria)

[wspr]

I'm a little confused here. The code to select a maths font is currently:

    \bool_if:NT \g_@@_init_bool \@@_fontspec_trial_font:
    \bool_if:NT \g_@@_init_bool \@@_declare_math_sizes:
    \@@_fontspec_select_font:

where the "trial font" is supposed to load the maths font for the first time, extract values of the different sizes, then use \DeclareMathSizes itself to set up all the proper sizes, and only then do the fp calculations to define the ranges.

Is this not what happens? As the explanation of the problem and the proposed fix implies that doesn't happen.

[RuixiZhang42]

@wspr Apparently not, I’m afraid… With the following example:

\documentclass{article}
\usepackage{unicode-math}
\setmathfont{texgyrepagella-math.otf}
\begin{document}
$E_{E_E}$\par
\makeatletter
\f@size\par\sf@size\par\ssf@size
\makeatother
\end{document}

I see that the target sizes are set up correctly at 10pt, 7.4pt and 5.5pt. So it is expected to have the ranges 8.7-, 6.45-8.7 and -6.45. However, with the working code (2019/02/10), I got the following log, which contains the ranges <8.5->, <6-8.5> and <-6> (clearly from the legacy 10pt, 7pt and 5pt):

Package fontspec Info: Font family 'texgyrepagella-math.otf(0)' created for
(fontspec)             font 'texgyrepagella-math.otf' with options
(fontspec)             [BoldItalicFont={},ItalicFont={},SmallCapsFont={},Script
=Math].
(fontspec)              
(fontspec)              This font family consists of the following NFSS
(fontspec)             series/shapes:
(fontspec)              
(fontspec)             - 'normal' (m/n) with NFSS spec.:
(fontspec)             <->"[texgyrepagella-math.otf]/OT:script=math;language=DF
LT;"
(fontspec)             - 'small caps'  (m/sc) with NFSS spec.: 


Package fontspec Info: Font family 'texgyrepagella-math.otf(1)' created for
(fontspec)             font 'texgyrepagella-math.otf' with options
(fontspec)             [BoldItalicFont={},ItalicFont={},SmallCapsFont={},Script
=Math,SizeFeatures={{Size=8.5-},{Size=6-8.5,Font=texgyrepagella-math.otf,Style=
MathScript},{Size=-6,Font=texgyrepagella-math.otf,Style=MathScriptScript}}].
(fontspec)              
(fontspec)              This font family consists of the following NFSS
(fontspec)             series/shapes:
(fontspec)              
(fontspec)             - 'normal' (m/n) with NFSS spec.:
(fontspec)             <8.5->"[texgyrepagella-math.otf]/OT:script=math;language
=DFLT;"<6-8.5>"[texgyrepagella-math.otf]/OT:script=math;language=DFLT;+ssty=0;"
<-6>"[texgyrepagella-math.otf]/OT:script=math;language=DFLT;+ssty=1;"
(fontspec)             - 'small caps'  (m/sc) with NFSS spec.: 

Although the order of \@@_fontspec_trial_font:, \@@_declare_math_sizes: and \@@_fontspec_select_font: seems to be correct, it appears that the new sizes from \@@_declare_math_sizes: are not passed to \@@_fontspec_select_font: in time. Perhaps an expansion issue?

[RuixiZhang42]

@wspr Wow, I think I finally figure something out. Please ignore the “fix” in the OP — it was written before I was aware of the order of \@@_declare_math_sizes: and \@@_fontspec_select_font:.

---

I modified \@@_declare_math_sizes: (forgive my uneducated LaTeX3 code) to see what went wrong:

\__um_cs_new:Nn \__um_declare_math_sizes:
  {
    \dim_compare:nF { \fontdimen 10 \g__um_trial_font == 0pt }
      {
        \DeclareMathSizes { \f@size } { \f@size }
          { \__um_fontdimen_to_scale:nN {10} \g__um_trial_font }
          { \__um_fontdimen_to_scale:nN {11} \g__um_trial_font }
        \__um_warning:n
          {
            \fp_eval:n { \__um_fontdimen_to_scale:nN {11} \g__um_trial_font }
            \space versus\space \ssf@size\space !!!
          }
      }
  }

I then compiled the example in my previous comment and got this message:

! LaTeX3 Error: Unknown message '5.5 versus 5 !!!' for module 'unicode-math'.

So I realized that the new sizes were not updated until the first formula was encountered.

---

A reasonable fix, based on 32.2 Math fonts setup of source2e.pdf, is thus the following:

\cs_new:Nn \@@_declare_math_sizes:
  {
    \dim_compare:nF { \fontdimen 10 \g_@@_trial_font == 0pt }
      {
        \DeclareMathSizes { \f@size } { \f@size }
          { \@@_fontdimen_to_scale:nN {10} \g_@@_trial_font }
          { \@@_fontdimen_to_scale:nN {11} \g_@@_trial_font }
        \check@mathfonts
      }
  }

which forces the sizes of the math font to be updated. Though there will be additional messages in the log:

LaTeX Font Info:    External font `cmex10' loaded for size
(Font)              <7.4> on input line 4.
LaTeX Font Info:    External font `cmex10' loaded for size
(Font)              <5.5> on input line 4.

[wspr]

Wow, of course! That was a pretty big oversight of mine… thanks so much for digging into this, I really appreciate it. So the next question is “what is correct?” I still think that by default the maths font setup should respect the design of the font, but it would be nice to provide an interface for the class designer. I don’t have much time tonight but I’ll put in the quick fix straight away...

[wspr]

I don't have time to debug now but unfortunately the \check@mathfonts command broke some other things. So I've indeed implemented your original suggestion.

[RuixiZhang42]

Too bad that \check@mathfonts didn’t work out. I’m glad my original proposal is improved and has made its way to v0.8n.

However, there was an overlooked issue in the original code — I didn’t check whether ScriptPercentScaleDown was available or not. Since \@@_declare_math_sizes: re-declares math sizes only if ScriptPercentScaleDown is present, I think so do \g_@@_size_tfsf_fp and \g_@@_size_sfssf_fp.

With a non-math font in 10pt article, we would get zero for both \@@_sf_size: and \@@_ssf_size:, and then 5 for \g_@@_size_tfsf_fp and 0 for \g_@@_size_sfssf_fp, which results in the following 3 ranges: 5-, 0-5 and -0. This does not cause “real” problem AFAICT, since unicode-math is able to fallback to LM Math. But I think we should do the check anyway: Something like

    \dim_compare:nTF { \fontdimen 10 \g_@@_trial_font == 0pt }
      {
        \fp_gset:Nn \g_@@_size_tfsf_fp   { (\f@size +\sf@size )/2 }
        \fp_gset:Nn \g_@@_size_sfssf_fp  { (\sf@size+\ssf@size)/2 }
      }
      {
        \fp_gset:Nn \g_@@_size_tfsf_fp   { (\f@size     +\@@_sf_size: )/2 }
        \fp_gset:Nn \g_@@_size_sfssf_fp  { (\@@_sf_size:+\@@_ssf_size:)/2 }
      }

[RuixiZhang42]

… or something better — to define \@@_sf_size: to be either \sf@size or \@@_fontdimen_to_scale:nN {10} \g_@@_trial_font; to define \@@_ssf_size: to be either \ssf@size or \@@_fontdimen_to_scale:nN {11} \g_@@_trial_font depending on the availability of ScriptPercentScaleDown.

[wspr]

I hadn’t thought of that, but luckily my code is smarter than I am! I assume that any maths font with a `math` OpenType script will always be “well defined”, so if \setmathfont is called on a non-math font, it bypasses the \DeclareMathSizes line: \bool_if:NT \g_@@_init_bool \@@_fontspec_trial_font: \bool_if:NT \g_@@_init_bool \@@_declare_math_sizes: So hopefully we’re good here :)

[RuixiZhang42]

For some reasons, after the initial trial fails, unicode-math will still try to load the non-math font one more time with the 5-, 0-5, -0 ranges calculated from 10, 0, 0.

\documentclass{article}
\usepackage{unicode-math}
\setmathfont{texgyretermes-regular.otf} % non-math font
\begin{document}
$E_{E_E}$
\end{document}

In the log:

Package fontspec Warning: Font "texgyretermes-regular" does not contain
(fontspec)                requested Script "Math".

Package fontspec Info: Font family 'texgyretermes-regular.otf(0)' created for
...

Package fontspec Info: Font family 'texgyretermes-regular.otf(1)' created for
(fontspec)             font 'texgyretermes-regular.otf' with options
(fontspec)             [BoldItalicFont={},ItalicFont={},SmallCapsFont={},Script
=Math,SizeFeatures={{Size=5-},{Size=0-5,Font=texgyretermes-regular.otf,Style=Ma
thScript},{Size=-0,Font=texgyretermes-regular.otf,Style=MathScriptScript}}].
...

In particular, the Size=5-, Size=0-5 and Size=-0 are maybe troublesome.

[wspr]

Loading a non-math font like this is no longer supported and produces warning: Package unicode-math Warning: The first font loaded by unicode-math must be an (unicode-math) OpenType Math font (with script=math). If you (unicode-math) simply want ‘the default’ before loading (unicode-math) supplementary fonts over the top for certain (unicode-math) ranges, use: (unicode-math) \setmathfont{latinmodern-math.otf} And the side-effect is that latin modern math is loaded anyway, which (I think) overwrites the funny size for texgyretermes. I should possibly make that warning an error...

[RuixiZhang42]

Ah yes, unicode-math is smart enough to fallback to Latin Modern Math ;-), which is why I keep emphasizing that “this is not a ‘real’ problem” and that “it may be troublesome”… Warning to error seems a bit overkill, though.