Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TermControl: force all ambiguous glyphs to be narrow #2928

Merged

Conversation

DHowett-MSFT
Copy link
Contributor

From Egmont Koblinger:

In terminal emulation, apps have to be able to print something and
keep track of the cursor, whereas they by design have no idea of the
font being used. In many terminals the font can also be changed runtime
and it's absolutely not feasible to then rearrange the cells. In some
other cases there is no font at all (e.g. the libvterm headless terminal
emulation library, or a detached screen/tmux), or there are multiple
fonts at once (a screen/tmux attached from multiple graphical
emulators).

The only way to do that is via some external agreement on the number
of cells, which is typically the Unicode EastAsianWidth, often accessed
via wcwidth(). It's not perfect (changes through Unicode versions, has
ambiguous characters, etc.) but is still the best we have.

glibc's wcwidth() reports 1 for ambiguous width characters, so the de
facto standard is that in terminals they are narrow.

If the glyph is wider then the terminal has to figure out what to do.
It could crop it (newer versions of Konsole, as far as I know), overflow
to the right (VTE), shrink it (Kitty I believe does this), etc.

See Also:
https://bugzilla.gnome.org/show_bug.cgi?id=767529
https://gitlab.freedesktop.org/terminal-wg/specifications/issues/9
https://www.unicode.org/reports/tr11/tr11-34.html

Salient point from proposed update to Unicode Standard Annex #11:

Note: The East_Asian_Width property is not intended for use by modern
terminal emulators without appropriate tailoring on a case-by-case
basis.

Fixes #2066
Fixes #2375

PR Checklist

  • Closes stuff
  • written by an employee
  • Testing sorta done
  • Requires documentation to be updated
  • I've discussed this with core contributors already. AT LENGTH.

Validation Steps Performed

This one has some followup issues. I went wild on it, but it does still have some issues. It makes #900 worse, and introduces a debug break you hit when the renderer and the column counter disagree. It might make the renderer discard the rest of the line it was working on.

From Egmont Koblinger:
> In terminal emulation, apps have to be able to print something and
keep track of the cursor, whereas they by design have no idea of the
font being used. In many terminals the font can also be changed runtime
and it's absolutely not feasible to then rearrange the cells. In some
other cases there is no font at all (e.g. the libvterm headless terminal
emulation library, or a detached screen/tmux), or there are multiple
fonts at once (a screen/tmux attached from multiple graphical
emulators).

> The only way to do that is via some external agreement on the number
of cells, which is typically the Unicode EastAsianWidth, often accessed
via wcwidth(). It's not perfect (changes through Unicode versions, has
ambiguous characters, etc.) but is still the best we have.

> glibc's wcwidth() reports 1 for ambiguous width characters, so the de
facto standard is that in terminals they are narrow.

> If the glyph is wider then the terminal has to figure out what to do.
It could crop it (newer versions of Konsole, as far as I know), overflow
to the right (VTE), shrink it (Kitty I believe does this), etc.

See Also:
https://bugzilla.gnome.org/show_bug.cgi?id=767529
https://gitlab.freedesktop.org/terminal-wg/specifications/issues/9
https://www.unicode.org/reports/tr11/tr11-34.html

Salient point from proposed update to Unicode Standard Annex #11:
> Note: The East_Asian_Width property is not intended for use by modern
terminal emulators without appropriate tailoring on a case-by-case
basis.

Fixes #2066
Fixes #2375
Copy link
Member

@zadjii-msft zadjii-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm okay with this, but I also really want #900 to get better in 0.6 if we're also making it aggressively worse.

@zadjii-msft zadjii-msft added Area-TerminalControl Issues pertaining to the terminal control (input, selection, keybindings, mouse interaction, etc.) Issue-Bug It either shouldn't be doing this or needs an investigation. Product-Terminal The new Windows Terminal. labels Sep 27, 2019
@DHowett-MSFT
Copy link
Contributor Author

@zadjii-msft i can produce a table of all the emoji that exist, and which ones are miscolumned. Would that be sufficient evidence to move #900 forward, do you reckon?

@zadjii-msft
Copy link
Member

@DHowett-MSFT Probably. I'm probably not going to be the one solving #900 though, that seems more like @miniksa's wheelhouse

@miniksa
Copy link
Member

miniksa commented Oct 14, 2019

@zadjii-msft Mike Griese FTE i can produce a table of all the emoji that exist, and which ones are miscolumned. Would that be sufficient evidence to move #900 forward, do you reckon?

@DHowett-MSFT Dustin Howett FTE Probably. I'm probably not going to be the one solving #900 though, that seems more like @miniksa Michael Niksa FTE's wheelhouse

I'm fine with building such a table, @DHowett-MSFT and resolving the columns. It looks like in #900 that Fcharlie already did something like that

static bool _EnsureStaticInitialization()
{
// use C++11 magic statics to make sure we only do this once.
static bool initialized = []() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const?

@DHowett-MSFT DHowett-MSFT merged commit 1925173 into master Oct 15, 2019
@DHowett-MSFT DHowett-MSFT deleted the dev/duhowett/lose_a_little_bit_of_weight_but_not_too_much branch October 15, 2019 21:55
@ghost
Copy link

ghost commented Oct 23, 2019

🎉Windows Terminal Preview v0.6.2951.0 has been released which incorporates this pull request.:tada:

Handy links:

ghost pushed a commit that referenced this pull request Jul 20, 2020
* send alt/F10 through the control
  We were not listening for WM_SYSKEY{UP,DOWN}
* extract the actual scancode during WM_CHAR, not the bitfield
  We were accidentally sending some of the additional keypress data in with
  the character event in Win32 Input Mode
* set default fg/bg to campbell
  The WPF control starts up in PowerShell blue even though it's not typically used
  in PowerShell blue.
* don't rely on the font to determine wideness
  This is a cross-port of #2928 to the WPF control
* deterministic shutdown
  In testing, I saw a handful of crashes on teardown because we were not shutting
  down the render thread properly.
* don't pass 10 for the font weight ...
  When Cascadia Code is set, it just looks silly.
* trigger render when selection is cleared, do it under lock

Fixes #6966.
DHowett added a commit that referenced this pull request Jul 20, 2020
* send alt/F10 through the control
  We were not listening for WM_SYSKEY{UP,DOWN}
* extract the actual scancode during WM_CHAR, not the bitfield
  We were accidentally sending some of the additional keypress data in with
  the character event in Win32 Input Mode
* set default fg/bg to campbell
  The WPF control starts up in PowerShell blue even though it's not typically used
  in PowerShell blue.
* don't rely on the font to determine wideness
  This is a cross-port of #2928 to the WPF control
* deterministic shutdown
  In testing, I saw a handful of crashes on teardown because we were not shutting
  down the render thread properly.
* don't pass 10 for the font weight ...
  When Cascadia Code is set, it just looks silly.
* trigger render when selection is cleared, do it under lock

Fixes #6966.

(cherry picked from commit 76de2ae)
DHowett added a commit that referenced this pull request Jul 20, 2020
* send alt/F10 through the control
  We were not listening for WM_SYSKEY{UP,DOWN}
* extract the actual scancode during WM_CHAR, not the bitfield
  We were accidentally sending some of the additional keypress data in with
  the character event in Win32 Input Mode
* set default fg/bg to campbell
  The WPF control starts up in PowerShell blue even though it's not typically used
  in PowerShell blue.
* don't rely on the font to determine wideness
  This is a cross-port of #2928 to the WPF control
* deterministic shutdown
  In testing, I saw a handful of crashes on teardown because we were not shutting
  down the render thread properly.
* don't pass 10 for the font weight ...
  When Cascadia Code is set, it just looks silly.
* trigger render when selection is cleared, do it under lock

Fixes #6966.

(cherry picked from commit 76de2ae)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-TerminalControl Issues pertaining to the terminal control (input, selection, keybindings, mouse interaction, etc.) Issue-Bug It either shouldn't be doing this or needs an investigation. Product-Terminal The new Windows Terminal.
Projects
None yet
4 participants