Render unicode emoji with image fallback

[jhildenbiddle]

(Moving discussion of rendering unicode emoji from #768 to a separate thread)

From #768:

According to caniemoji.com, unicode emoji are supported all the way back to Windows 7, macOS 10.7, Android 4.4, and iOS 5. Switching to unicode emoji would remove emoji <img> requests and render emoji native to each platform. Seems like a better route to go, so long as the unsupported OS+Browser combinations aren't a concern. This article will likely be helpful if/when the switch to unicode happens.

[timaschew]

What is your idea to support GitHub based aliases? I don't see any unique short names on this page: https://unicode.org/emoji/charts/full-emoji-list.html
On another I saw some other project which use the same source, but there are ambiguous keywords which are assigned to multiple emojis sometimes.

[jhildenbiddle]

My initial thought was to just leverage the GitHub emoji API to get both the alias and the unicode value:

// API
{
  "+1": "https://github.com/githubassets/images/icons/emoji/unicode/1f44d.png?v8",
  ...
  "thumbsup": "https://github.com/githubassets/images/icons/emoji/unicode/1f44d.png?v8",
  ...
}

// Alias  : "+1" or "thumbsup"
// Unicode: "1f44d"
// URL    : "https://github.com/githubassets/images/icons/emoji/unicode/1f44d.png?v8"

GitHub has already done the work of mapping aliases to unicode values and images (the file name is the unicode value), as well as adding logical duplicates like the example above.

As for the implementation:

• Emoji data should be embedded in docsify.js, not requested from the GitHub API by the client.

• GitHub data should be transformed to better fit docsify's usage and reduce client-side parsing. For example, instead of embedding GitHub's API data as-is:

  {
    "+1": "https://github.com/githubassets/images/icons/emoji/unicode/1f44d.png?v8",
    ...
  }
  
  // const emojiURL  = emoji["+1"];
  // const emojiCode = emojiURL.slice(emojiURL.lastIndexOf('/') + 1, emojiURL.lastIndexOf('.'));

  The data should be transformed to allow fast unicode and URL lookups:

  {
    "+1": {
      unicode: "1f44d",
      url: "https://github.com/githubassets/images/icons/emoji/unicode/1f44d.png?v8"
    }
    ...
  }
  
  // const emojiURL  = emoji["+1"].url
  // const emojiCode = emoji["+1"].unicode

• An automated task should be added that refreshes docsify's local copy of GitHub API data, transforms the data as described above, and stores the result in a file that docsify can import/require at build time.

• A test for emoji support will determine if a unicode character or image is rendered. I'd likely reference Modernizr's emoji test and build from that. The test only needs to done one time, so the performance impact should be negligible.

• Some minor CSS tweaks may be required for unicode emoji characters, and it will most likely make sense to wrap them in a <span> tag with a unique class.

These non-breaking changes will provide the following benefits:

1. Improved emoji rendering performance by removing image requests on platforms that support unicode emoji
2. Rendering of native emoji on platforms that support unicode emoji
3. Additional emoji alias support (docsify currently support 886, GitHub support 1508)
4. Simplified emoji support for docsify devs (via automated task described above)

How does that sound?

[jhildenbiddle]

BTW, there are quite a few GitHub emoji lists generated from the GitHub API that we can direct users to:

• https://github.com/ikatyang/emoji-cheat-sheet
• https://github-emoji-list.herokuapp.com/
• https://jzeferino.github.io/AllGithubEmojis/

We should either reference once of these or provide our own auto-generated table in the official documentation.

[timaschew]

Sounds very good! Thanks for providing all the details.

I see now that the unicode is available in the URL 🙈

One suggestion: Some emojis have multiple unicode units/blocks? for example flags:

{
  "norway": {
    "unicode": ["1f1f3", "1f1f4"],
    "url": "https://github.com/githubassets/images/icons/emoji/unicode/1f1f3-1f1f4.png?v8"
  }
}

I've just realized, that the emojis on unicode.org have sometimes more than two units, but on GitHub they have maximal two units always. For instance the hash has three units on unicode.org
and on GitHub only two: https://github.com/githubassets/images/icons/emoji/unicode/0023-20e3.png?v8

But both variants are working.

[timaschew]

Which milestone for this issues? 4.x or 5.x

[jhildenbiddle]

Depends on how important you think this is. My preference would be to focus on getting the repo up-to-date (PRs, IE compatibility, triaging issues, etc.) for the next release (4.x) and push enhancements (like this) off to the following release.

[trusktr]

@sy-records handled this in #1188

If we want to improve on it (f.e. make a bot to keep it updated, etc) let's open new issues for each update.

[jhildenbiddle]

Reopening since #1188 does not address the issue/enhancements mentioned.

There are multiple goals here:

1. Render native emoji where supported (which is basically everywhere in 2022).
2. Render all emoji without requiring site owners to use Docsify's emoji plugin.
3. Generate emoji data from GitHub API instead of manually updating (see chore: sync emojis #1745 as an example).

More details above (#779 (comment)). Automating the update process via a bot isn't necessary. A build task and appropriate automated tests would be sufficient, but these are implementation details and not the end goal.

Here is a screenshot showing two different emoji rendered using native emoji (macOS in the screenshot since that's what I'm running) and on a Docsify site both with and without the emoji.js plugin:

Things to note:

• Native emoji are more performant as they do not require a network request for a remote image
• Native emoji allow users to copy/paste individually or as blocks of text.
• Native emoji are more familiar to users as they are native to the platform they are using
• Docsify is able to render some emoji without the emoji.js plugin (:smile:) but not others (:1st_place_medal:). This is unnecessarily confusing for users. We can simplify this by either:
  • Deprecating the emoji.js plugin and allowing Docsify to render all emoji by default. Users that do not wish to render emoji can use the noEmoji option. This would be my preference given the popularity of emoji.
  • Removing the ability to render emoji from Docsify core and require the emoji.js plugin to render emoji. This would make the purpose of the emoji plugin clear and reduce the size of the core Docsify bundle. This would not be my preference, but I can see how kilobyte-sensitive people may prefer it.

@docsifyjs/core -- Thoughts?

[jhildenbiddle]

See #1746, which is currently a draft/POC PR that addresses items 1 and 2 in the above comment.