Encoding issues in newer Gradio versions

[BramVanroy]

Describe the bug

In 3.33.1, my gradio app could be launched perfectly fine on both Windows and Linux. However, when I upgraded to the most recent version, I was getting encoding issues left and right. I traced it back to the following line:

gradio/gradio/cli/commands/reload.py

Line 34 in 4d3aad3

app_text = Path(original_path).read_text()

If I change that to the following everything works as intended.

app_text = Path(original_path).read_text(encoding="utf-8")

I understand that people may DO want to use cp1252 on Windows, or any other encoding scheme, but it'd be great to allow for more control here. Either by an environment variable or even a launch argument where a user can specify the encoding to use.

Have you searched existing issues? 🔎

• I have searched and found no existing issues

Reproduction

I cloned this repo and tried to run it locally: https://huggingface.co/spaces/uonlp/open_multilingual_llm_leaderboard

Screenshot

No response

Logs

[notice] A new release of pip available: 22.3.1 -> 23.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip
(leaderboard) PS F:\python\open_multilingual_llm_leaderboard> gradio .\app.py
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ F:\python\open_multilingual_llm_leaderboard\.venv\lib\site-packages\gradio\cli\commands\reload.p │
│ y:97 in main                                                                                     │
│                                                                                                  │
│    94 │   demo_path: Path, demo_name: str = "demo", watch_dirs: Optional[List[str]] = None       │
│    95 ):                                                                                         │
│    96 │   # default execution pattern to start the server and watch changes                      │
│ ❱  97 │   filename, path, watch_dirs, demo_name = _setup_config(                                 │
│    98 │   │   demo_path, demo_name, watch_dirs                                                   │
│    99 │   )                                                                                      │
│   100 │   # extra_args = args[1:] if len(args) == 1 or args[1].startswith("--") else args[2:]    │
│                                                                                                  │
│ ╭────────────── locals ──────────────╮                                                           │
│ │  demo_name = 'demo'                │                                                           │
│ │  demo_path = WindowsPath('app.py') │                                                           │
│ │ watch_dirs = []                    │                                                           │
│ ╰────────────────────────────────────╯                                                           │
│                                                                                                  │
│ F:\python\open_multilingual_llm_leaderboard\.venv\lib\site-packages\gradio\cli\commands\reload.p │
│ y:34 in _setup_config                                                                            │
│                                                                                                  │
│    31 │   additional_watch_dirs: list[str] | None = None,                                        │
│    32 ):                                                                                         │
│    33 │   original_path = demo_path                                                              │
│ ❱  34 │   app_text = Path(original_path).read_text()                                             │
│    35 │                                                                                          │
│    36 │   patterns = [                                                                           │
│    37 │   │   f"with gr\\.Blocks\\(\\) as {demo_name}",                                          │
│                                                                                                  │
│ ╭─────────────────── locals ────────────────────╮                                                │
│ │ additional_watch_dirs = []                    │                                                │
│ │             demo_name = 'demo'                │                                                │
│ │             demo_path = WindowsPath('app.py') │                                                │
│ │         original_path = WindowsPath('app.py') │                                                │
│ ╰───────────────────────────────────────────────╯                                                │
│                                                                                                  │
│ C:\Users\bramv\.pyenv\pyenv-win\versions\3.10.10\lib\pathlib.py:1135 in read_text                │
│                                                                                                  │
│   1132 │   │   """                                                                               │
│   1133 │   │   encoding = io.text_encoding(encoding)                                             │
│   1134 │   │   with self.open(mode='r', encoding=encoding, errors=errors) as f:                  │
│ ❱ 1135 │   │   │   return f.read()                                                               │
│   1136 │                                                                                         │
│   1137 │   def write_bytes(self, data):                                                          │
│   1138 │   │   """                                                                               │
│                                                                                                  │
│ ╭──────────────────────────────── locals ─────────────────────────────────╮                      │
│ │ encoding = 'locale'                                                     │                      │
│ │   errors = None                                                         │                      │
│ │        f = <_io.TextIOWrapper name='app.py' mode='r' encoding='cp1252'> │                      │
│ │     self = WindowsPath('app.py')                                        │                      │
│ ╰─────────────────────────────────────────────────────────────────────────╯                      │
│                                                                                                  │
│ C:\Users\bramv\.pyenv\pyenv-win\versions\3.10.10\lib\encodings\cp1252.py:23 in decode            │
│                                                                                                  │
│    20                                                                                            │
│    21 class IncrementalDecoder(codecs.IncrementalDecoder):                                       │
│    22 │   def decode(self, input, final=False):                                                  │
│ ❱  23 │   │   return codecs.charmap_decode(input,self.errors,decoding_table)[0]                  │
│    24                                                                                            │
│    25 class StreamWriter(Codec,codecs.StreamWriter):                                             │
│    26 │   pass                                                                                   │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ final = True                                                                                 │ │
│ │ input = b'import json\r\nfrom collections import defaultdict\r\n\r\nimport pandas as         │ │
│ │         pd\r\nimport '+4982                                                                  │ │
│ │  self = <encodings.cp1252.IncrementalDecoder object at 0x000001780C99F2B0>                   │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 3554: character maps to <undefined>

System Info

Works with 3.33.1 but not with the most recent version.

Severity

I can work around it

[freddyaboulton]

Would you like to open a PR for this @BramVanroy ?

[BramVanroy]

Sure. Is it sufficient to update this line

gradio/gradio/cli/commands/reload.py

Line 34 in 4d3aad3

app_text = Path(original_path).read_text()

to something like the following, or should there also be an update to Interface or Blocks to make this easier to do without an envvar? (If so, where?)

app_text = Path(original_path).read_text(encoding=os.getenv("GRADIO_APP_ENCODING", "utf-8"))

Probably also a doc update but I'm not sure where to add this information. A dedicated page with a list of suitable environment variables would be useful. There are quite a lot here https://www.gradio.app/docs/interface#initialization

[abidlabs]

cc @aliabd with regards to the documentation, but we can do that in a separate PR @BramVanroy

[abidlabs]

@BramVanroy I think that change should be sufficient. If you could go ahead and open a PR, we're happy to review more thoroughly

[curiousRay]

Encountered the same situation when trying to run my script which contains Chinese characters.
A PR is initiated. @BramVanroy @abidlabs @freddyaboulton

[abidlabs]

Should be closed with #6622