dev.args encoding issue with unicode symbol

[sebkopf]

I ran into an issue on Mac OS X where the encoding in dev.args chunk option seems to be getting lost. To explain I am including a series of reprexs to highlight the issue . Any help much appreciated. Note that this is NOT an issue on Windows (tested with the same package versions).

When including unicode symbol per mille (u2030 = ‰), saving to a pdf throws the following mbcsToSbcs error if the encoding is not included (unicode symbol dagger u2021 = ‡ for comparison does not throw an error on its own):

# unicode symbol causes an error
library(ggplot2)
p <- ggplot() + 
 annotate(geom = 'text', x = 0, y = 1, label = 'permil: \u2030 \n dagger: \u2021')
ggsave(plot = p, filename = "test.pdf")
#> Saving 7 x 5 in image
#> Error in grid.Call.graphics(C_text, as.graphicsAnnot(x$label), x$x, x$y, : conversion failure from UTF-8 to latin1 on 'permil: ‰ ' in 'mbcsToSbcs'

^{Created on 2023-10-31 with reprex v2.0.2}

With the encoding included (e.g. WinAnsi, others work too), this issue is resolved:

# no error if proper encoding is included
library(ggplot2)
p <- ggplot() + 
 annotate(geom = 'text', x = 0, y = 1, label = ' dagger: \u2021 \n permil: \u2030')
ggsave(plot = p, filename = "test.pdf", encoding = "WinAnsi")
#> Saving 7 x 5 in image

^{Created on 2023-10-31 with reprex v2.0.2}

However, when now including automatic chunk plot saving in an RMarkdown document, the error reappears for saving to pdf. First how it should work (no issue when saving to png alone, included a dev.args parameter to illustrate its effect):

# generate a simple rmarkdown file
"---
title: test
format: html
---
```{r 'plot', dev = 'png', fig.path = 'test-', dev.args = list(bg = 'yellow')}
library(ggplot2)
ggplot() +
   annotate(geom = 'text', x = 0, y = 1, label = 'dagger: \\u2021 \\n permil: \\u2030') +
   theme(plot.background = element_blank())
```" |> cat(file = "example.Rmd")

# render the rmarkdown file
rmarkdown::render("example.Rmd")
#> processing file: example.Rmd
#> output file: example.knit.md
#> /Applications/RStudio.app/Contents/Resources/app/quarto/bin/tools/pandoc +RTS -K512m -RTS example.knit.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output example.html --lua-filter /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/rmarkdown/rmarkdown/lua/pagebreak.lua --lua-filter /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/rmarkdown/rmarkdown/lua/latex-div.lua --embed-resources --standalone --variable bs3=TRUE --section-divs --template /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library/rmarkdown/rmd/h/default.html --no-highlight --variable highlightjs=1 --variable theme=bootstrap --mathjax --variable 'mathjax-url=https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --include-in-header /var/folders/ns/x0fwvm6x22v8gf5tmj9r3yww0000gn/T//RtmptknJaz/rmarkdown-str16a411eae9e82.html
#> 
#> Output created: example.html

# check if png was created correctly
magick::image_read("test-plot-1.png")

^{Created on 2023-10-31 with reprex v2.0.2}

Actual Issue

But as soon as I save to pdf I get the mbcsToSBcs error even when defining the encoding in the dev.args:

# saving to pdf in the chunk fails even with encoding = "WinAnsi"
"---
title: test
format: html
---
```{r 'plot', dev = 'pdf', fig.path = 'test-', dev.args = list(encoding = 'WinAnsi')}
library(ggplot2)
ggplot() +
   annotate(geom = 'text', x = 0, y = 1, label = 'dagger: \\u2021 \\n permil: \\u2030') +
   theme(plot.background = element_blank())
```" |> cat(file = "example.Rmd")

# render the rmarkdown file
rmarkdown::render("example.Rmd")
#> processing file: example.Rmd
#> 
#> Quitting from lines 6-11 [plot] (example.Rmd)
#> Error in `grid.Call.graphics()`:
#> ! conversion failure from UTF-8 to latin1 on ' permil: ‰' in 'mbcsToSbcs'
#> Backtrace:
#>      ▆
#>   1. ├─rmarkdown::render("example.Rmd")
#>   2. │ └─knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
#>   3. │   └─knitr:::process_file(text, output)
#>   4. │     ├─knitr:::handle_error(...)
#>   5. │     │ └─base::withCallingHandlers(...)
#>   6. │     ├─base::withCallingHandlers(...)
#>   7. │     ├─knitr:::process_group(group)
#>   8. │     └─knitr:::process_group.block(group)
#>   9. │       └─knitr:::call_block(x)
#>  10. │         └─knitr:::block_exec(params)
#>  11. │           └─knitr:::eng_r(options)
#>  12. │             ├─knitr:::in_input_dir(...)
#>  13. │             │ └─knitr:::in_dir(input_dir(), expr)
#>  14. │             └─knitr (local) evaluate(...)
#>  15. │               └─evaluate::evaluate(...)
#>  16. │                 └─evaluate:::evaluate_call(...)
#>  17. │                   ├─base (local) handle(...)
#>  18. │                   ├─base::withCallingHandlers(...)
#>  19. │                   ├─base::withVisible(value_fun(ev$value, ev$visible))
#>  20. │                   └─knitr (local) value_fun(ev$value, ev$visible)
#>  21. │                     └─knitr (local) fun(x, options = options)
#>  22. │                       ├─base::withVisible(knit_print(x, ...))
#>  23. │                       ├─knitr::knit_print(x, ...)
#>  24. │                       └─knitr:::knit_print.default(x, ...)
#>  25. │                         └─evaluate (local) normal_print(x)
#>  26. │                           ├─base::print(x)
#>  27. │                           └─ggplot2:::print.ggplot(x)
#>  28. │                             ├─grid::grid.draw(gtable)
#>  29. │                             └─grid:::grid.draw.gTree(gtable)
#>  30. │                               └─grDevices::recordGraphics(drawGTree(x), list(x = x), getNamespace("grid"))
#>  31. ├─grid:::drawGTree(x)
#>  32. │ ├─grid::grid.draw(x$children[[i]], recording = FALSE)
#>  33. │ └─grid:::grid.draw.gTree(x$children[[i]], recording = FALSE)
#>  34. │   └─grDevices::recordGraphics(drawGTree(x), list(x = x), getNamespace("grid"))
#>  35. ├─grid:::drawGTree(x)
#>  36. │ ├─grid::grid.draw(x$children[[i]], recording = FALSE)
#>  37. │ └─grid:::grid.draw.gTree(x$children[[i]], recording = FALSE)
#>  38. │   └─grDevices::recordGraphics(drawGTree(x), list(x = x), getNamespace("grid"))
#>  39. ├─grid:::drawGTree(x)
#>  40. │ ├─grid::grid.draw(x$children[[i]], recording = FALSE)
#>  41. │ └─grid:::grid.draw.grob(x$children[[i]], recording = FALSE)
#>  42. │   └─grDevices::recordGraphics(drawGrob(x), list(x = x), getNamespace("grid"))
#>  43. └─grid:::drawGrob(x)
#>  44.   ├─grid::drawDetails(x, recording = FALSE)
#>  45.   └─grid:::drawDetails.text(x, recording = FALSE)
#>  46.     └─grid:::grid.Call.graphics(...)

^{Created on 2023-10-31 with reprex v2.0.2}

I'm totally stumped, especially because this used to work in the past. Any help why encoding does not fix this issue would be very much appreciated. It's possible that a recent OS X upgrade is the cause for this but that seems unfixable so I hope it's not the issue.

Here's my xfun::session_info('rmarkdown')

R version 4.3.1 (2023-06-16)
Platform: x86_64-apple-darwin20 (64-bit)
Running under: macOS Sonoma 14.0, RStudio 2023.6.2.561


Locale: en_US.UTF-8 / en_US.UTF-8 / en_US.UTF-8 / C / en_US.UTF-8 / en_US.UTF-8

time zone: America/Denver
tzcode source: internal

Package version:
  base64enc_0.1.3   bslib_0.5.1       cachem_1.0.8      cli_3.6.1         digest_0.6.33     ellipsis_0.3.2    evaluate_0.22     fastmap_1.1.1     fontawesome_0.5.2
  fs_1.6.3          glue_1.6.2        graphics_4.3.1    grDevices_4.3.1   highr_0.10        htmltools_0.5.6.1 jquerylib_0.1.4   jsonlite_1.8.7    knitr_1.45.2     
  lifecycle_1.0.3   magrittr_2.0.3    memoise_2.0.1     methods_4.3.1     mime_0.12         R6_2.5.1          rappdirs_0.3.3    rlang_1.1.1       rmarkdown_2.25.1 
  sass_0.4.7        stats_4.3.1       stringi_1.7.12    stringr_1.5.0     tinytex_0.48      tools_4.3.1       utils_4.3.1       vctrs_0.6.4       xfun_0.40        
  yaml_2.3.7       

Pandoc version: 3.1.1

Checklist

When filing a bug report, please check the boxes below to confirm that you have provided us with the information we need. Have you:

• [ x ] formatted your issue so it is easier for us to read?

• [ x ] included a minimal, self-contained, and reproducible example?

• [ x ] pasted the output from xfun::session_info('rmarkdown') in your issue?

• [ x ] upgraded all your packages to their latest versions (including your versions of R, the RStudio IDE, and relevant R packages)?

• [ x ] installed and tested your bug with the development version of the rmarkdown package using remotes::install_github("rstudio/rmarkdown")?

[yihui]

Should be fixed in knitr now. Thanks for the report! You can install the development version via

remotes::install_github('yihui/knitr')

[github-actions]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.