feature request: tweak non-escaped ampersand

[HedvigS]

---

By filing an issue to this repo, I promise that

• [ X] I have fully read the issue guide at https://yihui.org/issue/.
• [ X] I have provided the necessary information about my issue.
  • If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
  • If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('knitr'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('yihui/knitr').
  • If I have posted the same issue elsewhere, I have also mentioned it in this issue.
• [ X] I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

---

For reasons beyond my understanding, some bibtex entries generated by citation() from Description files have non-escaped ampersands - which causes problems with LaTeX later.

See for example spam2017 below.

> knitr::write_bib(x = "spam")
@Manual{R-spam,
  title = {spam: SPArse Matrix},
  author = {Reinhard Furrer and Florian Gerber and Roman Flury},
  year = {2022},
  note = {R package version 2.9-1},
  url = {https://www.math.uzh.ch/pages/spam/},
}

@Article{spam2010,
  title = {{spam}: A Sparse Matrix {R} Package with Emphasis on {MCMC} Methods for {G}aussian {M}arkov Random Fields},
  author = {Reinhard Furrer and Stephan R. Sain},
  journal = {Journal of Statistical Software},
  year = {2010},
  volume = {36},
  number = {10},
  pages = {1--25},
  doi = {10.18637/jss.v036.i10},
}

@Article{spam2015,
  title = {Pitfalls in the Implementation of {B}ayesian Hierarchical Modeling of Areal Count Data: An Illustration Using {BYM} and {L}eroux Models},
  author = {Florian Gerber and Reinhard Furrer},
  journal = {Journal of Statistical Software, Code Snippets},
  year = {2015},
  volume = {63},
  number = {1},
  pages = {1--32},
  doi = {10.18637/jss.v063.c01},
}

@Article{spam2017,
  title = {Extending {R} packages to support 64-bit compiled code: An illustration with spam64 and {GIMMS} {NDVI3g} data},
  author = {Florian Gerber and Kaspar Moesinger and Reinhard Furrer},
  journal = {Computer & Geoscience},
  year = {2017},
  volume = {104},
  pages = {109--119},
  issn = {0098-3004},
  doi = {10.1016/j.cageo.2016.11.015},
}


> xfun::session_info('knitr')
R version 4.3.3 (2024-02-29)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.4.1, RStudio 2023.12.1.402


Locale: en_US.UTF-8 / en_US.UTF-8 / en_US.UTF-8 / C / en_US.UTF-8 / en_US.UTF-8

time zone: Europe/Berlin
tzcode source: internal

Package version:
  evaluate_0.21   graphics_4.3.3  grDevices_4.3.3 highr_0.10      knitr_1.43      methods_4.3.3   stats_4.3.3     tools_4.3.3    
  utils_4.3.3     xfun_0.39       yaml_2.3.7  

Currently, I'm doing a fairly "hacky" solve to this by reading in the output from write_bib and then doing some find and replace with stringr.

knitr::write_bib(x = pkgs_to_cite, file = output_fn)

readLines(output_fn) %>% 
  str_replace_all("\\&", "\\\\&") %>%
  str_replace_all("\\\\\\\\&", "\\\\&") %>% 
    writeLines(output_fn)

Could knitr::write_bib please check for non-escpaped ampersands and do this tweak if necessary?

Thank you! Love the package!

[cderv]

Thanks for the report.

This is indeed an issue with natbib for example, but not with pandoc citeproc

---
title: "test"
output: 
  pdf_document:
    keep_tex: true
    citation_package: natbib
bibliography: packages.bib
---

```{r}
knitr::write_bib(x = "spam", "packages.bib")
```

@spam2017

leads to

! Misplaced alignment tab character &.
<argument> Computer &
                      Geoscience
l.11 \newblock \emph{Computer & Geoscience}
                                           , 104:\penalty0 109--119, 2017. 

while no error if the default is used (meaning Pandoc citeproc handles the citation processing for PDF)

For reasons beyond my understanding, some bibtex entries generated by citation() from Description files have non-escaped ampersands

Though, as you noticed, this comes directly from citation() converted by toBibTex() which does no escaping.

They have a escape = FALSE by default - see ?bibentry but it seems it covers only non-ASCII

escape : a logical indicating whether non-ASCII characters should be translated to LaTeX escape sequences.

It doesn't not help here with &

toBibtex(citation("spam"), espace = TRUE)
#> @Manual{,
#>   year = {2023},
#>   title = {{spam}: SPArse Matrix},
#>   author = {Reinhard Furrer and Roman Flury and Florian Gerber},
#>   note = {R package version 2.10-0},
#>   url = {https://CRAN.R-project.org/package=spam},
#> }
#> 
#> @Article{,
#>   title = {{spam}: A Sparse Matrix {R} Package with Emphasis on {MCMC} Methods for {G}aussian {M}arkov Random Fields},
#>   author = {Reinhard Furrer and Stephan R. Sain},
#>   journal = {Journal of Statistical Software},
#>   year = {2010},
#>   volume = {36},
#>   number = {10},
#>   pages = {1--25},
#>   doi = {10.18637/jss.v036.i10},
#> }
#> 
#> @Article{,
#>   title = {Pitfalls in the Implementation of {B}ayesian Hierarchical Modeling of Areal Count Data: An Illustration Using {BYM} and {L}eroux Models},
#>   author = {Florian Gerber and Reinhard Furrer},
#>   journal = {Journal of Statistical Software, Code Snippets},
#>   year = {2015},
#>   volume = {63},
#>   number = {1},
#>   pages = {1--32},
#>   doi = {10.18637/jss.v063.c01},
#> }
#> 
#> @Article{,
#>   title = {Extending {R} packages to support 64-bit compiled code: An illustration with spam64 and {GIMMS} {NDVI3g} data},
#>   author = {Florian Gerber and Kaspar Moesinger and Reinhard Furrer},
#>   journal = {Computer & Geoscience},
#>   year = {2017},
#>   volume = {104},
#>   pages = {109--119},
#>   issn = {0098-3004},
#>   doi = {10.1016/j.cageo.2016.11.015},
#> }

It seems to be an issue in R utils function after all. 🤔

@yihui what do you think ?

[yihui]

We can probably escape & when there isn't a backslash before it (using negative lookbehind (?<!) in Perl regex).

gsub('(?<!\\\\)&', '\\\\&', x, perl = TRUE)

We need to verify if pandoc-citeproc is also fine with \& in .bib. If that's the case, please feel free to submit a pull request. Thanks!

[HedvigS]

We can probably escape & when there isn't a backslash before it (using negative lookbehind (?<!) in Perl regex).

gsub('(?<!\\\\)&', '\\\\&', x, perl = TRUE)

We need to verify if pandoc-citeproc is also fine with \& in .bib. If that's the case, please feel free to submit a pull request. Thanks!

Ok! Is that an encouragement to me or @cderv to submit a PR?

[cderv]

Is that an encouragement to me or @cderv to submit a PR?

Either of us I would say :)

Feel free to contribute if you are interested. Do you want to give it a try ?

[HedvigS]

Is that an encouragement to me or @cderv to submit a PR?

Either of us I would say :)

Feel free to contribute if you are interested. Do you want to give it a try ?

I could... but.. I have a deadline for a paper on Sunday and 53 exams to grade and I haven't edited knitr source code before. I think you're probably going to do it better and faster ;)

[cderv]

I totally get the workload you're juggling right now – the paper deadline and all those exams to grade must be keeping you super busy! Don't worry; It was appreciated you were willing to. We'll do it. Thanks again for opening the issue

[HedvigS]

I totally get the workload you're juggling right now – the paper deadline and all those exams to grade must be keeping you super busy! Don't worry; It was appreciated you were willing to. We'll do it. Thanks again for opening the issue

Thanks!!