Warn against devtools::load_all() in _targets.R

[wlandau]

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.
• Format your code according to the tidyverse style guide.

Proposal

devtools::load_all() etc. are problematic for targets pipelines. It is best to fully install the packages using install.packages() or equivalent and set the packages and possibly imports options as described at https://books.ropensci.org/targets/practices.html#loading-and-configuring-r-packages and https://books.ropensci.org/targets/practices.html#packages-based-invalidation.

Issues with devtools::load_all() and friends:

1. The loaded package will not be available on parallel workers created by tar_make_clustermq() or tar_make_future().
2. If the goal is to load your custom functions for the pipeline instead of source("R/functions.R"), then these functions will live in a special package environment where targets does not find them by default. Unless you also set the imports option, tar_visnetwork(targets_only = FALSE) will not show them, and tar_make() will not rerun the correct targets when you change your functions.

There is not much we can do here except education, but we can at least try to statically crawl through _targets.R and throw a warning if symbols like load_all are detected. This warning should be suppressed if Sys.getenv("TAR_WARN") is "false".

[mdneuzerling]

Hi @wlandau. I suspect the links in the warning need to be updated. https://books.ropensci.org/targets/practices.html#loading-and-configuring-r-packages leads to a 404. Perhaps https://books.ropensci.org/targets/packages.html#loading-and-configuring-r-packages is the intended URL?

I can put a PR together for this tomorrow if you'd like.

[wlandau]

Thanks @mdneuzerling, 3e138e2 updates the links.

[gorgitko]

Hi @wlandau, do you think this partially simulates devtools::load_all() and bypasses its shortcomings mentioned in this PR?

# _targets.R in package root
# First load packages ("import()") and individual objects ("importFrom()") from the NAMESPACE file
for (imp in parseNamespaceFile(".", ".")$imports) {
  if (length(imp) == 1) {
    library(imp, character.only = TRUE)
  } else {
    import::from(imp[[1]], imp[[2]], .character_only = TRUE)
  }
}

# And then objects from R/
for (file in fs::dir_ls("R/")) {
  source(file, local = .GlobalEnv)
}

[wlandau]

To load scripts from within _targets.R, the current recommendation is tar_source(). For interactive/exploratory work outside the pipeline, you can use tar_load_globals() to load all the packages and functions.