-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Whitebox cannot create "settings.json" on shiny server after upgrade to 2.1.1 #67
Comments
Hello, It seems that I broke this by passing parameters to the WhiteboxTools binary even when they are not specified by the user. The whitebox_tools program will write updates to the options even if they aren't different from the current setting/defaults. It is normal for a non-root user to not be able to write to a folder owned by root. You were relying on default options before, which means you could run tools without whitebox creating a settings.json file. Now essentially any tool run will trigger an update to settings. Sorry about that I believe in the short term you'll need to install WhiteboxTools to a directory that all users have ability to write to and set the executable path option. Consider Something like: whitebox::install_whitebox(pkg_dir = "/opt/whitebox") Then when you go to use tools as a non-root user call this at beginning of session: whitebox::wbt_init(exe_path = "/opt/whitebox/WBT/whitebox_tools") Let me know if this resolves your immediate problem, and I will work on how to prevent passing of the default arguments that trigger this. |
You may also need to |
Thanks for your reply. It helped me better understand the problem, though I was not able to solve it so far. First I tested it on my local machine:
This makes the executables available to the R-package. If I execute any I applied the same code shown above there (as root) and installed the whitebox executable into the shiny user home. In order to make it writeable to the shiny user, I also changed permissions by comparing it to my local installation:
Executing R code as shiny user in a R terminal creates the settings.json file within Thanks, |
Thanks for your detailed response Hendrick, and thank you for raising this issue. You have alerted me to several things that need to be fixed and I got started on it last night. I am very interested in making sure whitebox package works well for your use case. I believe if you set the working directory option e.g. There is not currently an option to have settings.json stored somewhere that is not the WBT install location or user-set working directory via whitebox_tools command line interface (that I am aware of). I have committed a few things to master branch on my fork where I corrected the issues that I could see from your initial post: remotes::install_github("brownag/whiteboxR") If you have a chance to try this one out it would be great to know if it resolves your issues. As long as you don't change the default options in theory the above should work. I will need to set up a test environment to try and better understand this shiny use case. Thanks for bringing it to my attention. |
I installed the github version of the R package and also specified the working directory.
Still when I launch the app and create a WBT function call, the |
This should fix the issue ensuring that additional flags related to settings.json are not added to the command string if the user has not explicitly set them. I also did some digging into the settings.json. If we look here we see that
For your apps Here is a minimal example. I was wrong when I said an update to settings.json is always triggered; with a change in one or more options (verbosity, working directory, raster compression or maximum number of processes) an update to settings.json is triggered. # run with reprex::reprex(std_out_err = TRUE)
library(whitebox)
wbt_verbose(TRUE)
if (file.exists("settings.json"))
file.remove("settings.json")
wbt_slope(sample_dem_data(), output = "test.tif")
#> slope - Elapsed Time (excluding I/O): 0.5s
file.exists("settings.json")
#> [1] FALSE
wbt_slope(sample_dem_data(), output = "test.tif", compress_rasters = FALSE)
#> slope - Elapsed Time (excluding I/O): 0.4s
cat(readLines("settings.json", warn = FALSE))
#> { "verbose_mode": true, "working_directory": "", "compress_rasters": false, "max_procs": -1 }
wbt_slope(sample_dem_data(), output = "test.tif", compress_rasters = TRUE)
#> slope - Elapsed Time (excluding I/O): 0.6s
cat(readLines("settings.json", warn = FALSE))
#> { "verbose_mode": true, "working_directory": "", "compress_rasters": true, "max_procs": -1 }
file.exists("settings.json")
#> [1] TRUE
Sys.chmod("settings.json", "444")
# this doesn't work, it requires writing to settings.json
wbt_slope(sample_dem_data(), output = "test.tif", compress_rasters = FALSE)
#> thread 'main' panicked at 'Error creating output settings.json file.: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', whitebox-common/src/configs/mod.rs:57:46
#> note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
#>
#> Error running WhiteboxTools (slope)
#> whitebox.exe_path: '/home/andrew/workspace/whitebox-tools/target/release/whitebox_tools'; File exists? TRUE
#> Arguments: --run=slope --dem='/home/andrew/R/x86_64-pc-linux-gnu-library/4.1/whitebox/extdata/DEM.tif' --output='test.tif' --units=degrees --compress_rasters=FALSE -v
#> System command had status 101
#> slope - Elapsed Time: NA [did not run]
# this works, compress_rasters=TRUE is already set
wbt_slope(sample_dem_data(), output = "test.tif", compress_rasters = TRUE)
#> slope - Elapsed Time (excluding I/O): 0.6s
# with https://github.com/brownag/whiteboxR/commit/4f72b173bb59bb8368279a05936f71167ec719a3 this works,
# requires no change to existing (unwritable) settings.json
wbt_slope(sample_dem_data(), output = "test.tif")
#> slope - Elapsed Time (excluding I/O): 0.4s
# change working directory
dir.create('test')
setwd('test')
# this now works; creates a new settings.json in test
wbt_slope(sample_dem_data(), output = "test.tif", compress_rasters = FALSE)
#> slope - Elapsed Time (excluding I/O): 0.4s If I understand correctly even on 2.1.0 you were not able to set the settings.json related flags on shiny server if your app directory/settings.json is not writable--unless you change your R working directory to somewhere you can write. I think it does not matter where whitebox_tools is installed to, what matters is the directory it is being called from. If you can use an R working directory other than the app directory which is writable by shiny user you would have ability to control the settings stored in the file as needed. |
Here is an app I set up to test #68 on shiny server. With those fixes basic running of tools should work with the default R working directory (/srv/shiny-server/ app folder) as long as the settings.json options are not set. If the user home folder is used (uncomment @hewag1975: I think this issue has been resolved but feel free to comment further / re-open the issue if needed. library(shiny)
library(whitebox)
library(terra)
ui <- fluidPage(
titlePanel(paste("WhiteboxTools-Shiny Test", getwd())),
sidebarLayout(
sidebarPanel(
sliderInput("threshold", "Threshold:", min = 1, max = 40, value = 8),
checkboxInput("compress", "Compress Rasters?")
),
mainPanel(
plotOutput("slopePlot")
)
)
)
server <- function(input, output) {
# setwd("~")
# wbt_wd("")
output$slopePlot <- renderPlot({
# generate threshold slope map based on input$threshold from ui.R
x <- wbt_slope(sample_dem_data(destfile = "~/input.tif"), "~/output.tif")
# , compress_rasters = input$compress)
terra::plot(terra::rast("~/output.tif") > input$threshold)
unlink("~/output.tif")
})
}
shinyApp(ui = ui, server = server) |
Thanks for fixing this. |
Hi,
we are using whitebox-tools through the R-client from within a shiny-application.
Installation on our shiny-server was performed as documented here by
install.packages("whitebox")
as root (package directory is"/usr/local/lib/R/site-library"
)whitebox::install_whitebox()
likewise as root which installs the WBT executable into the folder of the R-packageSo far it was possible to use the app as a standard shiny (=non-root) user.
Today when calling
wbt_slope
(or any other function) the app crashed with the following error message in the log:thread 'main' panicked at 'Error creating output settings.json file.: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', whitebox-common/src/configs/mod.rs:57:46 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
Error running WhiteboxTools (slope) whitebox.exe_path: '/usr/local/lib/R/site-library/whitebox/WBT/whitebox_tools'; File exists? TRUE Arguments: --run=slope --dem='' --output='/home/shiny/slope.tif' --units=degrees --compress_rasters=FALSE -v --max_procs=-1
To make sure that this is not related to the application itself I executed the code below on the same server in an R session as root user and in another session as non-root user (typically shiny users do not have root privileges) outside the application.
In root mode the code calculates slope as expected and produces the output file.
In user mode I get the same error as when running the application.
The NEWS of the R-client states for version 2.1.1:
Updated how --compress_rasters parameter is passed via command line. Now the flag is added to all commands regardless of whether the value is TRUE or FALSE. This allows update of settings.json accordingly when --compress_rasters=FALSE. RE: https://github.com/jblindsay/whitebox-tools/issues/233#issuecomment-1065955783
Downgrading the R-package to version
2.1.0
resolves the issue.Can this be changed to also allow non-root users to call the functions?
Best,
Hendrik
The text was updated successfully, but these errors were encountered: