Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.2.0 dev #83

Merged
merged 68 commits into from
Jun 20, 2024
Merged

0.2.0 dev #83

merged 68 commits into from
Jun 20, 2024

Conversation

sigmafelix
Copy link
Collaborator

@sigmafelix sigmafelix commented May 30, 2024

  • Significant changes
    • Due to bug fix of process_modis_swath, previous calculations are invalidated and should be rerun
    • Ecoregion functions were refactored
    • Dropped doParallel and foreach dependency: future and future.apply in place
    • NLCD functions refactoring: two modes (exactextractr and terra based functions; the latter is slower but consumes less memory) and multicore mode by argument nthreads
      • inst/extdata/nlcd_classes.csv fix: missing category is added (Perennial Ice/Snow)
    • AQS processing function supports three modes (unique sites, full space-time points, and sparse space-time points [returning rows with available measurement data])
      • Handling cases with multiple event types. Keeping "Included" only
  • Bug fix
    • Tests on Date class is removed or replaced with POSIXt
    • HMS: empty/missing polygon error due to date sequence generator setting (i.e., sub_hyphen=TRUE)
    • calc_terraclimate: promise before evaluation error due to introducing geom argument
    • process_modis_swath: empty subdataset processing, clarified documentation on setting subdataset argument
    • process_modis_warp: first-phase mosaicking is done with stars::st_mosaic
    • Most calc_* functions for yearly updated raw data: explicit integer assignment complying to calc_check_time logic
  • Refactoring
    • Refactoring Refactoring #75
    • Edited calculate_covariates tests
    • Edited calc_gmted for succinct field names
    • Edited process_gmted for the raw data compatibility
    • File name change of nei test data (...17.csv -> ...2017.csv)
    • Styling roxygen2 and elaborating documentation
    • check_url_status: HTTP 200 and 206 are accepted
    • download_koppen_geiger tests: GET (actual download) to HEAD by accepting 206 and tracing the file origin
    • Ecoregion functions refactoring
      • process_ecoregion will return a fixed polygon around Tukey's Bridge in Portland, ME (see below; the red dot represents an AQS PM2.5 site)
image

mitchellmanware and others added 30 commits April 25, 2024 12:42
- calc_nlcd: locs can be sf (actually, anything being able to be converted to vect)
- process_gmted: added documentation
- download_data: added "ecoregions"
- To read shp files only
… calc_merra2, calc_gridet, calc_terraclimate
- To make calc_tri, calc_nlcd, calc_sedac_population abide by the protocol discussed in #68
- Now process_aqs supports three options
  - "full": locations * time regardless of attributes are available
  - "sparse": location-time pairs with available attributes
  - "location" unique locations existing the full input data
- calc_worker gets match_arg() for time_type argument
- Reflowing calc_nlcd
- Significant changes
  - Due to bug fix of process_modis_swath, previous calculations are invalidated and should be rerun
  - Dropped doParallel and foreach dependency: future and future.apply in place
- Bug fix
  - Tests on Date class is removed or replaced with POSIXt
  - HMS: empty/missing polygon error due to date sequence generator setting (i.e., sub_hyphen=TRUE)
  - calc_terraclimate: promise before evaluation error due to introducing geom argument
  - process_modis_swath: empty subdataset processing, clarified documentation on setting subdataset argument
  - process_modis_warp: first-phase mosaicking is done with stars::st_mosaic
  - Most calc_* functions for yearly updated raw data: explicit integer assignment complying to calc_check_time logic
- Refactoring #75
- Etc.
  - Edited calculate_covariates tests
  - Edited calc_gmted for succinct field names
  - Edited process_gmted for the raw data compatibility
  - File name change of nei test data (...17.csv -> ...2017.csv)
  - Styling roxygen2 and elaborating documentation
- Revised swath function: all NaN layers get corner pixels to retain layers in multi-layer SpatRaster
- Refactored calc_modis_par: try statement, error case treatment
- calc_modis_daily: time field is assigned
- us_main in calc_nlcd is removed
- Logic for available date filtering by number of files is fixed
- Due to the deprecation of `directory_to_download` argument in download functions
- All tests passed locally
- writeLines to cat
- Koppen-Geiger refers to the direct file link
- check_url_status: added 206 as a valid value since Koppen-Geiger S3 link only sends 206 Partial Content
- AQS test: pick only one link
- GET to HEAD for http tests
- 🤷🏻‍♂️ local tests passed
- extract_urls: added trimws to remove leading blanks
- workflow yaml to cat all error messages
- setup-r-dependencies error
- test coverage error printing -- 2nd try
- pkgdown.yaml: test coverage html
- color code for coverage rate: green/orange to green/yellow
- pkgdown: coverage inclusion test (2nd try)
- nhdplusTools memoisation led to fail in GH Action: temporarily excluding from code coverage
- rollback calc_lagged tests
- yaml
  - setup-r: use-public-rspm
- calc_lagged: date alignment test logic fix
@sigmafelix sigmafelix mentioned this pull request Jun 5, 2024
sigmafelix and others added 7 commits June 12, 2024 09:35
- process_aqs: cases with multiple event types are handled
  - Documentation is revised to describe the event processing
- logic fix: from now `date` cannot be NULL. Records will be filtered with a date sequence generated from `date`.
- download_koppen_geiger: dropped overspecified file removal that cannot be tested
- download_remove_zips: merging unlink() into the function, removed bare unlink()s in functions
- download_aqs: vapply to sapply
- download_remove_zips tests
- download_remove_zips documentation edited
- Non-testable lines in download_cropscape get nocov
Copy link
Collaborator

@mitchellmanware mitchellmanware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks great, thanks Insang.

@sigmafelix sigmafelix merged commit 971ef31 into main Jun 20, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants