From 269c6c2392031d6deb1500c39b09eb4b0059a341 Mon Sep 17 00:00:00 2001 From: --global Date: Thu, 23 Nov 2023 23:27:01 -0500 Subject: [PATCH] update manual code example --- docs/src/development_resources/shell.md | 35 ++++++++++++++++++++++--- 1 file changed, 31 insertions(+), 4 deletions(-) diff --git a/docs/src/development_resources/shell.md b/docs/src/development_resources/shell.md index 82e7ecb5..56815457 100644 --- a/docs/src/development_resources/shell.md +++ b/docs/src/development_resources/shell.md @@ -3,7 +3,7 @@ The shell is used to access app functions and data, such as Hobolink and USGS data and access to the database. -The reason why the shell is useful is because there may be cases where you want to play around with the app's functions. For example, maybe you see something that seems fishy in the data, so you want to have direct access to the function the website is running. You may also want to +The reason why the shell is useful is because there may be cases where you want to play around with the app's functions. For example, maybe you see something that seems fishy in the data, so you want to have direct access to the function the website is running. You may also want to The way Flask works makes it impossible to run the website's functions outside the Flask app context, which means importing the functions into a naked shell doesn't work as intended. The `flask shell` provides all the tools needed to let coders access the functions the exact same way the website does, except in a shell environment. @@ -49,16 +49,16 @@ And you should be good to go! The functions listed below should be available for - **`process_data`** (*(pd.DataFrame, pd.DataFrame) -> pd.DataFrame*): Combines the Hobolink and USGS tables. - **`compose_tweet`** (*() -> str*): - Generates a message for Twitter that represents the current status of the flagging program (note: this function does not actually send the Tweet to Twitter.com). + Generates a message for Twitter that represents the current status of the flagging program (note: this function does not actually send the Tweet to Twitter.com). Additionally, Pandas and Numpy are already pre-imported via `#!python import pandas as pd` and `#!python import numpy as np`. ???+ tip To add more functions and variables that pre-load in the Flask shell, simply add another entry to the dictionary returned by the function `#!python make_shell_context()` in `app/app.py:creat_app()`. - + ???+ tip All of the website's functions can be run in the Flask shell, even those that are not pre-loaded in the shell's global context. All you need to do is import it. For example, let's say you want to get the un-parsed request object from USGS.gov. You can import the function we use and run it like this: - + ```python # (in Flask shell) from app.data.usgs import request_to_usgs @@ -86,3 +86,30 @@ Let's say you want to preview a Tweet that would be sent out without actually se # (in Flask shell) print(compose_tweet()) ``` + +## Example 3: Manual load + +I had to run this once because we missed the 90 day cutoff. This uses a different API endpoint, but it returns the same data and seems to work fine. + +```python +import requests +from app.data.processing.hobolink import get_live_hobolink_data +from app.data.processing.predictive_models import v1 +from app.data.processing.usgs import parse_usgs_data + +begin_date = "2023-01-02" +end_date = "2023-11-09" + +# Usgs request +res = requests.get("https://nwis.waterdata.usgs.gov/usa/nwis/uv/", params={"cb_00060": "on", "cb_00065": "on", "format": "rdb", "site_no": "01104500", "legacy": "1", "period": "", "begin_date": begin_date, "end_date": end_date}) + +df_hobolink = get_live_hobolink_data("code_for_boston_export_180d") +df_usgs = parse_usgs_data(res) +df_combined = v1.process_data(df_hobolink=df_hobolink, df_usgs=df_usgs) +df_predictions = v1.all_models(df_combined) + +df_hobolink.to_csv(f"{end_date}-hobolink.csv") +df_usgs.to_csv(f"{end_date}-usgs.csv") +df_combined.to_csv(f"{end_date}-combined.csv") +df_predictions.to_csv(f"{end_date}-predictions.csv") +```