Have you ever complained about the weatherman failing to predict the weather correctly? That's what this project is about: the realiability of the weather forecasts. To do so we'll investigate the accuracy of the meteorological models.
- temperature
- pressure
- humidity
- wind speed
- wind direction
- London
- Madrid
- Sydney
- New York
- Sao Paulo
- Tokyo
- Moscow
- Asuncion
- Nairobi
- Lagos
Consists in assuming that the next value is the same as the one of the last period.
The tricky part is to identify what the last value is. For instance, if we measure the temperature every 3 hours and we want to predict the temperature today at 3pm, what is the last value: today's temperature at 12pm, yesterday's temperature at 3pm or maybe last year's temperature on the same day at 3pm?
Provided by OpenWeather API (https://openweathermap.org/technology)
Metrics are used to evaluate the accuracy of the models' predictions when compared to the actual values.
Mean absolute scaled error is a measure of the precision of a model compared to the naive forecast.
It is the mean absolute error of the forecast values, divided by the mean absolute error of the naive forecast.
Values greater than one indicate that the naive method performs better than the forecast values under consideration.
https://en.wikipedia.org/wiki/Mean_absolute_scaled_error
Average of the absolute value of the errors (the errors being the differences between predicted and real values)
Square root of the average of the square of the errors
It weighs outliers more heavily than MAE as a result of the squaring of each term, which effectively weighs large errors more heavily than small ones
Median of the absolute value of the errors.
It is robust to outliers
- Actual weather variables are measured (read from OpenWeather API) every 3 hours: 12am, 3am, 6am and so on.
- Every day, we get the forecast of those weather variables for each of the hours under consideration (12am, 3am, 6am...) over the next 5 days
- Metrics are calculated by comparing each actual value with the value forecasted 1 day before, 2 days before, etc.
This project comprises two Python 3.8 applications:
- data collection
- data analysis
Data collection consists of two different python modules (weather_spider.py
, forecast_spider
) that run as two separate lambda functions on AWS. Those modules share common functionality through common.py
The data collected is stored on an EFS (Elastic File System).
This data is obtained from https://openweathermap.org through the endpoints:
- current weather data
- 5 day forecast
Given that the 5 day forecast only include data every 3 hours (00:00, 03:00, 06:00, 09:00, 12:00, 15:00, 18:00, 21:00), those are the data points for which we get the current weather data too.
The data is recorded in 2 types of csv files:
weather_*.csv
forecast_*.csv
where *
represents each of the locations.
Dependencies corresponding to the production code of data collection must be kept separate in the file lambda_requirements.txt
. This file is used to generate the artifact to be deployed as a lambda function.
On the other hand, requirements.txt
has all the dependencies to run all modules and their corresponding tests.
Data analysis is carried out by the modules:
data_analysis.py
, to calculate metricsdata_explorer.py
, to explore the quality of the data collected (like missing datapoints)streamlit_app.py
, Streamlit dashboard to visualize data
Steps:
- join the records from
weather_*.csv
andforecast_*.csv
by the datetime fielddt
to match the actual measurement with each of the forecasts made over the 5 previous days; the result is stored in `join_*.csv`` - calculate the precision of the forecast according to the different metrics; the result is stored on
metrics_*.csv
The files weather_*.csv
and forecast_*.csv
need to be downloaded from the EFS
pytest --cov-branch --cov-report html --cov=roboclimate tests/
Coverage report is generated in the folder htmlcov
OPEN_WEATHER_API
Key to access OpenWeather API
ROBOCLIMATE_HOME
Path to the root folder of the project
ROBOCLIMATE_CSV_FILES_PATH
Path to the root folder containing the different csv files, e.g.
csv_files
├── forecast_london.csv
├── forecast_madrid.csv
├── humidity
│ ├── join_london.csv
│ ├── join_madrid.csv
│ ├── metrics_london.csv
│ ├── metrics_madrid.csv
├── pressure
│ ├── join_london.csv
│ ├── join_madrid.csv
│ ├── metrics_london.csv
│ ├── metrics_madrid.csv
├── temp
│ ├── join_london.csv
│ ├── join_madrid.csv
│ ├── metrics_london.csv
│ ├── metrics_madrid.csv
├── weather_london.csv
├── weather_madrid.csv
├── wind_deg
│ ├── join_london.csv
│ ├── join_madrid.csv
│ ├── metrics_london.csv
│ ├── metrics_madrid.csv
└── wind_speed
├── join_london.csv
├── join_madrid.csv
├── metrics_london.csv
├── metrics_madrid.csv
See deploy