PR: Refactoring of the tool to search for climate stations in the ECCC database. #82
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Now that searching for station is very quick, the QTableWidget cannot keep up to refresh the display when there is a lot of stations. We will have to implement a custom QTableView with a custom QAbstractTableModel instead that will be more efficient. |
Used the function defined in utils module instead
This is so it is importable more easily from other modules.
Reworked the GUI so it is more inline with how the search is performing now. It is more filter oriented now than research oriented.
This will allow to keep track of index ordering in column 0 instead.
Since WeatherStationDisplayTable was not used anymore in search_weather_data, it was moved to dwnld_weather_data where it is still used.
Since everything is now so quick, there is no neeed for a search button anymore and the station list can be updated live at runtime.
|
And here is a demo of the result. No need to click on a search button anymore. Everything is updated automatically.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #71
As suggested by @xamal2, instead of searching for weather station directly on the ECCC website (http://climate.weather.gc.ca/historical_data/search_historic_data_e.html), we now use instead the information contained within a csv that is published by ECCC on its ftp Tor server (ftp://client_climate@ftp.tor.ec.gc.ca/Pub/Get_More_Data_Plus_de_donnees/).
The data contained in the csv is read, formatted and saved in binary format on the disk. This make searching for weather station almost instantaneous. It is not required to do this in another thread anymore, nor to use a progress bar.