Tutorial for Conformal Prediction #597
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
Also thanks for fixing the issue related to level and prediction intervals🤗 |
| @@ -0,0 +1,14030 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,14030 @@ | |||
| { | |||
There was a problem hiding this comment.
these are uncalibrated, meaning that they tend to ignore seasonality.
I think we need a more accurate description of what an uncalibrated interval is. Here's a suggestion:
meaning that the actual frequency of observations falling within the interval does not align with the confidence level associated with it. For example, a calibrated 95% prediction interval should contain the true value 95% of the time in repeated sampling. An uncalibrated 95% prediction interval, on the other hand, might contain the true value only 80% of the time, or perhaps 99% of the time. In the first case, the interval is too narrow and underestimates the uncertainty, while in the second case, it is too wide and overestimates the uncertainty.
Next
Statistical methods also assume normality. Here, we talk about another method called conformal prediction that doesn't require any distributional assumptions. More information...
Reply via ReviewNB
| @@ -0,0 +1,14030 @@ | |||
| { | |||
There was a problem hiding this comment.
In the current version there's a problem displaying the plots.
Unable to display output for mime type(s): application/vnd.plotly.v1+json
Reply via ReviewNB
| @@ -0,0 +1,14030 @@ | |||
| { | |||
There was a problem hiding this comment.
Maybe we can move this to the next section? something like "we'll plot the different intervals using matplotlib for one unique_id. To obtain the ids in the dataset use.. ". Otherwise it might look a little random.
Reply via ReviewNB
There was a problem hiding this comment.
Oh this was me finding a good ID to plot. Will just delete.
| @@ -0,0 +1,14030 @@ | |||
| { | |||
There was a problem hiding this comment.
Before the references, we should include a future work and an acknowledgements section. Here are some ideas:
Future work
Conformal prediction has become a powerful framework for uncertainty quantification, providing well-calibrated prediction intervals without making any distributional assumptions. Its use has surged in both academia and industry over the past few years. We'll continue working on it, and future tutorials may include:
- Exploring larger datasets
- Incorporating industry-specific examples
- Investigating specialized methods like the jackknife+ that are closely related to conformal prediction (for details on the jackknife+ see [here](https://valeman.medium.com/jackknife-a-swiss-knife-of-conformal-prediction-for-regression-ce3b56432f4f)).
If you're interested in any of these, or in any other related topic, please let us know by opening an issue on [GitHub](https://github.com/Nixtla/statsforecast/issues)
Acknowledgements
We would like to thank [Kevin Kho](https://github.com/kvnkho) for writing this tutorial, and Valeriy [Manokhin](https://github.com/valeman) for his expertise on conformal prediction, as well as for promoting this work.
Reply via ReviewNB
No description provided.