Code for our paper about time series anomaly detection evaluation protocols published at TPCTC 2023 (Multivariate Time Series Anomaly Detection: Fancy Algorithms and Flawed Evaluation Methodology).
Please download the SWaT, WADI and PSM datasets and place original files in the
corresponding directory under notebooks. For WADI, run the provided script
(prepare_WADI.sh) to remove comments from data and shorten columns names.
- For SWaT and WADI: https://itrust.sutd.edu.sg/itrust-labs_datasets/dataset_info/
- For PSM: https://github.com/eBay/RANSynCoders
To reproduce the results, first install the conda environment (conda env create -f environment.yml).
We recommend that you start with the SWaT dataset as the notebook contains more comments.
| Dataset | F1 (point-wise) |
F1_c (composite) |
F1_ew (event-wise) |
|---|---|---|---|
| SWaT | 0.810 | 0.596 | 0.555 |
| WADI | 0.374 | 0.655 | 0.608 |
| PSM | 0.538 | 0.484 | 0.20 |
As mentioned in the notebooks, better F1_c scores can be achieved by disabling
score smoothing or by using a smaller smoothing window.