560 improve results of lime timeseries notebooks

[geek-yang]

We try to fine-tune the parameters of lime timeseries and improve the results in the notebooks (lime_timeseries_coffee.ipynb and lime_timeseries_weather.ipynb).

Note that because of the absence of strategic segmentation and multi-channels masking, the results and visualization are not perfect. But they got much improved than before (at least good enough to show during the surf event.)

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[stefsmeets]

Nice work! I noticed that you use explain_timeseries in the notebooks. But when using method='lime' or method='rise' gives different output. The interface is not consistant. Is this something that can be fixed in this PR or should it be a new issue?

Lime gives back an explanation instance:

>>> lime_exp = dianna.explain_timeseries(
>>>     run_expert_model, 
>>>     timeseries_data=data_extreme,
>>>     method='lime', 
>>>     labels=[0,1], 
>>>     class_names=["summer", "winter"],
>>>     num_features=len(data_extreme),
>>>     num_samples=10000,
>>>     num_slices=len(data_extreme), 
>>>     distance_method='euclidean',
>>>     mask_type=input_train_mean)
>>> rise_exp
<lime.explanation.Explanation at 0x7f8dbf2dfe20>

Rise gives back a numpy array:

>>> rise_exp = dianna.explain_timeseries(
>>>     run_expert_model, 
>>>     timeseries_data=data_extreme,
>>>     method='rise', 
>>>     labels=[0,1], 
>>>     p_keep=0.1,
>>>     n_masks=10000, 
>>>     mask_type=input_train_mean)
>>> rise_exp
array([[[7.700e-02],
        [3.000e-03],
        ...
        [4.000e-03]],

       [[9.650e-01],
        [9.820e-01],
        ...
       [1.053e+00]]])
>>> rise_exp.shape
(2, 28, 1)

[geek-yang]

Nice work! I noticed that you use explain_timeseries in the notebooks. But when using method='lime' or method='rise' gives different output. The interface is not consistant. Is this something that can be fixed in this PR or should it be a new issue?

Hi @stefsmeets , thanks a lot for your review and quick feedback 😄. Let me take a quick look into the interface, that's a bit unexpected. I will try to fix it in this PR, if possible.

Lime gives back an explanation instance:

>>> lime_exp = dianna.explain_timeseries(
>>>     run_expert_model, 
>>>     timeseries_data=data_extreme,
>>>     method='lime', 
>>>     labels=[0,1], 
>>>     class_names=["summer", "winter"],
>>>     num_features=len(data_extreme),
>>>     num_samples=10000,
>>>     num_slices=len(data_extreme), 
>>>     distance_method='euclidean',
>>>     mask_type=input_train_mean)
>>> rise_exp
<lime.explanation.Explanation at 0x7f8dbf2dfe20>

Rise gives back a numpy array:

>>> rise_exp = dianna.explain_timeseries(
>>>     run_expert_model, 
>>>     timeseries_data=data_extreme,
>>>     method='rise', 
>>>     labels=[0,1], 
>>>     p_keep=0.1,
>>>     n_masks=10000, 
>>>     mask_type=input_train_mean)
>>> rise_exp
array([[[7.700e-02],
        [3.000e-03],
        ...
        [4.000e-03]],

       [[9.650e-01],
        [9.820e-01],
        ...
       [1.053e+00]]])
>>> rise_exp.shape
(2, 28, 1)

This is because we use the lime_base function from the original implementation of lime to compute the scores. And it returns an explainer object, which contains LIME scores in explainer.local_exp. It is nice that you raise this point. I think we can simply only return explainer.local_exp, and convert it to a numpy array as well, to be consistent with rise. Let me fix this then.

[geek-yang]

@stefsmeets I just updated the returned values of LIME timeseries explainer and it is an array, which is consistent with RISE.

I also updated the notebook (lime_timeseries_weather.ipynb) to use explain_timeseries interface. The results from LIME timeseries are different from RISE, which is expected (not due to the difference in interface, but for several other reasons, e.g. the algorithms are different, also due to the absence of segmentation strategy #546).

Just take another look and let me know if you have more comments, thanks @stefsmeets !

[stefsmeets]

Thanks, Looks good to me! 🚀