546 masking time step segmentation #562
Conversation
dianna/utils/maskers.py
Outdated
|
|
||
|
|
||
| def generate_channel_masks(input_data: np.ndarray, number_of_masks: int, p_keep: float): |
There was a problem hiding this comment.
Reminder: merge this function with generate_time_step_masks to reduce code duplication, action point from PR #554
This may have introduced an error as the RISE time series tests no longer pass.
…ep-segmentation' into 546-masking-time-step-segmentation
cwmeijer
added this to In progress
in SS Sprint 3 - improve code quality, adding tabular funcitonality
introduce failing test
add tests refactor: rename parameter
The error made the rise_for_timeseries tests fail as it couldn't find the correct hot/cold days in the synthetic tests with this bug.
to have a more accurate expected masked step count in the statistical sense of the word expected (mean). This also means that the test for this is no longer valid. Also removes a duplicate parametric test.
…on' into 546-masking-time-step-segmentation
# Conflicts: # tests/methods/test_rise_timeseries.py
![github-advanced-security[bot]](https://github.com/avatars/in/57789?s=60&v=4){kind=small}
| ceil = np.ceil(mean) | ||
| if floor != ceil: | ||
| user_requested_steps = int( | ||
| np.random.choice([floor, ceil], 1, p=[ceil - mean, mean - floor])) |
Check notice
Code scanning / SonarCloud
numpy.random.Generator should be preferred to numpy.random.RandomState Low
|
|
||
| Returns: | ||
| The generated masks (np.ndarray) | ||
| """ | ||
| cell_size = np.ceil(np.array(input_size) / feature_res) | ||
| up_size = (feature_res + 1) * cell_size | ||
| grid = np.random.choice(a=(True, False), |
Check notice
Code scanning / SonarCloud
numpy.random.Generator should be preferred to numpy.random.RandomState Low
| up_size = (number_of_features + 1) * cell_size | ||
| masks = np.empty((number_of_masks, *input_size), dtype=np.float32) | ||
| for i in range(masks.shape[0]): | ||
| y_offset = np.random.randint(0, cell_size[0]) |
Check notice
Code scanning / SonarCloud
numpy.random.Generator should be preferred to numpy.random.RandomState Low
| masks = np.empty((number_of_masks, *input_size), dtype=np.float32) | ||
| for i in range(masks.shape[0]): | ||
| y_offset = np.random.randint(0, cell_size[0]) | ||
| x_offset = np.random.randint(0, cell_size[1]) |
Check notice
Code scanning / SonarCloud
numpy.random.Generator should be preferred to numpy.random.RandomState Low
| Returns: | ||
| The generated masks (np.ndarray) | ||
| """ | ||
| grid = np.random.random(size=(number_of_masks, number_of_features, |
Check notice
Code scanning / SonarCloud
numpy.random.Generator should be preferred to numpy.random.RandomState Low
Some examples of univariate masks of length 18 with 6 features: |
|
|
||
| masks = np.empty(masks_shape, dtype=np.float32) | ||
| for i_mask in range(masks.shape[0]): | ||
| offset = np.random.random() |
Check notice
Code scanning / SonarCloud
numpy.random.Generator should be preferred to numpy.random.RandomState Low
|
For anyone reviewing this PR. Please have a look first at my blog explaining this approach on a high level: https://medium.com/@cwmeijer/how-to-correctly-mask-time-series-data-to-use-with-xai-90247ac252b4 |
| from dianna.utils.maskers import generate_channel_masks | ||
| from dianna.utils.maskers import generate_masks | ||
| from dianna.utils.maskers import generate_time_step_masks |
There was a problem hiding this comment.
can we also test the function _generate_interpolated_float_masks_for_image?
There was a problem hiding this comment.
@cwmeijer thank you for implementing this. I read the blog and checked the implementation. The code can benefit from more documentation, but I saw that this will be addressed in a different issue later. I left some minor suggestions/comments.
Co-authored-by: SarahAlidoost <55081872+SarahAlidoost@users.noreply.github.com>
Work in progress: Fixes #546.
This should now result in nice masks with clustered time steps.
time masks
(0=masked, 1=unmasked)
1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1
1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1
1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1
1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1
1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1
1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1
^ channels/ time -->
(0=masked, 1=unmasked)
0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
^ channels/ time -->
(0=masked, 1=unmasked)
1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0
1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0
1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0
1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0
1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0
1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0
^ channels/ time -->