Is it possible to get the uncertainty values relative to the Oracle index?

[ColinS97]

Hi,

we are currently working on a University project evaluating DataAugmentation and Active Learning.

In our current setup we would need to retrieve Uncertainties relative to Oracle indices. Currently we only found a way to get the uncertainties relative to the pool as done in "active_loop.step()".
Is there a way to do the same for the Oracle indices?
If there is no such function we may need to implement it ourselves.

[Dref360]

The full dataset can be accessed throught active_dataset._datataset.

If you use ModelWrapper you can get the uncertainty for all items in the full dataset by doing:

wrapper: ModelWrapper = ... # your_model
al_ds: ActiveLearningDataset = ... # Your active dataset



predictions = wrapper.predict_on_dataset(al_ds._dataset, iterations=N_ITERATIONS, ...) 
uncertainty = BALD().get_uncertainties(predictions)

# `uncertainty` as shape (len(al_ds._dataset),)
# We can easily get the oracle indices sorted by uncertainty.
oracle_indices = uncertainty.argsort()

Let me know if this answer your question.

Also if you need to translate pool indices to oracle indices you can use al_dataset._pool_to_oracle_index.

[ColinS97]

Thanks a lot!
It took me some time to try it out but it seems to work for us as expected.