[MASK] appears in the augmented sentences

[saeedEnte]

If I try augmenting the sentences of my dataset (using word augmenter with contextual embedding), for the case of action=substitute, the string [MASK] starts to appear for some augmentation tries. Do you know why is it the case?

[makcedward]

may you share a sample inputs?

[saeedEnte]

@makcedward I have given the following command:
aug = naw. ContextualWordEmbsAug(model_path='distilbert-base-uncased', top_k=5, top_p=0.5, aug_p=0.4, action='substitute')
generated_sents.append(aug.augment(input_sent, n=1))

[makcedward]

Since all potential candidates are filtered, the final result includes [MASK]. Will fix it by providing the original token.

However, it may return original texts if top_p is too low. Suggest increasing top_p to 0.9 or 0.8 and have a try.