You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The default reference in IG is a zero scalar corresponding to each input tensor (effectively PAD for BERT). It can be customized by setting the 'baselines' parameter when calling the attribute function. For example (setting UNK as reference, assuming seq_len are the number of tokens in your input):
# Custom token for IGfromtransformersimportAutoTokenizerfromcaptum.attrimportTokenReferenceBasetokenizer=AutoTokenizer.from_pretrained('all-MiniLM-L6-v2') # Load your model's tokenizerref_token_id=tokenizer.unk_token_id# Choose the id of your desired token, you can call tokenizer.all_special_tokens for a list of all special tokens supported by your modeltoken_reference=TokenReferenceBase(reference_token_idx=ref_token_id) # Use Captum to generate a reference based on the number of tokens in your inputdevice=torch.device("cuda:0"iftorch.cuda.is_available() else"cpu")
ref=token_reference.generate_reference(seq_len,device=device).unsqueeze(0)
For sentence classification task using BERT, is the PAD token used in IG/Deeplift? or Unkown token? or it can be customized?
The text was updated successfully, but these errors were encountered: