Add a condition for nested_detach

[haikuoxin]

System Info

When use Trainer to train a VLM like 'internlm/internlm-xcomposer2-vl-7b' with evaluation_strategy, an AttributeError error will be triggered like below:

trainer.train()
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer.py", line 1553, in train
    return inner_training_loop(
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer.py", line 1927, in _inner_training_loop
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer.py", line 2254, in _maybe_log_save_evaluate
    metrics = self.evaluate(ignore_keys=ignore_keys_for_eval)
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer.py", line 2968, in evaluate
    output = eval_loop(
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer.py", line 3157, in evaluation_loop
    loss, logits, labels = self.prediction_step(model, inputs, prediction_loss_only, ignore_keys=ignore_keys)
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer.py", line 3347, in prediction_step
    labels = nested_detach(tuple(inputs.get(name) for name in self.label_names))
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer_pt_utils.py", line 166, in nested_detach
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer_pt_utils.py", line 166, in <genexpr>
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer_pt_utils.py", line 168, in nested_detach
    return type(tensors)({k: nested_detach(t) for k, t in tensors.items()})
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer_pt_utils.py", line 168, in <dictcomp>
    return type(tensors)({k: nested_detach(t) for k, t in tensors.items()})
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer_pt_utils.py", line 166, in nested_detach
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer_pt_utils.py", line 166, in <genexpr>
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/root/anaconda3/envs/sharegpt4v/lib/python3.10/site-packages/transformers/trainer_pt_utils.py", line 169, in nested_detach
    return tensors.detach()
AttributeError: 'str' object has no attribute 'detach'

Who can help?

@muellerzr

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

1. load the model and tokenizer with from_pretrained method
2. set training_args
3. build train_dataset and eval_dataset, the dataset return Dict[str, str]
4. create a trainer with Trainer API then trainer.train()

Expected behavior

Model should be train normally