Problem with getting predictions #287
Comments
|
This clearly is a prompt mistake, can you share your training config settings!! |
|
@balajiChundi here: resume_from_checkpoint_path: null
result_path: "./result"
pretrained_model_name_or_path: "naver-clova-ix/donut-base-finetuned-cord-v2"
# pretrained_model_name_or_path: "naver-clova-ix/donut-base"
dataset_name_or_paths: ["donut/data"]
sort_json_key: False
train_batch_sizes: [2]
val_batch_sizes: [4]
input_size: [1280, 960] # when the input resolution differs from the pre-training setting, some weights will be newly initialized (but the model training would be okay)
max_length: 768
align_long_axis: False
num_nodes: 2
seed: 2022
lr: 3e-5
warmup_steps: 300 # 800/8*30/10, 10%
num_training_samples_per_epoch: 400
max_epochs: 100
max_steps: -1
num_workers: 0 # it was 8, but changed due to error, experiment with number until it works and time is good
val_check_interval: 1.0
check_val_every_n_epoch: 3
gradient_clip_val: 1.0
verbose: Truewith proper prompt same output: from donut import DonutModel
import torch
my_model = DonutModel.from_pretrained("donut/result/train_opl/20240208_003824")
my_model.encoder.eval()
my_model.inference(image=image, prompt="<s_data>")
{'predictions': [{'text_sequence': ' <s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s>'}]} |
|
Exactly the same problem here, with the same base model. |
|
me too, and Ido not find any possible solution |
|
Hello, I also encountered the same problem earlier, but after adjustments, the current effect is not bad. I hope it can help you. First, I reconfigured the environment according to https://colab.research.google.com/drive/1NMSqoIZ_l39wyRD7yVjw2FIuU2aglzJi?usp=sharing and then used the pre-trained model donut-base-official https://huggingface.co/naver-clova-ix/donut-base/tree/official. I think the main issue lies in the versions of dependency packages and the pre-trained model. Here is my conda list for your reference |
|
Sorry to hear this, But I can't find any problem here. Can you guys check your dependencies as @2021xyl suggested. |
|
Had the the same issue with transformers version 4.38.2. Downgrading to the version 4.25.1 solved the issue! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, I am trying to fine-tune pretrained model:
naver-clova-ix/donut-base-finetuned-cord-v2with my custom dataset of invoices in jpeg format. I tried to usenaver-clova-ix/donut-basefor fine-tuning, and different parameters. When I try to get prediction one of documents I always get:My example metadata.jsonl line like this:
I am training model on macbook pro m1.
What am I doing wrong?
The text was updated successfully, but these errors were encountered: