Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PaliGemma #30814

Merged
merged 120 commits into from
May 14, 2024
Merged

Add PaliGemma #30814

merged 120 commits into from
May 14, 2024

Commits on Mar 4, 2024

  1. add new model like

    molbap committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    2fddcc9 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #9 from huggingface/update

    Update
    molbap committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    e536f6a View commit details
    Browse the repository at this point in the history

Commits on Mar 5, 2024

  1. Configuration menu
    Copy the full SHA
    6ca0bf7 View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2024

  1. Configuration menu
    Copy the full SHA
    21db4a7 View commit details
    Browse the repository at this point in the history
  2. fix

    molbap committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    7985fc4 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2024

  1. update

    molbap committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    bcb341d View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2024

  1. reorder loading/unpacking

    molbap committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    38ad70e View commit details
    Browse the repository at this point in the history

Commits on Mar 11, 2024

  1. clean up

    molbap committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    929746a View commit details
    Browse the repository at this point in the history

Commits on Mar 12, 2024

  1. add debug statements

    molbap committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    ae55ad9 View commit details
    Browse the repository at this point in the history
  2. change device

    molbap committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    9d5f8fb View commit details
    Browse the repository at this point in the history
  3. Merge branch 'add_palma' of github.com:huggingface/new-model-addition…

    …-palma into add_palma
    molbap committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    524a073 View commit details
    Browse the repository at this point in the history
  4. fix

    molbap committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    d171c4e View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2024

  1. debugging

    molbap committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    1638a73 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2024

  1. fix noncausal mask

    molbap committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    4aad850 View commit details
    Browse the repository at this point in the history
  2. fixup sdpa + causal mask

    molbap committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    2972674 View commit details
    Browse the repository at this point in the history
  3. fix activation function

    molbap committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    71fa912 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    94e4806 View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. add variants

    molbap committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    7b6f0b3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4e8e1c6 View commit details
    Browse the repository at this point in the history
  3. revert to non-debug sdpa

    molbap committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    ba8fb4e View commit details
    Browse the repository at this point in the history
  4. revert gemma modifications

    molbap committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    6c2348d View commit details
    Browse the repository at this point in the history
  5. add custom language modeling

    molbap committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    906e87f View commit details
    Browse the repository at this point in the history
  6. use Processor

    molbap committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    f26361d View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    b3e4a03 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    500a360 View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2024

  1. Update

    molbap committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    96a82e2 View commit details
    Browse the repository at this point in the history

Commits on Mar 19, 2024

  1. update mask

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    347df2c View commit details
    Browse the repository at this point in the history
  2. breakpoints galore

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    d01b502 View commit details
    Browse the repository at this point in the history
  3. remove conflict

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    bb8030c View commit details
    Browse the repository at this point in the history
  4. switch to left-padding

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    a6056c6 View commit details
    Browse the repository at this point in the history
  5. add incomplete model doc

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    dacd297 View commit details
    Browse the repository at this point in the history
  6. add paligemma global files

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    f133a1d View commit details
    Browse the repository at this point in the history
  7. batch rename paligemma

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    37c1368 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    9935e2e View commit details
    Browse the repository at this point in the history
  9. style

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    e7fd049 View commit details
    Browse the repository at this point in the history
  10. style

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    edcf1b1 View commit details
    Browse the repository at this point in the history
  11. remove copied from + doc

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    2a29a2e View commit details
    Browse the repository at this point in the history
  12. remove more copied from

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    5607c9d View commit details
    Browse the repository at this point in the history
  13. remove copy from projector

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    8ce77f3 View commit details
    Browse the repository at this point in the history
  14. minor fix

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    3563926 View commit details
    Browse the repository at this point in the history
  15. update config and style

    molbap committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    f48f61d View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. add readme - dummy

    molbap committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    b84d1c1 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. CORRECT image captioning

    molbap committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    deb35ba View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2024

  1. moving to args

    molbap committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    9275d53 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2024

  1. Configuration menu
    Copy the full SHA
    5487734 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7f9c479 View commit details
    Browse the repository at this point in the history
  3. remove breakpoint

    molbap committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    438b143 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. leverage AutoModel

    molbap committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    00d2922 View commit details
    Browse the repository at this point in the history
  2. fix input_ids slicing

    molbap committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    72697b6 View commit details
    Browse the repository at this point in the history
  3. make siglip head conditional

    molbap committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    17b30fd View commit details
    Browse the repository at this point in the history
  4. remove encoder_decoder value

    molbap committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    26cb46e View commit details
    Browse the repository at this point in the history
  5. remove unneeded modeling file

    molbap committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    f49f389 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    d1964ea View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2024

  1. FIXED generation with 4D mask

    molbap committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    6da4d2e View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2024

  1. Configuration menu
    Copy the full SHA
    ef8c0fb View commit details
    Browse the repository at this point in the history

Commits on Mar 31, 2024

  1. Update src/transformers/models/siglip/modeling_siglip.py

    Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
    molbap and ArthurZucker committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    9f261d7 View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2024

  1. fix left padding detection

    molbap committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    11c3488 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6d527e4 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    8b679b1 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2024

  1. Configuration menu
    Copy the full SHA
    fdba83a View commit details
    Browse the repository at this point in the history
  2. fix

    molbap committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    eee3c1e View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. Configuration menu
    Copy the full SHA
    e4869dd View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b973365 View commit details
    Browse the repository at this point in the history
  3. handle merging in processor

    molbap committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    8cab59c View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    9186b79 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. Configuration menu
    Copy the full SHA
    78398a1 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4ede86e View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. Configuration menu
    Copy the full SHA
    c899136 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. Configuration menu
    Copy the full SHA
    a912273 View commit details
    Browse the repository at this point in the history
  2. pass image processing args

    molbap committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    2c8a508 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    7201d43 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. add \n token separately

    molbap committed May 3, 2024
    Configuration menu
    Copy the full SHA
    c74f3a3 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2024

  1. Configuration menu
    Copy the full SHA
    23b12a3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4e4a957 View commit details
    Browse the repository at this point in the history
  3. fix tokenizer and prompts

    molbap committed May 9, 2024
    Configuration menu
    Copy the full SHA
    1f465c0 View commit details
    Browse the repository at this point in the history
  4. fix docstrings

    molbap committed May 9, 2024
    Configuration menu
    Copy the full SHA
    38a0401 View commit details
    Browse the repository at this point in the history
  5. change to camel

    molbap committed May 9, 2024
    Configuration menu
    Copy the full SHA
    16baabc View commit details
    Browse the repository at this point in the history
  6. fix casing

    molbap committed May 9, 2024
    Configuration menu
    Copy the full SHA
    9119db4 View commit details
    Browse the repository at this point in the history
  7. debug pos_ids and sdpa

    molbap committed May 9, 2024
    Configuration menu
    Copy the full SHA
    267a0da View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    3e32118 View commit details
    Browse the repository at this point in the history
  9. pass and use cache_position

    molbap committed May 9, 2024
    Configuration menu
    Copy the full SHA
    4ed0594 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. Configuration menu
    Copy the full SHA
    46c5c0b View commit details
    Browse the repository at this point in the history
  2. Update src/transformers/models/paligemma/processing_paligemma.py

    Co-authored-by: Merve Noyan <merveenoyan@gmail.com>
    molbap and merveenoyan committed May 10, 2024
    Configuration menu
    Copy the full SHA
    5b47a5b View commit details
    Browse the repository at this point in the history
  3. simplify conversion script

    molbap committed May 10, 2024
    Configuration menu
    Copy the full SHA
    b80c591 View commit details
    Browse the repository at this point in the history
  4. add copied from

    molbap committed May 10, 2024
    Configuration menu
    Copy the full SHA
    7d21409 View commit details
    Browse the repository at this point in the history

Commits on May 12, 2024

  1. Configuration menu
    Copy the full SHA
    b1eda0e View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. Update src/transformers/models/paligemma/modeling_paligemma.py

    Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
    molbap and pcuenca committed May 13, 2024
    Configuration menu
    Copy the full SHA
    404abd8 View commit details
    Browse the repository at this point in the history
  2. clean up

    molbap committed May 13, 2024
    Configuration menu
    Copy the full SHA
    a7e2446 View commit details
    Browse the repository at this point in the history
  3. Merge branch 'add_palma' of github.com:huggingface/new-model-addition…

    …-palma into add_palma
    molbap committed May 13, 2024
    Configuration menu
    Copy the full SHA
    2f6b17d View commit details
    Browse the repository at this point in the history
  4. Shift attention mask from 1:

    After discussion with @molbap
    pcuenca committed May 13, 2024
    Configuration menu
    Copy the full SHA
    60ad9c5 View commit details
    Browse the repository at this point in the history
  5. Merge pull request #6 from pcuenca/add_palma_shift_mask

    Shift mask from `1:`
    molbap committed May 13, 2024
    Configuration menu
    Copy the full SHA
    daccbe7 View commit details
    Browse the repository at this point in the history
  6. add docs, fix quality

    molbap committed May 13, 2024
    Configuration menu
    Copy the full SHA
    91c7aab View commit details
    Browse the repository at this point in the history
  7. Merge branch 'add_palma' of github.com:huggingface/new-model-addition…

    …-palma into add_palma
    molbap committed May 13, 2024
    Configuration menu
    Copy the full SHA
    372d566 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    63464ec View commit details
    Browse the repository at this point in the history
  9. fix more tests

    molbap committed May 13, 2024
    Configuration menu
    Copy the full SHA
    55187a1 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    5ea6c9f View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    ddb7ac7 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    7f32718 View commit details
    Browse the repository at this point in the history
  13. skip incompatible tests

    molbap committed May 13, 2024
    Configuration menu
    Copy the full SHA
    ac9fd9a View commit details
    Browse the repository at this point in the history
  14. quality

    molbap committed May 13, 2024
    Configuration menu
    Copy the full SHA
    cceb3d0 View commit details
    Browse the repository at this point in the history
  15. [broken archive maps]

    molbap committed May 13, 2024
    Configuration menu
    Copy the full SHA
    a264824 View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. Apply suggestions

    - remove archive lists
    - style
    - take shape of inputs_embeds for batch
    
    Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
    molbap and ArthurZucker committed May 14, 2024
    Configuration menu
    Copy the full SHA
    9310873 View commit details
    Browse the repository at this point in the history
  2. Update src/transformers/utils/dummy_pt_objects.py

    Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
    molbap and ArthurZucker committed May 14, 2024
    Configuration menu
    Copy the full SHA
    0711b12 View commit details
    Browse the repository at this point in the history
  3. simplify conversion script

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    e7ec216 View commit details
    Browse the repository at this point in the history
  4. add suggestions

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    8b0724d View commit details
    Browse the repository at this point in the history
  5. add suggestions

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    7bcea3e View commit details
    Browse the repository at this point in the history
  6. add copied from

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    498bbde View commit details
    Browse the repository at this point in the history
  7. fix

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    a8bd223 View commit details
    Browse the repository at this point in the history
  8. move labels out

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    04d962f View commit details
    Browse the repository at this point in the history
  9. revert

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    e7caa8a View commit details
    Browse the repository at this point in the history
  10. fix

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    ac5ed67 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    72f6fdc View commit details
    Browse the repository at this point in the history
  12. use cache_position

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    c824771 View commit details
    Browse the repository at this point in the history
  13. fix quality + docstrings

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    7a8e62e View commit details
    Browse the repository at this point in the history
  14. fix quality

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    4913c07 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    0c8f2c9 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    54fd284 View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    99c3ac5 View commit details
    Browse the repository at this point in the history
  18. fix config docstring

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    75c36c2 View commit details
    Browse the repository at this point in the history
  19. fix query and attn_mask dtype

    molbap committed May 14, 2024
    Configuration menu
    Copy the full SHA
    9b49838 View commit details
    Browse the repository at this point in the history