Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mask cache Performance Optimization for vllm #939

Merged
merged 5 commits into from
Jun 16, 2024
Merged

Mask cache Performance Optimization for vllm #939

merged 5 commits into from
Jun 16, 2024

Commits on Jun 14, 2024

  1. Add vllm mask cache

    pgrundmann committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    d676bb9 View commit details
    Browse the repository at this point in the history

  2. Fix broken commits

    @paul-grundmann
    paul-grundmann committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    1385324 View commit details
    Browse the repository at this point in the history

  3. Fix pre-commit checks, added dict type annotation to mask_cache

    @paul-grundmann
    paul-grundmann committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    c866632 View commit details
    Browse the repository at this point in the history

  4. Remove benchmark

    @paul-grundmann
    paul-grundmann committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    49f485b View commit details
    Browse the repository at this point in the history

  5. Fix move to scores device

    @paul-grundmann
    paul-grundmann committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    1a7079b View commit details
    Browse the repository at this point in the history