Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Branch][LLM Testing] Full Testing Harness for LLMs #1216

Merged
merged 20 commits into from
Sep 13, 2023
Merged

[Feature Branch][LLM Testing] Full Testing Harness for LLMs #1216

merged 20 commits into from
Sep 13, 2023

Commits on Aug 28, 2023

  1. initial commit

    @dbogunowicz
    dbogunowicz committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    b7133a0 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2023

  1. Merge branch 'main' of https://github.com/neuralmagic/deepsparse into… 

    … main
    @dbogunowicz
    dbogunowicz committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    6c9ab1d View commit details
    Browse the repository at this point in the history

  2. initial commit

    @dbogunowicz
    dbogunowicz committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    a47f977 View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2023

  1. Merge branch 'main' into feature/damian/llm_testing_feature_branch

    @dbogunowicz
    dbogunowicz committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    90382bd View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2023

  1. Merge branch 'main' into feature/damian/llm_testing_feature_branch

    @dbogunowicz
    dbogunowicz committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    37116cb View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. Merge branch 'main' into feature/damian/llm_testing_feature_branch

    @dbogunowicz
    dbogunowicz committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    87fe78b View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2023

  1. Merge branch 'main' into feature/damian/llm_testing_feature_branch

    @dbogunowicz
    dbogunowicz committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    afb2a2a View commit details
    Browse the repository at this point in the history

  2. [Feature Branch][LLM Testing] Create GroundTruthSource objects (#1219) 

    * initial commit

* finish creation of helper objects

* Update tests/conftest.py

* small refactor

* [Feature Branch][LLM Testing] LLM Testing Suite (#1227)

* Update README.md

* Update src/deepsparse/yolov8/README.md

* Update text_generation.py

* quality

* readability

* all tests passing

* added some full kv cache tests

* initial commit

* ready for review

* Delete tests/deepsparse/transformers/pipelines/proposal_text_generation_tests.md
    @dbogunowicz
    dbogunowicz committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    0477551 View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2023

  1. Merge branch 'main' into feature/damian/llm_testing_feature_branch

    @dbogunowicz
    dbogunowicz committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    a13d7a8 View commit details
    Browse the repository at this point in the history

  2. fix tests

    @dbogunowicz
    dbogunowicz committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    92163b9 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into feature/damian/llm_testing_feature_branch

    @dbogunowicz
    dbogunowicz committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    567337f View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2023

  1. Dipika's comments plus adjusting the script to renamed variables

    @dbogunowicz
    dbogunowicz committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    bfe1b62 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'feature/damian/llm_testing_feature_branch' of https://g… 

    …ithub.com/neuralmagic/deepsparse into feature/damian/llm_testing_feature_branch
    @dbogunowicz
    dbogunowicz committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    27b45e6 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2023

  1. remove ORT ground truth

    @dbogunowicz
    dbogunowicz committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    f2693ff View commit details
    Browse the repository at this point in the history

  2. add OPT tests

    @dbogunowicz
    dbogunowicz committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    dd270a2 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into feature/damian/llm_testing_feature_branch

    @dbogunowicz
    dbogunowicz committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    c22f5f2 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. Merge remote-tracking branch 'origin/main' into feature/damian/llm_te… 

    …sting_feature_branch
    @dbogunowicz
    dbogunowicz committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    a975402 View commit details
    Browse the repository at this point in the history

  2. rebase and disable tests in GHA

    @dbogunowicz
    dbogunowicz committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    646d6f5 View commit details
    Browse the repository at this point in the history

  3. quality

    @dbogunowicz
    dbogunowicz committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    17e63ee View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into feature/damian/llm_testing_feature_branch

    @dbogunowicz
    dbogunowicz committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    d1379b7 View commit details
    Browse the repository at this point in the history