Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add code eval #587

Merged
merged 29 commits into from
Sep 26, 2023
Merged

Add code eval #587

merged 29 commits into from
Sep 26, 2023

Commits on Sep 8, 2023

  1. add code eval

    @samhavens
    samhavens committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    a57dde8 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2023

  1. Create coding_tasks.yaml

    @samhavens
    samhavens authored Sep 12, 2023
    Configuration menu
    Copy the full SHA
    a63c4aa View commit details
    Browse the repository at this point in the history

  2. Update coding_tasks.yaml

    @samhavens
    samhavens authored Sep 12, 2023
    Configuration menu
    Copy the full SHA
    71de0bf View commit details
    Browse the repository at this point in the history

  3. Update tasks.yaml 

    add three shot to coding tasks
    @samhavens
    samhavens authored Sep 12, 2023
    Configuration menu
    Copy the full SHA
    85b727f View commit details
    Browse the repository at this point in the history

  4. Update eval_gauntlet.yaml 

    make 0 and 3 shot for gauntlet, add programming_lite
    @samhavens
    samhavens authored Sep 12, 2023
    Configuration menu
    Copy the full SHA
    b0670e8 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. update readme add code eval

    @samhavens
    samhavens committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    1e868d8 View commit details
    Browse the repository at this point in the history

  2. torch nograd eval_model

    @samhavens
    samhavens committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    c6e54ae View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2023

  1. integrate seed

    @mcarbin
    mcarbin committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    a2c877b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'mosaicml:sam/add-coding-eval' into sam/add-coding-eval

    @mcarbin
    mcarbin authored Sep 14, 2023
    Configuration menu
    Copy the full SHA
    071ddde View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2023

  1. pass_at_k threading

    @mcarbin
    mcarbin committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    12c0175 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2023

  1. remove extraneous print

    @mcarbin
    mcarbin committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    2755e8d View commit details
    Browse the repository at this point in the history

  2. update num_beams to default of 20

    @mcarbin
    mcarbin committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    06ae4fb View commit details
    Browse the repository at this point in the history

  3. update num_beamss for coding eval to 20 as the standard for pass@1

    @mcarbin
    mcarbin committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    bfbbf17 View commit details
    Browse the repository at this point in the history

  4. remove C from default coding tasks because it's too small

    @mcarbin
    mcarbin committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    e042eb7 View commit details
    Browse the repository at this point in the history

  5. update num_beams in full task list to default of 20

    @mcarbin
    mcarbin committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    1fa60ec View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2023

  1. Merge branch 'main' into sam/add-coding-eval

    @mcarbin
    mcarbin committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    9e241f5 View commit details
    Browse the repository at this point in the history

  2. fix break from merge and precommit

    @mcarbin
    mcarbin committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    4b5f2d9 View commit details
    Browse the repository at this point in the history

  3. yamllint

    @mcarbin
    mcarbin committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    c3819de View commit details
    Browse the repository at this point in the history

  4. remove code finetuning

    @mcarbin
    mcarbin committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    cb45e4b View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into sam/add-coding-eval

    @mcarbin
    mcarbin committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    e3d7dda View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2023

  1. Merge branch 'main' into sam/add-coding-eval

    @dakinggg
    dakinggg authored Sep 26, 2023
    Configuration menu
    Copy the full SHA
    0acec3d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into sam/add-coding-eval

    @dakinggg
    dakinggg authored Sep 26, 2023
    Configuration menu
    Copy the full SHA
    79d1d8c View commit details
    Browse the repository at this point in the history

  3. bump version for CI

    @mcarbin
    mcarbin committed Sep 26, 2023
    Configuration menu
    Copy the full SHA
    628e020 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'refs/remotes/mosaicml/sam/add-coding-ev… 

    …al' into sam/add-coding-eval
    @mcarbin
    mcarbin committed Sep 26, 2023
    Configuration menu
    Copy the full SHA
    797c690 View commit details
    Browse the repository at this point in the history

  5. update gauntlet

    @mcarbin
    mcarbin committed Sep 26, 2023
    Configuration menu
    Copy the full SHA
    0860f91 View commit details
    Browse the repository at this point in the history

  6. update tasks

    @mcarbin
    mcarbin committed Sep 26, 2023
    Configuration menu
    Copy the full SHA
    fbb7342 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into sam/add-coding-eval

    @mcarbin
    mcarbin authored Sep 26, 2023
    Configuration menu
    Copy the full SHA
    b10beaf View commit details
    Browse the repository at this point in the history

  8. remove torch.no_grad

    @mcarbin
    mcarbin committed Sep 26, 2023
    Configuration menu
    Copy the full SHA
    63470b4 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'main' into sam/add-coding-eval

    @dakinggg
    dakinggg authored Sep 26, 2023
    Configuration menu
    Copy the full SHA
    707a2e7 View commit details
    Browse the repository at this point in the history