Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding SFT training #14

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Adding SFT training #14

wants to merge 10 commits into from

Commits on Jul 27, 2024

  1. First prototype, let's jump padding free

    @TJ-Solergibert
    TJ-Solergibert committed Jul 27, 2024
    Configuration menu
    Copy the full SHA
    419b33a View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. This mess produces sames generations as hf

    @TJ-Solergibert
    TJ-Solergibert committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    03f4308 View commit details
    Browse the repository at this point in the history

  2. Added SFT generations check script

    @TJ-Solergibert
    TJ-Solergibert committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    c57533d View commit details
    Browse the repository at this point in the history

  3. Added masked LOSS check

    @TJ-Solergibert
    TJ-Solergibert committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    a66b0c6 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. Getting ready

    @TJ-Solergibert
    TJ-Solergibert committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    06af8cf View commit details
    Browse the repository at this point in the history

  2. RCP Working

    @TJ-Solergibert
    TJ-Solergibert committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    a8f979d View commit details
    Browse the repository at this point in the history

  3. Added todi scripts

    @TJ-Solergibert
    TJ-Solergibert committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    c026422 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. Added SFT docs

    @TJ-Solergibert
    TJ-Solergibert committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    38f3815 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'sft' of https://github.com/tj-solergibert/nanotron into… 

    … sft
    @TJ-Solergibert
    TJ-Solergibert committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    2d882db View commit details
    Browse the repository at this point in the history

  3. Dont predict EOText token

    @TJ-Solergibert
    TJ-Solergibert committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    d5228bb View commit details
    Browse the repository at this point in the history