Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Layer Normalization #2213

Merged
merged 13 commits into from
Oct 20, 2020
Merged

Add Layer Normalization #2213

merged 13 commits into from
Oct 20, 2020

Commits on Oct 11, 2020

  1. wip: layer normalization on cpu

    @arrufat
    arrufat committed Oct 11, 2020
    Configuration menu
    Copy the full SHA
    33e5969 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2020

  1. wip: add cuda implementation, nor working yet

    @arrufat
    arrufat committed Oct 12, 2020
    Configuration menu
    Copy the full SHA
    d135e19 View commit details
    Browse the repository at this point in the history

  2. wip: try to fix cuda implementation

    @arrufat
    arrufat committed Oct 12, 2020
    Configuration menu
    Copy the full SHA
    efb13b1 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2020

  1. swap grid_strid_range and grid_strid_range_y: does not work yet

    @arrufat
    arrufat committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    0bb3ceb View commit details
    Browse the repository at this point in the history

  2. fix CUDA implementation

    @arrufat
    arrufat committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    0d6276c View commit details
    Browse the repository at this point in the history

  3. implement cuda gradient

    @arrufat
    arrufat committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    f61b84e View commit details
    Browse the repository at this point in the history

  4. add documentation, move layer_norm, update bn_visitor

    @arrufat
    arrufat committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    a85779c View commit details
    Browse the repository at this point in the history

  5. add tests

    @arrufat
    arrufat committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    d28a9bd View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2020

  1. use stddev instead of variance in test (they are both 1, anyway)

    @arrufat
    arrufat committed Oct 14, 2020
    Configuration menu
    Copy the full SHA
    7896aa4 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2020

  1. add test for means and invstds on CPU and CUDA

    @arrufat
    arrufat committed Oct 18, 2020
    Configuration menu
    Copy the full SHA
    f0feb3d View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2020

  1. rename visitor to disable_duplicative_bias

    @arrufat
    arrufat committed Oct 19, 2020
    Configuration menu
    Copy the full SHA
    4a82abf View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2020

  1. handle more cases in the visitor_disable_input_bias

    @arrufat
    arrufat committed Oct 20, 2020
    Configuration menu
    Copy the full SHA
    09be870 View commit details
    Browse the repository at this point in the history

  2. Add tests for visitor_disable_input_bias

    @arrufat
    arrufat committed Oct 20, 2020
    Configuration menu
    Copy the full SHA
    c7f74b7 View commit details
    Browse the repository at this point in the history