LayerNorm using PyTorch #1069
Merged
LayerNorm using PyTorch #1069
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
frankfliu
reviewed
Jul 1, 2021
pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArrayEx.java
Show resolved
Hide resolved
tensorflow/tensorflow-engine/src/main/java/ai/djl/tensorflow/engine/TfNDArrayEx.java
Show resolved
Hide resolved
| @@ -144,6 +144,25 @@ JNIEXPORT jlong JNICALL Java_ai_djl_pytorch_jni_PyTorchLibrary_torchNNBatchNorm( | |||
| API_END_RETURN() | |||
| } | |||
|
|
|||
| JNIEXPORT jlong JNICALL Java_ai_djl_pytorch_jni_PyTorchLibrary_torchNNLayerNorm(JNIEnv* env, jobject jthis, | |||
There was a problem hiding this comment.
Please run:
./gradlew formatCpp
There was a problem hiding this comment.
I just did
gradlew formatCpp
Found C:\djl\\gradle\wrapper\gradle-wrapper.jar
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/7.0.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 1s
6 actionable tasks: 6 executed
but not seen an effect. Do I miss something?
There was a problem hiding this comment.
I see, formatCpp doesn't work on Windows
Here is what it should looks like:
JNIEXPORT jlong JNICALL Java_ai_djl_pytorch_jni_PyTorchLibrary_torchNNLayerNorm(
JNIEnv* env, jobject jthis, jlong jinput, jlongArray jnormalizedshape, jlong jweight, jlong jbias, jdouble jeps) {
There was a problem hiding this comment.
I copy/paste the two lines ... hope it fits ... looks like I should build djl on linux.
…Ex.java Co-authored-by: Frank Liu <frankfliu2000@gmail.com>
…DArrayEx.java Co-authored-by: Frank Liu <frankfliu2000@gmail.com>
…ngine/TfNDArrayEx.java Co-authored-by: Frank Liu <frankfliu2000@gmail.com>
Codecov Report
@@ Coverage Diff @@
## master #1069 +/- ##
============================================
+ Coverage 69.98% 70.02% +0.03%
- Complexity 5212 5227 +15
============================================
Files 510 511 +1
Lines 23255 23339 +84
Branches 2489 2492 +3
============================================
+ Hits 16276 16342 +66
- Misses 5650 5665 +15
- Partials 1329 1332 +3
Continue to review full report at Codecov.
|
frankfliu
reviewed
Jul 1, 2021
| @@ -144,6 +144,25 @@ JNIEXPORT jlong JNICALL Java_ai_djl_pytorch_jni_PyTorchLibrary_torchNNBatchNorm( | |||
| API_END_RETURN() | |||
| } | |||
|
|
|||
| JNIEXPORT jlong JNICALL Java_ai_djl_pytorch_jni_PyTorchLibrary_torchNNLayerNorm(JNIEnv* env, jobject jthis, | |||
There was a problem hiding this comment.
I see, formatCpp doesn't work on Windows
Here is what it should looks like:
JNIEXPORT jlong JNICALL Java_ai_djl_pytorch_jni_PyTorchLibrary_torchNNLayerNorm(
JNIEnv* env, jobject jthis, jlong jinput, jlongArray jnormalizedshape, jlong jweight, jlong jbias, jdouble jeps) {
| (PtNDArray) gamma, | ||
| (PtNDArray) beta, | ||
| eps)); | ||
| } |
There was a problem hiding this comment.
I have tried to fix it ... but could be that I am blind here ... I am using
gradlew formatJava
and
gradlew build
frankfliu
approved these changes
Jul 2, 2021
|
@lanking520 @stu1130 Please take a look. |
stu1130
approved these changes
Jul 5, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Similar to BatchNormalization there exist some other variants of normalizing data flowing through the network that have been implemented by the underlying ai frameworks. Here LayerNorm as one of them has been wired up to be used with PyTorch.
This PullRequest would close #1057.