Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ONNX (v1.2.2) preset #547

Merged
merged 60 commits into from
Aug 4, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
79076dd
Initial commit for ONNX 1.0.1 JavaCPP Preset
EmergentOrder Feb 1, 2018
3e2e5d9
Remove default constructors, reinstate attributes declaration, fix En…
EmergentOrder Feb 16, 2018
37ec5fd
Fix get_all_schemas and EnforceConsumed and AllowConsumed
EmergentOrder Feb 16, 2018
60fc3ac
Fix FillUsing
EmergentOrder Feb 16, 2018
3ea1303
Fix access to TypeConstraintParam by restoring default constructor
EmergentOrder Mar 30, 2018
f071d63
Add LICENSE / Copyright and latest generated code
EmergentOrder Apr 3, 2018
ed8103f
Update pom.xml and generated code to JavaCPP 1.4.2-SNAPSHOT
EmergentOrder Apr 3, 2018
19df87c
Merge branch 'master' of https://github.com/bytedeco/javacpp-presets
EmergentOrder Apr 3, 2018
51da3a9
Add ONNX preset to .travis.yml, install miniconda in Travis, patch sc…
EmergentOrder Apr 7, 2018
8fb5ea9
Add onnx project to top-level pom.xml; patch the correct schema.h
EmergentOrder Apr 8, 2018
2a711c4
Add onnx module for linux-x86_64 to pom.xml
EmergentOrder Apr 9, 2018
9e99098
Remove use of sudo from cppbuild.sh, put onnx .so in the right place …
EmergentOrder Apr 10, 2018
90c7f17
Merge branch 'master' into master
EmergentOrder Apr 10, 2018
6eb8cb3
Explicitly use python3
EmergentOrder Apr 11, 2018
2c7829e
Patch ONNX build script to use python3 as well
EmergentOrder Apr 12, 2018
6657ac7
put miniconda on PATH in cppbuild.sh just to be sure
EmergentOrder Apr 12, 2018
d2e3ba6
Use python2 instead of python3, try installing miniconda from within …
EmergentOrder Apr 13, 2018
5e1e366
attempt to temporarily disable other Travis builds
EmergentOrder Apr 13, 2018
81fc397
temporarily disable Appveyor builds, install correct protobuf with pr…
EmergentOrder Apr 14, 2018
97277b0
Merge branch 'master' into master
EmergentOrder Apr 14, 2018
ea0a777
export CFLAGS so protobuf headers are found; remove onnx section from…
EmergentOrder Apr 15, 2018
96ff7af
Merge branch 'master' of https://github.com/EmergentOrder/javacpp-pre…
EmergentOrder Apr 15, 2018
f807074
Account for older GCC version in Travis; Switch to protobuf3 which ca…
EmergentOrder Apr 15, 2018
1b48fe3
fix path
EmergentOrder Apr 15, 2018
6c8b0de
bump protobuf version
EmergentOrder Apr 15, 2018
cdd3cf9
add setup.py patchfile
EmergentOrder Apr 15, 2018
76a91f4
another CFLAG
EmergentOrder Apr 15, 2018
5d90c05
try to update Travis's gcc
EmergentOrder Apr 15, 2018
858bb58
try to get a newer gcc
EmergentOrder Apr 15, 2018
be52a71
use centos 7
EmergentOrder Apr 15, 2018
f504ed5
fix
EmergentOrder Apr 15, 2018
f56889f
fix
EmergentOrder Apr 15, 2018
332c93b
revert to protobuf2
EmergentOrder Apr 15, 2018
b738c56
fix
EmergentOrder Apr 16, 2018
742745b
add autogen.sh.patch
EmergentOrder Apr 16, 2018
4fd1d76
link protobuf
EmergentOrder Apr 16, 2018
3dadeeb
link to protobuf from JavaCPP
EmergentOrder Apr 16, 2018
3f71ef9
fix
EmergentOrder Apr 16, 2018
7e7bc95
use LIBRARY_PATH instead of LD_LIBRARY_PATH
EmergentOrder Apr 16, 2018
b0ec9a1
clean up
EmergentOrder Apr 16, 2018
b2299fc
more cleanup
EmergentOrder Apr 16, 2018
9fad665
restore appveyor and travis builds; update generated code
EmergentOrder Apr 16, 2018
eb63f55
Install miniconda to instead of /home/laptx
EmergentOrder Apr 18, 2018
b1952cd
Merge branch 'master' into master
EmergentOrder Apr 18, 2018
c3c2a61
Remove unneeded installation of Miniconda
saudet Apr 30, 2018
765ad5e
Merge remote-tracking branch 'upstream/master'
saudet May 2, 2018
7c5d20a
Install Python 3.5 on Travis CI for CentOS 7
saudet May 2, 2018
7f76f73
Fix installation for Python 3.5 on CentOS 7
saudet May 3, 2018
3f58a76
Merge branch 'master' into master
saudet May 23, 2018
72b9f86
Merge branch 'master' into master
EmergentOrder Jun 18, 2018
f03580f
Use Protobuf 3.3.2
EmergentOrder Jun 18, 2018
b3d02fa
Merge branch 'master' of https://github.com/EmergentOrder/javacpp-pre…
EmergentOrder Jun 18, 2018
ef45c79
Bump ONNX version to v1.1.2
EmergentOrder Jun 19, 2018
5825cc8
Merge branch 'master' into master
EmergentOrder Jul 5, 2018
7689163
Bump ONNX version to 1.2.2
EmergentOrder Jul 6, 2018
6a05a63
Merge branch 'master' into master
saudet Aug 1, 2018
3198726
Fix, clean up, add README.md with example, and adjust license
saudet Aug 1, 2018
70caffc
Merge branch 'master' into master
saudet Aug 2, 2018
93402b0
Fix build script and presets
saudet Aug 3, 2018
862477e
Merge branch 'master' into master
EmergentOrder Aug 3, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -691,6 +691,10 @@ matrix:
env: PROJ=ale OS=linux-x86_64
install: true
script: ./ci/install-travis.sh
- os: linux
env: PROJ=onnx OS=linux-x86_64
install: true
script: ./ci/install-travis.sh
- os: linux
env: PROJ=liquidfun OS=linux-x86_64
install: true
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@

* Add presets for ONNX 1.2.2 ([pull #547](https://github.com/bytedeco/javacpp-presets/pull/547))
* Define `std::vector<tensorflow::OpDef>` type to `OpDefVector` for TensorFlow
* Link HDF5 with zlib on Windows also ([issue deeplearning4j/deeplearning4j#6017](https://github.com/deeplearning4j/deeplearning4j/issues/6017))
* Enable MKL-DNN for TensorFlow
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ Each child module in turn relies by default on the included [`cppbuild.sh` scrip
* TensorFlow 1.10.0-rc1 https://github.com/tensorflow/tensorflow
* TensorRT 4.0 https://developer.nvidia.com/tensorrt
* The Arcade Learning Environment 0.6.0 https://github.com/mgbellemare/Arcade-Learning-Environment
* ONNX 1.2.2 https://github.com/onnx/onnx
* LiquidFun http://google.github.io/liquidfun/
* Skia https://skia.org
* System APIs of the build environments:
Expand Down
2 changes: 1 addition & 1 deletion ci/install-travis.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ if [ "$TRAVIS_OS_NAME" == "osx" ]; then export JAVA_HOME=$(/usr/libexec/java_hom
if [[ "$OS" == "linux-x86" ]] || [[ "$OS" == "linux-x86_64" ]] || [[ "$OS" =~ android ]]; then
CENTOS_VERSION=6
SCL_ENABLE="devtoolset-6 rh-maven33 python27"
if [[ "librealsense mxnet tensorflow skia " =~ "$PROJ " ]] || [[ "$OS" =~ android ]]; then
if [[ "librealsense mxnet tensorflow onnx skia " =~ "$PROJ " ]] || [[ "$OS" =~ android ]]; then
CENTOS_VERSION=7
SCL_ENABLE="rh-maven33 rh-python35"
fi
Expand Down
2 changes: 1 addition & 1 deletion cppbuild.sh
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ function sedinplace {
}

if [[ -z ${PROJECTS:-} ]]; then
PROJECTS=(opencv ffmpeg flycapture libdc1394 libfreenect libfreenect2 librealsense videoinput artoolkitplus chilitags flandmark hdf5 mkl mkl-dnn openblas arpack-ng cminpack fftw gsl llvm libpostal leptonica tesseract caffe cuda mxnet tensorflow tensorrt ale liquidfun skia systems)
PROJECTS=(opencv ffmpeg flycapture libdc1394 libfreenect libfreenect2 librealsense videoinput artoolkitplus chilitags flandmark hdf5 mkl mkl-dnn openblas arpack-ng cminpack fftw gsl llvm libpostal leptonica tesseract caffe cuda mxnet tensorflow tensorrt ale onnx liquidfun skia systems)
fi

for PROJECT in ${PROJECTS[@]}; do
Expand Down
12 changes: 12 additions & 0 deletions onnx/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Open Neural Network Exchange

Copyright (c) Facebook, Inc. and Microsoft Corporation.
All rights reserved.

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ""Software""), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
70 changes: 70 additions & 0 deletions onnx/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
JavaCPP Presets for ONNX
========================

Introduction
------------
This directory contains the JavaCPP Presets module for:

* ONNX 1.2.2 https://onnx.ai/

Please refer to the parent README.md file for more detailed information about the JavaCPP Presets.


Documentation
-------------
Java API documentation is available here:

* http://bytedeco.org/javacpp-presets/onnx/apidocs/


Sample Usage
------------
Here is a simple example of ONNX ported to Java from this C++ source file and for this data:

* https://github.com/onnx/onnx/issues/418#issuecomment-357596638
* https://github.com/onnx/onnx/blob/master/onnx/examples/resources/single_relu.onnx

We can use [Maven 3](http://maven.apache.org/) to download and install automatically all the class files as well as the native binaries. To run this sample code, after creating the `pom.xml` and `src/main/java/LoadModel.java` source files below, simply execute on the command line:
```bash
$ mvn compile exec:java
```

### The `pom.xml` build file
```xml
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>org.bytedeco.javacpp-presets.onnx</groupId>
<artifactId>loadmodel</artifactId>
<version>1.4.3-SNAPSHOT</version>
<properties>
<exec.mainClass>LoadModel</exec.mainClass>
</properties>
<dependencies>
<dependency>
<groupId>org.bytedeco.javacpp-presets</groupId>
<artifactId>onnx-platform</artifactId>
<version>1.2.2-1.4.3-SNAPSHOT</version>
</dependency>
</dependencies>
</project>
```

### The `src/main/java/LoadModel.java` source file
```java
import java.nio.file.*;
import org.bytedeco.javacpp.*;
import static org.bytedeco.javacpp.onnx.*;

public class LoadModel {
public static void main(String[] args) throws Exception {
OpSchemaRegistry.get_all_schemas();

byte[] bytes = Files.readAllBytes(Paths.get("examples/resources/single_relu.onnx"));

ModelProto model = new ModelProto();
ParseProtoFromBytes(model, new BytePointer(bytes), bytes.length);

System.out.println(model.graph().input_size());
}
}
```
60 changes: 60 additions & 0 deletions onnx/cppbuild.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
#!/bin/bash
# This file is meant to be included by the parent cppbuild.sh script
if [[ -z "$PLATFORM" ]]; then
pushd ..
bash cppbuild.sh "$@" onnx
popd
exit
fi

if [[ $PLATFORM == windows* ]]; then
#No Windows support yet
echo "Error: Platform \"$PLATFORM\" is not supported"
exit 1
fi

export ONNX=1.2.2
export PROTO=3.5.1
export PYBIND=2.2.3

download https://github.com/onnx/onnx/archive/v$ONNX.tar.gz onnx-$ONNX.tar.gz
download https://github.com/google/protobuf/releases/download/v$PROTO/protobuf-cpp-$PROTO.tar.gz protobuf-$PROTO.tar.gz
download https://github.com/pybind/pybind11/archive/v$PYBIND.tar.gz pybind11-$PYBIND.tar.gz

mkdir -p "$PLATFORM"
cd "$PLATFORM"
INSTALL_PATH=`pwd`

echo "Decompressing archives..."
tar --totals -xf ../onnx-$ONNX.tar.gz
tar --totals -xf ../protobuf-$PROTO.tar.gz
tar --totals -xf ../pybind11-$PYBIND.tar.gz

export LIBRARY_PATH="$INSTALL_PATH/lib"
export PATH="$PATH:$INSTALL_PATH/bin"
export CFLAGS="-I$INSTALL_PATH/include"
export CXXFLAGS="-I$INSTALL_PATH/include"

cd protobuf-$PROTO
./configure "--prefix=$INSTALL_PATH" CFLAGS=-fPIC CXXFLAGS=-fPIC --disable-shared
make -j $MAKEJ V=0
make install

cd ../onnx-$ONNX
rm -df third_party/pybind11
ln -sf $INSTALL_PATH/pybind11-$PYBIND third_party/pybind11

#to build with "Traditional ML" support. Untested.
#export ONNX_ML=1
export CMAKE_BUILD_DIR=.setuptools-cmake-build/
export CMAKE_ARGS=-DBUILD_SHARED_LIBS=ON
python3 setup.py build

mkdir -p ../include/onnx ../include/onnx/common ../include/onnx/defs
cp onnx/*.h ../include/onnx/
cp onnx/common/*.h ../include/onnx/common/
cp onnx/defs/*.h ../include/onnx/defs/
cp $CMAKE_BUILD_DIR/onnx/*.h ../include/onnx/
cp $CMAKE_BUILD_DIR/*.so ../lib

cd ../..
56 changes: 56 additions & 0 deletions onnx/platform/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>org.bytedeco</groupId>
<artifactId>javacpp-presets</artifactId>
<version>1.4.3-SNAPSHOT</version>
<relativePath>../../</relativePath>
</parent>

<groupId>org.bytedeco.javacpp-presets</groupId>
<artifactId>onnx-platform</artifactId>
<version>1.2.2-${project.parent.version}</version>
<name>JavaCPP Presets Platform for ONNX</name>

<properties>
<javacpp.moduleId>onnx</javacpp.moduleId>
</properties>

<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>${javacpp.moduleId}</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>${javacpp.moduleId}</artifactId>
<version>${project.version}</version>
<classifier>${javacpp.platform.linux-x86_64}</classifier>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<executions>
<execution>
<id>default-jar</id>
<configuration>
<archive>
<manifestEntries>
<Class-Path>${javacpp.moduleId}.jar ${javacpp.moduleId}-linux-x86_64.jar</Class-Path>
</manifestEntries>
</archive>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

</project>
63 changes: 63 additions & 0 deletions onnx/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>org.bytedeco</groupId>
<artifactId>javacpp-presets</artifactId>
<version>1.4.3-SNAPSHOT</version>
</parent>

<groupId>org.bytedeco.javacpp-presets</groupId>
<artifactId>onnx</artifactId>
<version>1.2.2-1.4.3-SNAPSHOT</version>
<name>JavaCPP Presets for ONNX</name>

<dependencies>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacpp</artifactId>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
</configuration>
</plugin>
<plugin>
<artifactId>maven-resources-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.bytedeco</groupId>
<artifactId>javacpp</artifactId>
<configuration>
<compilerOptions>
<compilerOption>-std=c++11</compilerOption>
<compilerOption>-fpermissive</compilerOption>
</compilerOptions>
</configuration>
</plugin>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-source-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-javadoc-plugin</artifactId>
</plugin>
</plugins>
</build>

</project>
Loading