Skip to content

Commit

Permalink
* Add presets for HDF5
Browse files Browse the repository at this point in the history
  • Loading branch information
saudet committed Oct 15, 2016
1 parent 8fc317e commit 5d4c975
Show file tree
Hide file tree
Showing 11 changed files with 13,915 additions and 2 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
* Fix `libdc1394` not properly linking with `libusb-1.0` on Mac OS X ([issue bytedeco/javacv#501](https://github.com/bytedeco/javacv/issues/501))
* Add presets for the `bioinspired` module of OpenCV 3.1 ([pull #282](https://github.com/bytedeco/javacpp-presets/pull/282))
* Include `tensorflow/core/graph/dot.h` header file from TensorFlow ([pull #272](https://github.com/bytedeco/javacpp-presets/pull/272))
* Add presets for OpenBLAS/MKL ([issue #112](https://github.com/bytedeco/javacpp-presets/issues/112))
* Add presets for HDF5 and OpenBLAS/MKL ([issue #112](https://github.com/bytedeco/javacpp-presets/issues/112))
* Make Caffe work on CPU-only machines ([issue #219](https://github.com/bytedeco/javacpp-presets/issues/219))
* Fix loading issue with `opencv_face` ([issue bytedeco/javacv#470](https://github.com/bytedeco/javacv/issues/470))
* Fix presets for CUDA on the `linux-ppc64le` platform
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ Each child module in turn relies on its corresponding native libraries being alr
* ARToolKitPlus 2.3.1 https://launchpad.net/artoolkitplus
* Chilitags https://github.com/chili-epfl/chilitags
* flandmark 1.07 http://cmp.felk.cvut.cz/~uricamic/flandmark/#download
* HDF5 1.10.0 https://support.hdfgroup.org/HDF5/
* OpenBLAS 0.2.18 http://www.openblas.net/
* FFTW 3.3.4 http://www.fftw.org/download.html
* GSL 2.1 http://www.gnu.org/software/gsl/#downloading
Expand Down
2 changes: 1 addition & 1 deletion cppbuild.sh
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ function download {
}

if [[ -z ${PROJECTS:-} ]]; then
PROJECTS=(opencv ffmpeg flycapture libdc1394 libfreenect videoinput artoolkitplus chilitags flandmark openblas fftw gsl llvm leptonica tesseract caffe cuda mxnet tensorflow)
PROJECTS=(opencv ffmpeg flycapture libdc1394 libfreenect videoinput artoolkitplus chilitags flandmark hdf5 openblas fftw gsl llvm leptonica tesseract caffe cuda mxnet tensorflow)
fi

for PROJECT in ${PROJECTS[@]}; do
Expand Down
98 changes: 98 additions & 0 deletions hdf5/COPYING
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@

Copyright Notice and License Terms for
HDF5 (Hierarchical Data Format 5) Software Library and Utilities
-----------------------------------------------------------------------------

HDF5 (Hierarchical Data Format 5) Software Library and Utilities
Copyright 2006-2016 by The HDF Group.

NCSA HDF5 (Hierarchical Data Format 5) Software Library and Utilities
Copyright 1998-2006 by the Board of Trustees of the University of Illinois.

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted for any purpose (including commercial purposes)
provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice,
this list of conditions, and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions, and the following disclaimer in the documentation
and/or materials provided with the distribution.

3. In addition, redistributions of modified forms of the source or binary
code must carry prominent notices stating that the original code was
changed and the date of the change.

4. All publications or advertising materials mentioning features or use of
this software are asked, but not required, to acknowledge that it was
developed by The HDF Group and by the National Center for Supercomputing
Applications at the University of Illinois at Urbana-Champaign and
credit the contributors.

5. Neither the name of The HDF Group, the name of the University, nor the
name of any Contributor may be used to endorse or promote products derived
from this software without specific prior written permission from
The HDF Group, the University, or the Contributor, respectively.

DISCLAIMER:
THIS SOFTWARE IS PROVIDED BY THE HDF GROUP AND THE CONTRIBUTORS
"AS IS" WITH NO WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED. In no
event shall The HDF Group or the Contributors be liable for any damages
suffered by the users arising out of the use of this software, even if
advised of the possibility of such damage.

-----------------------------------------------------------------------------
-----------------------------------------------------------------------------

Contributors: National Center for Supercomputing Applications (NCSA) at
the University of Illinois, Fortner Software, Unidata Program Center (netCDF),
The Independent JPEG Group (JPEG), Jean-loup Gailly and Mark Adler (gzip),
and Digital Equipment Corporation (DEC).

-----------------------------------------------------------------------------

Portions of HDF5 were developed with support from the Lawrence Berkeley
National Laboratory (LBNL) and the United States Department of Energy
under Prime Contract No. DE-AC02-05CH11231.

-----------------------------------------------------------------------------

Portions of HDF5 were developed with support from the University of
California, Lawrence Livermore National Laboratory (UC LLNL).
The following statement applies to those portions of the product and must
be retained in any redistribution of source code, binaries, documentation,
and/or accompanying materials:

This work was partially produced at the University of California,
Lawrence Livermore National Laboratory (UC LLNL) under contract
no. W-7405-ENG-48 (Contract 48) between the U.S. Department of Energy
(DOE) and The Regents of the University of California (University)
for the operation of UC LLNL.

DISCLAIMER:
This work was prepared as an account of work sponsored by an agency of
the United States Government. Neither the United States Government nor
the University of California nor any of their employees, makes any
warranty, express or implied, or assumes any liability or responsibility
for the accuracy, completeness, or usefulness of any information,
apparatus, product, or process disclosed, or represents that its use
would not infringe privately- owned rights. Reference herein to any
specific commercial products, process, or service by trade name,
trademark, manufacturer, or otherwise, does not necessarily constitute
or imply its endorsement, recommendation, or favoring by the United
States Government or the University of California. The views and
opinions of authors expressed herein do not necessarily state or reflect
those of the United States Government or the University of California,
and shall not be used for advertising or product endorsement purposes.

-----------------------------------------------------------------------------

HDF5 is available with the SZIP compression library but SZIP is not part
of HDF5 and has separate copyright and license terms. See “Szip Compression
in HDF Products” (www.hdfgroup.org/doc_resource/SZIP/) for further details.

-----------------------------------------------------------------------------

194 changes: 194 additions & 0 deletions hdf5/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,194 @@
JavaCPP Presets for HDF5
========================

Introduction
------------
This directory contains the JavaCPP Presets module for:

* HDF5 1.10.0-patch1 https://support.hdfgroup.org/HDF5/

Please refer to the parent README.md file for more detailed information about the JavaCPP Presets.


Documentation
-------------
Java API documentation is available here:

* http://bytedeco.org/javacpp-presets/hdf5/apidocs/


Sample Usage
------------
Here is a simple example of HDF5 ported to Java from this C++ source file:

* https://support.hdfgroup.org/ftp/HDF5/current/src/unpacked/c++/examples/h5tutr_cmprss.cpp

We can use [Maven 3](http://maven.apache.org/) to download and install automatically all the class files as well as the native binaries. To run this sample code, after creating the `pom.xml` and `src/main/java/H5tutrCmprss.java` source files below, simply execute on the command line:
```bash
$ mvn compile exec:java
```

### The `pom.xml` build file
```xml
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>org.bytedeco.javacpp-presets.hdf5</groupId>
<artifactId>H5tutrCmprss</artifactId>
<version>1.2.5-SNAPSHOT</version>
<properties>
<exec.mainClass>H5tutrCmprss</exec.mainClass>
</properties>
<dependencies>
<dependency>
<groupId>org.bytedeco.javacpp-presets</groupId>
<artifactId>hdf5</artifactId>
<version>1.10.0-patch1-1.2.5-SNAPSHOT</version>
</dependency>
</dependencies>
</project>
```

### The `src/main/java/H5tutrCmprss.java` source file
```java
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *
* terms governing use, modification, and redistribution, is contained in *
* the files COPYING and Copyright.html. COPYING can be found at the root *
* of the source code distribution tree; Copyright.html can be found at the *
* root level of an installed copy of the electronic HDF5 document set and *
* is linked from the top-level documents page. It can also be found at *
* https://support.hdfgroup.org/HDF5/doc/Copyright.html. If you do not have *
* access to either file, you may request a copy from help@hdfgroup.org. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */

/*
* This example illustrates how to create a compressed dataset.
* It is used in the HDF5 Tutorial.
*/

import java.io.*;
import org.bytedeco.javacpp.*;
import static org.bytedeco.javacpp.hdf5.*;

public class H5tutrCmprss {
static final String FILE_NAME = "h5tutr_cmprss.h5";
static final String DATASET_NAME = "Compressed_Data";
static final int DIM0 = 100;
static final int DIM1 = 20;

public static void main(String[] args) {
long[] dims = { DIM0, DIM1 }; // dataset dimensions
long[] chunk_dims = { 20, 20 }; // chunk dimensions
int[] buf = new int[DIM0 * DIM1];

// Try block to detect exceptions raised by any of the calls inside it
try {
// Turn off the auto-printing when failure occurs so that we can
// handle the errors appropriately
org.bytedeco.javacpp.hdf5.Exception.dontPrint();

// Create a new file using the default property lists.
H5File file = new H5File(FILE_NAME, H5F_ACC_TRUNC);

// Create the data space for the dataset.
DataSpace dataspace = new DataSpace(2, dims);

// Modify dataset creation property to enable chunking
DSetCreatPropList plist = new DSetCreatPropList();
plist.setChunk(2, chunk_dims);

// Set ZLIB (DEFLATE) Compression using level 6.
// To use SZIP compression comment out this line.
plist.setDeflate(6);

// Uncomment these lines to set SZIP Compression
// unsigned szip_options_mask = H5_SZIP_NN_OPTION_MASK;
// unsigned szip_pixels_per_block = 16;
// plist->setSzip(szip_options_mask, szip_pixels_per_block);

// Create the dataset.
DataSet dataset = new DataSet(file.asCommonFG().createDataSet(DATASET_NAME,
new DataType(PredType.STD_I32BE()), dataspace, plist));

for (int i = 0; i < DIM0; i++)
for (int j = 0; j < DIM1; j++)
buf[i * DIM1 + j] = i + j;

// Write data to dataset.
dataset.write(new IntPointer(buf), new DataType(PredType.NATIVE_INT()));

// Close objects and file. Either approach will close the HDF5 item.
dataspace.close();
dataset.close();
plist.close();
file.close();

// -----------------------------------------------
// Re-open the file and dataset, retrieve filter
// information for dataset and read the data back.
// -----------------------------------------------

int[] rbuf = new int[DIM0 * DIM1];
int numfilt;
long nelmts = 1, namelen = 1;
int[] flags = new int[1], filter_info = new int[1], cd_values = new int[1];
byte[] name = new byte[1];
int filter_type;

// Open the file and the dataset in the file.
file = new H5File();
file.openFile(FILE_NAME, H5F_ACC_RDONLY);
dataset = new DataSet(file.asCommonFG().openDataSet(DATASET_NAME));

// Get the create property list of the dataset.
plist = new DSetCreatPropList(dataset.getCreatePlist());

// Get the number of filters associated with the dataset.
numfilt = plist.getNfilters();
System.out.println("Number of filters associated with dataset: " + numfilt);

for (int idx = 0; idx < numfilt; idx++) {
nelmts = 0;

filter_type = plist.getFilter(idx, flags, new SizeTPointer(1).put(nelmts), cd_values, namelen, name, filter_info);

System.out.print("Filter Type: ");

switch (filter_type) {
case H5Z_FILTER_DEFLATE:
System.out.println("H5Z_FILTER_DEFLATE");
break;
case H5Z_FILTER_SZIP:
System.out.println("H5Z_FILTER_SZIP");
break;
default:
System.out.println("Other filter type included.");
}
}

// Read data.
IntPointer p = new IntPointer(rbuf);
dataset.read(p, PredType.NATIVE_INT());
p.get(rbuf);

plist.close();
dataset.close();
file.close(); // can be skipped

} // end of try block

// catch failure caused by the H5File, DataSet, and DataSpace operations
catch (RuntimeException error) {
System.err.println(error);
error.printStackTrace();
System.exit(-1);
}

System.exit(0); // successfully terminated
}
}
```
72 changes: 72 additions & 0 deletions hdf5/cppbuild.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
#!/bin/bash
# This file is meant to be included by the parent cppbuild.sh script
if [[ -z "$PLATFORM" ]]; then
pushd ..
bash cppbuild.sh "$@" hdf5
popd
exit
fi

HDF5_VERSION=1.10.0-patch1
download https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-$HDF5_VERSION/src/hdf5-$HDF5_VERSION.tar.bz2 hdf5-$HDF5_VERSION.tar.bz2

mkdir -p $PLATFORM
cd $PLATFORM
INSTALL_PATH=`pwd`
tar -xvf ../hdf5-$HDF5_VERSION.tar.bz2
cd hdf5-$HDF5_VERSION

case $PLATFORM in
android-arm)
patch -Np1 < ../../../hdf5-$HDF5_VERSION-android.patch
./configure --prefix=$INSTALL_PATH --host="arm-linux-androideabi" --with-sysroot="$ANDROID_ROOT" AR="$ANDROID_BIN-ar" RANLIB="$ANDROID_BIN-ranlib" CPP="$ANDROID_BIN-cpp" CC="$ANDROID_BIN-gcc" CXX="$ANDROID_BIN-g++" STRIP="$ANDROID_BIN-strip" CPPFLAGS="--sysroot=$ANDROID_ROOT -DANDROID -I$ANDROID_CPP/include/ -I$ANDROID_CPP/include/backward/ -I$ANDROID_CPP/libs/armeabi/include/ -fPIC -ffunction-sections -funwind-tables -fstack-protector -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fomit-frame-pointer -fstrict-aliasing -funswitch-loops -finline-limit=300" LDFLAGS="-L$ANDROID_ROOT/usr/lib/ -L$ANDROID_CPP/libs/armeabi/ -nostdlib -Wl,--fix-cortex-a8 -z text -L./" LIBS="-lgnustl_static -lgcc -ldl -lz -lm -lc" --without-getpwuid --enable-cxx
make -j $MAKEJ
make install-strip
;;
android-x86)
patch -Np1 < ../../../hdf5-$HDF5_VERSION-android.patch
./configure --prefix=$INSTALL_PATH --host="i686-linux-android" --with-sysroot="$ANDROID_ROOT" AR="$ANDROID_BIN-ar" RANLIB="$ANDROID_BIN-ranlib" CPP="$ANDROID_BIN-cpp" CC="$ANDROID_BIN-gcc" CXX="$ANDROID_BIN-g++" STRIP="$ANDROID_BIN-strip" CPPFLAGS="--sysroot=$ANDROID_ROOT -DANDROID -I$ANDROID_CPP/include/ -I$ANDROID_CPP/include/backward/ -I$ANDROID_CPP/libs/x86/include/ -fPIC -ffunction-sections -funwind-tables -mssse3 -mfpmath=sse -fomit-frame-pointer -fstrict-aliasing -funswitch-loops -finline-limit=300" LDFLAGS="-L$ANDROID_ROOT/usr/lib/ -L$ANDROID_CPP/libs/x86/ -nostdlib -z text -L." LIBS="-lgnustl_static -lgcc -ldl -lz -lm -lc" --without-getpwuid --enable-cxx
make -j $MAKEJ
make install-strip
;;
linux-x86)
./configure --prefix=$INSTALL_PATH CC="$OLDCC -m32" CXX="$OLDCXX -m32" --enable-cxx
make -j $MAKEJ
make install-strip
;;
linux-armhf)
./configure --prefix=$INSTALL_PATH --host="arm-linux-gnueabihf" CC="arm-linux-gnueabihf-gcc" CXX="arm-linux-gnueabihf-g++" --enable-cxx
make -j $MAKEJ
make install-strip
;;
linux-x86_64)
./configure --prefix=$INSTALL_PATH CC="$OLDCC -m64" CXX="$OLDCXX -m64" --enable-cxx
make -j $MAKEJ
make install-strip
;;
linux-ppc64le)
./configure --prefix=$INSTALL_PATH CC="$OLDCC -m64" CXX="$OLDCXX -m64" --enable-cxx
make -j $MAKEJ
make install-strip
;;
macosx-*)
./configure --prefix=$INSTALL_PATH --enable-cxx
make -j $MAKEJ
make install-strip
;;
windows-x86)
./configure --prefix=$INSTALL_PATH --host="i686-w64-mingw32" CC="gcc -m32" CXX="g++ -m32" --enable-cxx
make -j $MAKEJ
make install-strip
;;
windows-x86_64)
./configure --prefix=$INSTALL_PATH --host="x86_64-w64-mingw32" CC="gcc -m64" CXX="g++ -m64" --enable-cxx
make -j $MAKEJ
make install-strip
;;
*)
echo "Error: Platform \"$PLATFORM\" is not supported"
;;
esac

cd ../..
Loading

0 comments on commit 5d4c975

Please sign in to comment.