Skip to content
This repository has been archived by the owner on Jul 1, 2024. It is now read-only.

Commit

Permalink
Prepare for keras-mxnet 2.2 release (#117)
Browse files Browse the repository at this point in the history
* Document that "same" is inconsistent across backends with strides!=1 (#9629)

* Document that `"same"` is inconsistent across backends with `strides` != 1

* Use "[here](...)"

* Add h5py to dependencies

* import `pydot`, improve error messages about `pydot` and GraphViz, bump to `pydot >= 1.2.4` (#9904)

* REL: bump to `pydot >= 1.2.4` in `extras_require`

* MAI: import pydot (as required in `extras_require`)

* MAI: refine error messages for `pydot` and GraphViz

distinguish between absence of `pydot` and failure to find
the executables of GraphViz in the $PATH.

* DEV: ignore `.pytest_cache`

* Added note to manually install h5py where needed (#9830)

* Added notes to manually install h5py if needed

* Added FAQ entry on h5py

* deleted redundant remark about h5py

* updated FAQ to reflect dependency change

* fixed comment format to pass failing test

* removed new trailing whitespaces

* improved docstring format

* reverted callbacks.py

* fixed links in model.py

* updated faq.py

* link pointing to FAQ

* Added an error message for undefined shape on NASNet. (#9891)

* Added an error message for undefined shape on NASNet.

* Forgot that the message should be present only when loading imagenet weights.

* Changed the message.

* Fix PEP8

* Make conv_invalid_use and pooling_invalid_use efficient (#9944)

* Chenta/cntk bn (#9952)

* fix cntk static learning phase issue; add a test

* fix code style;add more comments

* add boolean support

* fix code style issue

* fixing typos (#10016)

* Add `separable_conv2d` for Theano (#10003)

* Add `separable_conv2d` for Theano

* Enable the layer test for `separable_conv2d`

* Fix the strides for 1x1 conv

* Refactor topological part of `engine` module (#10023)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Fix Bidirectional Regularization (#10012)

* Fix Bidirectional Regularization

Override the Wrapper `get_updates_for` and `get_losses_for` methods so that contributions from both the forward and backward layers are included in Bidirectional.

* Use Parent for Calculating Inner Inputs

Remove unneeded inner input calculations.

* Simplify Bidirectional Losses

* Add Bidirectional Unit Tests

Test Bidirectional updates and losses.

* Remove Trailing Whitespace

* Fix Bidirectional Loss Inputs

* Add Tests for Conditional Updates/Losses

* Remove Whitespace

* Refactor training part of `engine` module. (#10029)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Refactor training part of the Keras engine.

* Fix unit tests.

* Fix `int_shape` of Theano and Refactor associated lines (#10030)

* Add exceptions for `batch_dot` (#10020)

* Enable Xception to work on Theano and CNTK (#10024)

* Enable Xception to work on Theano and CNTK

* Fix different predictions over all the backends

* Add support for passthrough arguments to NumpyArrayIterator (#10035)

* Add support for second output to NumpyArrayIterator

* Enable Model subclassing API (#10046)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Refactor training part of the Keras engine.

* Fix unit tests.

* Refactor Network to prepare support for Model subclassing

* Finish enabling Model subclassing.

* Add `long_description` field in setup.py.

* Remove unused import

* Increase pytest duration from 10 mins to 20 mins (#10072)

* [RELNOTES] Simplify implementation of Sequential and remove legacy Merge support (#10077)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Refactor training part of the Keras engine.

* Fix unit tests.

* Refactor Network to prepare support for Model subclassing

* Finish enabling Model subclassing.

* Simplify Sequential implementation.

RELNOTES: This breaks weight loading and model loading/saving for models from Keras 0.* that used Merge layers. The Merge layer has been deprecated for 2 years.

* [RELNOTES] Remove support for Keras 0.* Merge layer and associated functionality, which was scheduled for 08/2017.

* fix typo (#10078)

* Add documentation to several activation functions (#10066)

* Add documentation to several activation functions

* Fix style issues.

* Small style fixes.

* [RELNOTES] Introduce `preprocessing.image.save_img` and remove deprecated imsave method in neural style transfer example (#9996)

* imsave method in scipy.misc package is deprecated - now using imageio

* updated save_img method to use array_to_img method.  also updated the neural style transfer example to use the new save_img method

* forgot to commit changes - updated save_img

* added file_format and **kwargs parameter to save_img and updated docstring

* removed space at the end of a line in save_img method.  updated instances of imsave in example scripts with save_img method.

* added kwargs to docstring.  removed additional whitespace lines

* removed trailing whitespace

* Extensive style fixes in `preprocessing.image`.

* [RELNOTES] Allow loading external backends (#10034)

* load external backend

* safety++

* Excpetion -> ValueError

* Revert TF version to 1.7 on Travis CI. (#10101)

* New Callback: EarlyBaselineStopping (#10061)

* Initial Commit

* Continued changes

* Alpha version

* Added support to make sure that previous epochs, which may pass the
baseline test, are kept in variable history.

* Code formatting

* Code formatting

* Code formatting to address CI errors

* Code formatting for CI errors

* Initial unit test code

* Adjust for epoch being zero-based

* Code formatting

* Unit tests added

* Code formatting

* Code formatting

* Factorized the unit test

* Code formatting

* Refactored to be part of EarlyStopping and modified unit tests

* Code formatting

* Adds MobiletNetV2 to applications (#10047)

* [RELNOTES] Allow symbolic tensors to be fed to models (with TF backend) (#10087)

* Allow symbolic tensors to be fed to models (with TF backend)

* Skip tensor feeding tests when TF>=1.8 not available.

* [RELNOTES] Fix EarlyStopping API

* Fix shape mismatch in `rnn()` of tensorflow_backend (#10038)

* Fix shape mismatch in `rnn()` of tensorflow_backend

Fix the shape mismatch in `rnn()` with `not unroll` and `mask is not None` of the Tensorflow backend.

Problem: If the rnn cell has any (recurrent) `states[i]` whose shape is different from that of the `output` (`states[0]`), there will raise an `ValueError` when updating that state.

Reason: This is because the `tiled_mask_t` is not updated correctly for each `states[i]` but instead is simply copied from that of the `output` (`states[0]`).

Solution: Tile the `mask_t` with the correct shape of each `states[i]`. Notice that in a similar situation with `unroll is True`, the `tiled_mask_t` is handled correctly.

* add unit_test for rnn() with states whose shape is different from that of the output.

* Revert "add unit_test for rnn() with states whose shape is different from that of the output."

This reverts commit f1df2a58ff635bbf698444e3d7403785a92dfed1.

* Simplify the unit_test for rnn with additional states

* [RELNOTES] Default `Flatten` layer’s `data_format` argument to `None`.

* #10080 Convert CuDNN weights in nested Model. (#10081)

* #10080 Convert CuDNN weights in nested Model.

- similar problem to nesting in Bidirectional (#8860)
- quick fix: just copy/paste. I'll refactor it later
- also convert biases from H5 Dataset to np.array for reshape() (#9662)

* Refactor keras.engine.saving.preprocess_weights_for_loading().

- less duplication
- extracted conversion methods for Bidirectional/Model/Sequential

* Format docstrings to follow the project code style.

* Move tests of CuDNN RNN weight conversion to a more proper place.

- move from cudnn_recurrent_test.py to test_model_saving.py
- adjust imports to be consistent

* Make better tests for CuDNN RNN weight conversion.

- add nested models (#10080)
- various model types: add Model, Sequential (not only Sequential)
- convert weights on model, not layer (not test nesting in models)
  - test just by save/load model weights instead of calling preprocess_weights_for_loading() directly

* Check GPU support via tf in pytest skipif with short-circuit evaluation.

It seems that multiple skipif decorators get evaluated eagerly, not lazily.
Thus it fails on theano and cntk backends.

I don't understand why the tests didn't fail until now.
Maybe some change in pytest?

* Refactor (deduplicate) skipif decorators for TensorFlow and GPU.

At least multiple occurrences within a test module.
Now we can't import from `tests/` and `keras.utils.test_utils` don't se pytest.
Otherwise we can define the marker only once at all.

* Fix PEP8 (whitespace).

* Add `separable_conv1d` (#10125)

* Add `separable_conv1d` for Theano

* Add `separable_conv1d` for CNTK

* Fix preprocess_input not working with int arrays (#10134)

* Make MobileNet tests random (#10132)

* Sample weighted ImageDataGenerator (#10092)

* Add support for sample_weight in ImageDataGenerator.flow

* Added test for sample weighted imagedatagen

* clarified docs + PEP8

* pep8 - blank line

* sample_weight argument after shuffle

* RNN docstring fixes; pass `maximum_iteration` argument to `while_loop` in TF.

* Use `embedding_lookup` for `gather` in TF backend (enables partitioned variables).

* Add `data_format` argument to Conv1D; docstring style fixes.

* Further fixes.

* Fix initializers (#9963)

* fix VarianceScaling

* Correct standard initializer testing

* Add test for truncated normal in backend

* Use correct theano function for truncated_normal

* Fix sloppiness in correction

* fix PEP

* improve style initialiser fix

* fix line length

* Use NCW/NWC for conv1d data format in TF backend.

* Model loading: do not return inside `try` block.

* Rely on `Flatten` layer to do data_format conversion in ResNet50, VGG16, VGG19.

* Remove potentially misleading warning messages in `applications`.

* Fix typo.

* Remove useless super delegation.

* Remove batch_size when doing stepwise fit (#10144)

`batch_size` is always `None` , because `steps_per_epoch` and `batch_size` are mutually exclusive.
Passing batch_size seems irrelevant as `None` is its default value in `test_loop`

* Add exceptions for `fit_loop` (#10145)

* Style fixes (#10148)

* Fix ConvLSTM2D unit_forget_bias initializer

* Avoid a warning by using  'x = x * y' instead of 'x *= y' (#10158)

* Fix undefined behaviour: preprocess_input copying/not copying the input arrays (#10153)

* Add copy option for image preprocessing

* Fix unnecessary import

* Fix style

* fix test error

* Make modifications in-place instead

* vae examples fixes (#10062)

* vae examples fixes

* white spaces around op

* comments and docstring fixes

* comments and docstring fixes

* comments and docstring fixes

* fixes on docs

* docs and spacing

* Merge 2 functions together in applications MobileNetV2 (#10163)

* merge 2 functions together

_inverted_res_block() and _first_inverted_res_block() are nearly the same. Merged them into one function
1. skip "Expand" part for 0 block
2. made layers names similar to original TF graph names

* added "mobl_" to prefix, pep8 fix

* remove dropout parameter

* concise prefix name

* Remove named argument from schedule function (#10178)

* Add missing doc (#10188)

* Best to set self.built=True at the end of build() (#10191)

When using Keras on TensorFlow (in graph mode), via `tf.keras`, then calling `self.add_weight()` fails if `self.built==True`, so it is best to encourage users to set `self.built=True` at the *end* of their `build()` method, rather than just "somewhere".

* Clean up preprocessing of `depthwise_kernel` for Theano (#10189)

* Allow dynamic backends in _need_convert_kernel (#10111)

* Fix _need_convert_kernel for external backends

* Don't assume external backend won't support NASNet

* Update from review comments

* fix TensorBoard callback with unit test (#10173)

* Allow project specific config files in Keras (#10183)

* Allow project specific config files in Keras

* Clearer code, updated comment

* Update cifar10_resnet.py (#10199)

change the comments on line 133.

* Adds height_shift_range to preprocessing.image doc and adds support for indentation in auto-generated doc (#10194)

* Update README.me

* Mobilenetv2 explanation  (#10174)

* i add mobilenetv2 to the table

* dd explanation for mobilenetv2

* add explanation for mibilenetv2

* Make multi_gpu_model serializable.

* Skip tests for multi gpu

* Fix slice namespace collision in backend.

* Skip multi_gpu tests for other backends than TF

* Replace np.ceil() with a faster operation (#10184)

* Replace ceil() with faster a operation

Replacing np.ceil() with faster operations, as suggested in #8488

* Revert "Replace ceil() with faster a operation"

This reverts commit 74fe60b445377b4f1be1d0a36b4777d7ed4f1c1b.

* Replace np.ceil() with a faster operation

* Fix syntax error

* Remove trailing whitespace

* Explicitly mention using a Sequence object in val (#10223)

* Grammatical error - "inputs channels" rather than "input". (#10217)

* Fix duplicated argname: num_gpus === parts (#10228)

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Fix NASNet (#10209)

* Fix NASNet

* Update weight files

* In-place split to avoid inter-device duplication (#10230)

New Benchmark by in-place split:

>> keras.application.Resnet50 224x224x3 (NCWH; NVidia Tesla P100 x 4)
 input_shape = 3x224x224, batch_size =  96 x 4: 392(images/sec) => 417(images/sec)
 input_shape = 3x299x299, batch_size =  64 x 4: 229(images/sec) => 244(images/sec)
 input_shape = 3x224x224, batch_size =   8 x 4: 148(images/sec) => 163(images/sec)

>> keras.application.InceptionV3 (NCWH; NVidia Tesla P100 x 4)
 input_shape = 3x224x224, batch_size = 128 x 4: 488(images/sec) => 526(images/sec)
 input_shape = 3x299x299, batch_size =  96 x 4: 270(images/sec) => 294(images/sec)
 input_shape = 3x224x224, batch_size =   8 x 4: 146(images/sec) => 158(images/sec)

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Increase test coverages by adding invalid CNTK usecases (#10236)

* Remove Sequential.model deprecation warning (#10256)

* Remove Sequential.model deprecation warning

* Remove dead line of code

* Increase test coverages by factorizing CNTK pads (#10259)

* Refactor ImageDataGenerator (#10130)

* Create get_random_transform and refactor

* Fix style and add tests

* Add more tests

* Fix documentation error

* Fix documentation style issue

* add apply_affine_transform

* document transformation dictionary

* Doc style fix

* Remove deprecated model.model from engine/saving (#10275)

* Typo in docstring for softplus (#10277)

Softplus docstring missing a parenthesis.

* Make Dot documentation inline with Concatenate (#10271)

Doc expects a list containing 2 tensors.

* Fixes automatic doc generation problem with nested lists. Adds a new test (#10212)

* Fixes automatic doc generation problem with indented lists. Adds a new test

* Some style fixes on doc automatic generation files

* Fixes a bad space in convolutional_recurrent.py

* Changes the test_doc_auto_generation in order to include a doc string taken from the codebase. Allows text lines following nested lists

* Use count_params function for non_trainable_count. (#10280)

* load_weights will fail if shape mismatch (#10266)

Fix for #10265

* Adds to and alphabetizes documentation of Layer base class. (#10282)

* Alphabetizes and adds to layers doc.

* Responding to @cais comments

* fix spacing.  Remove in(out)bound_nodes

* Non training Batch Norm operator has bad performance for it running into tensorflow's non fused batch norm API (#10207)

* When use tensorflow as backend, let batch norm run into fused batch norm as much as possible, which has better performance.

fix issue: http://github.com/keras-team/keras/issues/10058

* In Tensorflow backend, let batch norm call to FusedBatchNorm only NHWC format, also gamma and beta are not None.

Test result:
test env: with Tensorflow(commit a543d9471047ca3f6881c87105fcbe2cdff9207d Date:   Thu May 10 17:43:30 2018, local build), python3.4, centos7.4
test cases:
  "pytest  ./tests/keras/layers/normalization_test.py"  <all passed>
  "pytest  ./tests"      <keep same result as without this commit's modification on BN>

* fix code sytle.

* 1. Add axis parameter in backend's batch_normalization functions.
2. Refine the batch_normalization function in tensorflow backend, Let's it call to fused batch norm as much as possible.

Thanks the coments from fchollet.

* Trigger

* 1. add default value -1 for parameter axis in batch_normalization function in backend.
2. fix some code style.
Thanks the comments from fchollet.

* Handle capitalised extensions in list_pictures (#10220)

#10219

* Typo fix (#10293)

* Fix doc  (#10308)

* Fix naming convention

* Add missing doc

* Fix typo

* Improve docstrings of applications (#10310)

* Add pooling options in MobileNetV2 (#10313)

* Add pooling option

* Add pooling test

* Fix doc (#10327)

Fixed doc

* Handle `mask` in `TimeDistributed` wrapper. (#10242)

* equip TimeDistributed with mask and unspecified input length

* fix bugs in theano. add test on timedistributed + masking

* skip tests on cntk with multiple unspecified time lengths.

* move static shape inference to theano_backend, add docstring, etc.

* fix format

* Split `applications` and `preprocessing` modules. (#10339)

* Split `applications` and `preprocessing` modules.

* Fix dependencies.

* Move tests for applications (#10341)

* Improve the docstring of Conv3DTranspose (#10342)

* Add depth as third dimension in docstring of
  Conv3DTranspose in convolutional.py in keras.layers

* Reduce tests for applications (#10346)

* Reduce tests for applications

* Make selection over all models random

* Add an advanced activation layer for ReLU (#10322)

The max_value argument can not be used in a layer, except
custom layer or Lambda. Hence, similarly to LeakyReLU or
for example Softmax, this PR adds a layer for ReLU,
enabling also a capped ReLU to be used.

* FIX: Tensorboard callback only supports logging Embeddings layer weights (#7766)

* Embed layer-outputs rather than layer-weights in TensorBoard callback

* Update docstring and allow multiple inputs

* Fix tests

* Renaming

* Set learning phase

* Compute embeddings in batches

* Pass embedding data explicitly

* Actually process embeddings in batches

* Allow multiple inputs and validate input data

* Add example

* Delete utils.py

* Revert uncorrectly resolved merge-conflict

* Minor renaming

* Add comment clarifying the design choice

*  Fix HDF5Matrix issue when working in conjunction with TimeSeriesGenerator (#10334)

* Fix issue when working in conjunction with TimeSeriesGenerator

The TimeSeriesGenerator class uses xrange through six which caused an IndexError

* Add test

* Add corresponding test

* Fix for python3

* Simplified code

* Fix indent

* Fix test

* Supporting channels_first data format with crossentropy losses (#9715)

* Add error message when calling `summary` on unbuilt subclassed models.

* Prepare 2.2.0 release.

* Fix a version number (#10361)

* Update to Keras Applications 1.0.2 (fixes NASNet issue).

* Add tests for inputs set dynamically (#10367)

* CuDNN RNN layers nested in TimeDistributed are not converted when loading (#10357)

* Add a unit test for CuDNNGRU conversion with TimeDistributed.

* Extract duplicated function convert_model() to _convert_model_weights().

* #10356 Convert weights of CuDNN/plain RNN nested in TimeDistributed.

Same case as for Bidirectional, except that in TimeDistributed there's only
one nested layer instead of two.

* Style fix

* Update docs for 2.2.0.

* Add spatial dropout and 3D global pooling to docs (#10373)

* spatial dropout in docs

* 3d global pooling in docs

* Doc update (#10376)

A couple of variables are "used" in two examples without being defined. For consistency with other examples where auxiliary dimensions are defined, I think it would be better to explicitly assign them a value. I just used made up values, feel free to change to whatever makes more sense!

* Preserve input shape data when serializing deferred-build Sequential models.

* Add MXNet Backend (#59)

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Rebase to latest Keras - April 3, 2018

* Add build badges

* Fix multi_gpu API bug for CPU. Fix PEP. (#64)

* Fix multi_gpu API bug for CPU. Fix PEP.

* fix embedding layer bug (#61)

* fix embedding bug

* addressed comments, enabled more test cases

* add keras test

* reduce line length

* fix style, add blank lines

* Benchmark (#55)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* add benchmark scripts for resnet and imagenet data

* combine scripts

* fix args

* fix num of gpus

* update log

* multi_gpu_model only support tf

* add benchamrk scripts for synthetic data

* update read me and scripts

* add mxnet traing result table

* update on readme

* add cifar10 dataset and enable various resnet layers

* fix compile for mxnet multiple gpu

* update callbacks

* update synthetic data script, add credits

* undo new line

* update readme, addressed pr comments

* update readme

* benchmark scripts style fix (#66)

* style fix

* remove unused import, fix line too long

* adrressed pr comments

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend (#65)

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend

* Modified comments

* Addressed review comments and made the API more generic accross backends

* Removed shape check

* Modified comments

* Added edge cases

* moved helper method as nested

* Added RNN benchmark scripts (#69)

* Added RNN benchmark scripts

* Fixed new line in bash script

* Removed different backend code and modified comments

* Removed spacing

* Automated the wikiText2 download script

* Added dataset_util functionality to have more flexible code

* Added minor comments

* modified minor comments

* Fixed the multi-gpu context (#68)

* Update benchmark result (#70)

* update benchmark result

* update result

* simplify folder structure

* add image result

* add note

* add note

* Add MXNet Backend (#59)

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Rebase to latest Keras - April 3, 2018

* Add build badges

* Fix multi_gpu API bug for CPU. Fix PEP. (#64)

* Fix multi_gpu API bug for CPU. Fix PEP.

* fix embedding layer bug (#61)

* fix embedding bug

* addressed comments, enabled more test cases

* add keras test

* reduce line length

* fix style, add blank lines

* Benchmark (#55)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* add benchmark scripts for resnet and imagenet data

* combine scripts

* fix args

* fix num of gpus

* update log

* multi_gpu_model only support tf

* add benchamrk scripts for synthetic data

* update read me and scripts

* add mxnet traing result table

* update on readme

* add cifar10 dataset and enable various resnet layers

* fix compile for mxnet multiple gpu

* update callbacks

* update synthetic data script, add credits

* undo new line

* update readme, addressed pr comments

* update readme

* benchmark scripts style fix (#66)

* style fix

* remove unused import, fix line too long

* adrressed pr comments

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend (#65)

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend

* Modified comments

* Addressed review comments and made the API more generic accross backends

* Removed shape check

* Modified comments

* Added edge cases

* moved helper method as nested

* Added RNN benchmark scripts (#69)

* Added RNN benchmark scripts

* Fixed new line in bash script

* Removed different backend code and modified comments

* Removed spacing

* Automated the wikiText2 download script

* Added dataset_util functionality to have more flexible code

* Added minor comments

* modified minor comments

* Fixed the multi-gpu context (#68)

* Update benchmark result (#70)

* update benchmark result

* update result

* simplify folder structure

* add image result

* add note

* add note

* rebase to latest Keras - April 20, 2018, fix bug and unit tests

* Added detailed RNN results (#73)

* Added detailed RNN results

* Modified table content and added CUDA version

* fix keras examples (#72)

* fix auto encoder examples

* update other examples

* fix style and add ctc not implemented error

* Added Detailed RNN results (#77)

* Modified RNN benchmark document

* Added minor comments

* fixed broken image link

* Added API to extract metrics from a test and also added epoch parameter (#78)

* Add mxnet backend tutorial documents (#76)

* add performance tips document

* update warning

* add docs from wiki

* add initial multi gpu doc, simplified installation doc, fix benchmark doc typo

* update install steps

* add multi_gpu_model tutorial

* Support exporting model as MXNet model (sym, params). (#80)

* Support exporting model as MXNet model (sym, params).

* Return data_names and data_shapes

* add unit tests for mxnet model save API

* Add test with LSTM layer for mxnet model save API

* Add support for functional Model graphs in save_mxnet_model API

* Add additional logging for cnn benchmarks (#89)

* add extra logging

* add logging for cnn synthetic

* fix log name

* fix file name

* Log RNN benchmark results (#90)

* Make benchmark result logging available in RNN scripts

* Make log file name consistent across CNN and RNN benchmarks

* fix pytest errors (#93)

* Cherry pick keras-team/keras 2.1.6 missing 3 commits into awslabs/keras-apache-mxnet (#96)

* update multi_gpu api in benchmark scripts (#95)

* update multi_gpu

* update logging

* fix logging

* fix logging

* fix speed format

* remove learning rate log

* Revamp keras-mxnet docs (#82)

* Update main README and move mxnet_backend_docs under docs

* revisit installation mxnet backend docs

* revisit multi_gpu_training mxnet backend docs

* revisit performance_guide mxnet backend docs

* revisit using rnn with mxnet backend in mxnet backend docs

* add save_mxnet_model tutorials in mxnet backend docs

* Fixing review comments from aaron

* Resolve CR comments on save_mxnet_model tutorial

* Fix broken links, update tutorial links in the mxnet_backend code

* revamp benchmark results readme

* Benchmark results README page revamp

* Add library versions

* Remove too detailed benchmark results. Summarize in README

* Get back detailed results document

* Remove experiemental RNN benchmarks from README

* addressed review comments on benchmark results

* Set latest stable dependency of h5py to avoid warnings

* Rebase to latest Keras April 20 2018 (#71)

* Improve tests by designating dtype of sample data (#9834)

* Document that "same" is inconsistent across backends with strides!=1 (#9629)

* Document that `"same"` is inconsistent across backends with `strides` != 1

* Use "[here](...)"

* #9642 Add kwarg and documentation for dilation_rate to SeparableConvs (#9844)

* Add kwarg and documentation for dilation_rate to SeparableConvs

* Fix pep8 complaint

I forgot to check the style before committing. Pep8 was complaining about a missing whitespace after comma, now it's fixed.

* fit/evaluate_generator supporting native tensors (#9816)

Currently, `fit/evaluate_generator` don't support this case without this fix.
But framework-native data tensors are already supported by `_fit_loop` and `_test_loop`.

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Add h5py to dependencies

* Fixed typo. (#9866)

* Fix image_ocr.py example ValueError (#9869)

* Fixed the NASNet issue. (#9865)

* Fixed the NASNet issue.

* Nasnet doesn't require flatten.

* Updated documentation accordingly.

* Removed generate dropout ones from recurrent. (#9892)

* Removed generate dropout ones from recurrent.

* Fixed index issue.

* Fix `in_test_phase` of CNTK and Add its tests (#9902)

* Fix dtype designation for `variable` of CNTK and Add its tests (#9903)

* import `pydot`, improve error messages about `pydot` and GraphViz, bump to `pydot >= 1.2.4` (#9904)

* REL: bump to `pydot >= 1.2.4` in `extras_require`

* MAI: import pydot (as required in `extras_require`)

* MAI: refine error messages for `pydot` and GraphViz

distinguish between absence of `pydot` and failure to find
the executables of GraphViz in the $PATH.

* DEV: ignore `.pytest_cache`

* Fix documentation of flow_from_directory() (#9910)

The way the documentation is parsed for the Keras website made some lines of the documentation beginning with "Default:" look funny. Also changed the documentation of return value to be clear that it always returns a batch of images.

* ModelCheckpoint: print previous best (#9911)

* multi_gpu_model supporting legacy/fullCPU/fullGPU (#9638)

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Fix `batch_dot` of Theano when `axes=0` (#9920)

* Fix `batch_dot` of CNTK when `axes=None` (#9921)

* Fix `batch_dot` of TensorFlow when `axes=None` (#9922)

* Fix stateful metrics when passing dict to compile (#9894)

* Added note to manually install h5py where needed (#9830)

* Added notes to manually install h5py if needed

* Added FAQ entry on h5py

* deleted redundant remark about h5py

* updated FAQ to reflect dependency change

* fixed comment format to pass failing test

* removed new trailing whitespaces

* improved docstring format

* reverted callbacks.py

* fixed links in model.py

* updated faq.py

* link pointing to FAQ

* Add support for `constants` in Bidirectional wrapper (#9260)

* Add support fot `constants` in Bidirectional wrapper

* Add more tests for Bidirectional wrapper

* Fix `compute_mask` for Birectional with return_state=True

Fix `compute_mask` to properly support `return_state` introduced in Birectional with #8977

* Add test for Bidirectional with unknown timestamps

* Skip test for CNTK for unknown timestamps with Bidirectional

* avoid override the input constant when need broadcast sequential axis on rnn's constant

* Move _standardize_args to recurrent, remove duplication

* Fix  for Birectional when multiple masks are passed

* Updated for TF 1.7 (#9937)

* fix TimeSeriesGenerator glitch (#9899)

* Added an error message for undefined shape on NASNet. (#9891)

* Added an error message for undefined shape on NASNet.

* Forgot that the message should be present only when loading imagenet weights.

* Changed the message.

* Fix PEP8

* Allow shift_range to be 1-D array-like or int (#8869)

* Allow shift_range to be 1-D array-like or int

* Add docstrings

* Fix conflict resolution merge minor disaster

* remove stray line from merge

* Remove extra "tabs"

* Exclude multi-gpu utils when reporting coverages (#9942)

* Make conv_invalid_use and pooling_invalid_use efficient (#9944)

* Chenta/cntk bn (#9952)

* fix cntk static learning phase issue; add a test

* fix code style;add more comments

* add boolean support

* fix code style issue

* Immigrate reference operations to a separate module (#9948)

* Add MXNet Backend (#59)

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - royw…
  • Loading branch information
sandeep-krishnamurthy committed Aug 13, 2018
1 parent 44231cf commit b6cfa49
Show file tree
Hide file tree
Showing 26 changed files with 585 additions and 79 deletions.
2 changes: 1 addition & 1 deletion ISSUE_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Please make sure that the boxes below are checked before you submit your issue.
Thank you!

- [ ] Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/keras-team/keras.git --upgrade --no-deps
pip install git+git://github.com/awslabs/keras-apache-mxnet.git --upgrade --no-deps

- [ ] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found [here](https://www.tensorflow.org/get_started/os_setup).

Expand Down
4 changes: 0 additions & 4 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,6 @@ All contributions by Amazon:
Copyright (c) 2017 Amazon.com, Inc. or its affiliates
All rights reserved.

All contributions by Amazon:
Copyright (c) 2017 Amazon.com, Inc. or its affiliates
All rights reserved.

All other contributions:
Copyright (c) 2015 - 2018, the respective contributors.
All rights reserved.
Expand Down
2 changes: 1 addition & 1 deletion benchmark/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -260,4 +260,4 @@ For TensorFlow backend benchmarks:
## References

* [TensorFlow Keras Benchmarks](https://github.com/tensorflow/benchmarks/tree/keras-benchmarks/scripts/keras_benchmarks)
* [lstm_text_generation.py](https://github.com/keras-team/keras/blob/master/examples/lstm_text_generation.py)
* [lstm_text_generation.py](https://github.com/keras-team/keras/blob/master/examples/lstm_text_generation.py)
2 changes: 0 additions & 2 deletions benchmark/scripts/benchmark_resnet.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
'''Trains a ResNet on the ImageNet/CIFAR10 dataset.
Credit:
Script modified from examples/cifar10_resnet.py
Reference:
ResNet v1
[a] Deep Residual Learning for Image Recognition
https://arxiv.org/pdf/1512.03385.pdf
ResNet v2
[b] Identity Mappings in Deep Residual Networks
https://arxiv.org/pdf/1603.05027.pdf
Expand Down
2 changes: 1 addition & 1 deletion benchmark/scripts/run_mxnet_backend.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ echo -e "Running tests with the following config:\n$(cat ~/.keras/keras.json)"

dir=`pwd`

python $dir/run_benchmark.py --pwd=$dir --mode="$1" --model_name="$2" --dry_run=True --inference="$3" --epochs="$4"
python $dir/run_benchmark.py --pwd=$dir --mode="$1" --model_name="$2" --dry_run=True --inference="$3" --epochs="$4"
2 changes: 1 addition & 1 deletion benchmark/scripts/run_tf_backend.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ echo -e "Running tests with the following config:\n$(cat ~/.keras/keras.json)"

dir=`pwd`

python $dir/run_benchmark.py --pwd=$dir --mode="$1" --model_name="$2" --dry_run=True --inference="$3" --epochs="$4"
python $dir/run_benchmark.py --pwd=$dir --mode="$1" --model_name="$2" --dry_run=True --inference="$3" --epochs="$4"
1 change: 1 addition & 0 deletions examples/babi_memnn.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
'''
from __future__ import print_function

from keras import backend as K
from keras.models import Sequential, Model
from keras.layers.embeddings import Embedding
from keras.layers import Input, Activation, Dense, Permute, Dropout
Expand Down
4 changes: 4 additions & 0 deletions examples/cifar10_cnn_capsule.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@
from keras.layers import *
from keras.preprocessing.image import ImageDataGenerator

if K.backend() == 'mxnet':
raise RuntimeError('This example cannot run with MXNet backend.'
'MXNet backend does not support auto infer of shape in Convolution operators.')


# the squashing function.
# we use 0.5 in stead of 1 in hinton's paper.
Expand Down
90 changes: 90 additions & 0 deletions examples/cifar10_mobilenet.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
'''Train a MobileNet on the CIFAR10 small images dataset.
Credit: scripts modified from examples/cifar10_cnn.py
'''

from __future__ import print_function
import keras
from keras.datasets import cifar10
from keras.preprocessing.image import ImageDataGenerator
from keras.applications import MobileNet
import os

batch_size = 32
num_classes = 10
epochs = 100
data_augmentation = True
num_predictions = 20
save_dir = os.path.join(os.getcwd(), 'saved_models')
model_name = 'keras_cifar10_trained_model.h5'

# The data, split between train and test sets:
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# Convert class vectors to binary class matrices.
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

# MobileNet only supports channels last format
model = MobileNet(input_shape=(32, 32, 3), weights=None, classes=num_classes)

# initiate RMSprop optimizer
opt = keras.optimizers.rmsprop(lr=0.0001, decay=1e-6)

# Let's train the model using RMSprop
model.compile(loss='categorical_crossentropy',
optimizer=opt,
metrics=['accuracy'])

x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255

if not data_augmentation:
print('Not using data augmentation.')
model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
validation_data=(x_test, y_test),
shuffle=True)
else:
print('Using real-time data augmentation.')
# This will do preprocessing and realtime data augmentation:
datagen = ImageDataGenerator(
featurewise_center=False, # set input mean to 0 over the dataset
samplewise_center=False, # set each sample mean to 0
featurewise_std_normalization=False, # divide inputs by std of the dataset
samplewise_std_normalization=False, # divide each input by its std
zca_whitening=False, # apply ZCA whitening
rotation_range=0, # randomly rotate images in the range (degrees, 0 to 180)
width_shift_range=0.1, # randomly shift images horizontally (fraction of total width)
height_shift_range=0.1, # randomly shift images vertically (fraction of total height)
horizontal_flip=True, # randomly flip images
vertical_flip=False) # randomly flip images

# Compute quantities required for feature-wise normalization
# (std, mean, and principal components if ZCA whitening is applied).
datagen.fit(x_train)

# Fit the model on the batches generated by datagen.flow().
model.fit_generator(datagen.flow(x_train, y_train,
batch_size=batch_size),
epochs=epochs,
validation_data=(x_test, y_test),
workers=4)

# Save model and weights
if not os.path.isdir(save_dir):
os.makedirs(save_dir)
model_path = os.path.join(save_dir, model_name)
model.save(model_path)
print('Saved trained model at %s ' % model_path)

# Score trained model.
scores = model.evaluate(x_test, y_test, verbose=1)
print('Test loss:', scores[0])
print('Test accuracy:', scores[1])
22 changes: 14 additions & 8 deletions examples/cifar10_resnet_multi_gpu.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
"""Trains a ResNet on the CIFAR10 dataset.
"""Trains a ResNet on the CIFAR10 dataset with multi-GPU.
Modified from examples/cifar10_resnet.py
ResNet v1
[a] Deep Residual Learning for Image Recognition
Expand Down Expand Up @@ -26,13 +28,15 @@
from keras.utils import multi_gpu_model

# Training parameters
batch_size = 32 # orig paper trained all networks with batch_size=128
num_gpus = 4
batch_size = 32 * num_gpus # orig paper trained all networks with batch_size=128
epochs = 200
data_augmentation = True
data_augmentation = False
data_normalization = False
num_classes = 10

# Subtracting pixel mean improves accuracy
subtract_pixel_mean = True
subtract_pixel_mean = False

# Model parameter
# ----------------------------------------------------------------------------
Expand All @@ -48,7 +52,8 @@
# ResNet164 |27(18)| ----- | 94.07 | ----- | 94.54 | ---(---)
# ResNet1001| (111)| ----- | 92.39 | ----- | 95.08+-.14| ---(---)
# ---------------------------------------------------------------------------
n = 3
# use ResNet56
n = 9

# Model version
# Orig paper: version = 1 (ResNet v1), Improved ResNet: version = 2 (ResNet v2)
Expand All @@ -70,8 +75,9 @@
input_shape = x_train.shape[1:]

# Normalize data.
x_train = x_train.astype('float32') / 255
x_test = x_test.astype('float32') / 255
if data_normalization:
x_train = x_train.astype('float32') / 255
x_test = x_test.astype('float32') / 255

# If subtract pixel mean is enabled
if subtract_pixel_mean:
Expand Down Expand Up @@ -335,7 +341,7 @@ def resnet_v2(input_shape, depth, num_classes=10):
model = resnet_v1(input_shape=input_shape, depth=depth)

# use 4 gpus to train the model
model = multi_gpu_model(model, gpus=4)
model = multi_gpu_model(model, gpus=num_gpus)
model.compile(loss='categorical_crossentropy',
optimizer=Adam(lr=lr_schedule(0)),
metrics=['accuracy'])
Expand Down
4 changes: 4 additions & 0 deletions examples/conv_filter_visualization.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@
from keras.applications import vgg16
from keras import backend as K

if K.backend() == 'mxnet':
raise RuntimeError('This example cannot run with MXNet backend.'
'MXNet backend does not Gradient operator yet!')

# dimensions of the generated pictures for each filter.
img_width = 128
img_height = 128
Expand Down
2 changes: 1 addition & 1 deletion keras/backend/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
_keras_base_dir = '/tmp'
_keras_dir = os.path.join(_keras_base_dir, '.keras')

# Default backend: TensorFlow.
# Default backend: MXNet.
_BACKEND = 'mxnet'

# Attempt to read Keras config file.
Expand Down
Loading

0 comments on commit b6cfa49

Please sign in to comment.