TinyTensor is an open-source deep learning library built from scratch in Python, inspired by Joel Grus' approach. This project includes 2 examples for creating a Non-Linear Neural Network and training it in order to predict the common XOR and FizzBuzz problems.
To use this deep learning library locally, you have 2 methods available:
- Install locally
- Use the Docker Image
git clone https://github.com/michaelradu/tinytensor.git
# Example of basic imports for training a Neural Network
from tinytensor.train import train
from tinytensor.nn import NeuralNet
from tinytensor.layers import Linear, Tanh
from tinytensor.optim import SGD
import pickle
with open('./models/trained_pipeline-0.1.0.pk1', 'wb') as f:
pickle.dump(net, f)
git clone https://github.com/michaelradu/tinytensor.git
cd tinytensor
docker build -t tinytensor-app .
docker run tinytensor-app
# Example of basic imports for training a Neural Network
from tinytensor.train import train
from tinytensor.nn import NeuralNet
from tinytensor.layers import Linear, Tanh
from tinytensor.optim import SGD
import pickle
with open('./models/trained_pipeline-0.1.0.pk1', 'wb') as f:
pickle.dump(net, f)
- Numpy
- Docker (Optional)
- tinytensor: This directory contains the deep learning library files.
- tinytensor/_init_.py: Default init file for python libraries.
- tinytensor/data.py: Tools for iterating over data in batches.
- tinytensor/layers.py: Layers for training Neural Networks.
- tinytensor/loss.py: File containing available loss functions.
- tinytensor/nn.py: Neural Network base class for instantiating objects. In essence, a collection of layers.
- tinytensor/optim.py: Optimization functions available for the training process.
- tinytensor/tensor.py: Cheat tensor class implementation.
- tinytensor/train.py: Default Training function for fitting models.
- models: This directory contains exported pre-trained models, optional to the actual library.
- examples: Directory showcasing example neural networks built with tinytensor.
- fizzbuzz.py: Example tinytensor Neural Network trained on the popular fizzbuzz problem with a default number of 5000 epochs.
- xor.py: Example non-linear tinytensor Neural Network trained on predicting valid xor values.
- xor-export.py:
xor.py
example with pickle model export included for use within other apps.
Fitting or training a model is a straightforward process. To achieve this:
- Import or create a new dataset.
- Edit your file to incorporate the new dataset.
- Implement any model pipeline according to your requirements.
- (Optional) Export your model with pickle.
- Run your python file to train the model.
- Update the version, and a new model, named trained_pipeline-0.1.1.pkl, will be saved in the models directory.
For more code-wise information see the provided examples and play around with them.
Contributions to this project are highly encouraged! If you have ideas for improvements, bug fixes, or new features, feel free to open an issue or submit a pull request. Let's collaborate to make this library even better and learn new concepts together!
This code is licensed under the GNU General Public License, version 3 (GPL-3.0). See the LICENSE file for more details.
Special thanks to Joel Grus and the open-source community for their amazing contributions to Machine Learning and for their copious amounts of educational content, making projects like this possible.
Feel free to explore, experiment, modify, rewrite, and integrate this library into your applications. Happy coding!
-
- Cheat Numpy Implementation
-
- MSE (Mean Squared Error), although the current implementation is more similar to Total Squared Erorr.
-
- Linear
- Tanh Activation
-
- Convolutional
-
- SGD (Stochastic Gradient Descent)
-
- Batch Iterators
-
- Helper Training Function
-
- XOR Example
- XOR + Model Export Example
- FizzBuzz Example