Classical ML Toolkit

This repo contains a collection of classical machine learning algorithms and tools implemented in Python. The goal is to provide a simple and easy to understand implementation of the algorithms, so that they can be used as a reference implementation for students and practitioners alike.

Contents

• Dimensionality Reduction:
  • Principal Component Analysis
  • Linear Discriminant Analysis
• Cross Validation:
  • K-Fold Cross Validation
  • Leave-One-Out Cross Validation
• Gaussian Classifiers:
  • Multivariate Gaussian Classifier (Full Covariance)
  • Multivariate Gaussian Classifier (Naive Bayes Diagonal Covariance)
  • Multivariate Gaussian Classifier (Tied Full Covariance)
  • Multivariate Gaussian Classifier (Tied Naive Bayes Diagonal Covariance)
• Logistic Regression:
  • Logistic Regression
• Support Vector Machines:
  • Linear SVM
  • Polynomial Kernel SVM
  • Radial Basis Function Kernel SVM
• Coming (somewhat) Soon:
  • Gaussian Mixture Models
  • K-Means Clustering
  • Perceptron and basic neural networks
  • Backpropagation
  • Examples
  • Documentation
  • And more!

Usage

The algorithms are implemented as classes, with the following methods:

• train(X, y): Train the model on the given data.
• predict(X): Predict the labels for the given data.

Implementation may vary slightly between algorithms, but the general idea is the same.