Awesome Gradient Boosting

A curated list of Gradient Boosting resources for Data Scientists

List of content

1. Intro
2. Implementations
3. Parameter Tuning
4. Posts
5. Talks
6. Notebooks
7. Datasets

Intro

• Wikipedia - Gradient Boosting
• XGBoost - Introduction to Boosted Trees
• Kaggle - a kaggle master explains gradient boosting

Implementations

• XGBoost
  • XGBoost Github
  • XGBoost Documentation
  • XGBoost Paper - XGBoost: A Scalable Tree Boosting System
• LightGBM
  • LightGBM Github
  • LightGBM Documentation
  • LightGBM Paper - LightGBM: A Highly Efficient Gradient Boosting Decision Tree
• Catboost
  • Catboost Github
  • Catboost Documentation
  • Catboost Paper - CatBoost: unbiased boosting with categorical features
• Other
  • sklearn - Gradient Boosting Guide
  • H20 - GBM

Parameter Tuning

• Hyperspace - Hyperparameters Spaces for Optimization
• XGBoost - Complete Guide to Parameter Tuning in XGBoost (with codes in Python)
• LightGBM - parameters tuning guides for different scenarios
• Catboost - some tips on the possible parameter settings

Posts

• Gradient Boosting Explained - by Brilliantly wrong thoughts on science and programming
• Gradient Boosting From Scratch - by ML Review
• CatBoost vs. Light GBM vs. XGBoost - by Alvira Swalin on Towards Data Science
• Mastering The New Generation of Gradient Boosting - by Tal Peretz on Towards Data Science

Talks

• Catboost - Anna Veronika Dorogush on pydata
• Can one do better than XGBoost? - Mateusz Susik on pydata

Notebooks

• Kaggle XGBoost
• Catboost + LightGBM + XGBoost - comparison of the 3 implementations on categorical dataset.

Datasets

• Catboost Datasets

License

To the extent possible under law, Tal Peretz has waived all copyright and related or neighboring rights to this work.