Skip to content

Welcome to the Mixtral 8x7B offloading demo repository! This project aims to demonstrate the seamless execution of Mixtral-8x7B models on Colab or consumer desktops.

License

Notifications You must be signed in to change notification settings

tushar2704/ColabMixtral

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Colab Mixtral 8x7B Offloading Demo

Welcome to the Mixtral 8x7B offloading demo repository! This project aims to demonstrate the seamless execution of Mixtral-8x7B models on Colab or consumer desktops.

Screenshot 2023-12-30 191022

Getting Started

To explore and experience the demo, follow these steps:

  1. Open the demo notebook located at ./notebooks/demo.ipynb.

  2. Follow the instructions within the notebook to run the Mixtral-8x7B offloading demo.

Overview

This repository provides a hands-on experience for users interested in leveraging Mixtral-8x7B models for various applications. The demo notebook serves as a comprehensive guide, offering insights into the capabilities of Mixtral-8x7B and providing a platform for experimentation.

Requirements

Make sure you have the necessary dependencies installed before running the demo. Refer to the notebook for specific requirements.

Issues and Contributions

If you encounter any issues or have suggestions for improvement, feel free to open an issue or contribute to the project. We welcome collaboration from the community to enhance this demo.

Thank you for exploring the Mixtral 8x7B offloading demo! Enjoy experimenting with the capabilities of Mixtral-8x7B in your projects.

Refrence:

1)https://arxiv.org/abs/2312.17238

Upcoming application to demonstrate!

About

Welcome to the Mixtral 8x7B offloading demo repository! This project aims to demonstrate the seamless execution of Mixtral-8x7B models on Colab or consumer desktops.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published