Pinned Loading

1. ModelCloud/GPTQModel ModelCloud/GPTQModel Public

   An easy-to-use LLM quantization and inference toolkit based on GPTQ algorithm (weight-only quantization).

   Python 66 12

2. sgl-project/sglang sgl-project/sglang Public

   SGLang is a fast serving framework for large language models and vision language models.

   Python 4.6k 296

3. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 25.5k 3.7k

4. AutoGPTQ/AutoGPTQ AutoGPTQ/AutoGPTQ Public

   An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

   Python 4.3k 453

5. flashinfer-ai/flashinfer flashinfer-ai/flashinfer Public

   FlashInfer: Kernel Library for LLM Serving

   Cuda 1k 92

6. Dao-AILab/flash-attention Dao-AILab/flash-attention Public

   Fast and memory-efficient exact attention

   Python 13.1k 1.2k