v0.4.11

What's Changed

• Add distribute=None to python-api.md by @rasbt in #1676
• Make LitGPT LLM API compatible with PyTorch Lightning Trainer 1/2 by @rasbt in #1667
• Auto device handling in LLM API by @rasbt in #1677
• Fix KV cache issue in LLM API by @rasbt in #1678
• Improved benchmark utils by @rasbt in #1679
• Add PR benchmark util for internal use by @rasbt in #1680
• Added git hash to benchmark utility. by @apaz-cli in #1681
• Spelling fix by @rasbt in #1685
• Add Microsoft Phi 3.5 checkpoint by @rasbt in #1687
• Update check_nvlink_connectivity by @sanderland in #1684
• Make number of generated tokens consistent with CLI by @rasbt in #1690
• Avoid error when executing benchmark util outside a git folder by @rasbt in #1691
• Combine generate() functions by @apaz-cli in #1675
• Bumb version to 0.4.11 by @rasbt in #1695
• Fix falcon prompt template by @rasbt in #1696

New Contributors

• @sanderland made their first contribution in #1684

Full Changelog: v0.4.10...v0.4.11

v0.4.10

What's Changed

• Support Tensor Parallel in Python API by @rasbt in #1661
• Swap old Llama model with Phi-3 by @rasbt in #1666
• Update azure-gpu-test.yml by @rasbt in #1669
• Support the refactored API in litgpt serve by @rasbt in #1668
• Multi-gpu serving by @rasbt in #1670
• Add Mistral Large 123B by @rasbt in #1673
• Bumb version to 0.4.10 for next release by @rasbt in #1674

Full Changelog: v0.4.9...v0.4.10

v0.4.9

What's Changed

• Update LitServe version and tests by @rasbt in #1654
• Support for using large models in the Python API via sequential generation by @rasbt in #1637
• Add a PyTorch Lightning example by @rasbt in #1656
• Refactor Python API to introduce new distribute method (part of a larger refactor for PTL support) by @rasbt in #1657
• Fix some issues with circular and relative imports by @rasbt in #1658
• Optionally return benchmark info in Python API by @rasbt in #1660
• Bumb version for 0.4.9 release by @rasbt in #1664

Full Changelog: v0.4.8...v0.4.9

v0.4.8

What's Changed

• Adds unit test to test for parity between streaming and non-streaming API by @rasbt in #1650
• Add Gemma 2 2B by @rasbt in #1651
• Pin litserve version by @rasbt in #1652
• Version bumb for Gemma 2 2B release by @rasbt in #1653

Full Changelog: v0.4.7...v0.4.8

v0.4.7

What's Changed

• Apply prompt style for tp.py and sequentially.py by @Andrei-Aksionov in #1629
• Fix prompt docstring in Python API by @rasbt in #1635
• Update windows cpu-tests.yml by @rasbt in #1630
• Remove NumPy < 2.0 pin by @rasbt in #1631
• Fix kv-cache issue in Python API streaming mode by @rasbt in #1633
• Updates installation requirements to install minimal required packages for basic use by @rasbt in #1634
• Faster safetensors conversion when downloading model by @awaelchli in #1624
• Add Sebastian as code owner by @awaelchli in #1641
• Add missing super() call in data modules by @awaelchli in #1639
• Update Lightning version to 2.4.0 pre by @awaelchli in #1640
• Add tunable kvcache with error handling for nonsense inputs. by @apaz-cli in #1636
• Use Python API in serve code by @rasbt in #1644
• Fix autodownload + conversion issue by @rasbt in #1645
• Properly clear kv-cache by @rasbt in #1647
• Fix error raising where max_returned_tokens > max_seq_length_setting by @rasbt in #1648
• Add quantization support to litgpt serve by @rasbt in #1646
• Bump for 0.4.7 release by @rasbt in #1649

Full Changelog: v0.4.6...v0.4.7

v0.4.6

What's Changed

• Change default top_k to 50 everywhere for consistency by @rasbt in #1592
• Fix kv-cache clearing in Python API and Serve by @rasbt in #1596
• dynamic KV Cache batching by @aniketmaurya in #1600
• Remove non-used eos_id in Python API by @rasbt in #1594
• Add quantization test and revert lightning version by @rasbt in #1605
• Dynamically set kv-cache size in serve by @rasbt in #1602
• Update LitData version and restore previous LitData assertions in tests by @awaelchli in #1609
• Gemma 2: 9b and 27b versions by @Andrei-Aksionov in #1545
• Update config hub table qlora sections by @rasbt in #1611
• max_returned_tokens -> max_new_tokens by @rasbt in #1612
• Add warning about pretrain preprocessing by @rasbt in #1618
• Print warning about unsupported repo_ids by @rasbt in #1617
• Restore capability to load alternative weights by @rasbt in #1620
• Enable unbalanced number of layers in sequential generation by @awaelchli in #1623
• Llama 3.1 8B and 70B checkpoints by @rasbt in #1619
• Add Llama 3.1 405B config by @awaelchli in #1622
• Bumb version to 0.4.6 for next release (Gemma 2 and Llama 3.1) by @rasbt in #1626

Full Changelog: v0.4.5...v0.4.6

v0.4.5

What's Changed

• Update LitData to latest version 0.2.16 by @awaelchli in #1573
• Fixed warning during model download and conversion. by @bhargavyagnik in #1574
• Bump lightning version by @awaelchli in #1579
• chat/base.py: extend checkpoint_dir before accessing it by @Andrei-Aksionov in #1575
• Chat: dynamically set kv-cache size by @Andrei-Aksionov in #1583
• Add slow interconnect warning by @rasbt in #1577
• Mathstral checkpoints by @rasbt in #1587
• Make sure quantization is used correctly by @rasbt in #1585
• Improve mathstral tests by @rasbt in #1589
• Fixes an issue where the LitGPT Python API was consuming too much memory by @rasbt in #1590
• Increase version number for 0.4.5 release by @rasbt in #1591

New Contributors

• @bhargavyagnik made their first contribution in #1574

Full Changelog: v0.4.4...v0.4.5

v0.4.4

What's Changed

• Tokenizer: fix issue when decoding a single token at a time by @Andrei-Aksionov in #1559
• Filter out false positive PyTorch scheduler warning by @rasbt in #1562
• Add an explanation on how to evaluate on a custom test set by @rasbt in #1564
• Remove duplicated bos_token for CodeLlama by @alealv in #1566
• Fix SFTDataset sample generation by @alealv in #1568
• Add Python API section to 0 to LitGPT docs by @rasbt in #1570
• Fix multi-turn prompting error handling and extra newline by @apaz-cli in #1569
• Add automatic downloading to CLI by @rasbt in #1571
• Bumb version for 0.4.4 release by @rasbt in #1572

New Contributors

• @alealv made their first contribution in #1566
• @apaz-cli made their first contribution in #1569

Full Changelog: v0.4.3...v0.4.4

v0.4.3

What's Changed

• Fix Twine Build Issue by @rasbt in #1530
• Fix Mixtral prompt style by @janEbert in #1531
• Raise warning when loading a large model on a CPU device by @rasbt in #1532
• Tutorial convert_from_litgpt doc fix to remove output_dir command argument by @jentfoo in #1533
• Test against eager attention on GPTNeoX by @rasbt in #1537
• Bump to 0.4.3.dev0 version by @rasbt in #1536
• Fix for LM Eval harness 0.4.3 by @rasbt in #1542
• Add Phi-3-mini-4k-instruct checkpoint by @rasbt in #1341
• Update phi-3 prompt template by @rasbt in #1544
• Add progress bar for weight loading by @rasbt in #1534
• Remove redundant code in checkpoint conversion script by @rasbt in #1547
• Update how evaluate tasks are listed by @rasbt in #1550
• Fix generate issue in finetuning script when finetuning with small datasets by @rasbt in #1551
• Tokenizer: check that bos and eos tokens are applied w.r.t. to a provided config/args by @Andrei-Aksionov in #1527
• Add num_nodes argument by @janEbert in #1546
• Add Phi-3 Configs by @rasbt in #1553
• Update to version 0.4.3 for new release by @rasbt in #1554

New Contributors

• @janEbert made their first contribution in #1531
• @jentfoo made their first contribution in #1533

Full Changelog: v0.4.2...v0.4.3

v0.4.2

What's Changed

• Disable "validating ..." comment if initial_validation is False by @rasbt in #1499
• Add simple issue templates by @rasbt in #1501
• Add note about dependencies by @rasbt in #1502
• Update test_readme.py by @Andrei-Aksionov in #1505
• Print message when unnecessary can't be deleted by @rasbt in #1507
• Bumb version by @rasbt in #1509
• LitGPT Python API v1 by @rasbt in #1463
• Fix pip upgrade command on windows by @rasbt in #1513
• add LLM.generate with streaming by @aniketmaurya in #1511
• Fix and update Python API docs by @rasbt in #1514
• Update LLM import and make stream/non-stream option consistent by @rasbt in #1515
• Extend Python API to initialize model with random weights by @rasbt in #1512
• Add Python API download option by @rasbt in #1516
• Add Rope group size as constexpr to fix thunder tests by @rasbt in #1520
• Explain how to access pretraining dataset options by @rasbt in #1510
• Bump thunder by @rasbt in #1519
• Add automatic checkpoint downloading in Python API by @rasbt in #1521
• Tokenizer: prefer HF Tokenizer by @Andrei-Aksionov in #1518
• iterate response from streaming server by @aniketmaurya in #1523
• Add new python API to readme by @williamFalcon in #1525
• Remove init="local" option by @rasbt in #1524
• Readme improvements by @rasbt in #1526
• Add evaluate to Readme by @rasbt in #1528

Full Changelog: v0.4.1...v0.4.2