Skip to content

Pull requests: huggingface/text-generation-inference

New pull request New
29 Open 1,038 Closed
29 Open 1,038 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

feat: add release and sha tagged images
#2360 opened Aug 5, 2024 by drbh Loading…
1
WIP: Update ROCM libs
#2358 opened Aug 5, 2024 by mht-sharma • Draft
4 tasks done
[TENSORRT-LLM] - Implement new looper thread based backend
#2357 opened Aug 2, 2024 by mfuntowicz • Draft
Add FlashInfer support
#2354 opened Aug 2, 2024 by danieldk Loading…
5 tasks
Unsigned integer underflow in max_batch_size
#2352 opened Aug 1, 2024 by maxdebayser Loading…
fix: fix num_ln_in_parallel_attn attribute name typo in RWConfig
#2350 opened Aug 1, 2024 by almersawi Loading…
3 of 5 tasks
1
fix EleutherAI/gpt-neox-20b does not work in tgi
#2346 opened Aug 1, 2024 by sywangyi Loading…
fix: allocate tmp based on sgmv kernel if available
#2345 opened Jul 31, 2024 by drbh Loading…
More fixes trtllm
#2342 opened Jul 31, 2024 by mfuntowicz Loading…
hotfix: fix xpu crash brought by code refine. torch.xpu rely on impor…
#2337 opened Jul 31, 2024 by sywangyi Loading…
3
fix: improve completions to send a final chunk with usage details
#2336 opened Jul 30, 2024 by drbh Loading…
add numa to improve cpu inference perf
#2330 opened Jul 30, 2024 by sywangyi Loading…
3
Update vLLM dependency to 0.5.3.post1
#2317 opened Jul 26, 2024 by danieldk • Draft
5 tasks
Add model_load_time metric
#2311 opened Jul 26, 2024 by Edwinhr716 Loading…
2 of 5 tasks
no repeat ngram size ci
#2308 opened Jul 25, 2024 by ErikKaum Loading…
8
Fix missing model id in rocm warmup
#2298 opened Jul 24, 2024 by almersawi Loading…
2 of 5 tasks
1
Using HF_HOME instead of CACHE to get token read in addition to models.
#2288 opened Jul 23, 2024 by Narsil Loading…
5 tasks
feat(router): drop permit after batching
#2284 opened Jul 23, 2024 by OlivierDehaene • Draft
adding max_token_capacity metric
#2279 opened Jul 22, 2024 by Edwinhr716 Loading…
2 of 5 tasks
1
feat: refactor lora linear and remove adapter layers
#2251 opened Jul 18, 2024 by drbh • Draft
Feature: During Generation add support for no_repeat_ngram_size
#2232 opened Jul 15, 2024 by njbrake Loading…
2 tasks
1
@ErikKaum
6
Don't error on OpenAI valid top_p values.
#2231 opened Jul 15, 2024 by ErikKaum Loading…
@ErikKaum
7
doc: Add metrics documentation and add a 'Reference' section documentation Improvements or additions to documentation
#2230 opened Jul 15, 2024 by Hugoch Loading…
2 of 5 tasks
@Hugoch
1
feat: Add load tests
#2217 opened Jul 11, 2024 by Hugoch Loading…
1 of 5 tasks
@Hugoch
9
added tie_weights support to mlp speculator
#2215 opened Jul 10, 2024 by JRosenkranz Loading…
5 tasks
ProTip! What’s not been updated in a month: updated:<2024-07-06.