-
Notifications
You must be signed in to change notification settings - Fork 168
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[KV Cache Interface] Text Generation & Decoder Engine Implementation #1089
[KV Cache Interface] Text Generation & Decoder Engine Implementation #1089
Commits on Jun 5, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 48ac0ac - Browse repository at this point
Copy the full SHA 48ac0acView commit details -
Configuration menu - View commit details
-
Copy full SHA for cf7f2b9 - Browse repository at this point
Copy the full SHA cf7f2b9View commit details
Commits on Jun 6, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 832630a - Browse repository at this point
Copy the full SHA 832630aView commit details
Commits on Jun 7, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 9958c83 - Browse repository at this point
Copy the full SHA 9958c83View commit details -
Configuration menu - View commit details
-
Copy full SHA for e6d2b03 - Browse repository at this point
Copy the full SHA e6d2b03View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7f9935b - Browse repository at this point
Copy the full SHA 7f9935bView commit details
Commits on Jun 8, 2023
-
Configuration menu - View commit details
-
Copy full SHA for b1cf01b - Browse repository at this point
Copy the full SHA b1cf01bView commit details -
[Codegen][ORT][Static Seq Length] TextGenerationPipeline (#946)
* initial commit * coreys simplifications * finishing the second model static * ready, time for beautification * ready for review * moved the code to examples * fix eos logic * add argument num_tokens_to_generate
Configuration menu - View commit details
-
Copy full SHA for 0a3f48d - Browse repository at this point
Copy the full SHA 0a3f48dView commit details -
[CodeGen][Documentation] (#956)
* initial commit * coreys simplifications * finishing the second model static * ready, time for beautification * ready for review * moved the code to examples * fix eos logic * add argument num_tokens_to_generate * initial commit * change order * Update examples/codegen/README.md Co-authored-by: corey-nm <109536191+corey-nm@users.noreply.github.com> --------- Co-authored-by: corey-nm <109536191+corey-nm@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for add4625 - Browse repository at this point
Copy the full SHA add4625View commit details -
Configuration menu - View commit details
-
Copy full SHA for 22d2746 - Browse repository at this point
Copy the full SHA 22d2746View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7f1651d - Browse repository at this point
Copy the full SHA 7f1651dView commit details -
Configuration menu - View commit details
-
Copy full SHA for b85746d - Browse repository at this point
Copy the full SHA b85746dView commit details -
Configuration menu - View commit details
-
Copy full SHA for aadc608 - Browse repository at this point
Copy the full SHA aadc608View commit details -
Pipeline fully refactored, time to test engine support. Note: Sliding…
… window not yet implemented!
Configuration menu - View commit details
-
Copy full SHA for 58bc2b0 - Browse repository at this point
Copy the full SHA 58bc2b0View commit details -
Configuration menu - View commit details
-
Copy full SHA for d538444 - Browse repository at this point
Copy the full SHA d538444View commit details -
Configuration menu - View commit details
-
Copy full SHA for e19676b - Browse repository at this point
Copy the full SHA e19676bView commit details -
ORT agrees with the Engine. But they both give not entirely correct r…
…esult. Hey, this is good news still
Configuration menu - View commit details
-
Copy full SHA for 7908b74 - Browse repository at this point
Copy the full SHA 7908b74View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4bc3472 - Browse repository at this point
Copy the full SHA 4bc3472View commit details -
Configuration menu - View commit details
-
Copy full SHA for c07f7ed - Browse repository at this point
Copy the full SHA c07f7edView commit details -
Configuration menu - View commit details
-
Copy full SHA for fb77838 - Browse repository at this point
Copy the full SHA fb77838View commit details -
adjust shapes and slicing to enable static autoregressive pass - ISSU…
…E: tokens past the base seq len are repeated
Configuration menu - View commit details
-
Copy full SHA for 2097463 - Browse repository at this point
Copy the full SHA 2097463View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5eb10a9 - Browse repository at this point
Copy the full SHA 5eb10a9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9213f29 - Browse repository at this point
Copy the full SHA 9213f29View commit details -
Configuration menu - View commit details
-
Copy full SHA for d9af004 - Browse repository at this point
Copy the full SHA d9af004View commit details -
Configuration menu - View commit details
-
Copy full SHA for 476f25d - Browse repository at this point
Copy the full SHA 476f25dView commit details -
Configuration menu - View commit details
-
Copy full SHA for fab44e4 - Browse repository at this point
Copy the full SHA fab44e4View commit details -
Configuration menu - View commit details
-
Copy full SHA for d454e2f - Browse repository at this point
Copy the full SHA d454e2fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1613e25 - Browse repository at this point
Copy the full SHA 1613e25View commit details -
Stop saving tmp files, otherwise the engine looks for external files …
…in the wrong place
Configuration menu - View commit details
-
Copy full SHA for b61055c - Browse repository at this point
Copy the full SHA b61055cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6ee25fc - Browse repository at this point
Copy the full SHA 6ee25fcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5d3004b - Browse repository at this point
Copy the full SHA 5d3004bView commit details -
Configuration menu - View commit details
-
Copy full SHA for ace6fa5 - Browse repository at this point
Copy the full SHA ace6fa5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 388586d - Browse repository at this point
Copy the full SHA 388586dView commit details -
Configuration menu - View commit details
-
Copy full SHA for afd0139 - Browse repository at this point
Copy the full SHA afd0139View commit details -
Configuration menu - View commit details
-
Copy full SHA for 30eeda7 - Browse repository at this point
Copy the full SHA 30eeda7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5882b56 - Browse repository at this point
Copy the full SHA 5882b56View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4bbe33d - Browse repository at this point
Copy the full SHA 4bbe33dView commit details -
Configuration menu - View commit details
-
Copy full SHA for afa5746 - Browse repository at this point
Copy the full SHA afa5746View commit details -
Configuration menu - View commit details
-
Copy full SHA for e2bb78c - Browse repository at this point
Copy the full SHA e2bb78cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2299009 - Browse repository at this point
Copy the full SHA 2299009View commit details
Commits on Jun 9, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 2935b77 - Browse repository at this point
Copy the full SHA 2935b77View commit details
Commits on Jun 11, 2023
-
Configuration menu - View commit details
-
Copy full SHA for b89b156 - Browse repository at this point
Copy the full SHA b89b156View commit details
Commits on Jun 13, 2023
-
Configuration menu - View commit details
-
Copy full SHA for dc3d61b - Browse repository at this point
Copy the full SHA dc3d61bView commit details -
Configuration menu - View commit details
-
Copy full SHA for a294265 - Browse repository at this point
Copy the full SHA a294265View commit details -
Configuration menu - View commit details
-
Copy full SHA for af97f2b - Browse repository at this point
Copy the full SHA af97f2bView commit details -
Configuration menu - View commit details
-
Copy full SHA for c117788 - Browse repository at this point
Copy the full SHA c117788View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4ad5f49 - Browse repository at this point
Copy the full SHA 4ad5f49View commit details -
Merge branch 'feature/damian/do_not_save_to_tmp' of https://github.co…
…m/neuralmagic/deepsparse into feature/damian/do_not_save_to_tmp
Configuration menu - View commit details
-
Copy full SHA for 9e816bb - Browse repository at this point
Copy the full SHA 9e816bbView commit details -
Configuration menu - View commit details
-
Copy full SHA for f97467f - Browse repository at this point
Copy the full SHA f97467fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6be8d87 - Browse repository at this point
Copy the full SHA 6be8d87View commit details -
Configuration menu - View commit details
-
Copy full SHA for e2f088d - Browse repository at this point
Copy the full SHA e2f088dView commit details -
Merge remote-tracking branch 'origin/feature/damian/do_not_save_to_tm…
…p' into feature/damian/codegen_pipeline_clean
Configuration menu - View commit details
-
Copy full SHA for 9fc6c64 - Browse repository at this point
Copy the full SHA 9fc6c64View commit details -
Configuration menu - View commit details
-
Copy full SHA for a610faf - Browse repository at this point
Copy the full SHA a610fafView commit details -
Configuration menu - View commit details
-
Copy full SHA for 347d1fb - Browse repository at this point
Copy the full SHA 347d1fbView commit details -
Configuration menu - View commit details
-
Copy full SHA for e11027c - Browse repository at this point
Copy the full SHA e11027cView commit details -
Configuration menu - View commit details
-
Copy full SHA for a950910 - Browse repository at this point
Copy the full SHA a950910View commit details -
Configuration menu - View commit details
-
Copy full SHA for c1d02dc - Browse repository at this point
Copy the full SHA c1d02dcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 711cdfb - Browse repository at this point
Copy the full SHA 711cdfbView commit details
Commits on Jun 14, 2023
-
Configuration menu - View commit details
-
Copy full SHA for e602662 - Browse repository at this point
Copy the full SHA e602662View commit details
Commits on Jun 16, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 06b5246 - Browse repository at this point
Copy the full SHA 06b5246View commit details
Commits on Jun 21, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 5d59d23 - Browse repository at this point
Copy the full SHA 5d59d23View commit details -
Configuration menu - View commit details
-
Copy full SHA for 765a5f7 - Browse repository at this point
Copy the full SHA 765a5f7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 15586a4 - Browse repository at this point
Copy the full SHA 15586a4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4d35779 - Browse repository at this point
Copy the full SHA 4d35779View commit details -
Configuration menu - View commit details
-
Copy full SHA for 775c648 - Browse repository at this point
Copy the full SHA 775c648View commit details -
Merge branch 'feature/damian/codegen_pipeline_clean' of https://githu…
…b.com/neuralmagic/deepsparse into feature/damian/codegen_pipeline_clean
Configuration menu - View commit details
-
Copy full SHA for 54aec69 - Browse repository at this point
Copy the full SHA 54aec69View commit details -
Configuration menu - View commit details
-
Copy full SHA for 25cdd38 - Browse repository at this point
Copy the full SHA 25cdd38View commit details
Commits on Jun 22, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 830a85e - Browse repository at this point
Copy the full SHA 830a85eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 388e7ab - Browse repository at this point
Copy the full SHA 388e7abView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3970a7a - Browse repository at this point
Copy the full SHA 3970a7aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7cdf939 - Browse repository at this point
Copy the full SHA 7cdf939View commit details -
Configuration menu - View commit details
-
Copy full SHA for 950c653 - Browse repository at this point
Copy the full SHA 950c653View commit details -
Configuration menu - View commit details
-
Copy full SHA for ea82e99 - Browse repository at this point
Copy the full SHA ea82e99View commit details -
Revert "remove files to make review easier"
This reverts commit ea82e99.
Configuration menu - View commit details
-
Copy full SHA for 016cac1 - Browse repository at this point
Copy the full SHA 016cac1View commit details -
Merge DecoderKVCache with KVCacheORT (KVCacheORT will not exist, it i…
…s just an abstraction) * rebase * add tests
Configuration menu - View commit details
-
Copy full SHA for c6efccd - Browse repository at this point
Copy the full SHA c6efccdView commit details -
Configuration menu - View commit details
-
Copy full SHA for a19cf2e - Browse repository at this point
Copy the full SHA a19cf2eView commit details -
Configuration menu - View commit details
-
Copy full SHA for c59da37 - Browse repository at this point
Copy the full SHA c59da37View commit details -
DecoderKVCache that manipulates cache state and additionally passes i…
…nfo to the engine via KVCache object
Configuration menu - View commit details
-
Copy full SHA for 6d40c03 - Browse repository at this point
Copy the full SHA 6d40c03View commit details
Commits on Jun 23, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 741f452 - Browse repository at this point
Copy the full SHA 741f452View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7b27abe - Browse repository at this point
Copy the full SHA 7b27abeView commit details
Commits on Jun 26, 2023
-
Configuration menu - View commit details
-
Copy full SHA for db6b54b - Browse repository at this point
Copy the full SHA db6b54bView commit details -
Configuration menu - View commit details
-
Copy full SHA for b3fb3b8 - Browse repository at this point
Copy the full SHA b3fb3b8View commit details -
Merge remote-tracking branch 'origin/feature/damian/kv_cache_ort' int…
…o feature/damian/decoder_engine
Configuration menu - View commit details
-
Copy full SHA for 47c0c4b - Browse repository at this point
Copy the full SHA 47c0c4bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 76e332d - Browse repository at this point
Copy the full SHA 76e332dView commit details -
Merge remote-tracking branch 'origin/feature/damian/kv_cache_ort' int…
…o feature/damian/decoder_engine
Configuration menu - View commit details
-
Copy full SHA for 4791ed3 - Browse repository at this point
Copy the full SHA 4791ed3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8c5734b - Browse repository at this point
Copy the full SHA 8c5734bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6c5daab - Browse repository at this point
Copy the full SHA 6c5daabView commit details
Commits on Jun 27, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 812408c - Browse repository at this point
Copy the full SHA 812408cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 952abda - Browse repository at this point
Copy the full SHA 952abdaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2ff4987 - Browse repository at this point
Copy the full SHA 2ff4987View commit details -
Configuration menu - View commit details
-
Copy full SHA for 725a210 - Browse repository at this point
Copy the full SHA 725a210View commit details -
Configuration menu - View commit details
-
Copy full SHA for 108596e - Browse repository at this point
Copy the full SHA 108596eView commit details -
Configuration menu - View commit details
-
Copy full SHA for f6a9baf - Browse repository at this point
Copy the full SHA f6a9bafView commit details -
Configuration menu - View commit details
-
Copy full SHA for b25886a - Browse repository at this point
Copy the full SHA b25886aView commit details -
Merge remote-tracking branch 'origin/feature/damian/kv_cache_ort' int…
…o feature/damian/decoder_engine
Configuration menu - View commit details
-
Copy full SHA for 53d7b70 - Browse repository at this point
Copy the full SHA 53d7b70View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0b0f74a - Browse repository at this point
Copy the full SHA 0b0f74aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4d6860a - Browse repository at this point
Copy the full SHA 4d6860aView commit details -
Configuration menu - View commit details
-
Copy full SHA for d68f045 - Browse repository at this point
Copy the full SHA d68f045View commit details
Commits on Jun 28, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 759dc93 - Browse repository at this point
Copy the full SHA 759dc93View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3e1d32f - Browse repository at this point
Copy the full SHA 3e1d32fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4c39d7f - Browse repository at this point
Copy the full SHA 4c39d7fView commit details