Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lmdeploy tis python backend model #1014

Merged
merged 3 commits into from
Apr 23, 2024

Conversation

ispobock
Copy link
Contributor

@ispobock ispobock commented Apr 1, 2024

Motivation

Propose evaluation code for LMDeploy TIS Python backend.

Modification

Add evaluation model and config files.

Use cases

python run.py configs/eval_internlm_chat_lmdeploy_tis.py -w outputs

Checklist

Before PR:

  • Pre-commit or other linting tools are used to fix the potential lint issues.
  • Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
  • The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

  • If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects.
  • CLA has been signed and all committers have signed the CLA in this PR.

@ispobock
Copy link
Contributor Author

ispobock commented Apr 1, 2024

@lvhan028 @Leymore Could you help review?

@ispobock
Copy link
Contributor Author

ispobock commented Apr 7, 2024

Hi @lvhan028, this is the evaluation code for TIS python backend. Shall we add it here?

@lvhan028
Copy link
Contributor

lvhan028 commented Apr 7, 2024

Hi @lvhan028, this is the evaluation code for TIS python backend. Shall we add it here?

yes. I'll review it tomorrow. Thanks a lot

@tonysy
Copy link
Collaborator

tonysy commented Apr 16, 2024

Hi @lvhan028, this is the evaluation code for TIS python backend. Shall we add it here?

yes. I'll review it tomorrow. Thanks a lot

@lvhan028 Can we merge this PR currently?

@lvhan028
Copy link
Contributor

Oh, sorry.
My bad. I am going to review it right now

@lvhan028
Copy link
Contributor

LGTM @tonysy
Thanks a lot @ispobock and sorry for the late response

@ispobock
Copy link
Contributor Author

Hi @tonysy Could you help merge this PR?

Copy link
Collaborator

@tonysy tonysy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
Also it would be great if you can update the documentation .

@@ -0,0 +1,41 @@
from mmengine.config import read_base
from opencompass.models.lmdeploy_tis import LmdeployTisModel
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you like to update the documentation to prodive more introduction of different backend? @ispobock @lvhan028

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lvhan028 Shall we add a note for this backend here?

- If you evaluate the InternLM 7B model, please modify `eval_internlm_turbomind.py` or `eval_internlm_chat_turbomind.py` by changing to the setting `models = [internlm_7b]` in the last line.

@tonysy tonysy merged commit 81d0e4d into open-compass:main Apr 23, 2024
9 checks passed
liuyaox pushed a commit to liuyaox/opencompass that referenced this pull request Jun 26, 2024
* add lmdeploy tis python backend model

* fix pr check

* update
Leymore pushed a commit to Leymore/opencompass that referenced this pull request Jul 12, 2024
* add lmdeploy tis python backend model

* fix pr check

* update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants