Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Switch to python runner for single GPU #1308

Merged
merged 1 commit into from
Aug 23, 2024

Conversation

xu-song
Copy link
Contributor

@xu-song xu-song commented Jul 11, 2024

Motivation

There is no need to use torchrun for single GPU inference.
Besides, python runner is more friendly for debugging.
The debugging snippet is as follows

# run in subprocess if starts with torchrun etc.
if 'python3 ' in cmd or 'python ' in cmd:
# If it is an infer type task do not reload if
# the current model has already been loaded.
if 'infer' in self.task_cfg.type.lower():
# If a model instance already exists,
# do not reload it.
task.run(cur_model=getattr(self, 'cur_model',
None),
cur_model_abbr=getattr(
self, 'cur_model_abbr', None))
self.cur_model = task.model
self.cur_model_abbr = model_abbr_from_cfg(
task.model_cfg)
else:
task.run()

Copy link
Collaborator

@MaiziXiao MaiziXiao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@MaiziXiao MaiziXiao changed the title Switch to python runner for single GPU [Feature] Switch to python runner for single GPU Aug 23, 2024
@MaiziXiao MaiziXiao merged commit ad3931a into open-compass:main Aug 23, 2024
9 checks passed
liushz pushed a commit to liushz/opencompass that referenced this pull request Sep 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants