Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/sg 747 support predict video full pipeline master #829

Merged

Conversation

Louis-Dupont
Copy link
Contributor

@Louis-Dupont Louis-Dupont commented Apr 10, 2023

Adding support for image folder and video as part of the base predict function.
Also adding support for predict_webcam.
This also introduced some refactoring on the Pipeline to support lazy batch processing.

@BloodAxe
Copy link
Collaborator

Regarding processing video/image folder, I wonder whether we can extract common logic via generators:

def process_series(self, images: Iterable[np.ndarray]) -> Iterable[Result]:
   for image in images:
      yield self.process(image)

So processing logic for a collection of images is in one place, regardless what if it is an image folder or video.

So now even if we want to process and save entire LoTR movie we can do this without storing all frames in memory:

class VideoFileIterator:
  def __init__(self, input_path):
     self.cap = cv2.VideoCapture(input_path)

  def __next__(self):
     success, frame = self.cap.grab()
     if not success:
        raise StopIteration()
     return frame

def process_video(self, input_path, output_path):
    input = VideoFileIterator(input_path)
    output = VideoWriter(output_path)
    for result in process_series(input):
        output.write_frame(result)

Copy link
Collaborator

@shaydeci shaydeci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 small comment then LGTM on my end.

src/super_gradients/training/transforms/processing.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@shaydeci shaydeci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Louis-Dupont Louis-Dupont merged commit a114860 into master Apr 16, 2023
@Louis-Dupont Louis-Dupont deleted the feature/SG-747-support_predict_video_full_pipeline_master branch April 16, 2023 09:15
Louis-Dupont added a commit that referenced this pull request Apr 17, 2023
* wip

* move to imageprocessors

* wip

* add back changes

* making it work fully for yolox and almost for ppyoloe

* minor change

* working for det

* cleaning

* clean

* undo

* replace empty with none

* add _get_shift_params

* minor doc change

* cleaning wip

* working for multiple images

* add ppyoloe

* replace pydantic with dataclasses and fix typing

* add docstrings

* doc improvment and use get_shift_params in transforms

* add tests

* improve comment

* rename

* wip

* add option to keep ratio in rescale

* make functions private

* remove DetectionPaddedRescale

* fix doc

* big commit with wrong things

* try undo bad change

* doc

* minor doc

* add a lot of doc

* fix comment

* minor change

* first draft of load_video

* adding save_video, some parts are still to be checked

* wip

* add __init__.py to pipelines

* replace size with shape

* wip

* cleaning

* wip

* fix rgb to bgr and remove check

* almost working, missing batch

* proposal of predict_video

* wip working on dete

* add yolox

* add flag to visualize

* update

* add streaming

* improve streaming code

* docstring update

* fix stream example

* rename Results

* cleaning

* rename stream to predict_webcam

* doc fixes

* improve docstring and homogenize some names

* rename _images_prediction_lst

* improve doc

* add doc

* minore change

* fix image

* fix ci

* fix merge

* reverse channel properly
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants