Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add autoware_rosbag2_anonymizer usage #557

Merged

Conversation

StepTurtle
Copy link
Contributor

@StepTurtle StepTurtle commented May 24, 2024

Description

This PR updates the dataset documentation and adds instructions for the data anonymizer tool.

Related issue:

Repo Link:

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The Reviewers must check the checkboxes below before approval.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

Signed-off-by: Barış Zeren <bzeren1819@gmail.com>
@StepTurtle StepTurtle added the tag:deploy-docs Mark for deploy-docs action generation. (used-by-ci) label May 24, 2024
@StepTurtle StepTurtle self-assigned this May 24, 2024
docs/datasets/data-anonymization/index.md Outdated Show resolved Hide resolved
docs/datasets/data-anonymization/index.md Outdated Show resolved Hide resolved
docs/datasets/data-anonymization/index.md Outdated Show resolved Hide resolved
docs/datasets/data-anonymization/index.md Outdated Show resolved Hide resolved
docs/datasets/data-anonymization/index.md Outdated Show resolved Hide resolved
Signed-off-by: Barış Zeren <bzeren1819@gmail.com>
…utoware-documentation into feat/add_rosbag2_anonymizer
Signed-off-by: Barış Zeren <bzeren1819@gmail.com>
@StepTurtle StepTurtle marked this pull request as ready for review May 29, 2024 07:57
Signed-off-by: Barış Zeren <bzeren1819@gmail.com>
@mitsudome-r
Copy link
Member

I will test this tool this week.

@mitsudome-r
Copy link
Member

@StepTurtle
I'm trying to follow the instructions, but I had to install the following additional dependencies to run python3 main.py config/anonymize_with_unified_model.yaml --anonymize_with_unified_model:

  • open_clip_torch
  • groundingdino-py
  • ultralytics

@StepTurtle
Copy link
Contributor Author

StepTurtle commented Aug 6, 2024

@StepTurtle I'm trying to follow the instructions, but I had to install the following additional dependencies to run python3 main.py config/anonymize_with_unified_model.yaml --anonymize_with_unified_model:

* open_clip_torch

* groundingdino-py

* ultralytics

Thanks @mitsudome-r for your reply, can you share output of the python3 -m pip install . command?

@mitsudome-r
Copy link
Member

@StepTurtle
Here's the output:

Defaulting to user installation because normal site-packages is not writeable
Processing /home/mitsudome-r/autoware_rosbag2_anonymizer
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: cv-bridge in /opt/ros/humble/local/lib/python3.10/dist-packages (from autoware_rosbag2_anonymizer==0.1.0) (3.2.1)
Requirement already satisfied: huggingface-hub in /home/mitsudome-r/.local/lib/python3.10/site-packages (from autoware_rosbag2_anonymizer==0.1.0) (0.24.5)
Requirement already satisfied: numpy in /home/mitsudome-r/.local/lib/python3.10/site-packages (from autoware_rosbag2_anonymizer==0.1.0) (1.26.4)
Requirement already satisfied: opencv-python in /home/mitsudome-r/.local/lib/python3.10/site-packages (from autoware_rosbag2_anonymizer==0.1.0) (4.10.0.84)
Requirement already satisfied: Pillow in /home/mitsudome-r/.local/lib/python3.10/site-packages (from autoware_rosbag2_anonymizer==0.1.0) (10.4.0)
Requirement already satisfied: PyYAML in /home/mitsudome-r/.local/lib/python3.10/site-packages (from autoware_rosbag2_anonymizer==0.1.0) (6.0.1)
Requirement already satisfied: segment-anything in /home/mitsudome-r/.local/lib/python3.10/site-packages (from autoware_rosbag2_anonymizer==0.1.0) (1.0)
Requirement already satisfied: supervision>0.18.0 in /home/mitsudome-r/.local/lib/python3.10/site-packages (from autoware_rosbag2_anonymizer==0.1.0) (0.22.0)
Requirement already satisfied: torch in /home/mitsudome-r/.local/lib/python3.10/site-packages (from autoware_rosbag2_anonymizer==0.1.0) (2.4.0)
Requirement already satisfied: torchvision in /home/mitsudome-r/.local/lib/python3.10/site-packages (from autoware_rosbag2_anonymizer==0.1.0) (0.19.0)
Collecting open-clip-torch (from autoware_rosbag2_anonymizer==0.1.0)
  Using cached open_clip_torch-2.26.1-py3-none-any.whl.metadata (31 kB)
Collecting groundingdino-py (from autoware_rosbag2_anonymizer==0.1.0)
  Using cached groundingdino_py-0.4.0-py2.py3-none-any.whl
Requirement already satisfied: defusedxml<0.8.0,>=0.7.1 in /usr/lib/python3/dist-packages (from supervision>0.18.0->autoware_rosbag2_anonymizer==0.1.0) (0.7.1)
Requirement already satisfied: matplotlib>=3.6.0 in /home/mitsudome-r/.local/lib/python3.10/site-packages (from supervision>0.18.0->autoware_rosbag2_anonymizer==0.1.0) (3.9.0)
Requirement already satisfied: opencv-python-headless>=4.5.5.64 in /home/mitsudome-r/.local/lib/python3.10/site-packages (from supervision>0.18.0->autoware_rosbag2_anonymizer==0.1.0) (4.10.0.84)
Requirement already satisfied: scipy<2.0.0,>=1.10.0 in /home/mitsudome-r/.local/lib/python3.10/site-packages (from supervision>0.18.0->autoware_rosbag2_anonymizer==0.1.0) (1.12.0)
Requirement already satisfied: addict in /home/mitsudome-r/.local/lib/python3.10/site-packages (from groundingdino-py->autoware_rosbag2_anonymizer==0.1.0) (2.4.0)
Requirement already satisfied: pycocotools in /home/mitsudome-r/.local/lib/python3.10/site-packages (from groundingdino-py->autoware_rosbag2_anonymizer==0.1.0) (2.0.8)
INFO: pip is looking at multiple versions of groundingdino-py to determine which version is compatible with other requirements. This could take a while.
  Using cached groundingdino-py-0.3.0.tar.gz (82 kB)
  Preparing metadata (setup.py) ... done
  Using cached groundingdino-py-0.2.0.tar.gz (82 kB)
  Preparing metadata (setup.py) ... done
  Using cached groundingdino-py-0.1.2.tar.gz (82 kB)
  Preparing metadata (setup.py) ... done
  Using cached groundingdino-py-0.1.0.tar.gz (78 kB)
  Preparing metadata (setup.py) ... done
ERROR: Cannot install autoware-rosbag2-anonymizer==0.1.0 and autoware_rosbag2_anonymizer because these package versions have conflicting dependencies.

The conflict is caused by:
    autoware-rosbag2-anonymizer 0.1.0 depends on supervision>0.18.0
    groundingdino-py 0.4.0 depends on supervision==0.6.0
    autoware-rosbag2-anonymizer 0.1.0 depends on supervision>0.18.0
    groundingdino-py 0.3.0 depends on supervision==0.6.0
    autoware-rosbag2-anonymizer 0.1.0 depends on supervision>0.18.0
    groundingdino-py 0.2.0 depends on supervision==0.6.0
    autoware-rosbag2-anonymizer 0.1.0 depends on supervision>0.18.0
    groundingdino-py 0.1.2 depends on supervision==0.6.0
    autoware-rosbag2-anonymizer 0.1.0 depends on supervision>0.18.0
    groundingdino-py 0.1.0 depends on supervision==0.6.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip to attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

@mitsudome-r
Copy link
Member

Since the error was around supervision, I manually installed groundingdino, open_clip_torch, and ultralytics, then reinstalled supervision 0.22.0 manually.

@StepTurtle
Copy link
Contributor Author

@StepTurtle I'm trying to follow the instructions, but I had to install the following additional dependencies to run python3 main.py config/anonymize_with_unified_model.yaml --anonymize_with_unified_model:

* open_clip_torch

* groundingdino-py

* ultralytics

Fixed with this commit: autowarefoundation/autoware_rosbag2_anonymizer@9445da4

  • I forked groundingdino-py library to leo-drive organization and I just changed the supervision version because upper version of supervision is need from tool but any version is not necessary for groundingdino-py.

StepTurtle and others added 3 commits August 17, 2024 17:36
Signed-off-by: Barış Zeren <bzeren1819@gmail.com>
Signed-off-by: Barış Zeren <bzeren1819@gmail.com>
@StepTurtle
Copy link
Contributor Author

H, @mitsudome-r

I think there are no more comment or request for this PR. Is there anything we should wait for?

@mitsudome-r
Copy link
Member

@StepTurtle Thanks for the ping. I'm doing another test to make sure that there are no issues left.

Signed-off-by: Barış Zeren <baris@leodrive.ai>
StepTurtle and others added 3 commits September 3, 2024 13:16
Signed-off-by: Barış Zeren <baris@leodrive.ai>
Signed-off-by: Barış Zeren <baris@leodrive.ai>
Copy link
Member

@mitsudome-r mitsudome-r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mitsudome-r mitsudome-r merged commit 5a3d76c into autowarefoundation:main Sep 10, 2024
8 of 9 checks passed
@StepTurtle StepTurtle deleted the feat/add_rosbag2_anonymizer branch September 10, 2024 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tag:deploy-docs Mark for deploy-docs action generation. (used-by-ci)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants