Errors running "python -m tracklab.main -cn soccernet" giving final GS-HOTA = 0% #11
Comments
|
Hi @frj555, this is a strange issue that is difficult to analyse without further information.
|
|
Thank you for your quick answer: I agree it seems no detection is performed regardless of the program running for more than 11 hours in my Intel(R) Core(TM) i7-9700K CPU 3.60 GHz 64 RAM, NVIDIA GForce RTX 2080 running the complete pipeline.
Actually I am planning to keep trying new tests with this reduced input to see if I manage to avoid some of the error messages I received above. A) The first set of warnings, I can see also in your explanotary video (Youtube "SoccerNet 2024 Live Tutorials - ft. Vladimir Somers, Victor Joos, and Jan Held"), so I guess they are not criticals: B) My doubt is about :"** The following layers are discarded due to unmatched keys or layer size: ['global_identity_classifier.classifier.weight', 'background_identity_classifier.classifier.weight', 'foreground_identity_classifier.classifier.weight', Please note in the output folder REID/0 folder is empty after running every test!! C) The final warnings. "INFO Saved state at : X:\Pycharmproj\Sngamestate\sn-gamestate\outputs\sn-gamestate\2024-05-27\17-44-38\states\sn-gamestate.pklz main.py:66 Cheers |
|
Hi @frj555 , Can you make sure your "trackeval" library is up to date? you should install the latest version from "https://github.com/SoccerNet/sn-trackeval". Also, can you share the current version of your dataset? I should be the latest one, i.e. "1.3", you can check this number at the beginning one of the .json annotation file "Labels-GameState.json". And what do you mean by "modifying Labels-GameState.json accordingly"? If you want to perform tracking on less images, you can use the "nframes: 150" config next to the "nvid: 1" config below the "dataset" key. |
|
About error B, this does not explain why you don't see any detection or tracking result. Error C is indeed strange. In "tracklab/main.py", could you put a breakpoint after "tracking_engine.track_dataset()", on the line "evaluate(cfg, evaluator, tracker_state)". There, you can have a look at what is inside the "tracker_state.detections_pred" dataframe. I should contain detections, but from what I see with the error you get, it seems this dataframe is empty for you. It would be helpful to know if this dataframe is empty or not after tracking is done. |
|
Hi.
"modifying Labels-GameState.json accordingly": I mean keeping info only from image 151 to 300, and changing labels .json file info to: |
|
Starting again from scratch, and reviewing (tracklab) conda environment I found Error messages reagarding trackeval and mmcv. "ERROR: Ignored the following versions that require a different python version: 0.0.1 Requires-Python ==3.7.0 at conda environment(tracklab). Many thanks for your time |
|
Hi frj555, Could you update tracklab to the latest version ? Some changes made in sn-trackeval changed the way we had to specify it as a dependency. The "mim: command not found" is probably just an error due to the fact that the first installation (which installs mim) did not succeed. |
|
please, can you advice how proceeding to update? I have just relaunched the project with the last git clones as per: "mkdir soccernet and then install dependencies as per your instructions. Then conda environment and working in Pycharm. This is supposed to be tracklab latest version?? Many thanks |
|
Yes, this should indeed give you the latest version of tracklab. Do you still have errors when running any of the installation commands ? ("pip install -e ." or "pip install -e ../tracklab" or "mim install ....") |
|
The last run I reinstall everything again (Windows11, Pycharm, Conda Environment) and no errors occurred in trackeval – not sure about mim-), but no tracking occurring after running the complete pipeline in one video (reid/0 folder empty and states/.PKLZ file almost 2 Gb). Of course any tracking evidence in the output video (predictions map is empty).
Maybe is better idea trying a new Project only with tracking workflow and outside sn-gamestate Project??.
Any recommendation about which Project and workflow I can try?
Thank you
Enviado desde Correo<https://go.microsoft.com/fwlink/?LinkId=550986> para Windows
…________________________________
De: Victor Joos ***@***.***>
Enviado: Monday, June 3, 2024 10:13:09 AM
Para: SoccerNet/sn-gamestate ***@***.***>
Cc: frj555 ***@***.***>; Mention ***@***.***>
Asunto: Re: [SoccerNet/sn-gamestate] Errors running "python -m tracklab.main -cn soccernet" giving final GS-HOTA = 0% (Issue #11)
Yes, this should indeed give you the latest version of tracklab.
Do you still have errors when running any of the installation commands ? ("pip install -e ." or "pip install -e ../tracklab" or "mim install ....")
—
Reply to this email directly, view it on GitHub<#11 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BIS52QPLFWXF5BAZSM6WBKLZFQQRLAVCNFSM6AAAAABIML5ZPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBUGU2TCNJXGQ>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
(tracklab) PS X:\Pycharmproj\Sngamestate\sn-gamestate> python -m tracklab.main -cn soccernet
[2024-05-27 17:44:38,788][main][INFO] - Using device: 'cuda'.
Loading SoccerNetGS 'train' set videos ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:06
Loading SoccerNetGS 'valid' set videos ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:06
Loading SoccerNetGS 'test' set videos ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:06
Loading SoccerNetGS 'challenge' set videos ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:06
Overwriting current config with config loaded from X://Pycharmproj/Sngamestate/sn-gamestate/pretrained_models/reid/prtreid-soccernet-baseline.pth.tar
Diff from default config :
{'batch_size': 32,
'ce': 0.0,
'dim_reduce_output': 256,
'hrnet_pretrained_path': 'X://Pycharmproj/Sngamestate/sn-gamestate/pretrained_models/reid',
'id': 0.0,
'load_config': True,
'mask_filtering_testing': False,
'max_epoch': 20,
'preprocess': 'id',
'test_embeddings': "['globl']",
'tr': 0.0,
'train_sampler': 'PrtreidSampler',
'train_sampler_t': 'PrtreidSampler'}
Loads checkpoint by http backend from path: https://download.openmmlab.com/mmocr/textdet/dbnet/dbnet_resnet18_fpnc_1200e_icdar2015/dbnet_resnet18_fpnc_1200e_icdar2015_20220825_221614-7c0e94f2.pth
The model and loaded state dict do not match exactly
unexpected key in source state_dict: data_preprocessor.mean, data_preprocessor.std
05/27 17:45:15 - mmengine - WARNING - Failed to search registry with scope "mmocr" in the "function" registry tree. As a workaround, the current "function" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmocr" is a correct scope, or whether the registry is initialized.
Loads checkpoint by http backend from path: https://download.openmmlab.com/mmocr/textrecog/sar/sar_resnet31_parallel-decoder_5e_st-sub_mj-sub_sa_real/sar_resnet31_parallel-decoder_5e_st-sub_mj-sub_sa_real_20220915_171910-04eb4e75.pth
The model and loaded state dict do not match exactly
unexpected key in source state_dict: data_preprocessor.mean, data_preprocessor.std
Loads checkpoint by http backend from path: https://download.openmmlab.com/mmocr/textdet/dbnet/dbnet_resnet18_fpnc_1200e_icdar2015/dbnet_resnet18_fpnc_1200e_icdar2015_20220825_221614-7c0e94f2.pth
The model and loaded state dict do not match exactly
unexpected key in source state_dict: data_preprocessor.mean, data_preprocessor.std
Loads checkpoint by http backend from path: https://download.openmmlab.com/mmocr/textrecog/sar/sar_resnet31_parallel-decoder_5e_st-sub_mj-sub_sa_real/sar_resnet31_parallel-decoder_5e_st-sub_mj-sub_sa_real_20220915_171910-04eb4e75.pth
The model and loaded state dict do not match exactly
unexpected key in source state_dict: data_preprocessor.mean, data_preprocessor.std
[05/27/24 17:45:17] INFO Pipeline: YOLOv8 -> PRTReId -> BPBReIDStrongSORT -> TVCalib_Segmentation -> TVCalib -> MMOCR -> MajorityVoteTracklet -> TrackletTeamClustering -> TrackletTeamSideLabeling module.py:68
INFO Starting tracking operation on valid set. main.py:47
INFO Saving TrackerState to X:\Pycharmproj\Sngamestate\sn-gamestate\outputs\sn-gamestate\2024-05-27\17-44-38\states\sn-gamestate.pklz tracker_state.py:45
INFO Pipeline has been validated module.py:85
building model on device cuda
=> init weights from normal distribution
Loading pretrained ImageNet HRNet32 model at X://Pycharmproj/Sngamestate/sn-gamestate/pretrained_models/reid\hrnetv2_w32_imagenet_pretrained.pth
=> loading pretrained model X://Pycharmproj/Sngamestate/sn-gamestate/pretrained_models/reid\hrnetv2_w32_imagenet_pretrained.pth
Successfully loaded pretrained weights from "X://Pycharmproj/Sngamestate/sn-gamestate/pretrained_models/reid/prtreid-soccernet-baseline.pth.tar"
** The following layers are discarded due to unmatched keys or layer size: ['global_identity_classifier.classifier.weight', 'background_identity_classifier.classifier.weight', 'foreground_identity_classifier.classifier.weight',
'concat_parts_identity_classifier.classifier.weight', 'parts_identity_classifier.0.classifier.weight']
Building train transforms ...
Building test transforms ...
Tracking videos (SNGS-021) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/1 0:52:08 • 0:00:00
[05/27/24 18:40:13] INFO Starting evaluation. main.py:107
INFO Starting evaluation using TrackEval library (https://github.com/JonathonLuiten/TrackEval) trackeval_evaluator.py:28
INFO Tracking predictions saved in SoccerNetGS format in eval\pred\SoccerNetGS-valid\tracklab trackeval_evaluator.py:45
INFO Tracking ground truth saved in SoccerNetGS format in eval\pred\SoccerNetGS-valid\tracklab trackeval_evaluator.py:65
Initializing the dataset class for the SoccerNet Game State Reconstruction task.
IMPORTANT: The official evaluation metric for the task, i.e. the 'GS-HOTA' will appear under the 'HOTA' name in the evaluation script output.
This happen because GS-HOTA mainly uses the same logic as the HOTA metric, the HOTA evaluation class is therefore not forked but re-used.
The key practical difference between the GS-HOTA and the HOTA is actually the similarity metric used to match predictions with ground truth.Since this similarity score is computed outside the HOTA class (i.e. inside the SoccerNetGS dataset class), there was no need to fork it into a GS-HOTA class.
Please refer to the official paper for more information.
Using a sigma of 2.042694913268175 for the gaussian similarity metric, based on a distance tolerance of 5 meters.
Evaluating 1 tracker(s) on 1 sequence(s) for 1 class(es) on SoccerNetGS dataset using the following metrics: HOTA, Identity, Count
Evaluating tracklab
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 14.70it/s]
HOTA: tracklab-cls_comb_cls_av HOTA DetA AssA DetRe DetPr AssRe AssPr LocA OWTA HOTA(0) LocA(0) HOTALocA(0)
COMBINED 0 0 0 0 0 0 0 100 0 0 100 0
Identity: tracklab-cls_comb_cls_av IDF1 IDR IDP IDTP IDFN IDFP
COMBINED 0 0 0 0 2067 0
Count: tracklab-cls_comb_cls_av Dets GT_Dets IDs GT_IDs
COMBINED 0 2067 0 19
HOTA: tracklab-cls_comb_det_av HOTA DetA AssA DetRe DetPr AssRe AssPr LocA OWTA HOTA(0) LocA(0) HOTALocA(0)
COMBINED 0 0 0 0 0 0 0 100 0 0 100 0
Identity: tracklab-cls_comb_det_av IDF1 IDR IDP IDTP IDFN IDFP
COMBINED 0 0 0 0 2067 0
Count: tracklab-cls_comb_det_av Dets GT_Dets IDs GT_IDs
COMBINED 0 2067 0 19
[05/27/24 18:40:15] INFO SoccerNet Game State Reconstruction performance GS-HOTA = 0% (config: EVAL_SPACE=pitch, USE_JERSEY_NUMBERS=True, USE_TEAMS=True, USE_ROLES=True, EVAL_DIST_TOL=5) soccernet_game_state.py:48
INFO Have a look at 'tracklab/tracklab/configs/dataset/soccernet_gs.yaml' for more details about the GS-HOTA metric and the evaluation configuration. soccernet_game_state.py:49
INFO Saved state at : X:\Pycharmproj\Sngamestate\sn-gamestate\outputs\sn-gamestate\2024-05-27\17-44-38\states\sn-gamestate.pklz main.py:66
[W C:\cb\pytorch_1000000000000\work\torch\csrc\CudaIPCTypes.cpp:95] Producer process tried to deallocate over 1000 memory blocks referred by consumer processes. Deallocation might be significantly slowed down. We assume it will never going to be the case, but if it is, please file but to https://github.com/pytorch/pytorch
[W C:\cb\pytorch_1000000000000\work\torch\csrc\CudaIPCTypes.cpp:15] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]
[W CUDAGuardImpl.h:46] Warning: CUDA warning: driver shutting down (function uncheckedGetDevice)
....
The text was updated successfully, but these errors were encountered: