How to get detection and tracking data from each frame of the stream (like box location, id, class, etc.)?

[SenchaBrest]

Hello. I have been working on my project which was based on yolov5. There, using yolov5, I detected objects on the stream and tracked them using the tracking algorithm. I recently decided to upgrade to a newer version, yolov8, which already has botsort and bytetrack. Due to the fact that there are already built-in algorithms for tracking, I will not need to embed extraneous ones.

However, I can't figure out how to do it. I need to know the information after detection and tracking from each frame and enter it into the database, but I can't find how to do it, because for me the function call scheme inside the repository looks very confusing.

Could you tell me how to get detection and tracking data from each frame of the stream (like box location, id, class, etc.)

[glenn-jocher]

@SenchaBrest see https://docs.ultralytics.com/modes/track for details.

Object tracking is a task that involves identifying the location and class of objects, then assigning a unique ID to
that detection in video streams.

The output of tracker is the same as detection with an added object ID.

Available Trackers

The following tracking algorithms have been implemented and can be enabled by passing tracker=tracker_type.yaml

• BoT-SORT - botsort.yaml
• ByteTrack - bytetrack.yaml

The default tracker is BoT-SORT.

Tracking

Use a trained YOLOv8n/YOLOv8n-seg model to run tracker on video streams.

!!! example ""

=== "Python"

    ```python
    from ultralytics import YOLO
    
    # Load a model
    model = YOLO('yolov8n.pt')  # load an official detection model
    model = YOLO('yolov8n-seg.pt')  # load an official segmentation model
    model = YOLO('path/to/best.pt')  # load a custom model
    
    # Track with the model
    results = model.track(source="https://youtu.be/Zgi9g1ksQHc", show=True) 
    results = model.track(source="https://youtu.be/Zgi9g1ksQHc", show=True, tracker="bytetrack.yaml") 
    ```
=== "CLI"

    ```bash
    yolo track model=yolov8n.pt source="https://youtu.be/Zgi9g1ksQHc"  # official detection model
    yolo track model=yolov8n-seg.pt source=...   # official segmentation model
    yolo track model=path/to/best.pt source=...  # custom model
    yolo track model=path/to/best.pt  tracker="bytetrack.yaml" # bytetrack tracker

    ```

As in the above usage, we support both the detection and segmentation models for tracking and the only thing you need to
do is loading the corresponding (detection or segmentation) model.

Configuration

Tracking

Tracking shares the configuration with predict, i.e conf, iou, show. More configurations please refer
to predict page.
!!! example ""

=== "Python"

    ```python
    from ultralytics import YOLO
    
    model = YOLO('yolov8n.pt')
    results = model.track(source="https://youtu.be/Zgi9g1ksQHc", conf=0.3, iou=0.5, show=True) 
    ```
=== "CLI"

    ```bash
    yolo track model=yolov8n.pt source="https://youtu.be/Zgi9g1ksQHc" conf=0.3, iou=0.5 show

    ```

Tracker

We also support using a modified tracker config file, just copy a config file i.e custom_tracker.yaml
from ultralytics/tracker/cfg and modify
any configurations(expect the tracker_type) you need to.
!!! example ""

=== "Python"

    ```python
    from ultralytics import YOLO
    
    model = YOLO('yolov8n.pt')
    results = model.track(source="https://youtu.be/Zgi9g1ksQHc", tracker='custom_tracker.yaml') 
    ```
=== "CLI"

    ```bash
    yolo track model=yolov8n.pt source="https://youtu.be/Zgi9g1ksQHc" tracker='custom_tracker.yaml'
    ```

Please refer to ultralytics/tracker/cfg
page

[dh031200]

@SenchaBrest

I don't know if this is the right way, but you can get information like this.

model = YOLO("yolov8n.pt")
results = model.track(source=src, tracker="botsort.yaml", verbose=False, stream=True)
for r in results:
    for tracker in detect_model.predictor.trackers:
        print(tracker.__dict__) # tracker information
        print()
        for strack in tracker.tracked_stracks:
            print(strack.__dict__)  # Each obeject information

Output:

Tracker information

{'tracked_stracks': [OT_1_(1-1), OT_2_(1-1), OT_3_(1-1), OT_4_(1-1), OT_5_(1-1)], 
 'lost_stracks': [],
 'removed_stracks': [],
 'frame_id': 1,
 'args': IterableSimpleNamespace(tracker_type='botsort', track_high_thresh=0.5, track_low_thresh=0.1, new_track_thresh=0.6, track_buffer=30, 
  match_thresh=0.8, cmc_method='sparseOptFlow', proximity_thresh=0.5, appearance_thresh=0.25, with_reid=False),
 'max_time_lost': 30,
 'kalman_filter': <ultralytics.tracker.utils.kalman_filter.KalmanFilterXYWH object at 0x7f59c844f370>,
 'proximity_thresh': 0.5,
 'appearance_thresh': 0.25,
 'gmc': <ultralytics.tracker.utils.gmc.GMC object at 0x7f59c844c550>}

Obeject information

{'_tlwh': array([     1260.5,      583.32,      124.22,      267.34], dtype=float32), 'kalman_filter': <ultralytics.tracker.utils.kalman_filter.KalmanFilterXYWH object at 0x7f59c844f370>, 'mean': array([     1322.6,      716.99,      124.22,      267.34,           0,           0,           0,           0], dtype=float32), 'covariance': array([[     154.32,           0,           0,           0,           0,           0,           0,           0],
       [          0,      714.69,           0,           0,           0,           0,           0,           0],
       [          0,           0,      154.32,           0,           0,           0,           0,           0],
       [          0,           0,           0,      714.69,           0,           0,           0,           0],
       [          0,           0,           0,           0,       60.28,           0,           0,           0],
       [          0,           0,           0,           0,           0,      279.17,           0,           0],
       [          0,           0,           0,           0,           0,           0,       60.28,           0],
       [          0,           0,           0,           0,           0,           0,           0,      279.17]]), 'is_activated': True, 'score': 0.91442204, 'tracklet_len': 0, 'cls': 15.0, 'idx': 0.0, 'smooth_feat': None, 'curr_feat': None, 'features': deque([], maxlen=50), 'alpha': 0.9, 'track_id': 1, 'state': 1, 'frame_id': 1, 'start_frame': 1}