[Support]: The review still image prioritizes showing a person at the top left of the frame instead of the object triggering the review in the bottom half

[Reverse256]

Checklist

• I have updated to the latest available Frigate version.
• I have cleared the cache of my browser.
• I have tried a different browser to see if it is related to my browser.
• I have tried reproducing the issue in incognito mode to rule out problems with any third party extensions or plugins I have installed.

Describe the problem you are having

There is more than 1 person in the frame. 1 or more is in the alert/detection zone and 1 or more is far outside. The still photo shown on the review page is of the person(s) outside of the zones. I have only seen this twice. Both times the person outside of the zones is at the top left of the frame and frigate prioritizes zooming in on them instead of the person triggering the alert/detection whom is in the bottom half of the frame.

Steps to reproduce

1. Set up an alert or detection zone in the bottom half of the camera frame.
2. Have someone walk in the top 3/4 of the frame.
3. Have someone walk inside of the alert or detection zone.
4. Person from step 3 triggers the review
5. Frigate zooms in on the person from step 2--outside of the alert/detection zone--for the still image seen on review.
6. (Note: Frigate has to successfully detect both people.)
   ...

Version

0.14.1-f4f3cfa

In which browser(s) are you experiencing the issue with?

Firefox 131.0.3 , Chromium 129.0.6668.100 (Official Build) Arch Linux (64-bit)

Frigate config file

mqtt:
  # Optional: Enable publishing snapshot via mqtt for camera (default: shown below)
  # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
  # All other messages will still be published.
  enabled: true
  host: 10.1.2.6
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: xxxxx
  password: xxxxxx

cameras:
#  dummy_camera: # <--- this will be changed to your actual camera later
#    enabled: False
#    ffmpeg:
#      inputs:
#        - path: rtsp://127.0.0.1:554/rtsp
#          roles:
#            - detect
  Yard:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Yard
          input_args: preset-rtsp-restream
          roles:
            - audio
            - record
            - detect
  #      - path: rtsp://127.0.0.1:8554/Yard_Sub
  #        input_args: preset-rtsp-restream
 #         roles:
 #           - detect 
    zones:
      yard:
        coordinates: 0,1,1,1,1,0.388,1,0.264,0.93,0.222,0.916,0.391,0.735,0.376,0.265,0.387,0,0.402
        inertia: 3
        loitering_time: 0
      sidewalk:
        coordinates: 0,0.296,0.517,0.278,0.924,0.295,0.918,0.394,0.718,0.378,0.263,0.387,0.001,0.405
        loitering_time: 0
        inertia: 3
    snapshots:
      required_zones:
        - yard
        - sidewalk
    review:
      alerts:
        required_zones:
          - yard
      detections:
        required_zones:
          - sidewalk
    motion:
      mask: 0.92,0,0.918,0.085,1,0.167,1,0
    objects:
      mask: 0.921,0.505,0.935,0.352,0.978,0.34,0.985,0.447
  Garage:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Garage
          input_args: preset-rtsp-restream
          roles:
            - audio
            - record
            - detect
     #   - path: rtsp://127.0.0.1:8554/Garage_Sub
     #     input_args: preset-rtsp-restream
     #     roles:
    #        - detect
    zones:
      driveway:
        coordinates: 1,0.617,1,1,0,1,0,0.369,0.639,0.407,0.785,0.536
        inertia: 3
        loitering_time: 0
      yard:
        coordinates: 0.637,0.407,0.782,0.54,1,0.62,1,0.318,0.975,0.317,0.945,0.422
        loitering_time: 0
        inertia: 3
      sidewalk:
        coordinates: 
          0.973,0.32,0.947,0.423,0.655,0.408,0,0.372,0.001,0.324,0.845,0.38,0.919,0.388,0.945,0.308
        loitering_time: 0
        inertia: 3
    review:
      alerts:
        required_zones:
          - driveway
          - yard
      detections:
        required_zones:
          - sidewalk
    snapshots:
      required_zones:
        - driveway
        - yard
        - sidewalk
    motion:
      mask:
        - 0,0,0,0.12,1,0.22,1,0
        - 0.918,0.252,1,0.269,1,0.322,0.911,0.305
objects:
  track:
    - person
    - face
    - license_plate
    - dog
    - cat
#    - car
    - amazon
    - fedex
    - ups
    - package
    - deer

audio:
  enabled: False
  # Optional: Configure the amount of seconds without detected audio to end the event (default: 30)
  max_not_heard: 30
  # Optional: Configure the min rms volume required to run audio detection
  # As a rule of thumb:
  #  - 200 - high sensitivity
  #  - 500 - medium sensitivity (default)
  #  - 1000 - low sensitivity
  min_volume: 500
  # Optional: Types of audio to listen for (default: bark, fire_alarm, scream, speech, yell)
  listen:
    - fire_alarm
    - scream
    - speech
    - yell
  # Optional: Filters to configure detection.
  filters:
    # Label that matches label in listen config.
    speech:
      # Minimum score that triggers an audio event (default: shown below)
      threshold: 0.8
    scream:
      threshold: 0.6
    yell:
      threshold: 0.6

go2rtc:
  streams:
    Yard:
      - rtsp://xxxxx:xxxxx@10.1.2.11:554/Streaming/Channels/101
    #  - "ffmpeg:Yard#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc
    Yard_Sub:
      - rtsp://xxxxx:xxxxx@10.1.2.11:554/Streaming/Channels/102
    #  - "ffmpeg:Yard#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc
    Garage:
      - rtsp://xxxxx:xxxxx@10.1.2.12:554/Streaming/Channels/101
    #  - "ffmpeg:Garage#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc
    Garage_Sub:
      - rtsp://xxxxx:xxxxx@10.1.2.12:554/Streaming/Channels/102
    #  - "ffmpeg:Garage#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc


  webrtc:
    candidates:
      - 10.1.2.45:8555
      - stun:8555

record:
############################## True if record ######################################
  enabled: true
  sync_recordings: true
  retain:
    days: 9
    mode: all
  events:
    retain:
      default: 90
      mode: motion


snapshots:
  enabled: true
  retain:
    default: 30


birdseye:
  enabled: true
  mode: continuous

ffmpeg:
  hwaccel_args: preset-vaapi
  output_args:
    record: preset-record-generic-audio-copy
#  hwaccel_args: preset-nvidia-h264

detectors:
  ov:
    type: openvino
    device: CPU
#  tensorrt:
#    type: tensorrt
#    device: 0 #This is the default, select the first GPU


###### temp intel GPU model while no custom model for gpu ##########
#model:
#  width: 300
#  height: 300
#  input_tensor: nhwc
#  input_pixel_format: bgr
#  path: /openvino-model/ssdlite_mobilenet_v2.xml
#  labelmap_path: /openvino-model/coco_91cl_bkgr.txt


model:
  path: plus://xxxxxxxx

#  input_tensor: nchw
#  input_pixel_format: rgb
#  width: 640
#  height: 640
version: 0.14
camera_groups:
  asdf:
    order: 1
    icon: LuActivitySquare
    cameras:
      - Garage
      - Yard

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "400mb" ## 3 cameras: 1080p=115MB; 4k=375MB
    networks:
      frigate_NET:
        ipv4_address: 172.19.0.2

#    deploy:
#      resources:
#        reservations:
#          devices:
#            - driver: nvidia
#              device_ids: ['0'] # this is only needed when using multiple GPUs
#              count: 1 # number of GPUs
#              capabilities: [gpu]
    devices:
#      - /dev/bus/usb:/dev/bus/usb  # Passes the USB Coral, needs to be modified for other versions
#      - /dev/video19:/dev/video19
#      - /dev/video20:/dev/video20
#      - /dev/video21:/dev/video21
#      - /dev/video22:/dev/video22
#      - /dev/video23:/dev/video23
#      - /dev/video24:/dev/video24
#      - /dev/video25:/dev/video25
#      - /dev/video26:/dev/video26
#      - /dev/video27:/dev/video27
#      - /dev/video28:/dev/video28
#      - /dev/video29:/dev/video29
#      - /dev/video30:/dev/video30
#      - /dev/video31:/dev/video31
#      - /dev/video32:/dev/video32
#      - /dev/video33:/dev/video33
#      - /dev/video34:/dev/video34
#      - /dev/video35:/dev/video35
#      - /dev/video36:/dev/video36
#      - /dev/apex_0:/dev/apex_0    # Passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
#      - /dev/video11:/dev/video11  # For Raspberry Pi 4B
      - /dev/dri/renderD128:/dev/dri/renderD128 # For intel hwaccel, needs to be updated for your hardware
    volumes:
 #      - ./resolv.conf:/etc/resolv.conf
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
      - ./frigate/config:/config
#      - ./frigate/storage:/media/frigate
      - /home/user/Videos/camera:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1.8g
#          size: 1800000000 #1.8GB
    ports:
#      - "127.0.0.1:5000:5000"
#      - "127.0.0.1:8554:8554" # RTSP feeds
#      - "127.0.0.1:8555:8555/tcp" # WebRTC over tcp
#      - "127.0.0.1:8555:8555/udp" # WebRTC over udp
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      - PLUS_API_KEY=xxxxxxxx
      - FRIGATE_RTSP_PASSWORD:"xxxxxxx"
#      - YOLO_MODELS=yolov7x-640
#      - USE_FP16="true"
      - TZ=xxxxxxxx

networks:
  frigate_NET:
    driver: bridge
#    driver_opts:
#      com.docker.network.bridge.host_binding_ipv4: "10.1.2.45"
    ipam:
      config:
        - subnet: 172.19.0.0/16
          gateway: 172.19.0.1
          ip_range: 172.19.0.0/24
#    external: true

Relevant Frigate log output

2024-10-15 13:33:58.699777477  127.0.0.1 - - [15/Oct/2024:13:33:58] "" 400 0 "-" "-" "-"
2024-10-15 13:34:58.716533275  127.0.0.1 - - [15/Oct/2024:13:34:58] "" 400 0 "-" "-" "-"
2024-10-15 13:35:58.732974647  127.0.0.1 - - [15/Oct/2024:13:35:58] "" 400 0 "-" "-" "-"
2024-10-15 13:36:58.750907233  127.0.0.1 - - [15/Oct/2024:13:36:58] "" 400 0 "-" "-" "-"

Relevant go2rtc log output

No recent logs

Operating system

Other Linux

Install method

Docker Compose

Network connection

Wired

Camera make and model

Hikvision DS-2CD2186G2-ISU

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[hawkeye217]

Is the person who is supposedly outside of the zone in the actual clip of the review item? If so, it could mean that Frigate marked them as having entered the zone.

How much time passed between person 1 and person 2?

The next version of Frigate has many helpful tools to debug this kind of situation.

[Reverse256]

How much time passed between person 1 and person 2?

11 seconds

Is the person who is supposedly outside of the zone in the actual clip of the review item?

yes, although after clicking a review item, I can watch indefinitely (which I very much appreciate. version 13 didn't do this).

If so, it could mean that Frigate marked them as having entered the zone.

Possibly, although the frigate+ model is great with bounding boxes in the daytime. Also, I do not have any frigate+ snapshots of the people outside of the zone. So I think that means frigate didn't detect them inside of the zones.

1:35:36 - Person exits their vehicle in the driveway but only their torso is visible so frigate sees the person as "in the middle of the road" (outside of my zones)
1:35:44 - driveway person detect by frigate in alert zone
1:35:55 - The frame with the woman and child outside of the zone

Zones

Review still photo 1:35:55

review trigger 1:35:44 (The review clip starts 5 seconds before this so that confirms it) Also, there are no other reviews for the alert person below

[NickM-27]

the review images prioritize the thumbnail that contains the most active objects, not necessarily the single object that started the activity. For example, just because a person enters the driveway doesn't mean the snapshot right then is the most helpful. Once they get in a car and start moving the car is likely a more relevant thumbnail. Perhaps the logic can be more advanced but this is working normally

[Reverse256]

That makes sense. Thank you everyone for your time.