tasks/pose/

[giscus[bot]]

tasks/pose/

Learn how to use Ultralytics YOLOv8 for pose estimation tasks. Find pretrained models, learn how to train, validate, predict, and export your own.

https://docs.ultralytics.com/tasks/pose/

[KingAsperin]

Do you already have a documentation on how to implement yolov8 landmark detection on android development? if none is it posible for you to make a thread to teach us how to implement yolov8 model into android development please

[pderrenger]

@KingAsperin hey there! 👋

We don't currently have a dedicated guide for implementing YOLOv8 landmark detection on Android, but it's a great suggestion! 🚀

In the meantime, you can export a YOLOv8 pose model to ONNX or TensorFlow formats, which are more compatible with Android development. For example, you can use the following command to export a model:

yolo export model=yolov8n-pose.pt format=onnx

Once you have the ONNX model, you can use tools like ONNX Runtime Mobile to run it on Android devices. For TensorFlow, you can convert the model to TensorFlow Lite and use the TensorFlow Lite Android Support Library.

I hope this helps! If you have more questions or need further assistance, feel free to ask. Happy coding! 😊

[leonidasavila]

How can i calculate angle position between three points? Thanks

[pderrenger]

@leonidasavila hey there! 👋 To calculate the angle between three points, you can use the law of cosines. Here's a quick example in Python:

import numpy as np

def calculate_angle(p1, p2, p3):
    # p1, p2, p3 are the points in format [x, y]
    # Calculate the vectors
    v1 = np.array(p1) - np.array(p2)
    v2 = np.array(p3) - np.array(p2)
    
    # Calculate the angle in radians
    angle_rad = np.arccos(np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2)))
    
    # Convert to degrees
    angle_deg = np.degrees(angle_rad)
    
    return angle_deg

# Example points
p1 = [0, 0]
p2 = [1, 0]
p3 = [1, 1]

angle = calculate_angle(p1, p2, p3)
print(f"The angle is {angle} degrees")

This calculates the angle at p2 formed by the line segments p1p2 and p2p3. Hope this helps! Let me know if you have any other questions. 😊

[leonidasavila]

Thanks @pderrenger, for your quickly response! I´m trying to migrate our startup app from mediapipe to yolo 😍

Do you have any tutorial or short example to define specifics landmarks to make calculates? (4 example push-up)

Best regards!
Leonidas

[pderrenger]

Hey Leonidas! 🌟

It's awesome to hear about your transition from MediaPipe to YOLO for your startup app! For specific landmark detection like push-ups, you can leverage our YOLOv8 pose models. Here's a quick example to get you started:

from ultralytics import YOLO

# Load a pretrained pose model
model = YOLO('yolov8n-pose.pt')

# Predict keypoints on an image
results = model('path/to/your/image.jpg')

# Access keypoints
keypoints = results[0].keypoints

# Now, you can use these keypoints to calculate angles or any other metrics you need

This snippet loads a pretrained YOLOv8 pose model and predicts keypoints on your image. You can then use these keypoints for further calculations, like angles between joints during push-ups.

For more detailed guidance, check out our Pose Estimation page. If you have any more questions or need further assistance, feel free to ask. Happy coding! 😊

[ozankavcu38]

By the way sir after loading the model how could we dissapear the pose estimatet lines like boxes=False argument

[pderrenger]

Hey there! 😊 To hide the pose estimation lines (similar to using boxes=False for bounding boxes), you can simply not plot them. When you're working with the results from the pose estimation model, you have control over what gets visualized. Here's a quick example on how to display the keypoints without drawing the connecting lines:

from ultralytics import YOLO

# Load your pose model
model = YOLO('yolov8n-pose.pt')

# Run prediction
results = model('path/to/image.jpg')

# Access keypoints
keypoints = results[0].keypoints

# Now, you can visualize keypoints without drawing lines between them
# This part depends on how you're visualizing. For example, using OpenCV to plot keypoints:
import cv2
image = cv2.imread('path/to/image.jpg')
for kpt in keypoints.xy:
    x, y = int(kpt[0]), int(kpt[1])
    cv2.circle(image, (x, y), radius=3, color=(0, 255, 0), thickness=-1)

cv2.imshow("Keypoints without lines", image)
cv2.waitKey(0)
cv2.destroyAllWindows()

This way, you'll see the keypoints on your image without the estimated pose lines. Hope this helps, and good luck with your app migration! If you have any more questions, feel free to ask. 🚀

[ozankavcu38]

Sir i just wanna ask you something about live training . I was making some calculations and getting some results that is okay by the way yolov8 is very good about pose estimation issues. I just wanna ask when i am training mp4 files ( my main is live training however i am just using for the testing this ) . The model is drawing everything like boxes poses conf values etc. I dont want to see nothing on the video however i just want to keep operating the results by the way . In addition to this i just want to write something on video like i have counted 3 people on the top of right corner of that video how could i do this i did not see any answer . when i turn off results = model(source="C://Users//Hp//Desktop//videolar//operation_room_1.mp4", show=True, conf=0.3, save=False,
verbose=False, stream=True,show_conf=True). like show false i did not see anything about video it is completely dissapeared or when i turn off stream yeah my calculations is continue however the video is dissapeared . I want to show the video appeared however there must be nothing as image , poses , boxes .

[glenn-jocher]

Hello! It sounds like you're doing some interesting work with live training and pose estimation using YOLOv8. To address your concerns about visualizing the video without drawing boxes, poses, or confidence values, while still being able to operate on the results and add custom text annotations, here's a simplified approach:

1. To process the video without visualizing detections: You can set show=False to prevent displaying the video with detections. This will still allow you to access the detection results programmatically.

2. To add custom text annotations: You can use OpenCV to write text on the video frames. Here's a small code snippet to help you get started:

import cv2
from ultralytics import YOLO

# Load your model
model = YOLO('path/to/your/model.pt')

# Open the video
cap = cv2.VideoCapture("C://Users//Hp//Desktop//videolar//operation_room_1.mp4")

# Process each frame
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Run model prediction
    results = model(frame, show=False)

    # Example: Count people and add text annotation
    people_count = sum(1 for *_, cls in results.xyxy[0] if cls == 0)  # Assuming class 0 is 'person'
    cv2.putText(frame, f"People Count: {people_count}", (50, 50), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2)

    # Show the frame
    cv2.imshow('Frame', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

This script processes each frame to count the number of people (assuming class 0 corresponds to 'person') and displays the count on the top right corner of the video. Adjust the class index as per your model's class labels.

Remember, the show=False parameter in the model() call ensures that the original video is processed without overlaying the detections, while cv2.putText() is used to add custom text annotations.

I hope this helps! Let me know if you have any more questions. 😊

[samverh]

Hi, is it possible to run this model on the OAK-D camera with VPU?

[pderrenger]

Hey there! 😊

After converting your YOLOv8 model for use with the OAK camera, the output format indeed changes, and parsing the results might require some adjustments. For handling the output layer and parsing keypoints correctly, I recommend checking out the Pose Estimation documentation on the Ultralytics Docs page. It provides insights into how keypoints are represented and might give you a good starting point.

For a quick example, if you're working with an ONNX model, you might use something like this to parse the output:

import numpy as np

# Assuming 'output' is the model's output tensor
keypoints = np.where(output > confidence_threshold)  # Apply confidence threshold
# Process 'keypoints' as needed

This is a simplified example, and the exact implementation will depend on your model's output structure. If you need more specific guidance, feel free to ask! 🚀

[samverh]

Hi, your help is really appreciated.

I am still struggling with parsing the output in the proper way. I inspected the model using Netron and the output layer should have a shape of (1, 56, 8400). The actual shape is (470400, ) so I perform the following conversion:

outputs = data.reshape(1, 56, 8400)
outputs = outputs.transpose(0, 2, 1)

Now I assume the bounding boxes can be extracted by iterating over the data as outputs[0][i][:4] where i is the row index. Then, I convert the boxes with the following function and filter the most relevant boxes using NMS.

def xywh2box(bbox_vals):

    center_x = bbox_vals[0]
    center_y = bbox_vals[1]
    w = bbox_vals[2] 
    h = bbox_vals[3]
    x = center_x - 0.5 * w # center_x to top_left_x
    y = center_y - 0.5 * h # center_y to top_left_y
    box = [x, y, x + w, y + h]

    return box

When plotting the results including keypoint coordinates I am getting very small boxes. It seems that my data conversion is incorrect or that I should apply scaling, but the image sizes and input shapes are both (640, 640).

What am I missing here?

[pderrenger]

@samverh hey there! 👋 It looks like you're on the right track with reshaping and transposing the output. However, if the boxes appear too small, it might indeed be a scaling issue. When working with models converted to OpenVINO and running on devices like the OAK-D, remember that the output coordinates are normalized. You might need to scale them back to the original image dimensions.

Here's a quick tip: Ensure you're scaling the bounding box coordinates by the original image width and height before converting them with your xywh2box function. For example:

def scale_boxes(outputs, img_width, img_height):
    scaled_outputs = []
    for i in range(outputs.shape[1]):  # Iterate over each detection
        bbox = outputs[0][i][:4]
        bbox[0] *= img_width  # Scale x
        bbox[1] *= img_height # Scale y
        bbox[2] *= img_width  # Scale width
        bbox[3] *= img_height # Scale height
        scaled_outputs.append(xywh2box(bbox))
    return scaled_outputs

Make sure to replace img_width and img_height with the actual dimensions of your input images. This should help correct the size of the bounding boxes. If you're still facing issues, double-check the model's input and output layer specifications and ensure they align with your preprocessing steps. Keep up the great work! 🚀

[samverh]

The problem was the input from the OAK-D camera. Frames from ColorCamera/MonoCamera are in U8 data type [0, 255]. To obtain [0, 1] values that can be passed to the model, rescaling is needed by setting mean=0 and scale=255. This can be done when converting the model to blob format:

import blobconverter

# convert onnx to blob
blobconverter.from_onnx(
    model=f"{yolo_modelname}.onnx", 
    output_dir=f"{folder}/",
    data_type="FP16",
    shaves=6,
    use_cache=False,

    optimizer_params=[
        "--mean_values=[0,0,0]",
        "--scale_values=[255,255,255]",
    ]
)

Thanks again for your help! 🙌

[pderrenger]

Hey there! 🙌 It looks like you've done a great job converting the model and reshaping the output for use with your OAK-D camera. The issue with small bounding boxes might indeed stem from not scaling the output coordinates back to the original image dimensions.

Given the output shape (1, 56, 8400) and after your reshaping to (1, 8400, 56), it's crucial to remember that the model outputs are normalized to [0, 1]. To map these back to the original image size, you'll need to scale the bounding box coordinates (x, y, width, height) by the width and height of the input image.

Here's a quick adjustment to your xywh2box function to include scaling:

def xywh2box(bbox_vals, img_width, img_height):
    center_x = bbox_vals[0] * img_width
    center_y = bbox_vals[1] * img_height
    w = bbox_vals[2] * img_width
    h = bbox_vals[3] * img_height
    x = center_x - 0.5 * w  # Convert center_x to top_left_x
    y = center_y - 0.5 * h  # Convert center_y to top_left_y
    box = [x, y, x + w, y + h]
    return box

Make sure to pass the img_width and img_height of your original image to this function. This should correct the size of your bounding boxes. If you're still encountering issues, double-check the output layer's interpretation to ensure you're extracting the bounding box and keypoint information correctly.

Keep up the great work, and don't hesitate to reach out if you have more questions! 😊

[shekharsingh8811]

Hi,
Could you please let me know which approach does the yolov8-pose model take? And, how does it output the bounding boxes?
Is it,

1. Top-down approach — which involves a person detector that outputs the bounding boxes and then a pose is calculated for each of these bounding boxes.
   OR
2. Bottom-up approach — which involves detecting all parts of each person within an image and then associating or grouping the parts that belong to each individual. Finally, the estimated pose is then used to calculate the bounding box thus encapsulating all the detected keypoints within it.

My motive here is to figure out, if there would be any difference between the bounding box output that I would get from yolov8 (detection) model and the yolov8-pose model. Would there be any?

Thank You!!

[glenn-jocher]

@shekharsingh8811 hey there! 😊

Absolutely, you can use bounding boxes from another model as input for YOLOv8-pose for pose estimation. While YOLOv8-pose is designed for a top-down approach, integrating external bounding boxes requires a bit of manual preprocessing. Here's a quick example on how you might do it:

from ultralytics import YOLO
import cv2

# Load your YOLOv8-pose model
model = YOLO('yolov8n-pose.pt')

# Assuming 'external_boxes' is a list of bounding boxes [x1, y1, x2, y2] from another model
# and 'image' is the image you're working with
for box in external_boxes:
    # Crop the person from the image using the bounding box
    x1, y1, x2, y2 = box
    person_crop = image[y1:y2, x1:x2]

    # Run pose estimation on the cropped image
    results = model(person_crop)

    # Now, results contain keypoints for the pose within the cropped image
    # You can plot these keypoints directly on the original image or the cropped image

This way, you're manually handling the bounding box to keypoint part of the top-down approach. The bounding boxes from YOLOv8 detection and those used for YOLOv8-pose might differ, but this method allows you to use any source of bounding boxes!

Hope this helps! Let me know if you have more questions.

[shekharsingh8811]

Hi @glenn-jocher,

I'm currently using the yolov8s-pose.engine model on my dataset which comprises of 640 x 640 frames / images. The person in the scene is at times at the edges of the frame and thus only partially visible, irrespective of that the model always outputs 17 keypoints and tries to squash the non-visible keypoints towards the edges of the frame. Is this expected?
On the other hand, I expected the keypoint coordinates to be in the range (0.0 , 639.0) but for these edge cases the model at times outputs the keypoint x or y coordinate value as 640.0 which it shouldn't right?

Could you please your thoughts on this.

Thank You!

[glenn-jocher]

Hi @shekharsingh8811,

Thank you for your detailed question! 😊

The behavior you're observing, where the YOLOv8-pose model outputs 17 keypoints even for partially visible persons and occasionally produces keypoint coordinates at the edge (640.0), is expected. This is because the model is designed to predict all keypoints regardless of visibility, and it attempts to place non-visible keypoints as close to the edge of the frame as possible.

Regarding the keypoint coordinates reaching 640.0, this can happen due to the model's bounding box normalization and rounding during inference. While the coordinates should ideally be within the image dimensions (0.0 to 639.0), slight deviations can occur.

To address this, you can post-process the keypoints to ensure they fall within the expected range. Here's a quick example in Python:

import numpy as np

# Assuming 'results' is the output from the model
for result in results:
    keypoints = result.keypoints.xy
    keypoints = np.clip(keypoints, 0, 639)  # Ensure keypoints are within the image bounds
    # Further processing or visualization

If you haven't already, please ensure you're using the latest versions of torch and ultralytics. You can update them using:

pip install --upgrade torch ultralytics

If the issue persists, could you provide a minimum reproducible example? This will help us investigate further. You can find guidelines for creating one here: Minimum Reproducible Example.

Feel free to reach out if you have more questions or need further assistance!

[shekharsingh8811]

Hi @glenn-jocher,

Thank you for the response!
I'm already clipping the keypoints for my use case, so this resolves the issue for now.

[glenn-jocher]

Hi @shekharsingh8811,

Thank you for your response! 😊

I'm glad to hear that clipping the keypoints resolves the issue for your use case. The behavior you described, where the model outputs keypoints even for partially visible persons and occasionally produces coordinates at the edge (640.0), is indeed expected due to the model's bounding box normalization and rounding during inference.

If you have any further questions or need additional assistance, feel free to ask. For more detailed information on pose estimation, you can refer to our Pose Estimation Documentation.

Happy coding! 🚀

[StavrosAnaniadis]

Hello! I am trying to output all the points detected by yolov8 into a excel or csv file in a video, and am having trouble with it. I am using the code:
from ultralytics import YOLO

model = YOLO('yolov8m-pose.pt')
video_path = #video path here

results = model(source=video_path, show=True, conf = 0.3, save=True)

Is there any way I can save all of the coordinates for the 2 people in the video to an excel file?

Thank you,
Stavros Ananiadis

[pderrenger]

@StavrosAnaniadis hi there! 🙌

For saving the keypoints to an Excel or CSV file, including their labels like "left arm", "right wrist" etc., you'll need to modify the approach slightly. Based on your requirement and the given structure of the Keypoints object, here's a concise example using pandas to save the data in a more structured format, including the labels:

import pandas as pd
from ultralytics import YOLO

# Define keypoint labels for your pose model (order should match the model's output)
labels = ["nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder",
          "right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist",
          "left_hip", "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle"]

model = YOLO('yolov8m-pose.pt')
results = model(source=video_path, show=True, conf=0.3, save=True)

# Initialize an empty list to store data
data = []

# Loop through results and store keypoints data
for frame_results in results:
    for person_id, person in enumerate(frame_results.keypoints):
        for label, (x, y, conf) in zip(labels, person.numpy()):
            data.append({
                "person_id": person_id,
                "label": label,
                "x": x,
                "y": y,
                "confidence": conf
            })

# Create a DataFrame and save it to an Excel file
df = pd.DataFrame(data)
df.to_excel("keypoints.xlsx", index=False)

This script runs through the keypoints detected on each person in every frame, collecting their labels, coordinates, and confidence levels into a structured format that's then written to an "keypoints.xlsx" Excel file.

Hope this serves your project well, especially with the fascinating application of machine learning in analyzing sword fights! If you need more info on the keypoint indexes and labels, referring to this Ultralytics docs page might be helpful. Good luck with your analysis! 🤺👀

[StavrosAnaniadis]

Thank you so much for your help.

When I run this code, I get the error:
for label, (x, y, conf) in zip(labels, person.numpy()):

ValueError: not enough values to unpack (expected 3, got 1)
Referring to line 26: for label, (x, y, conf) in zip(labels, person.numpy()):

Do you know why this would not be working?

Thank you,
Stavros Ananiadis

[pderrenger]

Hi Stavros,

It looks like the error stems from unpacking the keypoints incorrectly. The keypoints are structured as xy, xyn, and conf for the coordinates, normalized coordinates, and confidences respectively, according to the Ultralytics documentation. Here's how you can modify your loop for correct unpacking:

import pandas as pd

data = []
# Assuming 'results' is from your prediction
for frame_id, result in enumerate(results):
    for person_id, person in enumerate(result.keypoints):
        keypoints = person.xy  # Get keypoints
        confidences = person.conf  # Get confidences
        for kp_id, (xy, conf) in enumerate(zip(keypoints, confidences)):
            label = labels[kp_id]  # Get the label from predefined labels list
            data.append([frame_id, person_id, label, *xy, conf])

# Convert the data into a pandas DataFrame
df = pd.DataFrame(data, columns=['frame_id', 'person_id', 'label', 'x', 'y', 'confidence'])

# Save to an Excel file
df.to_excel('keypoints.xlsx', index=False)

Remember, for this to work, ensure you have labels list correctly ordered as per your model's keypoints output.

This adaptation should resolve the unpacking issue and store coordinates with their corresponding labels like "left arm", "right wrist", etc. 🚀

Best of luck with your project on providing insights to sword fighters. It sounds like a cutting-edge (pun intended) application of AI! If you have more questions or hit another snag, feel free to ask.

[StavrosAnaniadis]

Hello! Thank you for your continued assistance. On running the code, a new error appeared:
ValueError: 6 columns passed, passed data had 21 columns
My labels look like:

Define keypoint labels for your pose model (order should match the model's output)

labels = ["nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder",
"right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist",
"left_hip", "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle"]
Would you happen to know whats causing that?

Thank you,
Stavros Ananiadis

[pderrenger]

Hello Stavros,

It seems the issue arises because there’s a mismatch in the structure of the data being written to the CSV and the expected format. The error indicates that the DataFrame construction is attempting to handle more columns than specified.

Given the ValueError related to column mismatch and considering the structure of the keypoints in the YOLOv8 pose estimation model, it appears we need to adjust how the keypoints are unpacked before saving them. Specifically, the xy, conf values should be handled correctly.

The correct usage for processing keypoints and confidences involves accessing these attributes directly from the person object as .xy or .conf and ensuring each keypoint and its confidence are correctly represented:

import pandas as pd
from ultralytics import YOLO

model = YOLO('yolov8m-pose.pt')
results = model(source=video_path, conf=0.3)

data = []

for frame_id, (frame_results) in enumerate(results):
    for person_id, person in enumerate(frame_results.keypoints):
        xy = person.xy.numpy()  # keypoints coordinates
        conf = person.conf.numpy()  # confidence scores of keypoints
        for kp_id, (xy, kp_conf) in enumerate(zip(xy, conf)):
            label = labels[kp_id]  # Using the predefined labels list
            x, y = xy
            data.append({"frame_id": frame_id, "person_id": person_id, "label": label, "x": x, "y": y, "confidence": kp_conf})

df = pd.DataFrame(data)
df.to_csv('keypoints.csv', index=False)

Make sure the labels list matches your model's output order. This script aims to correctly handle the extraction of keypoint coordinates and confidence scores, complementing your innovative approach to analyzing sword fighting techniques with AI. Keep pushing the boundaries! 🤺🚀

If you have more questions or need further assistance, feel free to reach out.

[arav1ndb]

Where can I find the mapping for the xy coordinates obtained by the algorithm, as in which point pertains to which body part?

[arav1ndb]

I got it here, for anyone else searching for it

# Define keypoint labels for your pose model (order should match the model's output)
labels = ["nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder",
          "right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist",
          "left_hip", "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle"]

[glenn-jocher]

@arav1ndb great find! 🌟 For anyone else curious about the keypoint labels for pose estimation models, here they are:

labels = [
    "nose", "left_eye", "right_eye", "left_ear", "right_ear",
    "left_shoulder", "right_shoulder", "left_elbow", "right_elbow",
    "left_wrist", "right_wrist", "left_hip", "right_hip",
    "left_knee", "right_knee", "left_ankle", "right_ankle"
]

These labels correspond to the output keypoints of your pose model. When interpreting [x, y] coordinate predictions, you can match each point to the respective body part listed above. Happy coding! 😊

[dolfMo]

Hello! I would like to ask if it is possible to output the coordinate information of the specified key point, which is the coordinate point of the wrist

[glenn-jocher]

@dolfMo hello! 😊 Yes, it's definitely possible to get the coordinate information of specific keypoints, such as the wrist, using YOLOv8's pose estimation models. If you're working with a model like yolov8n-pose.pt, the keypoint coordinates, including those for the wrist, are outputted as part of the model's predictions.

Here's a quick Python example on how to access these keypoints:

from ultralytics import YOLO

# Load the trained pose estimation model
model = YOLO('yolov8n-pose.pt')

# Run predictions
results = model('path/to/your/image.jpg')

# Access keypoints for each detected person
for person in results.keypoints:
    wrist_coords = person.xy[person.names.index('wrist')]  # Assuming 'wrist' is part of your model's keypoints
    print(f'Wrist coordinates: {wrist_coords}')

Make sure to replace 'path/to/your/image.jpg' with the path to your image. The keypoint names (like 'wrist') and their exact order depend on your model and dataset, so you might need to adjust the person.names.index('wrist') part to fit your specifics.

Let me know if there's anything else I can help with! 🌟

[dolfMo]

Hello, I encountered the following error message after executing your code. I would like to know why?
for person in results.keypoints: AttributeError: 'list' object has no attribute 'keypoints'

[jirivchi1]

result = results[0]
Extract keypoints
keypoints = result.keypoints

It works for me in the previous code but I am got a new error

[glenn-jocher]

Hey there! 👋 It looks like you're encountering an attribute error because results is a list, not directly accessing attributes. For pose estimation with YOLOv8, you'll need to interact with the list of results to get the keypoints. Here's how you can correctly access keypoints:

from ultralytics import YOLO

# Load your model
model = YOLO('yolov8n-pose.pt') 

# Run your prediction
results = model('path/to/image.jpg')

# Correct way to access keypoints 
for result in results:
    if hasattr(result, 'keypoints'):
        for person in result.keypoints:
            wrist_coords = person.xy[person.names.index('wrist')]
            print(f'Wrist coordinates: {wrist_coords}')

This snippet checks if the keypoints attribute exists in each result and then iterates through each person detected to find the wrist coordinates. Let me know if that helps or if you have any more questions!

Keep on exploring! 🚀

[pyrat]

Hi,

I have trained a model based on YOLOv8l-pose, when I infer the model I get None for confidence and the keypoints just have x and y. Is there a reason why there is no confidence? here is contents of the keypoints object:

proto-proto-1 | Initializing LandmarkDetector
proto-proto-1 | YOLOv8l-pose summary (fused): 287 layers, 44464206 parameters, 0 gradients, 168.6 GFLOPs
proto-proto-1 |
proto-proto-1 | 0: 192x640 1 fish, 56.9ms
proto-proto-1 | Speed: 1.3ms preprocess, 56.9ms inference, 164.3ms postprocess per image at shape (1, 3, 192, 640)
proto-proto-1 | ultralytics.engine.results.Keypoints object with attributes:
proto-proto-1 |
proto-proto-1 | conf: None
proto-proto-1 | data: tensor([[[ 7.0377, 94.4488],
proto-proto-1 | [ 35.9414, 107.0425],
proto-proto-1 | [ 1.5750, 111.7595],
proto-proto-1 | [ 55.6107, 92.0907],
proto-proto-1 | [101.0829, 49.3980],
proto-proto-1 | [147.8248, 93.4525],
proto-proto-1 | [112.3623, 138.2682],
proto-proto-1 | [153.1546, 141.8253],
proto-proto-1 | [277.1523, 16.1814],
proto-proto-1 | [343.0081, 22.3252],
proto-proto-1 | [324.8737, 135.0928],
proto-proto-1 | [429.2652, 111.5511],
proto-proto-1 | [470.5245, 98.0572],
proto-proto-1 | [457.7650, 36.1689],
proto-proto-1 | [479.3454, 46.8262],
proto-proto-1 | [507.4113, 54.2151],
proto-proto-1 | [532.2663, 45.4843],
proto-proto-1 | [542.9807, 83.6471],
proto-proto-1 | [521.8376, 102.8704],
proto-proto-1 | [506.2361, 90.8436]]], device='cuda:0')
proto-proto-1 | has_visible: False
proto-proto-1 | orig_shape: (154, 576)
proto-proto-1 | shape: torch.Size([1, 20, 2])
proto-proto-1 | xy: tensor([[[ 7.0377, 94.4488],
proto-proto-1 | [ 35.9414, 107.0425],
proto-proto-1 | [ 1.5750, 111.7595],
proto-proto-1 | [ 55.6107, 92.0907],
proto-proto-1 | [101.0829, 49.3980],
proto-proto-1 | [147.8248, 93.4525],
proto-proto-1 | [112.3623, 138.2682],
proto-proto-1 | [153.1546, 141.8253],
proto-proto-1 | [277.1523, 16.1814],
proto-proto-1 | [343.0081, 22.3252],
proto-proto-1 | [324.8737, 135.0928],
proto-proto-1 | [429.2652, 111.5511],
proto-proto-1 | [470.5245, 98.0572],
proto-proto-1 | [457.7650, 36.1689],
proto-proto-1 | [479.3454, 46.8262],
proto-proto-1 | [507.4113, 54.2151],
proto-proto-1 | [532.2663, 45.4843],
proto-proto-1 | [542.9807, 83.6471],
proto-proto-1 | [521.8376, 102.8704],
proto-proto-1 | [506.2361, 90.8436]]], device='cuda:0')
proto-proto-1 | xyn: tensor([[[0.0122, 0.6133],
proto-proto-1 | [0.0624, 0.6951],
proto-proto-1 | [0.0027, 0.7257],
proto-proto-1 | [0.0965, 0.5980],
proto-proto-1 | [0.1755, 0.3208],
proto-proto-1 | [0.2566, 0.6068],
proto-proto-1 | [0.1951, 0.8978],
proto-proto-1 | [0.2659, 0.9209],
proto-proto-1 | [0.4812, 0.1051],
proto-proto-1 | [0.5955, 0.1450],
proto-proto-1 | [0.5640, 0.8772],
proto-proto-1 | [0.7453, 0.7244],
proto-proto-1 | [0.8169, 0.6367],
proto-proto-1 | [0.7947, 0.2349],
proto-proto-1 | [0.8322, 0.3041],
proto-proto-1 | [0.8809, 0.3520],
proto-proto-1 | [0.9241, 0.2954],
proto-proto-1 | [0.9427, 0.5432],
proto-proto-1 | [0.9060, 0.6680],
proto-proto-1 | [0.8789, 0.5899]]], device='cuda:0')

[glenn-jocher]

Hey there! 🌟 It looks like the keypoints are indeed present but the confidence associated with each keypoint isn't being returned directly in your output. This behavior depends on how the model was trained and how the inference script processes the outputs.

For YOLOv8l-pose, confidence scores are typically associated with the detection rather than individual keypoints. If you need per-keypoint confidence, consider modifying the model's architecture or post-processing script to include this information.

As for the keypoints, x and y in your output represent the normalized positions of each keypoint detected by your model. This is standard for pose estimation tasks, where the main goal is to locate key positional landmarks.

If your application requires confidence scores for each keypoint, you may explore customizing the post-processing step to calculate these based on the model's predictions or the detection's overall confidence score.

For now, you can work with the provided keypoints as they give you the essential information about the position of each landmark detected in the image. Happy coding! 😊

[PolinKun]

can we model pose estimation for excavator work?

[glenn-jocher]

@PolinKun absolutely! Pose estimation can be utilized for modeling excavator work by identifying and tracking keypoints on the excavator. For instance, keypoints could represent the excavator's arm joints, bucket, and other significant parts to analyze its movements and work dynamics.

Here's a quick start on doing so with a YOLOv8n-pose model:

from ultralytics import YOLO

# Load a pretrained pose estimation model
model = YOLO('yolov8n-pose.pt')

# Predict on an image
results = model('path/to/excavator/image.jpg')

# Display the keypoints
results.show()

This example loads a pretrained YOLOv8n-pose model and runs it on an excavator image to detect keypoints. You might want to train the model on a dataset specific to excavators for optimal results. Training on the excavator-specific dataset would involve collecting images of excavators, annotating them with keypoints, and following the training process as detailed in our docs.

Feel free to explore the Pose Estimation section for more insights and guidance on custom training for your specific needs. Happy modeling! 😊

[shukurullo2004]

hello, i hope you can help i am struggled to create logic human falling based on shoulder and knee distance if shoulder is below than knee that means human is fallen, but i can not get keypoints 😭of them and calculate distance, becouse it is first time to using yolov pose estimation. thank you

[shukurullo2004]

another question: can i find velocity of shoulder and knees using keypoints ?
if yes, how can i ?

[pderrenger]

Hello! 😊 Yes, you can calculate the velocity of shoulders and knees using keypoints from pose estimation. Velocity is essentially the change in position over time. If you have keypoints for shoulders and knees detected over sequential frames, you can calculate the distance they move between frames and divide by the time elapsed to get their velocity.

Here's a simple example on how you might approach this:

from ultralytics import YOLO

# Load your pose model
model = YOLO('yolov8n-pose.pt')

# Assume frame1 and frame2 are consecutive frames from your video
results_frame1 = model(frame1)
results_frame2 = model(frame2)

# Extract keypoint locations for shoulders and knees for both frames
# For demonstration, let's say the IDs for shoulders and knees are 2 and 3 respectively
shoulder_frame1 = results_frame1.keypoints[2]
knee_frame1 = results_frame1.keypoints[3]
shoulder_frame2 = results_frame2.keypoints[2]
knee_frame2 = results_frame2.keypoints[3]

# Calculate the distance moved by each keypoint (simple Euclidean distance for demonstration)
distance_shoulder = ((shoulder_frame2[0] - shoulder_frame1[0])**2 + (shoulder_frame2[1] - shoulder_frame1[1])**2)**0.5
distance_knee = ((knee_frame2[0] - knee_frame1[0])**2 + (knee_frame2[1] - knee_frame1[1])**2)**0.5

# Assuming you know the time elapsed between the two frames (time_elapsed), calculate velocity
velocity_shoulder = distance_shoulder / time_elapsed
velocity_knee = distance_knee / time_elapsed

Please adjust the keypoint IDs according to the specific model outputs and ensure the correct handling of the coordinate system used in your data. The velocity calculation here is a basic illustration. Depending on your exact needs, especially if you're dealing with video data, you might need a more robust approach to handle varying frame rates and more complex movement patterns. Good luck with your project!

[shukurullo2004]

thank you helping me, with your help i can detect delta distance but how can i measure time ?

i am doing project human fall detection with using yolov8-pose.pt model, so i can find keypoints but logical part is being difficult, my logic is that comparing velocity of shoulder if shoulder velocity > knee velocity AND if position of shoulder below than knee points that means human is fallen.

so can you help me write this logic

thank you a lot

[pderrenger]

@shukurullo2004 hey there! 😃 I'm glad to hear you've made progress with keypoint detection using the YOLOv8-pose model. For measuring the time, especially in videos, you can use the frame rate (FPS) of your video. If you're processing frame by frame, the time between frames would be 1/FPS. Here's a simplified way to include it in your logic:

fps = 30  # Example FPS of your video
time_elapsed = 1 / fps  # Time between each frame

# Your logic for comparing shoulder and knee velocities
# Assuming you already calculated velocities velocity_shoulder and velocity_knee
if (velocity_shoulder > velocity_knee) and (shoulder_position < knee_position):
    print("Human has fallen")

Remember to adjust fps according to your video's frame rate. This way, you can incorporate time measurement into your logic seamlessly. Best of luck with your project on human fall detection! Feel free to share any further questions. 🚀

[mihkriss]

Hello, I want to ask you for help
My task is to determine in the video whether there is a fight or not, using a model based on key points,
Could you tell me how to implement this using yolov8-pose ^_^

[pderrenger]

Hello there! 😄 It's awesome that you're looking to use YOLOv8-pose for detecting fights in videos based on key points. Here's a simple way to get started:

First, make sure you've installed the Ultralytics YOLOv8 package. If not, you can do so using pip:

pip install ultralytics

Next, you’ll want to use a YOLOv8 pose estimation model. Since your task is quite specific, I’d recommend starting with our pretrained yolov8n-pose.pt model to detect human poses and then apply your logic to determine if a fight is happening based on the poses/keypoints patterns observed.

Here’s how you can run predictions on a video file:

from ultralytics import YOLO

# Load a pretrained pose model
model = YOLO('yolov8n-pose.pt')

# Run predictions on a video file
results = model.predict('path/to/your/video.mp4', stream=True)

# Process your results here to analyze key points and infer fighting scenarios

You might need to develop a custom algorithm that analyzes the keypoints for each person detected in the video frames, looking for patterns that might indicate a fight (like proximity of people, rapid changes in pose, etc.).

Please note, identifying fights based on motion and pose in videos can be complex and may require further fine-tuning or training the model on a dataset specific to fighting scenarios if the results aren’t satisfactory.

Good luck with your project, and feel free to reach out if you have more questions! 👍

[mihkriss]

Do I understand correctly that I need to first determine the key points (their coordinates)? How to do this with video? and how will they be identified if there are several people there?

[pderrenger]

@mihkriss hey there! Yes, you've got the basic idea right. 😊 To work with videos and detect key points for multiple people, here's a streamlined approach using YOLOv8-pose models, like yolov8n-pose.pt:

1. Model Prediction on Video Frames: Pass each frame of your video to the model. It will automatically detect people and their keypoints.

   from ultralytics import YOLO
   
   # Load a pretrained model
   model = YOLO('yolov8n-pose.pt')
   
   # Predict on a video
   results = model.predict('path/to/your/video.mp4', stream=True)

2. Handling Multiple People: The model outputs keypoints for all detected people in a frame. Each person will have their own set of keypoints.

3. Iterating Through Results: Use a loop to iterate through each result's keypoints. You can differentiate between people by their unique set of keypoints.

Here's an example of accessing keypoints for further processing:

for result in results:
    for person in result.keypoints:
        keypoints = person.xy  # Access X, Y coordinates
        # Your logic to analyze keypoints here

YOLOv8's pose estimation models are pretrained on the COCO keypoints dataset and are optimized to handle multiple people in a scene efficiently. They provide the keypoints' locations along with confidence scores, making it easier to analyze complex scenes. 🚀

Hope this helps! Feel free to reach out if you have more questions.

[mihkriss]

Hello! I have a task to detect fights by key points, I extracted points from the dataset to train the model to detect a fight. I have several questions:

1. In what format should points be stored so that they can be trained later?
2. And once I save the points in the desired format, how can I train the model to determine the fighting pose?
3. And in the final output I need to display only a frame with a label on the video that there is a fight going on here
   Please help me deal with these questions :(

[glenn-jocher]

Hello! It sounds like an interesting project you're working on. Let's address your questions to help get you started with detecting fights using key points:

1. Point Format: The keypoints should be stored in a format that represents the x, y coordinates (and possibly the visibility) of each keypoint. For YOLOv8 pose estimation, keypoints are typically encoded in a list or array per object instance, each element holding the [x, y, visible] information.

2. Training for Fighting Pose: With your points appropriately formatted and saved, you can train a model like yolov8n-pose to recognize fighting poses. Start by organizing your data according to the YOLO pose dataset format, then train the model using command:

   yolo pose train data=your_dataset.yaml model=yolov8n-pose.pt epochs=100 imgsz=640

   Make sure your your_dataset.yaml points to your training and validation data, configured similarly to the coco-pose.yaml.

3. Displaying Frames with a Label: For the output, after prediction, you can process the results to check if a fighting pose is detected and then annotate the frame with a label indicating the fight. Here's a snippet:

   from ultralytics import YOLO
   model = YOLO('path/to/your_trained_model.pt')
   results = model('path/to/video_or_image')
   if 'fight' in results.pandas().xyxy[0]['name'].values:  # assuming 'fight' is a class name
       # This code assumes you're working with frames;
       # you can annotate the frame here to indicate a fight is detected.

I hope this helps clarify your workflow! If you have any more questions or need further assistance, don't hesitate to ask. Happy coding! 😊

[mihkriss]

Why when withdrawing
result.boxes.xyxy
displays
tensor([[ 85., 335., 755., 1521.]])
which does not correspond to the format from 0 to 1

[glenn-jocher]

Hey there! When you get result.boxes.xyxy, it actually returns the bounding boxes in pixel coordinates, not normalized ones. 😊 If you're looking to work with coordinates normalized between 0 and 1, you can do so by dividing by the image dimensions. Here's a quick example:

# Assuming 'result' is your prediction output and 'img' is your input image.
normalized_boxes = result.boxes.xyxy / torch.tensor([img.shape[1], img.shape[0], img.shape[1], img.shape[0]]).to(result.boxes.device)

This will give you the coordinates in the format you're looking for!

For pose estimation using YOLOv8, keypoints are usually the focus, but knowing how to handle bounding boxes is definitely useful too. If there's anything else you'd like to dive into, feel free to ask! 🌟

[gustavoortiz7]

hola buenas
estoy haciendo un proyecto de correción deportiva , remates de voley , y bueno para poder lograr esto intento usar yolov8-pose para , poder obtener los puntos clases de la persona y luego convertirlos en secuencias para una lstm , quería saber cómo puedo obtener los puntos claves pero solo de la persona que hace el remate , por que yolo de detecta a todas las personas que ve en el frame y los se quiere hacer es que en ves de mandarle un vídeo , mandarle una dirección para así poder leer todos los fotogramas de distintas acciones , bueno en este caso solo son dos (rematebueno y remate malo) no trabajo con video por que , saca muchos fotogramas repetidas aunque al final mi proyecto debe detectar en tiempo real el error que cometió , agradecería mucho si puedes proporcionarme como convertir los puntos clases en secuencias y si tiene algún concejos de cómo abordar el problema de mejor manera gracias

[pderrenger]

¡Hola! Para focalizar la detección de puntos clave únicamente en la persona que realiza el remate en vóley, una estrategia sería primero identificar a esa persona específica usando algún criterio (como la posición en el marco) y luego ejecutar la estimación de pose solamente en esa persona detectada.

Aquí un breve ejemplo de cómo podrías hacerlo:

from ultralytics import YOLO

# Carga tu modelo de estimación de pose
model = YOLO('yolov8n-pose.pt')

# Para procesar cada imagen de tu directorio
for imagen in tus_imagenes:
    # Realiza detecciones
    results = model(imagen)
    
    # De alguna manera determina cuál de los detectados es el jugador haciendo el remate
    # Esto podría implicar filtrar por ubicación si siempre rematan desde una posición similar
    jugador_remate = filtrar_jugador(results)
    
    if jugador_remate:
        # Convierte los puntos clave (keypoints) del jugador a secuencias para tu LSTM
        keypoints = jugador_remate.keypoints
        secuencia = convertir_a_secuencia(keypoints)
        # Usa secuencia para tu LSTM...

Este código es solo un esqueleto para darte una idea. La función filtrar_jugador sería personalizada para identificar al jugador correcto, y convertir_a_secuencia dependerá de cómo requieras estructurar los datos para tu LSTM.

Recuerda, el rendimiento efectivo dependerá mucho de la precisión de tu modelo YOLOv8-pose y cómo de bien puedes filtrar al jugador que realiza el remate. Podría serte útil recortar previamente la región de interés (en donde el jugador lleva a cabo el remate) antes de pasarle la imagen al modelo de pose.

Para leer los fotogramas desde una dirección específica, puedes usar algún método para iterar sobre los archivos en ese directorio. Python y las bibliotecas de manejo de archivos te permitirán hacer esto de manera bastante sencilla.

Finalmente, para el análisis en tiempo real, después de haber entrenado tu modelo LSTM con suficientes datos, podrás clasificar los remates en tiempo real procesando individualmente los frames de vídeo capturado.

Espero que esto te ayude en tu proyecto de corrección deportiva. ¡La combinación de YOLOv8-pose y una LSTM suena prometedora!🚀

[Bekbo01]

Hi there! Is it possible to train pose model for 2 or more classes? If it possible can you share with examples?

[pderrenger]

@Bekbo01 hey there! 🌟 Yes, you can have a .yaml configuration for training models with two or more classes. Here's a simplified example for a pose estimation task with two classes:

# custom-pose-dataset.yaml
train: /datasets/coco128/images/train2017/  # path to training images
val: /datasets/coco128/images/val2017/      # path to validation images

nc: 2  # number of classes
names: ['Class1', 'Class2']  # class names

# Define keypoints for each class
keypoints:
  - [nose, left_eye, right_eye, left_ear, right_ear, left_shoulder, right_shoulder, left_elbow, right_elbow, left_wrist, right_wrist, left_hip, right_hip, left_knee, right_knee, left_ankle, right_ankle]  # Class1 keypoints
  - [point1, point2, point3, point4]  # Class2 keypoints
  
# Example for specific keypoints count per class if different
num_keypoints:
  - 17  # Number of keypoints for Class1
  - 4   # Number of keypoints for Class2

Make sure the keypoints and num_keypoints sections reflect the actual structure of your dataset for each class. You'll need to customize paths and names accordingly. This configuration allows you to define different sets of keypoints for each class, which is especially useful in diverse datasets where different objects require different keypoints for accurate pose estimation. 🚀

[Bekbo01]

hey there! In wich version of yolo it works? I got error KeyError: 'kpt_shape' (if add just kpt_shape it will work inly for one class)

[pderrenger]

Hey there! 👋 It seems you encountered a KeyError: 'kpt_shape' because the kpt_shape parameter might be missing or improperly set for multiple classes in your dataset configuration. For pose estimation with multiple classes in YOLOv8, you should define num_keypoints for each class within your .yaml dataset configuration file, not just kpt_shape.

Here's a quick fix using an example .yaml config tailored for two classes:

# Your custom-pose-dataset.yaml
train: /path/to/train/images
val: /path/to/val/images

nc: 2  # number of classes
names: ['Class1', 'Class2']  # Class names

# Define keypoints for each class
num_keypoints:
  - 17  # Number of keypoints for Class1
  - 5   # Number of keypoints for Class2

Make sure to adjust the train, val paths, names, and num_keypoints according to your specific dataset and classes. This configuration will help you avoid the KeyError and work seamlessly with multiple classes. Hope this helps! Let me know if you have more questions! 🚀

[lilin19890401]

When we training multiple classes and the .yaml as you mentioned above，two classes with different number of keypoints. How do we modify nc and kpt_shape in yolov8n-pose.yaml ?

[pderrenger]

@lilin19890401 hello! For training a YOLOv8 pose model with multiple classes where each class has a different number of keypoints, you don't need to modify nc or kpt_shape directly in the yolov8n-pose.yaml. Instead, you should specify num_keypoints for each class in your dataset configuration YAML file. Here’s how you can structure it:

nc: 2  # Number of classes
names: ['Class1', 'Class2']  # Class names

# Specify keypoints for each class
num_keypoints:
  - 17  # Number of keypoints for Class1
  - 5   # Number of keypoints for Class2

This setup tells the model how many keypoints to expect for each class, allowing for different configurations per class. Make sure your dataset labels match this configuration! Let me know if you need further assistance. 😊

[AdadAlShabab]

Hi. I want to measure the length of detected line after running the pose detection model. How can I do it? Please let me know.

[glenn-jocher]

Hi! To measure the length of a detected line after running the pose detection model, you can compute the Euclidean distance between keypoints. If you have keypoints' coordinates, you can use the following Python code example:

import numpy as np

def calculate_distance(pt1, pt2):
    return np.sqrt((pt2[0] - pt1[0])**2 + (pt2[1] - pt1[1])**2)

# Example keypoints coordinates
keypoint1 = [x1, y1]  # Replace x1, y1 with your actual coordinates
keypoint2 = [x2, y2]  # Replace x2, y2 with your actual coordinates

# Calculate distance
length = calculate_distance(keypoint1, keypoint2)
print('The length of the line is:', length)

Just replace x1, y1, x2, y2 with the actual coordinates of your keypoints. This will give you the length between these two points which can represent a line in your use case. Hope this helps! 😊

[coopzoop]

Is it possible to convert into acceleration data of x y and z? Of the wrist specifically?

[pderrenger]

Currently, Ultralytics YOLOv8 models, including those for pose estimation, do not directly calculate or provide acceleration data for wrist movements in terms of x, y, and z axes. These models focus on identifying and locating keypoints in images, such as wrist positions in 2D space.

However, you can derive approximate acceleration from consecutive frames if you track the wrist's position changes over time and apply basic physics formulas. Here’s a simple approach using the difference in positions across frames:

1. Use a YOLOv8 pose model to detect wrist positions in consecutive video frames.
2. Calculate the wrist's velocity changes between frames.
3. Derive acceleration from the velocity changes.

Here's a very basic example in Python, assuming you have the wrist coordinates for three consecutive frames:

def calculate_acceleration(pos1, pos2, pos3, time_interval):
    # Calculate velocity
    velocity1 = (pos2 - pos1) / time_interval
    velocity2 = (pos3 - pos2) / time_interval
    
    # Calculate acceleration
    acceleration = (velocity2 - velocity1) / time_interval
    return acceleration

# Positions (pos1, pos2, pos3) should be numpy arrays or similar that represent wrist coordinates
# 'time_interval' is the time between frames in seconds

This code does not account for complexities like changes in frame rate or noise in detection, but it outlines a starting point. For precise calculations, especially in a dynamic environment, consider further enhancements such as filtering and more sophisticated kinematic models.

[pderrenger]

Yes, it's possible to derive acceleration data of the wrist (x, y, and z coordinates) by tracking the position over time and applying basic physics calculations. You'll need to track the wrist keypoints across frames to calculate the velocity and subsequently the acceleration.

Here's a basic outline of how you might approach this:

1. Track wrist keypoints across successive frames using a pose estimation model like yolov8n-pose.pt.
2. Calculate velocity between frames as the change in position over time.
3. Derive acceleration as the change in velocity over time.

If you're using Python, you'd capture the keypoints from the model prediction, then apply differences and time deltas:

from ultralytics import YOLO

# Load a model
model = YOLO('yolov8n-pose.pt')

# Predict with the model
results = model('path_to_video_or_stream')

# Assuming you have wrist keypoint results for sequential frames
# Calculate velocity and acceleration
# for simplicity, consider results is a list of wrist keypoints (x, y, visible)
velocities = [(results[i] - results[i-1])/frame_time for i in range(1, len(results))]
accelerations = [(velocities[i] - velocities[i-1])/frame_time for i in range(1, len(velocities))]

Ensure you handle cases where the wrist might not be detected in frames (visible flag in keypoints). This is a simplified explanation. Depending on your precision requirements, further refinements might be necessary!

[seagochen]

Hi,

I am currently using C/C++ to deploy YoloPose (the model I used is "yolov8s-pose") on some device for my personal research topic. But I am little confused that comparing with other normal Yolo models, the size of output from inference is different. I have checked the size of the output tensor, it is [1, 56, 8400].

So, I don't know the definition of the output tensor. As I have checked the documents you guys provided in this site, there is nothing about the dimension definition. Can you tell me something? And it's always appreciate if you can help.

Best regards

[glenn-jocher]

Hello!

For the YOLOv8s-pose model, the output tensor size [1, 56, 8400] you're seeing corresponds to the pose estimation outputs. Here's a breakdown of these dimensions:

• 1: Represents the batch size, which is 1 in this case.
• 56: This is the number of channels. For pose estimation, this typically includes the coordinates for keypoints (x, y) and possibly additional data like confidence scores.
• 8400: Represents the flattened spatial dimensions of the output grid. Each point in this grid corresponds to potential keypoints detected by the model across the input image.

Each keypoint detected by the model has an associated set of attributes (like x, y coordinates and confidence), which are encoded in the 56 channels.

If you need to convert these outputs to a more interpretable format (like extracting specific keypoints), you'll typically reshape or process this tensor according to the specifics of how keypoints are organized in those 56 channels.

For more detailed handling and visualization of keypoints, you might find our pose estimation documentation helpful: Pose Estimation Guide.

Best regards!

[Madhanbs67]

Hi I need to create small project it has to calculate the length of shoulder can you please anyone help me in this ,I need code for this

[pderrenger]

Hi there! To calculate the length of the shoulder using YOLOv8 pose estimation, you can follow these steps:

1. Load the YOLOv8 Pose Model:

   from ultralytics import YOLO
   
   model = YOLO("yolov8n-pose.pt")  # Load the pretrained pose model

2. Run Predictions:

   results = model("path/to/your/image.jpg")  # Predict on your image

3. Extract Keypoints:

   keypoints = results[0].keypoints.xy  # Get keypoints
   shoulder_length = ((keypoints[5][0] - keypoints[6][0])**2 + (keypoints[5][1] - keypoints[6][1])**2)**0.5
   print(f"Shoulder Length: {shoulder_length}")

This will give you the distance between the left and right shoulder keypoints. For more details, check out the Pose Estimation Docs. Good luck with your project! 😊

[Madhanbs67]

I'm getting Index error:index 5 is out of bounds for dimension 0 with Size 1

[pderrenger]

Hi there! It looks like you're encountering an IndexError. This usually happens when the model doesn't detect enough keypoints in the image. Make sure the image has clear and visible keypoints for the model to detect.

Here's a quick check:

1. Ensure the image quality is good and keypoints are visible.
2. Verify the model is correctly loaded and used.

For more details on pose estimation, you can refer to the Pose Estimation Docs.

If the issue persists, please share more details about your input image and code snippet. 😊

[xmdys]

[pderrenger]

For setting the visibility value in pose estimation tasks with YOLOv8, you can follow these guidelines for the three situations you described:

1. Point Outside the Image: Set the visibility to 0. This indicates that the keypoint is not visible and is outside the image boundaries.
2. Point Inside the Image and Visible: Set the visibility to 2. This shows that the keypoint is clearly visible and within the image.
3. Point Inside the Image but Not Visible: Set the visibility to 1. Use this to denote that the keypoint is occluded or not visible even though it's within the image frame.

These settings help the model understand the state of each keypoint during training and inference, improving the accuracy of pose estimation. If you need further details on configuring your pose estimation tasks, feel free to check the documentation or ask more questions here! 😊

[polmagri]

I'm trying to map a person's pose in 3D using a depth camera. I have successfully extracted the key points of interest, but when I try to calculate them in 3D using this function and then plot them, I get strange plots. Is there an error in the function, or is the camera unable to accurately distinguish the depth of the keypoints?

def calculate_3d(x, y, depth_frame, intrinsics, w, h, window_size=50):
min_depth = float('inf')

# Iterate over the pixels within the circular window around the keypoint
for i in range(-window_size // 2, window_size // 2 + 1):
    for j in range(-window_size // 2, window_size // 2 + 1):
        # Calculate the coordinates of the current pixel
        x_pixel = int(x) + i
        y_pixel = int(y) + j


        # Check if the current pixel is within the image bounds
        if 0 <= x_pixel < w and 0 <= y_pixel < h:
            # Get the depth value at the current pixel
            depth = depth_frame.get_distance(x_pixel, y_pixel)

            # Update the minimum depth value if the current depth is valid and smaller than the current minimum
            if depth != 0 and depth < min_depth:
                min_depth = depth

# If no valid depth value is found within the circular window, return zeros for all coordinates and angles
if min_depth == float('inf'):
    return 0.0, 0.0, 0.0, 0.0

x_normalized = (x - intrinsics.ppx) / intrinsics.fx
y_normalized = (y - intrinsics.ppy) / intrinsics.fy

# Compute 3D point coordinates
x_3d = min_depth * x_normalized
y_3d = min_depth * y_normalized
z_3d = np.sqrt(min_depth**2 - x_3d**2 - y_3d**2) if min_depth > 0 else 0  

return x_3d, y_3d, z_3d, min_depth

[pderrenger]

It sounds like you're encountering issues with depth accuracy in your 3D pose estimation. This could be due to several factors, including the quality of the depth data from your camera or the method used to calculate the 3D coordinates.

In your function, you're calculating the minimum depth within a window around each keypoint, which is generally a good approach. However, ensure that your depth camera provides accurate and consistent depth measurements. Inaccuracies in depth data can lead to the strange plots you're observing.

Additionally, consider checking the calibration of your camera's intrinsics, as any error in these parameters can significantly affect the conversion from 2D to 3D coordinates.

If you continue to experience issues, you might want to experiment with different window sizes or methods for selecting the depth value (e.g., median depth within the window instead of minimum) to see if there's an improvement.

For more detailed guidance on pose estimation with YOLOv8, you can refer to our Pose Estimation Documentation.

[mobin3399]

Hello.
I want to work with pose for the first time.
I had a few questions. In my images, I want to mark the intersection of two lines with a point, which may have several intersections in each image.
I would be grateful if you could give me some advice about annotation in this way.
The next question is, is it possible to draw the specified points during predict in a binary environment?
Thank you for taking the time to answer this question.

[mobin3399]

0 0.4837239583333333 0.5 0.9075520833333334 0.9259259259259259 0.1171875 0.17592592592592593 0.2747395833333333 0.4228395061728395 0.2994791666666667 0.43209876543209874 0.3528645833333333 0.37037037037037035 0.3606770833333333 0.1882716049382716 0.4505208333333333 0.1882716049382716 0.5559895833333334 0.27469135802469136 0.51171875 0.16049382716049382 0.6640625 0.23148148148148148 0.7682291666666666 0.36419753086419754 0.81640625 0.38271604938271603 0.5872395833333334 0.5462962962962963 0.5768229166666666 0.4382716049382716 0.5169270833333334 0.3271604938271605 0.3658854166666667 0.6234567901234568 0.5989583333333334 0.6574074074074074 0.82421875 0.47530864197530864

[mobin3399]

I hope it helps.
Thankful

[glenn-jocher]

Hello @mobin3399,

Thank you for sharing the output from your script. Based on the data you provided, it seems that the format is mostly correct, but there might be an issue with how the keypoints are being formatted or interpreted by the model during training.

The format you've shown:

0 0.4837239583333333 0.5 0.9075520833333334 0.9259259259259259 0.1171875 0.17592592592592593 0.2747395833333333 0.4228395061728395 0.2994791666666667 0.43209876543209874 0.3528645833333333 0.37037037037037035 0.3606770833333333 0.1882716049382716 0.4505208333333333 0.1882716049382716 0.5559895833333334 0.27469135802469136 0.51171875 0.16049382716049382 0.6640625 0.23148148148148148 0.7682291666666666 0.36419753086419754 0.81640625 0.38271604938271603 0.5872395833333334 0.5462962962962963 0.5768229166666666 0.4382716049382716 0.5169270833333334 0.3271604938271605 0.3658854166666667 0.6234567901234568 0.5989583333333334 0.6574074074074074 0.82421875 0.47530864197530864

appears to correctly list the class index followed by the normalized bounding box coordinates and then the keypoints. However, the error message about needing 119 columns suggests there might be a misunderstanding about the number of keypoints or additional data expected.

Here are a few steps to troubleshoot and resolve this issue:

1. Check Keypoint Count: Ensure that the number of keypoints (and their visibility if applicable) matches what your model configuration expects. If your model expects more data points per keypoint (like visibility), you need to include that in your label files.

2. Validate Format: Double-check that there are no extra spaces or missing values in your .txt files. Each line should strictly follow the format without additional spaces at the end.

3. Model Configuration: Review your model's configuration file (usually a .yaml file) to confirm that the number of keypoints and other parameters like kpt_shape are correctly defined.

4. Manual Inspection: Sometimes manually inspecting a few label files can help spot inconsistencies or formatting errors that might not be obvious.

If you continue to experience issues, it might be helpful to share more details about your model configuration and a few more examples of your label files. This way, I can provide more targeted assistance.

Thank you for your patience, and looking forward to getting this resolved!

[mobin3399]

Thank you for your good support.
I used roboflow for annotation and was able to train the model without any problems. So probably the problem was with the previous site, which I used.
I have some questions about pose.
I only used point detection, in my pictures I specify the intersection points of the lines with these points, and finally, I can determine the type of lines according to a series of parameters and these detected points. It takes a lot of precision to determine the type of lines.
Can pose do this?
And tell me the right way to achieve the highest precision and the best result?
Thank you very much for the time and energy you put into answering my questions. 😊

[glenn-jocher]

Hello!

I'm glad to hear that switching to Roboflow for annotation resolved your training issues! Regarding your questions about using pose estimation for precision line type determination based on intersection points:

1. Precision and Use Case: YOLOv8 pose models are quite adept at detecting keypoints with high precision, which includes intersection points if they are defined as keypoints during the training phase. However, they are generally optimized for human pose estimation scenarios. For your specific use case of line type determination based on precise point intersections, additional customization might be necessary.

2. Achieving High Precision: To maximize precision:

   • High-Quality Data: Ensure your training data is very accurately annotated. Precision in training translates to precision in prediction.
   • Adequate Training: Use a sufficiently large and representative dataset to train the model. More data generally helps the model generalize better but also learn specifics more accurately.
   • Model Choice: Consider using higher capacity models (like YOLOv8x-pose) if your computational resources allow, as they generally achieve better precision.

3. Post-Processing: After detecting keypoints, you might need to implement additional algorithms to analyze these points to determine line types based on your specific criteria. This could involve geometric analysis or even training another model to classify line types based on the configurations of points.

For the highest precision and best results, consider iteratively testing and refining your model and post-processing steps based on the performance you observe in your validation results.

Thank you for your engaging questions, and I'm here to help if you need further assistance! 😊

[sherry202311]

can we use the 3D dataset？

[pderrenger]

Currently, our YOLOv8 pose models are optimized for 2D keypoints detection and do not directly support 3D datasets. They are trained primarily on 2D keypoints from the COCO dataset. For tasks requiring 3D pose estimation, you might need to look into specialized models or frameworks designed for 3D spatial data. If you have further questions or need guidance on a specific use case, feel free to ask! 🌐

[sherry202311]

Got it. Thanks for the prompt reply.

[pderrenger]

You're welcome! If you have any more questions or need further assistance as you work with pose estimation tasks, feel free to reach out. Happy modeling! 😊

[seasuun]

Hi, I have been having problems when printing a specific keypoint in real time with a web cam, at least the lower body ones are not printed but if I print all the detected keypoints I get a tensor.

[pderrenger]

Hello,

It sounds like you're experiencing issues with real-time keypoint detection, particularly with the lower body keypoints not being displayed. This could be related to the visibility and confidence thresholds set for detecting these keypoints.

Here are a few things you might want to check:

1. Visibility and Confidence: Ensure that the keypoints have a high enough confidence score and are marked as visible in the output tensor.
2. Model Output: Double-check the model's output tensor structure to ensure you're correctly interpreting the keypoints' data.
3. Threshold Adjustments: Consider adjusting the confidence and visibility thresholds in your model's prediction settings.

If you continue to face issues, could you please provide a snippet of how you're processing and printing the keypoints? This might help in diagnosing the issue more effectively.

[Brenda-Caroline]

Hello, is there a configuration file where the reference points in the poses are defined? And when pose estimation predictions are made, is a file created that saves the coordinates of the tested image with each reference point?

[pderrenger]

Hello,

Great questions! Let’s dive into the details:

Keypoint Coordinates

The x and y values for keypoints are indeed derived from the pixel coordinates of the image. These coordinates are normalized to be between 0 and 1, relative to the image dimensions. This normalization ensures that the model can generalize across different image sizes.

Visualization of Keypoints

To visualize the Cartesian plane and identify the coordinates of each keypoint, you can plot the keypoints on the image using a simple script. Here’s an example in Python:

import cv2
from ultralytics import YOLO

# Load a pretrained YOLOv8n-pose model
model = YOLO("yolov8n-pose.pt")

# Run predictions on an image
results = model("path/to/your/image.jpg")

# Visualize keypoints
for result in results:
    img = result.orig_img
    for keypoint in result.keypoints:
        x, y = keypoint[:2]
        cv2.circle(img, (int(x * img.shape[1]), int(y * img.shape[0])), 5, (0, 255, 0), -1)
    cv2.imshow("Keypoints", img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

Analyzing Bounding Boxes and Keypoints

To analyze whether the bounding boxes and keypoints overlap, you can compare the coordinates. If the keypoints fall within the bounding box, they overlap. Additionally, you can check if the lines connecting the keypoints intersect with the bounding box. Here’s a basic approach:

import cv2
from ultralytics import YOLO

# Load a pretrained YOLOv8n-pose model
model = YOLO("yolov8n-pose.pt")

# Run predictions on an image
results = model("path/to/your/image.jpg")

# Analyze keypoints and bounding boxes
for result in results:
    img = result.orig_img
    for box in result.boxes:
        x, y, w, h = box.xywh
        bbox = (int(x - w/2), int(y - h/2), int(x + w/2), int(y + h/2))
        cv2.rectangle(img, bbox, (255, 0, 0), 2)
        
    for keypoint in result.keypoints:
        x, y = keypoint[:2]
        cv2.circle(img, (int(x * img.shape[1]), int(y * img.shape[0])), 5, (0, 255, 0), -1)
        
    # Check for intersections (simplified example)
    for i in range(len(result.keypoints) - 1):
        x1, y1 = result.keypoints[i][:2]
        x2, y2 = result.keypoints[i + 1][:2]
        cv2.line(img, (int(x1 * img.shape[1]), int(y1 * img.shape[0])), (int(x2 * img.shape[1]), int(y2 * img.shape[0])), (0, 255, 0), 2)
    
    cv2.imshow("Analysis", img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

This script will draw the bounding boxes and keypoints on the image, and you can visually inspect the overlaps.

For more detailed information on pose estimation, you can refer to the Pose Estimation documentation.

If you have any further questions or need additional assistance, feel free to ask. Happy coding! 🚀

[Brenda-Caroline]

Hello, I didn't understand the repetition loop in these codes. But first, I would like to review the way I obtain data from both keypoints and boxes, to see if I understand correctly what each one returns. I will put here how I understood the results, if my interpretation is wrong, please correct me.

Related to keypoints: There are two ways to return this data:

1. Using the save_txt= True attribute, for example:
   results = model(source = "/content/asd2-250.jpg", save= True, save_txt= True, save_conf= True, save_crop= True)

With this code, a new image is created with the keypoints, the text file referring to the keypoints, the confidence of each keypoint and the confidence of the class classification. And with the save_crop attribute, an image is created with the crop of each object found.

Text file result

0 0.432552 0.590278 0.203646 0.515741 0.48459 0.475335 0.791891 0 0 0.220644 0.477587 0.458518 0.864895 0 0 0.0619751 0.451167 0.446709 0.906585 0.435616 0.467884 0.888522 0.433296 0.507795 0.987799 0.442328 0.522698 0.682794 0.446462 0.63474 0.98396 0.473841 0.559181 0.713864 0.490251 0.697015 0.967138 0.386416 0.635135 0.97207 0.377665 0.65725 0.989551 0.477954 0.67088 0.93601 0.453298 0.711505 0.979735 0.391314 0.717061 0.874492 0.360123 0.77071 0.933644 0.893211

Looking at the return of the text file, is it correct to say that it has the format of dimension 3? Where returns
<class-index><x><y><width><height><px1><py1><p1-visibility><pxn><pyn><pn-visibility>

The first value in bold is <class-index>, the values ​​in italics are <x><y><width><height>, followed by <pxn><pyn> and <pn-visibility> in bold and italics (it was done in just a few values ​​as an example). This third visibility value, does it mean the confidence of the keypoint detection or its visibility in the image? And the last value in bold in the file is the confidence of the class classification?

2. The second way is using, for example, the example code you provided me:

with open("keypoints_output.txt", "w") as f:
    for result in results:
        for keypoint in result.keypoints:
            f.write(f"{keypoint}\n")

The result of this text file is:

ultralytics.engine.results.Keypoints object with attributes:

conf: tensor([[0.7919, 0.2206, 0.8649, 0.0620, 0.9066, 0.8885, 0.9878, 0.6828, 0.9840, 0.7139, 0.9671, 0.9721, 0.9896, 0.9360, 0.9797, 0.8745, 0.9336]])
data: tensor([[[9.3041e+02, 5.1336e+02, 7.9189e-01],
         [0.0000e+00, 0.0000e+00, 2.2064e-01],
         [9.1697e+02, 4.9520e+02, 8.6489e-01],
         [0.0000e+00, 0.0000e+00, 6.1975e-02],
         [8.6624e+02, 4.8245e+02, 9.0659e-01],
         [8.3638e+02, 5.0531e+02, 8.8852e-01],
         [8.3193e+02, 5.4842e+02, 9.8780e-01],
         [8.4927e+02, 5.6451e+02, 6.8279e-01],
         [8.5721e+02, 6.8552e+02, 9.8396e-01],
         [9.0977e+02, 6.0392e+02, 7.1386e-01],
         [9.4128e+02, 7.5278e+02, 9.6714e-01],
         [7.4192e+02, 6.8595e+02, 9.7207e-01],
         [7.2512e+02, 7.0983e+02, 9.8955e-01],
         [9.1767e+02, 7.2455e+02, 9.3601e-01],
         [8.7033e+02, 7.6843e+02, 9.7974e-01],
         [7.5132e+02, 7.7443e+02, 8.7449e-01],
         [6.9144e+02, 8.3237e+02, 9.3364e-01]]])
has_visible: True
orig_shape: (1080, 1920)
shape: torch.Size([1, 17, 3])
xy: tensor([[[930.4127, 513.3622],
         [  0.0000,   0.0000],
         [916.9666, 495.1992],
         [  0.0000,   0.0000],
         [866.2413, 482.4456],
         [836.3835, 505.3146],
         [831.9274, 548.4190],
         [849.2693, 564.5138],
         [857.2072, 685.5188],
         [909.7747, 603.9150],
         [941.2827, 752.7763],
         [741.9194, 685.9456],
         [725.1160, 709.8296],
         [917.6708, 724.5505],
         [870.3325, 768.4257],
         [751.3220, 774.4260],
         [691.4368, 832.3667]]])
xyn: tensor([[[0.4846, 0.4753],
         [0.0000, 0.0000],
         [0.4776, 0.4585],
         [0.0000, 0.0000],
         [0.4512, 0.4467],
         [0.4356, 0.4679],
         [0.4333, 0.5078],
         [0.4423, 0.5227],
         [0.4465, 0.6347],
         [0.4738, 0.5592],
         [0.4903, 0.6970],
         [0.3864, 0.6351],
         [0.3777, 0.6572],
         [0.4780, 0.6709],
         [0.4533, 0.7115],
         [0.3913, 0.7171],
         [0.3601, 0.7707]]])

The way I understood the attributes:

conf: A tensor with 17 values, which represents the detection confidence of each keypoint.

data: A tensor of 17 rows and 3 columns. Each line represents a keypoint and contains the x, y and visibility coordinates (the first column being x, the second column y and the third column being visibility, which are also the same values ​​as in conf).

has_visible: A boolean that indicates whether any keypoint is visible in the image.

orig_shape: A tuple that indicates the original height and width of the image.

shape: A tensor that indicates the shape of the data tensor, which is [1, 17, 3] in the case of pose estimation.

xy: A tensor of 17 rows and 2 columns. Each line represents a keypoint and contains the x (column 1), y (column 2) coordinates of the keypoint in the original image.

xyn: Represents the same thing as xy, but with normalized values ​​between 0 and 1.

Now, the question related to the Visualization of Keypoints code:

for result in results:
   img = result.orig_img
   for keypoint in result.keypoints:
      x, y = keypoint[:2]
      cv2.circle(img, (int(x * img.shape[1]), int(y * img.shape[0])), 5, (0, 255, 0), -1)
      cv2.imshow("Keypoints", img)
      cv2.waitKey(0)
      cv2.destroyAllWindows()

In the repeat loop, for each detected object, a clipping of the object will be saved in the img variable. In the internal repetition loop, for each keypoint found, in the code x, y = keypoint[:2] is the intention to save the values ​​of the x and y coordinates in their respective variables? for example, in the return shown above, would it be accessing the values ​​from keypoint.xy and placing the values ​​in column 1 in x and column 2 in y? Or is the intention to access keypoint.data?

As for the code to draw the keypoint circle, img -> will be the image cutout, and img.shape[1] -> will be the cutout height, while img.shape[0] -> will be the cutout width .

[pderrenger]

@Brenda-Caroline hello,

Thank you for your detailed question! Let's clarify your understanding and address your queries.

Keypoints Data Interpretation

1. Using save_txt=True:
   Your interpretation is mostly correct. The text file format for pose estimation with dimension 3 is:

   <class-index> <x> <y> <width> <height> <px1> <py1> <p1-visibility> <px2> <py2> <p2-visibility> ... <pxn> <pyn> <pn-visibility>
   

   • <class-index>: The class of the object.
   • <x> <y> <width> <height>: Bounding box coordinates.
   • <pxn> <pyn> <pn-visibility>: Keypoint coordinates and their visibility/confidence.

   The visibility value indicates the confidence of the keypoint detection. The last value in bold is indeed the confidence of the class classification.

2. Using the provided code:
   Your understanding of the attributes is correct:

   • conf: Confidence scores for each keypoint.
   • data: Keypoint coordinates and visibility.
   • has_visible: Indicates if any keypoint is visible.
   • orig_shape: Original image dimensions.
   • shape: Shape of the keypoint data tensor.
   • xy: Keypoint coordinates in the original image.
   • xyn: Normalized keypoint coordinates.

Visualization of Keypoints

In the provided code snippet:

for result in results:
   img = result.orig_img
   for keypoint in result.keypoints:
      x, y = keypoint[:2]
      cv2.circle(img, (int(x * img.shape[1]), int(y * img.shape[0])), 5, (0, 255, 0), -1)
      cv2.imshow("Keypoints", img)
      cv2.waitKey(0)
      cv2.destroyAllWindows()

• x, y = keypoint[:2] extracts the x and y coordinates from the keypoint data. This line accesses the keypoint.xy values, not keypoint.data.

Analyzing Bounding Boxes and Keypoints

To analyze the overlap between bounding boxes and keypoints, you can check if the keypoints fall within the bounding box and if the lines connecting keypoints intersect with the bounding box. Here’s an example:

import cv2
from ultralytics import YOLO

# Load a pretrained YOLOv8n-pose model
model = YOLO("yolov8n-pose.pt")

# Run predictions on an image
results = model("path/to/your/image.jpg")

# Analyze keypoints and bounding boxes
for result in results:
    img = result.orig_img
    for box in result.boxes:
        x, y, w, h = box.xywh
        bbox = (int(x - w/2), int(y - h/2), int(x + w/2), int(y + h/2))
        cv2.rectangle(img, bbox, (255, 0, 0), 2)
        
    for keypoint in result.keypoints:
        x, y = keypoint[:2]
        cv2.circle(img, (int(x * img.shape[1]), int(y * img.shape[0])), 5, (0, 255, 0), -1)
        
    # Check for intersections (simplified example)
    for i in range(len(result.keypoints) - 1):
        x1, y1 = result.keypoints[i][:2]
        x2, y2 = result.keypoints[i + 1][:2]
        cv2.line(img, (int(x1 * img.shape[1]), int(y1 * img.shape[0])), (int(x2 * img.shape[1]), int(y2 * img.shape[0])), (0, 255, 0), 2)
    
    cv2.imshow("Analysis", img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

This script will draw the bounding boxes and keypoints on the image, allowing you to visually inspect the overlaps.

For more detailed information, you can refer to the Pose Estimation documentation.

If you have any further questions or need additional assistance, feel free to ask. Happy coding! 🚀

[Brenda-Caroline]

Thank you for the return. I was able to access the data regarding the bounding boxes, and the data regarding the keypoints. But I still don't understand it, nor was I able to run the part # Check for intersections (simplified example)

A new question: is it possible to analyze the heuristic: is there interaction between people if the pose estimate of one overlaps the pose estimate of the other?

[pderrenger]

Hello,

Thank you for your detailed questions! Let's address them step-by-step.

Understanding Keypoints and Bounding Boxes

Your understanding of the keypoints and bounding boxes data is correct. The keypoints are represented as [x, y, visibility], where visibility indicates the confidence of the keypoint detection.

Running the Intersection Check

To run the intersection check, you can use the following code snippet. This will help you visualize the keypoints and bounding boxes, and check for intersections:

import cv2
from ultralytics import YOLO

# Load a pretrained YOLOv8n-pose model
model = YOLO("yolov8n-pose.pt")

# Run predictions on an image
results = model("path/to/your/image.jpg")

# Analyze keypoints and bounding boxes
for result in results:
    img = result.orig_img
    for box in result.boxes:
        x, y, w, h = box.xywh
        bbox = (int(x - w/2), int(y - h/2), int(x + w/2), int(y + h/2))
        cv2.rectangle(img, bbox, (255, 0, 0), 2)
        
    for keypoint in result.keypoints:
        x, y = keypoint[:2]
        cv2.circle(img, (int(x * img.shape[1]), int(y * img.shape[0])), 5, (0, 255, 0), -1)
        
    # Check for intersections (simplified example)
    for i in range(len(result.keypoints) - 1):
        x1, y1 = result.keypoints[i][:2]
        x2, y2 = result.keypoints[i + 1][:2]
        cv2.line(img, (int(x1 * img.shape[1]), int(y1 * img.shape[0])), (int(x2 * img.shape[1]), int(y2 * img.shape[0])), (0, 255, 0), 2)
    
    cv2.imshow("Analysis", img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

Analyzing Interaction Between People

To analyze if there is interaction between people based on pose estimation, you can check if the keypoints or the lines connecting keypoints of one person overlap with those of another person. Here's a simplified approach:

import cv2
from ultralytics import YOLO

# Load a pretrained YOLOv8n-pose model
model = YOLO("yolov8n-pose.pt")

# Run predictions on an image
results = model("path/to/your/image.jpg")

# Analyze keypoints and bounding boxes
for result in results:
    img = result.orig_img
    keypoints_list = []
    
    for keypoint in result.keypoints:
        keypoints_list.append(keypoint.xy)
    
    # Check for interactions
    for i in range(len(keypoints_list)):
        for j in range(i + 1, len(keypoints_list)):
            for k in range(len(keypoints_list[i])):
                if keypoints_list[i][k] == keypoints_list[j][k]:
                    print(f"Interaction detected between person {i} and person {j} at keypoint {k}")
    
    cv2.imshow("Analysis", img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

This script will help you identify if there is any overlap between the keypoints of different people, indicating interaction.

For more detailed information, you can refer to the Pose Estimation documentation.

If you have any further questions or need additional assistance, feel free to ask. Happy coding! 🚀

[adityal10]

Hello team,
I am working on building a badminton analysis system and I have to detect the badminton court keypoints and i have taken a dataset from robolow (https://universe.roboflow.com/himani-jain/badmintonc/dataset/4#) and downloaded it accordoing to yolov5.

So is it possible for me to detect the badminton court keypoints with yolov5 and if not what would you recommend my another approach to this problem.

YT tutorial refernce: https://www.youtube.com/watch?v=L23oIHZE14w&t=2125s

[glenn-jocher]

@adityal10 hello,

Thank you for reaching out with your query about detecting badminton court keypoints using YOLOv5. It's great to hear about your project on badminton analysis!

To address your question, YOLOv5 is primarily designed for object detection and not specifically for keypoint detection or pose estimation. However, you can use Ultralytics YOLOv8, which includes specialized models for pose estimation that might be more suitable for your task.

Recommended Approach

1. Using YOLOv8 for Pose Estimation:
YOLOv8 offers pretrained models specifically for pose estimation tasks, such as yolov8n-pose.pt. These models are designed to detect keypoints and can be trained on custom datasets for specific applications like detecting badminton court keypoints.

Here’s how you can get started with YOLOv8 for your task:

Training a YOLOv8 Pose Model:

1. Prepare your dataset in the Ultralytics YOLO format. Each image should have a corresponding .txt file with keypoints annotations.
2. Create a dataset YAML file specifying the paths to your images and annotations, and the keypoints configuration.

Example YAML configuration:

path: ../datasets/badminton
train: images/train
val: images/val
kpt_shape: [4, 2]  # Assuming 4 keypoints for the court, with x, y coordinates
names:
  0: court

3. Train the model using the following Python code:

from ultralytics import YOLO

# Load a pretrained pose model
model = YOLO("yolov8n-pose.pt")

# Train the model on your dataset
results = model.train(data="badminton.yaml", epochs=100, imgsz=640)

Or using the CLI:

yolo pose train data=badminton.yaml model=yolov8n-pose.pt epochs=100 imgsz=640

Validating and Predicting:

After training, you can validate and predict using the trained model:

# Validate the model
metrics = model.val()

# Predict with the model
results = model("path/to/your/image.jpg")

Or using the CLI:

yolo pose val model=path/to/best.pt
yolo pose predict model=path/to/best.pt source='path/to/your/image.jpg'

Additional Resources

For more detailed information on preparing your dataset and training a pose estimation model, please refer to the Pose Estimation Datasets Guide.

If you encounter any issues or have further questions, please ensure you are using the latest versions of torch and ultralytics. If the issue persists, providing a minimum reproducible example will help us assist you better. You can find more details on creating a reproducible example here.

Best of luck with your badminton analysis system! Feel free to reach out if you have any more questions.

[AhmadShaik]

Hi,

I am trying to develop a head pose orientation solution for an IP camera. The process I have used is as follows.

• Used a Yolov8-pose model to detect the 2D skeletons
• Used MotionBERT to convert the skeletons from 2D space to 3D space
• Applied few conditional logics and ML model to detect the head orientation.

This apporach is giving consistent results, when the person is looking stright towards the camera, but its failing when person is looking at the other sides. One thing I have observed was, whenever some body parts are not visible the skeleton points locations are not accurate because of that the orientation detection is not accurate.

Here are few questions I have.

1. Is the head pose estimation pipeline good or are there anything missing
2. Is there any way to improve the accuracy when bodyparts are not visible. Or occlusion will be a limitation for 2D skeleton detection.

[glenn-jocher]

@AhmadShaik hi there,

Thank you for sharing your approach and the detailed description of your head pose orientation solution. It's great to see the innovative use of YOLOv8-pose and MotionBERT in your pipeline. Let's address your questions:

1. Pipeline Evaluation:
   Your pipeline for head pose estimation seems well-structured. Using YOLOv8-pose for 2D skeleton detection and MotionBERT for converting these to 3D space is a solid approach. However, the accuracy of the 2D keypoints is crucial for the subsequent steps. If the 2D keypoints are not accurate, especially under occlusion, it will affect the 3D pose estimation and the final head orientation detection.

2. Improving Accuracy Under Occlusion:
   Occlusion is indeed a common challenge in pose estimation. Here are a few suggestions to improve accuracy:

   • Data Augmentation: Enhance your training data with augmented images that simulate occlusions. This can help the model learn to handle partial visibility better.
   • Model Enhancement: Consider using models that are specifically designed to handle occlusions better. Some advanced pose estimation models incorporate mechanisms to predict occluded keypoints more accurately.
   • Post-Processing: Implement post-processing techniques to refine the keypoints. For instance, using temporal smoothing if you're working with video data can help stabilize the keypoints over time.
   • Confidence Scores: Utilize the confidence scores provided by YOLOv8-pose to weigh the keypoints. Lower confidence scores can indicate less reliable keypoints, which can be handled differently in your logic.

For further details on pose estimation tasks and datasets, you can refer to our Pose Estimation Documentation.

If you have any more specific issues or need further assistance, feel free to provide additional details or a minimum reproducible example. This will help us better understand and address your concerns.

Best of luck with your project! 😊

[AhmadShaik]

@glenn-jocher - Thanks fot the reply. We are trying to enhance the model using data augmentation. And I like that post-processing approach to smooth the keypoint detections. I will incorporate that.

I am interested to take a part in the development or this repo. Please let me know if there is any way I can contribute.

[glenn-jocher]

Hi @AhmadShaik,

Thank you for your enthusiasm and for considering contributing to the Ultralytics repo! We're always excited to welcome new contributors to our community. Here are a few ways you can get involved:

1. Bug Reports and Feature Requests: If you encounter any issues or have ideas for new features, feel free to open an issue on our GitHub repo. Providing detailed descriptions and reproducible examples will help us address them more efficiently.

2. Code Contributions: You can contribute by fixing bugs, adding new features, or improving documentation. Please check our Contributing Guide for detailed instructions on how to submit a Pull Request (PR).

3. Community Support: Engage with other users by answering questions and providing support on GitHub issues. Your expertise can be invaluable to others facing similar challenges.

4. Documentation: Help us improve our documentation by suggesting edits or adding new content. Clear and comprehensive documentation is crucial for helping users get the most out of our tools.

For your current project, incorporating data augmentation and post-processing techniques should indeed help improve the accuracy of your head pose estimation, especially under occlusions. If you have any specific questions or need further assistance, feel free to ask!

Looking forward to your contributions and thank you for being a part of the YOLO community! 😊

[AhmadShaik]

That's a cool idea @glenn-jocher. I will experiment with the suggested changes and will share my experiences. Thanks for the support.

[glenn-jocher]

Hi @AhmadShaik,

Thank you for your positive feedback! I'm glad to hear that the suggestions were helpful. Enhancing your model with data augmentation and incorporating post-processing techniques should indeed improve the accuracy of your head pose estimation, especially under occlusions.

If you encounter any issues or have further questions as you experiment with these changes, feel free to reach out. We're here to help!

Regarding your interest in contributing to the Ultralytics repo, that's fantastic! We welcome contributions from the community. Here are a few ways you can get involved:

1. Bug Reports and Feature Requests: If you find any bugs or have ideas for new features, please open an issue on our GitHub repo. Detailed descriptions and reproducible examples are always appreciated.

2. Code Contributions: You can contribute by fixing bugs, adding new features, or improving documentation. Please check our Contributing Guide for detailed instructions on how to submit a Pull Request (PR).

3. Community Support: Engage with other users by answering questions and providing support on GitHub issues. Your expertise can be invaluable to others facing similar challenges.

4. Documentation: Help us improve our documentation by suggesting edits or adding new content. Clear and comprehensive documentation is crucial for helping users get the most out of our tools.

Looking forward to your contributions and thank you for being a part of the YOLO community! 😊

[seasuun]

Hi, i have a model for human keypoint detection pretrain yolov8 and another that i train for a 6 DOF robot, is there a way that i can use both of them at the same time for a specific purpose? for example i want the robot to stop when the hand of a person is near the tool.

[glenn-jocher]

Hi there!

Thank you for your question. Combining datasets and models for different tasks can indeed be challenging but also very rewarding. Here are a few strategies to help you optimize this process:

1. Merging Datasets: You can merge your human keypoint dataset with your 6 DOF robot dataset. This approach allows you to train a single model that can handle both tasks. However, as you mentioned, training on the entire COCO dataset can be time-consuming. One way to optimize this is to use a subset of the COCO dataset that focuses on the keypoints relevant to your application.

2. Transfer Learning: Since you already have a pretrained model on the COCO dataset, you can fine-tune this model on your 6 DOF robot dataset. This approach leverages the knowledge the model has already gained, reducing the amount of training time needed.

3. Multi-Task Learning: Another approach is to train a multi-task model that can simultaneously handle human keypoint detection and robot pose estimation. This can be more complex but allows for a more integrated solution.

Here’s a simplified example of how you might fine-tune a pretrained model with your custom dataset:

from ultralytics import YOLO

# Load the pretrained model
model = YOLO("yolov8n-pose.pt")

# Fine-tune the model on your custom dataset
results = model.train(data="path/to/your-6dof-robot-dataset.yaml", epochs=50, imgsz=640)

For real-time inference, you can run both models in parallel and integrate their outputs to make decisions, such as stopping the robot when a human hand is near the tool:

import cv2
from ultralytics import YOLO

# Load models
human_model = YOLO("path/to/human-keypoint-model.pt")
robot_model = YOLO("path/to/robot-pose-model.pt")

# Open webcam
cap = cv2.VideoCapture(0)

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Run inference
    human_results = human_model(frame)
    robot_results = robot_model(frame)

    # Extract keypoints
    human_keypoints = human_results[0].keypoints.xy if human_results[0].keypoints else []
    robot_keypoints = robot_results[0].keypoints.xy if robot_results[0].keypoints else []

    # Check distance between human hand and robot tool
    if human_keypoints and robot_keypoints:
        human_hand = human_keypoints[0]  # Assuming the first keypoint is the hand
        robot_tool = robot_keypoints[0]  # Assuming the first keypoint is the tool

        distance = ((human_hand[0] - robot_tool[0]) ** 2 + (human_hand[1] - robot_tool[1]) ** 2) ** 0.5

        if distance < threshold:  # Define your threshold
            print("Stop the robot!")
            # Add your robot stopping code here

    # Display the frame
    cv2.imshow("Frame", frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

For more detailed information on pose estimation and dataset preparation, you can refer to the Ultralytics Pose Estimation Documentation.

Feel free to reach out if you have any more questions. Happy coding! 😊

[seasuun]

Hi, is there a way i can get both model keypoints displays? i only see the screen without any detection or keypoint.

[glenn-jocher]

Hi there!

To display keypoints from both models simultaneously, you can run inference for both models on each frame and then overlay the keypoints on the image. Here’s a concise example to help you get started:

import cv2
from ultralytics import YOLO

# Load models
human_model = YOLO("path/to/human-keypoint-model.pt")
robot_model = YOLO("path/to/robot-pose-model.pt")

# Open webcam
cap = cv2.VideoCapture(0)

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Run inference
    human_results = human_model(frame)
    robot_results = robot_model(frame)

    # Plot results
    if human_results:
        frame = human_results[0].plot()
    if robot_results:
        frame = robot_results[0].plot()

    # Display the frame
    cv2.imshow("Frame", frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Ensure you have the latest versions of torch and ultralytics installed. If you encounter any issues, please provide a minimum reproducible example so we can assist you better. You can find more details on creating a reproducible example here.

Feel free to reach out if you have any more questions. Happy coding! 😊

[seasuun]

It really help but just for one model, but i change a few things and it works. thank u

[glenn-jocher]

Hi there!

I'm glad to hear that you found the initial guidance helpful and were able to make it work for one model. To display keypoints from both models simultaneously, you can run inference for both models on each frame and then overlay the keypoints on the image. Here's an enhanced example to help you achieve this:

import cv2
from ultralytics import YOLO

# Load models
human_model = YOLO("path/to/human-keypoint-model.pt")
robot_model = YOLO("path/to/robot-pose-model.pt")

# Open webcam
cap = cv2.VideoCapture(0)

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Run inference
    human_results = human_model(frame)
    robot_results = robot_model(frame)

    # Plot results
    if human_results:
        frame = human_results[0].plot()
    if robot_results:
        frame = robot_results[0].plot()

    # Display the frame
    cv2.imshow("Frame", frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

This script captures frames from your webcam, runs inference using both models, and overlays the keypoints on the frame. If you need more detailed control over the keypoints and their visualization, you can manually draw the keypoints and connections using OpenCV functions.

For further optimization and detailed information on pose estimation, you can refer to the Ultralytics Pose Estimation Documentation.

Feel free to reach out if you have any more questions or need further assistance. Happy coding! 😊

[mobin3399]

Hello to the ultralytics group.
When estimating the pose in an image, consider an original, standard size image.
Now I cut this image into two different sizes.
If we give the cropped images directly to the algorithm. Definitely, in the first step, there will be a mistake in recognizing the object class. And in the second stage, there are errors and mistakes in placing the points on the object again.
My solution was to take the size of the cropped image and by adding black pixels, taking into account that the aspect ratio of the image would not be messed up, I would resize the cropped image to the standard size of the image and then give it to the network.
Here I was partially successful because according to the percentage of confidence in the original image, here it displayed a lower percentage and also accurately and correctly recognizes the class name and the location of the points.
Now, if I change the black color with which the new pixel will be made to white or another color, the name of the class will be wrong, but to a large extent, the location of the correct points will be predicted.
Now, the size of the cropped photo has a great effect on the above.

Thank you for finding my problem and providing the right solution.

[glenn-jocher]

@mobin3399 hello!

Thank you for sharing your detailed observations and solution approach regarding pose estimation with cropped images. Your method of adding black pixels to maintain the aspect ratio is indeed a practical approach to ensure consistency in object detection and keypoint placement.

To address your concern about the impact of different padding colors (black vs. white) on the model's performance, it's important to understand that the model's training data likely influences its sensitivity to background colors. Models trained on datasets with predominantly dark or black backgrounds might perform better with black padding, while those trained with varied or lighter backgrounds might handle white padding differently.

Here are a few suggestions to further improve your results:

1. Consistent Padding: Stick with a padding color that closely matches the background color distribution in your training dataset. If the original dataset has varied backgrounds, consider using a neutral color like gray.

2. Data Augmentation: During training, include data augmentation techniques that simulate different padding scenarios. This can help the model generalize better to different background colors.

3. Model Fine-Tuning: Fine-tune your model with a subset of images that include your specific padding technique. This can help the model adapt to the new input format.

4. Experiment with Different Models: Try using different YOLOv8-pose models (e.g., yolov8s-pose.pt, yolov8m-pose.pt) to see if a larger model with more parameters handles the padding variations better.

Here’s a quick example of how you can fine-tune a YOLOv8-pose model with your custom dataset:

from ultralytics import YOLO

# Load a pretrained model
model = YOLO("yolov8n-pose.pt")

# Fine-tune the model with your custom dataset
results = model.train(data="your-custom-dataset.yaml", epochs=50, imgsz=640)

For more detailed guidance on training and validating pose models, you can refer to our Pose Estimation Documentation.

I hope this helps! Feel free to share any further insights or questions you might have. 😊

[mobin3399]

Hi again.
Thank you for your guidance. Another important point, I trained a model for segmentation and called the code from the ultralutics document to cut the boundary of the object in two cases: without background at all or black background.
The point is that when I used images without a background for the pose estimation network, the model automatically added a background to my photo, and the background was the original photo before the segmentation and cropping model.
But this did not happen in the images that had a completely black background.

[glenn-jocher]

Hello again!

Thank you for your detailed follow-up. It’s great to see your progress and the insights you’ve gathered regarding the background handling in pose estimation.

Background Handling in Pose Estimation

From your observations, it seems that the model's behavior varies based on the background color of the input images. Here are a few points to consider:

1. Model Training and Backgrounds: The model's performance can be influenced by the types of backgrounds it was trained on. If the training dataset had a variety of backgrounds, the model might generalize better to different background colors. However, if it was predominantly trained on images with specific background colors, it might perform better with similar backgrounds during inference.

2. Padding and Background Color: When you pad images with black pixels, it seems to work better because the model might be more accustomed to such padding from the training phase. Using other colors might introduce artifacts that the model interprets differently, affecting the class prediction and keypoint placement.

Recommendations

1. Consistent Backgrounds: Stick to a consistent background color that closely matches the training data. If black padding works better, continue using it.

2. Data Augmentation: During training, include images with various background colors and padding to help the model generalize better. This can be achieved through data augmentation techniques.

3. Fine-Tuning: Fine-tune your model with a subset of images that include your specific padding technique. This can help the model adapt to the new input format.

Example Code for Fine-Tuning

Here’s an example of how you can fine-tune your model with custom data:

from ultralytics import YOLO

# Load a pretrained model
model = YOLO("yolov8n-pose.pt")

# Fine-tune the model with your custom dataset
results = model.train(data="your-custom-dataset.yaml", epochs=50, imgsz=640)

Further Reading

For more detailed guidance on pose estimation, you can refer to our Pose Estimation Documentation.

I hope this helps! Feel free to share any further insights or questions you might have. 😊

[xooso]

Is it possible to export the coordinates of the prediction box and keypoints?

[pderrenger]

@xooso yes, it is indeed possible to export the coordinates of the prediction boxes and keypoints using Ultralytics YOLOv8. Here's how you can achieve this:

Python Example

You can use the results object returned by the model's prediction method to access the coordinates of the bounding boxes and keypoints. Here's a concise example:

from ultralytics import YOLO

# Load a model
model = YOLO("yolov8n-pose.pt")  # load an official model

# Predict with the model
results = model("https://ultralytics.com/images/bus.jpg")  # predict on an image

# Extract coordinates
for result in results:
    boxes = result.boxes.xyxy  # Bounding box coordinates
    keypoints = result.keypoints.xy  # Keypoint coordinates
    print("Boxes:", boxes)
    print("Keypoints:", keypoints)

CLI Example

If you prefer using the command line interface, you can run predictions and save the results to a text file:

yolo pose predict model=yolov8n-pose.pt source='https://ultralytics.com/images/bus.jpg' save_txt=True

This command will save the bounding box and keypoint coordinates to a text file in the runs/predict/exp directory.

For more detailed information on pose estimation, you can refer to the Pose Estimation Documentation.

Feel free to reach out if you have any more questions! 😊