Ultralytics
integrations/tfjs/ #13963
giscus[bot]
bot
Replies: 2 comments 13 replies
-
|
Hello, i tried to use exported model with tfjs but it threw an error. This is error message: Can you help me? |
Beta Was this translation helpful? Give feedback.
-
|
Hello, Thank you for reaching out! It looks like you're encountering an issue with running inference using a YOLOv8 model exported to TensorFlow.js. Currently, YOLOv8 does not support direct inference with TF.js models within the Ultralytics Python environment, which is why you're seeing the To proceed, you can use the exported TF.js model directly in a web or Node.js environment. Here’s a brief guide on how to do that:
For more detailed instructions on deploying TF.js models, you can refer to the TensorFlow.js documentation. If you encounter any further issues, please ensure you are using the latest versions of the packages and provide a minimum reproducible example as outlined here. I hope this helps! If you have any more questions, feel free to ask. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
I reply, that executeAsync(tensor) doesn't give me a prediction of model but tensor datatype like this : |
Beta Was this translation helpful? Give feedback.
-
|
@salim4n hello, Thank you for your detailed feedback. It appears that the Here's a step-by-step guide to help you process the tensor output and obtain the predictions:
Here's a complete example in JavaScript: async function runInference() {
// Load the model
const model = await tf.loadGraphModel('/path/to/yolov8n_web_model/model.json');
// Load and preprocess the image
const img = document.getElementById('input-image');
const tensor = tf.browser.fromPixels(img).expandDims(0).toFloat().div(tf.scalar(255));
// Run inference
const predictions = await model.executeAsync(tensor);
// Process the output tensors
const [boxes, scores, classes] = predictions;
const boxesData = boxes.dataSync();
const scoresData = scores.dataSync();
const classesData = classes.dataSync();
// Filter predictions by confidence threshold
const confidenceThreshold = 0.5;
const validDetections = [];
for (let i = 0; i < scoresData.length; i++) {
if (scoresData[i] > confidenceThreshold) {
validDetections.push({
box: boxesData.slice(i * 4, (i + 1) * 4),
score: scoresData[i],
class: classesData[i]
});
}
}
// Display results
validDetections.forEach(detection => {
const [x, y, width, height] = detection.box;
const score = detection.score;
const className = detection.class;
// Draw bounding box and label on the image
// (Implement your drawing logic here)
});
}
window.onload = runInference;For more detailed information on deploying TensorFlow.js models, you can refer to the TensorFlow.js documentation. If you encounter any further issues, please ensure you are using the latest versions of the packages and provide a minimum reproducible example as outlined here. I hope this helps! If you have any more questions, feel free to ask. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
@glenn-jocher thank you for your time. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @salim4n, Thank you for your feedback and for sharing your experience. I understand that integrating TensorFlow.js (TF.js) models directly into a Chrome extension can be challenging, especially when dealing with custom models. To address your concerns:
Example Code for IntegrationHere’s a basic example to load and use the exported TF.js model in a Chrome extension: Export the Model (Python)from ultralytics import YOLO
# Load the YOLOv8 model
model = YOLO("yolov8n.pt")
# Export the model to TF.js format
model.export(format="tfjs") # creates '/yolov8n_web_model'Load and Use the Model (JavaScript)import * as tf from '@tensorflow/tfjs';
async function loadModel() {
const model = await tf.loadGraphModel('/path/to/yolov8n_web_model/model.json');
return model;
}
async function runInference(imageElement) {
const model = await loadModel();
const tensor = tf.browser.fromPixels(imageElement).expandDims(0);
const predictions = await model.executeAsync(tensor);
console.log(predictions);
}
// Example usage
const imageElement = document.getElementById('input-image');
runInference(imageElement);Additional ResourcesFor more detailed instructions on deploying TF.js models, you can refer to: If you encounter any issues, please ensure you are using the latest version of the Ultralytics package and provide a minimum reproducible example if possible. This helps us diagnose and resolve any problems more effectively. You can find more information on creating a reproducible example here. Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
i have the same problem about how to get the my html code is below: <!DOCTYPE html>
<html>
<head>
<title>YOLOv8 Model Inference</title>
</head>
<body>
<!-- 用于显示输入图片和绘制推理结果 -->
<canvas id="input-image" width="640" height="640"></canvas>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
<script>
async function runModel() {
const model = await tf.loadGraphModel("./best_web_model/model.json");
// 1. 准备图片数据（这里使用示例图片）
const imgElement = document.getElementById("input-image");
const tensor = tf.browser
.fromPixels(imgElement)
.expandDims(0)
.toFloat()
.div(tf.scalar(255));
// 2. 进行推理
const predictions = await model.executeAsync([tensor]);
}
</script>
<script>
// 获取 canvas 元素
const canvas = document.getElementById("input-image");
const ctx = canvas.getContext("2d");
// 创建新的图片对象
const img = new Image();
// 图片加载完成后绘制到 canvas 上
img.onload = function () {
ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
runModel();
};
// 设置图片的来源
img.src = "./imgs/20217191442296037.jpg"; // 替换为您的图片路径
</script>
</body>
</html>the output of predictions is below: {
"kept": false,
"isDisposedInternal": false,
"shape": [
1,
5,
8400
],
"dtype": "float32",
"size": 42000,
"strides": [
42000,
8400
],
"dataId": {
"id": 1212
},
"id": 701,
"rankType": "3",
"scopeId": 879
} |
Beta Was this translation helpful? Give feedback.
-
|
Hi @wuhonglei, Thank you for your detailed question and for sharing your code! 😊 To extract the Here’s how you can modify your code to get the Updated JavaScript Codeimport * as tf from '@tensorflow/tfjs';
async function runModel() {
const model = await tf.loadGraphModel('./best_web_model/model.json');
// Prepare image data
const imgElement = document.getElementById('input-image');
const tensor = tf.browser.fromPixels(imgElement).expandDims(0).toFloat().div(tf.scalar(255));
// Run inference
const predictions = await model.executeAsync(tensor);
// Extract boxes, scores, and classes
const [boxes, scores, classes] = await Promise.all([
predictions[0].array(),
predictions[1].array(),
predictions[2].array()
]);
console.log('Boxes:', boxes);
console.log('Scores:', scores);
console.log('Classes:', classes);
// Optionally, draw boxes on the canvas
drawBoxes(imgElement, boxes, scores, classes);
}
function drawBoxes(imgElement, boxes, scores, classes) {
const canvas = document.getElementById('input-image');
const ctx = canvas.getContext('2d');
ctx.clearRect(0, 0, canvas.width, canvas.height);
ctx.drawImage(imgElement, 0, 0, canvas.width, canvas.height);
boxes.forEach((box, i) => {
const [y, x, height, width] = box;
ctx.strokeStyle = 'red';
ctx.lineWidth = 2;
ctx.strokeRect(x, y, width, height);
ctx.fillStyle = 'red';
ctx.fillText(`Class: ${classes[i]}, Score: ${scores[i].toFixed(2)}`, x, y > 10 ? y - 5 : 10);
});
}
// Load image and run model
const img = new Image();
img.onload = function () {
const canvas = document.getElementById('input-image');
const ctx = canvas.getContext('2d');
ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
runModel();
};
img.src = './imgs/20217191442296037.jpg'; // Replace with your image pathExplanation
This should help you extract and visualize the For more detailed information, you can refer to our TF.js integration guide. Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
thanks very much for your detailed response, In order to better assist future developers, I will paste the code that I have successfully run in the end。 <!DOCTYPE html>
<html>
<head>
<title>YOLOv8 Object Detection with TensorFlow.js</title>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
</head>
<body>
<input type="file" id="upload" accept="image/*" />
<canvas id="canvas"></canvas>
<script>
async function loadModel() {
// 加载模型
const model = await tf.loadGraphModel("./best_web_model/model.json");
return model;
}
async function predict(model, imageElement) {
// 将图像转换为张量
const tensor = tf.browser
.fromPixels(imageElement)
.resizeNearestNeighbor([640, 640]) // 调整大小到模型输入大小
.toFloat()
.expandDims(0)
.div(tf.scalar(255.0));
// inference
const predictions = await model.executeAsync(tensor);
// predictions shape is [1, 5, 8400]
const reshapedPredictions = predictions.reshape([5, 8400]);
const boxesTensor = reshapedPredictions.slice([0, 0], [4, 8400]);
const scoresTensor = reshapedPredictions
.slice([4, 0], [1, 8400])
.squeeze();
// Assuming that the category label of each box is fixed (you can adjust it as needed)
const classesTensor = tf.fill([8400], 0);
return {
boxes: boxesTensor,
scores: scoresTensor,
classes: classesTensor,
};
}
function drawBoxes(imageElement, canvas, boxes, scores, classes) {
const ctx = canvas.getContext("2d");
canvas.width = imageElement.width;
canvas.height = imageElement.height;
ctx.drawImage(imageElement, 0, 0, canvas.width, canvas.height);
const labels = ["企鹅"]; // Replace with actual label
const validBoxes = [];
for (let i = 0; i < scores.length; i++) {
if (scores[i] > 0.5) {
const x_c = boxes[0][i] / 640;
const y_c = boxes[1][i] / 640;
const w = boxes[2][i] / 640;
const h = boxes[3][i] / 640;
const x = x_c - w / 2;
const y = y_c - h / 2;
validBoxes.push([x, y, w, h, scores[i], classes[i]]);
}
}
validBoxes.forEach((box) => {
const [x, y, w, h, score, className] = box;
ctx.strokeStyle = "red";
ctx.lineWidth = 2;
ctx.strokeRect(
x * canvas.width,
y * canvas.height,
w * canvas.width,
h * canvas.height
);
const text = `${className}: ${(score * 100).toFixed(2)}%`;
ctx.fillText(text, x * canvas.width, y * canvas.height - 10);
});
}
document
.getElementById("upload")
.addEventListener("change", async (event) => {
const file = event.target.files[0];
const imageElement = new Image();
const reader = new FileReader();
reader.onload = () => {
imageElement.src = reader.result;
imageElement.onload = async () => {
const canvas = document.getElementById("canvas");
const model = await loadModel();
const { boxes, scores, classes } = await predict(
model,
imageElement
);
drawBoxes(
imageElement,
canvas,
boxes.arraySync(),
scores.arraySync(),
classes.arraySync()
);
};
};
reader.readAsDataURL(file);
});
</script>
</body>
</html>the the chrome html running result is below: |
Beta Was this translation helpful? Give feedback.
-
|
Hi @wuhonglei, Thank you for sharing your detailed implementation and the final working code! It's fantastic to see how you've successfully integrated YOLOv8 with TensorFlow.js for penguin detection in your web application. 🐧 For those looking to extract Key Steps:
Additional Tips:
For more detailed information on deploying YOLOv8 models with TensorFlow.js, you can refer to our TF.js integration guide. If you encounter any issues or have further questions, feel free to reach out. We're here to help! 😊 |
Beta Was this translation helpful? Give feedback.
-
integrations/tfjs/
Convert your Ultralytics YOLOv8 models to TensorFlow.js for high-speed, local object detection. Learn how to optimize ML models for browser and Node.js apps.
https://docs.ultralytics.com/integrations/tfjs/
Beta Was this translation helpful? Give feedback.
All reactions