Cannot load image, STB Reason: expected marker

[lengly]

When I try to train it on COCO:

./darknet detector train cfg/coco.data cfg/yolo.cfg weights/darknet19_448.conv.23 -gpus 0,1,2,3

I see such error

Region Avg IOU: 0.495555, Class: 0.534914, Obj: 0.116917, No Obj: 0.007393, Avg Recall: 0.468750,  count: 32
Region Avg IOU: 0.268316, Class: 0.294134, Obj: 0.117317, No Obj: 0.008529, Avg Recall: 0.222222,  count: 36
Region Avg IOU: 0.384779, Class: 0.296443, Obj: 0.101438, No Obj: 0.007914, Avg Recall: 0.343750,  count: 32
Syncing... Cannot load image "/home/walle/lyd/datasets/COCO/coco/images/train2014/COCO_train2014_000000167126.jpg"
STB Reason: expected marker
Done!
2544: 32.557770, 34.544533 avg, 0.004000 rate, 3.863461 seconds, 325632 images
Loaded: 0.000039 seconds

This error happend randomly, anyone knows how to solve it?

[RichardYinguo]

Have you solved the problem? I meet the same

[lengly]

@RichardYinguo Something wrong with that image, please delete it in the list. don't know why...

[RichardYinguo]

thanks

[jeremy-rutman]

These appear to be images that got truncated. opencv loads them fine with original dimensions, (so a check in parse.py won't prevent the problem) but the darknet code dies on them.

I found 2 examples and uploaded to
https://ibb.co/njvyf8
https://ibb.co/fQ3Uno

[mhaghighat]

I faced the same issue. Is the COCO_train2014_000000167126.jpg the only image that causes this error or are there others down the road? If this is the only one, we can simply remove it from the trainvalno5k.txt.

UPDATE:
Yes, it is the only damaged image. You can simply remove it from the list.

[jeremy-rutman]

any truncated image will do this due to the c code used for loading the image. You can add a check to the code eg. a try/catch, or hope there are no more truncated images in your dataset

[leicao-me]

If you take a look at the image that can’t be loaded - COCO_train2014_000000167126.jpg, you will see it’s damaged.

Solution:
Download the undamaged version of the image and replace the damaged
https://msvocds.blob.core.windows.net/images/262993_z.jpg

You are welcome. :)

References:
Damaged Image:
https://blog.csdn.net/chengyq116/article/details/80258821
Replacement:
karpathy/neuraltalk2#4

[usb1508]

Did someone solve the problem, in which the solution doesn't involve redownloading the damaged image?
Because I initially used the same images to train, and it worked fine. But the next time I reset the session of VM, this particular error pops up.
Is there some way this could be avoided as currently, I need to unzip the dataset every time I use a new session.

[jeremy-rutman]

You can fix the original C code which is lacking a check for this error

…

On Thu, Jun 27, 2019 at 10:41 AM utkarsh1508 ***@***.***> wrote: Did someone solve the problem, in which the solution doesn't involve redownloading the damaged image? Because I initially used the same images to train, and it worked fine. But the next time I reset the session of VM, this particular error pops up. Is there some way this could be avoided as currently, I need to unzip the dataset every time I use a new session. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#312?email_source=notifications&email_token=AA5PJQEN4QDHBRQKL46JD3TP4RVMFA5CNFSM4EDZ2AD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYWHXRY#issuecomment-506231751>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA5PJQH7E2K2QQ5JNDL3IA3P4RVMFANCNFSM4EDZ2ADQ> .

[chenjiahe01]

I've work on it all day.Finally I make it!!
Find "darknet/src/image.h" and open it in editor.
change the function
image load_image_stb(char *filename, int channels)
{
int w, h, c;
unsigned char *data = stbi_load(filename, &w, &h, &c, channels);
if (!data) {
fprintf("shit!")//stderr, "Cannot load image \"%s\"\nSTB Reason: %s\n", filename, stbi_failure_reason());
exit(0);
}
if(channels) c = channels;
int i,j,k;
image im = make_image(w, h, c);
for(k = 0; k < c; ++k){
for(j = 0; j < h; ++j){
for(i = 0; i < w; ++i){
int dst_index = i + w*j + w*h*k;
int src_index = k + c*i + c*w*j;
im.data[dst_index] = (float)data[src_index]/255.;
}
}
}
free(data);
return im;
}
to:
image load_image_stb(char *filename, int channels)
{
int w, h, c;
unsigned char *data = stbi_load(filename, &w, &h, &c, channels);
if (!data) {
//fprintf("shit!")//stderr, "Cannot load image \"%s\"\nSTB Reason: %s\n", filename, stbi_failure_reason());
//exit(0);
image i;
return i;
}
if(channels) c = channels;
int i,j,k;
image im = make_image(w, h, c);
for(k = 0; k < c; ++k){
for(j = 0; j < h; ++j){
for(i = 0; i < w; ++i){
int dst_index = i + w*j + w*h*k;
int src_index = k + c*i + c*w*j;
im.data[dst_index] = (float)data[src_index]/255.;
}
}
}
free(data);
return im;
}
Now, if the image to be loaded is truncated. The load_image program will return an empty image instead of exit.
Then we can useif(im.w==0) to skip the image automatically or what.

[dmitrygashnikov]

Find "darknet/src/image.h" and open it in editor.
change the function

Thx for solution, will try it later today. One correction:
not find those class in "darknet/src/image.h" by myself.
In my version its located in "darknet/src/image.c"
And reminder for thouse, who will try this solution - do not foorget to rebuild project after making changes.

P.S. Sorry for my english

[dranaivo]

I've work on it all day.Finally I make it!!
Find "darknet/src/image.h" and open it in editor.
change the function
image load_image_stb(char *filename, int channels)
{
int w, h, c;
unsigned char *data = stbi_load(filename, &w, &h, &c, channels);
if (!data) {
fprintf("shit!")//stderr, "Cannot load image \"%s\"\nSTB Reason: %s\n", filename, stbi_failure_reason());
exit(0);
}
if(channels) c = channels;
int i,j,k;
image im = make_image(w, h, c);
for(k = 0; k < c; ++k){
for(j = 0; j < h; ++j){
for(i = 0; i < w; ++i){
int dst_index = i + w*j + w*h*k;
int src_index = k + c*i + c*w*j;
im.data[dst_index] = (float)data[src_index]/255.;
}
}
}
free(data);
return im;
}
to:
image load_image_stb(char *filename, int channels)
{
int w, h, c;
unsigned char *data = stbi_load(filename, &w, &h, &c, channels);
if (!data) {
//fprintf("shit!")//stderr, "Cannot load image \"%s\"\nSTB Reason: %s\n", filename, stbi_failure_reason());
//exit(0);
image i;
return i;
}
if(channels) c = channels;
int i,j,k;
image im = make_image(w, h, c);
for(k = 0; k < c; ++k){
for(j = 0; j < h; ++j){
for(i = 0; i < w; ++i){
int dst_index = i + w*j + w*h*k;
int src_index = k + c*i + c*w*j;
im.data[dst_index] = (float)data[src_index]/255.;
}
}
}
free(data);
return im;
}
Now, if the image to be loaded is truncated. The load_image program will return an empty image instead of exit.
Then we can useif(im.w==0) to skip the image automatically or what.

@chenjiahe01 As I am not really good with C, can you elaborate a bit on the use of the condition if (im.w==0) to skip the image, namely where in the file can you use it and if possible an example of snippet. Thank you!

[HtR212]

@dranaivo You can check if (im.w==0) in the function test_detector inside "darknet/examples/detector.c". You also have to modify the load_image function inside "darknet/src/image.c" to make it return out before the if statement to avoid the resizing part.