Error when processing training images - No such file or directory IOError

[sahiliitm]

Hi.

I have installed all the dependencies. I was trying to run prepro.py as mentioned in the documentation. I ran into an issue which I believe is different from issue #4 mentioned in the documentation.

Here are the contents of my coco folder after running the ipython tutorial:

>>> ls coco/*
coco/captions_train-val2014.zip  coco/coco_preprocess.ipynb  coco/coco_raw.json  coco/cocotalk.h5

coco/annotations:
captions_train2014.json  captions_val2014.json

coco/images:
captions_train2014.json  captions_val2014.json

When I run prepro.py I get the following error:

parsed input parameters:
{
  "output_json": "coco/cocotalk.json", 
  "images_root": "coco/images", 
  "input_json": "coco/coco_raw.json", 
  "word_count_threshold": 5, 
  "max_length": 16, 
  "output_h5": "coco/cocotalk.h5", 
  "num_test": 5000, 
  "num_val": 5000
}
example processed tokens:
['a', 'woman', 'riding', 'a', 'bike', 'down', 'a', 'bike', 'trail']
...
top words and their counts:
(1019751, 'a')
(224731, 'on')
...
(35371, 'woman')
total words: 6447836
number of bad words: 20059/29625 = 67.71%
number of words in vocab would be 9566
number of UNKs: 34543/6447836 = 0.54%
max length sentence in raw data:  49
sentence length distribution (count, number of words):
 0:          0   0.000000%
 1:          0   0.000000%
 ... 
 49:          4   0.000649%
inserting the special UNK token
assigned 5000 to val, 5000 to test.
encoded captions to array of size  (616767, 16)
Traceback (most recent call last):
  File "prepro.py", line 240, in <module>
    main(params)
  File "prepro.py", line 185, in main
    I = imread(os.path.join(params['images_root'], img['file_path']))
  File "/usr/local/lib/python2.7/dist-packages/scipy/misc/pilutil.py", line 154, in imread
    im = Image.open(name)
  File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 1955, in open
    fp = __builtin__.open(fp, "rb")
IOError: [Errno 2] No such file or directory: u'coco/images/train2014/COCO_train2014_000000152328.jpg'

Could someone please help me out? Am I missing something here?

Thanks

[soumith]

"No such file or directory: u'coco/images/train2014/COCO_train2014_000000152328.jpg'"

[sahiliitm]

Got it. I think there was an error in the way I ran the ipython tutorial and the images folder was not populated correctly.

Thanks!

[gotope]

Hi @sahiliitm, where can I get the COCO_train2014_000000152328.jpg？
I have executed the coco/ folder and run the IPython notebook several times.

[sedhasukhdeep]

@sahiliitm [Urgent]

Getting stuck here,

Can you please share what was the solution?

[sedhasukhdeep]

Solution for COCO_train2014_000000152328.jpg is as following:

1. Create a new dir under coco called images
2. unzip train2014 data and move folder to images, it will look something like coco/images/train_2014
3. unzip val2014 data and do the same thing as above.
4. Make sure you have replaced the corrupted file as mentioned by other users.
5. Now run the ipynb file to generate coco_raw.json
6. Now run the prepro script as mentioned.