dataloading issue

[rwnck]

Hi! Thanks for uploading the code. I am facing the above error when running : "python train.py"
Looks like there is something wrong with the folder structure maintained by me or may be I need to provide the folder structure path as input somewhere. Can you kindly shed light on how to resolve this issue?

[likyoo]

you can modify the path in metadata.json

[rwnck]

Thank you! That has worked.

However I am getting another error now. Can't understand what this means...

(siamese_nested_unet) PS G:\NeuroPixel\segmentation\unet_plus_plus\siamese_nested_unet\Siam-NestedUNet-master> python train.py
INFO:root:GPU AVAILABLE? True
INFO:root:STARTING Dataset Creation
INFO:root:STARTING Dataloading
INFO:root:LOADING Model
INFO:root:STARTING training
INFO:root:SET model mode to train!
0%| | 0/625 [00:00<?, ?it/s]INFO:root:GPU AVAILABLE? True
INFO:root:STARTING Dataset Creation
INFO:root:STARTING Dataloading
INFO:root:LOADING Model
INFO:root:STARTING training
INFO:root:SET model mode to train!
0%| | 0/625 [00:00<?, ?it/s]
Traceback (most recent call last):
File "", line 1, in
File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
0%| | 0/625 [00:03<?, ?it/s] File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 114, in _main

Traceback (most recent call last):

prepare(preparation_data) File "train.py", line 79, in

  File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 225, in prepare

for batch_img1, batch_img2, labels in tbar:
_fixup_main_from_path(data['init_main_from_path']) File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\tqdm\std.py", line 1185, in iter

File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
for obj in iterable:run_name="mp_main")

File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 359, in iter
File "F:\Anaconda\envs\siamese_nested_unet\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "F:\Anaconda\envs\siamese_nested_unet\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "F:\Anaconda\envs\siamese_nested_unet\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "G:\NeuroPixel\segmentation\unet_plus_plus\siamese_nested_unet\Siam-NestedUNet-master\train.py", line 79, in
for batch_img1, batch_img2, labels in tbar:
File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\tqdm\std.py", line 1185, in iter
for obj in iterable:
File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 359, in iter
return self._get_iterator()
File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator
return self._get_iterator()
return _MultiProcessingDataLoaderIter(self) File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator

  File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 918, in __init__

return _MultiProcessingDataLoaderIter(self)
File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 918, in init
w.start()
File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\process.py", line 105, in start
w.start()
File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)self._popen = self._Popen(self)

File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\context.py", line 223, in _Popen
File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\context.py", line 322, in _Popen
return _default_context.get_context().Process._Popen(process_obj)return Popen(process_obj)

File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\context.py", line 322, in _Popen
File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\popen_spawn_win32.py", line 33, in init
return Popen(process_obj)
File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\reduction.py", line 60, in dump
prep_data = spawn.get_preparation_data(process_obj._name)
File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
_check_not_importing_main()
File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.
ForkingPickler(file, protocol).dump(obj)

BrokenPipeError: [Errno 32] Broken pipe

[Youskrpig]

Thank you! That has worked.

However I am getting another error now. Can't understand what this means...

(siamese_nested_unet) PS G:\NeuroPixel\segmentation\unet_plus_plus\siamese_nested_unet\Siam-NestedUNet-master> python train.py INFO:root:GPU AVAILABLE? True INFO:root:STARTING Dataset Creation INFO:root:STARTING Dataloading INFO:root:LOADING Model INFO:root:STARTING training INFO:root:SET model mode to train! 0%| | 0/625 [00:00<?, ?it/s]INFO:root:GPU AVAILABLE? True INFO:root:STARTING Dataset Creation INFO:root:STARTING Dataloading INFO:root:LOADING Model INFO:root:STARTING training INFO:root:SET model mode to train! 0%| | 0/625 [00:00<?, ?it/s] Traceback (most recent call last): File "", line 1, in File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) 0%| | 0/625 [00:03<?, ?it/s] File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 114, in _main

Traceback (most recent call last):

prepare(preparation_data) File "train.py", line 79, in

  File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 225, in prepare

for batch_img1, batch_img2, labels in tbar: _fixup_main_from_path(data['init_main_from_path']) File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\tqdm\std.py", line 1185, in iter

File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path for obj in iterable:run_name="mp_main")

File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 359, in iter File "F:\Anaconda\envs\siamese_nested_unet\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "F:\Anaconda\envs\siamese_nested_unet\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "F:\Anaconda\envs\siamese_nested_unet\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "G:\NeuroPixel\segmentation\unet_plus_plus\siamese_nested_unet\Siam-NestedUNet-master\train.py", line 79, in for batch_img1, batch_img2, labels in tbar: File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\tqdm\std.py", line 1185, in iter for obj in iterable: File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 359, in iter return self._get_iterator() File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator return self._get_iterator() return _MultiProcessingDataLoaderIter(self) File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator

  File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 918, in __init__

return _MultiProcessingDataLoaderIter(self) File "F:\Anaconda\envs\siamese_nested_unet\lib\site-packages\torch\utils\data\dataloader.py", line 918, in init w.start() File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\process.py", line 105, in start w.start() File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self)self._popen = self._Popen(self)

File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\context.py", line 223, in _Popen File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\context.py", line 322, in _Popen return _default_context.get_context().Process._Popen(process_obj)return Popen(process_obj)

File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\context.py", line 322, in _Popen File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\popen_spawn_win32.py", line 33, in init return Popen(process_obj) File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\popen_spawn_win32.py", line 65, in init reduction.dump(process_obj, to_child) File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\reduction.py", line 60, in dump prep_data = spawn.get_preparation_data(process_obj._name) File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 143, in get_preparation_data _check_not_importing_main() File "F:\Anaconda\envs\siamese_nested_unet\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.
ForkingPickler(file, protocol).dump(obj)

BrokenPipeError: [Errno 32] Broken pipe

Reduce the num of workers when you load the dataset.

[grndng]

Even when reducing the number of workers to 1, a freeze_support() error is being displayed:

RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.