How to do inference by multiple GPUs？

[ZhengMengbin]

I want to using more than one GPU to do inference, what should I do? Using the format as training?

[tianzhi0549]

Yes.

[super-wcg]

@ZhengMengbin How to set multi GPU to train the model? Can you show me your code?

[tianzhi0549]

@super-wcg please use the following command line for multi-GPU inference.

python -m torch.distributed.launch \
    --nproc_per_node=8 \
    --master_port=$((RANDOM + 10000)) \
    tools/test_net.py \
    --config-file configs/fcos/fcos_R_50_FPN_1x.yaml \
    MODEL.WEIGHT FCOS_R_50_FPN_1x.pth \
    TEST.IMS_PER_BATCH 8

[LIUhansen]

How to choose a specific GPU？

[tianzhi0549]

@LIUhansen please use export CUDA_VISIBLE_DEVICES="<GPU_ID>".

[Tigerwander]

But the dataloader speed is hard to say, i test coco_2017_val, but the tqdm kept as 0%, and when i debug, it seems stucked by enumerate(tqdm(dataloader)).