-
Notifications
You must be signed in to change notification settings - Fork 84
NCCL benchmark: Bagua Net vs google fastsocket vs baseline
Xiangru Lian edited this page Sep 15, 2021
·
1 revision
baseline
#
# nThread 1 nGpus 1 minBytes 8 maxBytes 134217728 step: 2(factor) warmup iters: 5 iters: 20 validation: 1
#
# Using devices
# Rank 0 Pid 146066 on bjlt03-hi13 device 0 [0x1a] Tesla V100-SXM2-32GB
# Rank 1 Pid 67084 on bjlt03-hi52 device 0 [0x1a] Tesla V100-SXM2-32GB
# Rank 2 Pid 34899 on bjlt03-hi65 device 0 [0x1a] Tesla V100-SXM2-32GB
# Rank 3 Pid 34368 on bjlt03-hi89 device 0 [0x1a] Tesla V100-SXM2-32GB
#
# out-of-place in-place
# size count type redop time algbw busbw error time algbw busbw error
# (B) (elements) (us) (GB/s) (GB/s) (us) (GB/s) (GB/s)
8 2 float sum 300.4 0.00 0.00 1e-07 235.1 0.00 0.00 0e+00
16 4 float sum 245.7 0.00 0.00 3e-08 218.9 0.00 0.00 3e-08
32 8 float sum 195.7 0.00 0.00 3e-08 167.1 0.00 0.00 3e-08
64 16 float sum 151.3 0.00 0.00 3e-08 159.7 0.00 0.00 3e-08
128 32 float sum 158.0 0.00 0.00 3e-08 154.3 0.00 0.00 3e-08
256 64 float sum 154.2 0.00 0.00 3e-08 142.4 0.00 0.00 3e-08
512 128 float sum 144.7 0.00 0.01 3e-08 170.9 0.00 0.00 1e-08
1024 256 float sum 237.5 0.00 0.01 1e-07 260.9 0.00 0.01 1e-07
2048 512 float sum 357.5 0.01 0.01 1e-07 409.6 0.01 0.01 1e-07
4096 1024 float sum 376.4 0.01 0.02 2e-07 404.2 0.01 0.02 2e-07
8192 2048 float sum 421.2 0.02 0.03 2e-07 434.7 0.02 0.03 2e-07
16384 4096 float sum 421.4 0.04 0.06 2e-07 357.2 0.05 0.07 2e-07
32768 8192 float sum 443.5 0.07 0.11 2e-07 484.8 0.07 0.10 2e-07
65536 16384 float sum 494.6 0.13 0.20 2e-07 521.9 0.13 0.19 2e-07
131072 32768 float sum 530.4 0.25 0.37 2e-07 553.9 0.24 0.35 2e-07
262144 65536 float sum 645.5 0.41 0.61 2e-07 640.4 0.41 0.61 2e-07
524288 131072 float sum 584.3 0.90 1.35 2e-07 649.6 0.81 1.21 2e-07
1048576 262144 float sum 956.7 1.10 1.64 2e-07 972.9 1.08 1.62 2e-07
2097152 524288 float sum 1401.9 1.50 2.24 2e-07 1391.4 1.51 2.26 2e-07
4194304 1048576 float sum 2407.7 1.74 2.61 2e-07 2392.8 1.75 2.63 2e-07
8388608 2097152 float sum 4737.8 1.77 2.66 2e-07 4709.8 1.78 2.67 2e-07
16777216 4194304 float sum 10279 1.63 2.45 2e-07 10588 1.58 2.38 2e-07
33554432 8388608 float sum 20209 1.66 2.49 2e-07 20327 1.65 2.48 2e-07
67108864 16777216 float sum 40024 1.68 2.52 2e-07 39941 1.68 2.52 2e-07
134217728 33554432 float sum 80513 1.67 2.50 2e-07 81373 1.65 2.47 2e-07
# Out of bounds values : 0 OK
# Avg bus bandwidth : 0.87012
#
bagua-net
#
# nThread 1 nGpus 1 minBytes 8 maxBytes 134217728 step: 2(factor) warmup iters: 5 iters: 20 validation: 1
#
# Using devices
# Rank 0 Pid 146410 on bjlt03-hi13 device 0 [0x1a] Tesla V100-SXM2-32GB
# Rank 1 Pid 67550 on bjlt03-hi52 device 0 [0x1a] Tesla V100-SXM2-32GB
# Rank 2 Pid 35367 on bjlt03-hi65 device 0 [0x1a] Tesla V100-SXM2-32GB
# Rank 3 Pid 34794 on bjlt03-hi89 device 0 [0x1a] Tesla V100-SXM2-32GB
#
# out-of-place in-place
# size count type redop time algbw busbw error time algbw busbw error
# (B) (elements) (us) (GB/s) (GB/s) (us) (GB/s) (GB/s)
8 2 float sum 290.4 0.00 0.00 1e-07 299.9 0.00 0.00 0e+00
16 4 float sum 296.8 0.00 0.00 3e-08 297.2 0.00 0.00 3e-08
32 8 float sum 296.8 0.00 0.00 3e-08 298.1 0.00 0.00 3e-08
64 16 float sum 296.3 0.00 0.00 3e-08 295.5 0.00 0.00 3e-08
128 32 float sum 296.5 0.00 0.00 3e-08 296.3 0.00 0.00 3e-08
256 64 float sum 297.7 0.00 0.00 3e-08 296.1 0.00 0.00 3e-08
512 128 float sum 272.4 0.00 0.00 3e-08 286.4 0.00 0.00 1e-08
1024 256 float sum 287.8 0.00 0.01 1e-07 365.8 0.00 0.00 1e-07
2048 512 float sum 360.9 0.01 0.01 1e-07 330.6 0.01 0.01 1e-07
4096 1024 float sum 382.5 0.01 0.02 2e-07 400.0 0.01 0.02 2e-07
8192 2048 float sum 469.9 0.02 0.03 2e-07 524.6 0.02 0.02 2e-07
16384 4096 float sum 494.8 0.03 0.05 2e-07 488.9 0.03 0.05 2e-07
32768 8192 float sum 529.1 0.06 0.09 2e-07 523.0 0.06 0.09 2e-07
65536 16384 float sum 480.2 0.14 0.20 2e-07 467.3 0.14 0.21 2e-07
131072 32768 float sum 473.2 0.28 0.42 2e-07 474.4 0.28 0.41 2e-07
262144 65536 float sum 701.0 0.37 0.56 2e-07 727.9 0.36 0.54 2e-07
524288 131072 float sum 786.8 0.67 1.00 2e-07 821.5 0.64 0.96 2e-07
1048576 262144 float sum 1013.8 1.03 1.55 2e-07 1035.9 1.01 1.52 2e-07
2097152 524288 float sum 1139.9 1.84 2.76 2e-07 1112.1 1.89 2.83 2e-07
4194304 1048576 float sum 1731.0 2.42 3.63 2e-07 1779.1 2.36 3.54 2e-07
8388608 2097152 float sum 2725.0 3.08 4.62 2e-07 2915.8 2.88 4.32 2e-07
16777216 4194304 float sum 5586.3 3.00 4.50 2e-07 5632.8 2.98 4.47 2e-07
33554432 8388608 float sum 10860 3.09 4.63 2e-07 10412 3.22 4.83 2e-07
67108864 16777216 float sum 20490 3.28 4.91 2e-07 20323 3.30 4.95 2e-07
134217728 33554432 float sum 41678 3.22 4.83 2e-07 41141 3.26 4.89 2e-07
# Out of bounds values : 0 OK
# Avg bus bandwidth : 1.35005
#
google-fastsocket
#
# nThread 1 nGpus 1 minBytes 8 maxBytes 134217728 step: 2(factor) warmup iters: 5 iters: 20 validation: 1
#
# Using devices
# Rank 0 Pid 146098 on bjlt03-hi13 device 0 [0x1a] Tesla V100-SXM2-32GB
# Rank 1 Pid 67132 on bjlt03-hi52 device 0 [0x1a] Tesla V100-SXM2-32GB
# Rank 2 Pid 34947 on bjlt03-hi65 device 0 [0x1a] Tesla V100-SXM2-32GB
# Rank 3 Pid 34416 on bjlt03-hi89 device 0 [0x1a] Tesla V100-SXM2-32GB
#
# out-of-place in-place
# size count type redop time algbw busbw error time algbw busbw error
# (B) (elements) (us) (GB/s) (GB/s) (us) (GB/s) (GB/s)
8 2 float sum 201.8 0.00 0.00 1e-07 238.0 0.00 0.00 0e+00
16 4 float sum 155.5 0.00 0.00 3e-08 177.4 0.00 0.00 3e-08
32 8 float sum 161.2 0.00 0.00 3e-08 152.8 0.00 0.00 3e-08
64 16 float sum 177.1 0.00 0.00 3e-08 153.9 0.00 0.00 3e-08
128 32 float sum 159.9 0.00 0.00 3e-08 201.8 0.00 0.00 3e-08
256 64 float sum 201.7 0.00 0.00 3e-08 168.2 0.00 0.00 3e-08
512 128 float sum 169.1 0.00 0.00 3e-08 152.9 0.00 0.01 1e-08
1024 256 float sum 142.6 0.01 0.01 1e-07 145.5 0.01 0.01 1e-07
2048 512 float sum 176.8 0.01 0.02 1e-07 159.4 0.01 0.02 1e-07
4096 1024 float sum 151.4 0.03 0.04 2e-07 163.5 0.03 0.04 2e-07
8192 2048 float sum 189.8 0.04 0.06 2e-07 182.6 0.04 0.07 2e-07
16384 4096 float sum 202.3 0.08 0.12 2e-07 190.1 0.09 0.13 2e-07
32768 8192 float sum 277.7 0.12 0.18 2e-07 280.7 0.12 0.18 2e-07
65536 16384 float sum 329.9 0.20 0.30 2e-07 347.2 0.19 0.28 2e-07
131072 32768 float sum 402.3 0.33 0.49 2e-07 393.1 0.33 0.50 2e-07
262144 65536 float sum 470.2 0.56 0.84 2e-07 478.1 0.55 0.82 2e-07
524288 131072 float sum 523.4 1.00 1.50 2e-07 561.0 0.93 1.40 2e-07
1048576 262144 float sum 778.4 1.35 2.02 2e-07 844.8 1.24 1.86 2e-07
2097152 524288 float sum 1014.9 2.07 3.10 2e-07 977.6 2.15 3.22 2e-07
4194304 1048576 float sum 1501.6 2.79 4.19 2e-07 1549.8 2.71 4.06 2e-07
8388608 2097152 float sum 12722 0.66 0.99 2e-07 2922.5 2.87 4.31 2e-07
16777216 4194304 float sum 5016.7 3.34 5.02 2e-07 4994.6 3.36 5.04 2e-07
33554432 8388608 float sum 9430.3 3.56 5.34 2e-07 9842.1 3.41 5.11 2e-07
67108864 16777216 float sum 38740 1.73 2.60 2e-07 28907 2.32 3.48 2e-07
134217728 33554432 float sum 69368 1.93 2.90 2e-07 98598 1.36 2.04 2e-07
# Out of bounds values : 0 OK
# Avg bus bandwidth : 1.24595