shred could use fewer syscalls

[tertsdiepraam]

Discovered while investigating #4488

Compare these excerpts from strace:

uutils

write(3, "A.\270\25~9\7\351\217 \357\277\250\360\17\255\34\263\343\314\17\314y$s\354\364Gb\236%\326"..., 512) = 512
write(3, "\366K\346\33\24\251nW\342\234tW\304\326\25b\201M&s\7\315m6\350J\17'\2041\17\275"..., 512) = 512
write(3, "!_\313\0103E\267W\326E\322\21Y\223\25\351Z=\225\212j&\1x\244\2643N\f\324|\221"..., 512) = 512
write(3, "\273\241\354\320#\323\374\365y%\345\231f\0245\31+p\330G\344\27\210\17\303%T\272\275\272\260\304"..., 512) = 512
... (continues like this for a long time)

GNU

write(3, "\352\355^\257\n\3\226\366\27\362\f\232!\357\221\354J\10-\245\272d\377\270V\213CC\336\205Z\331"..., 65536) = 65536
write(3, "\3672k\355Nv\\\16\6\370lM\347\267u\0\372\324LG\211d;G\310\265\364\371\335\205\347\20"..., 12288) = 12288

GNU uses a much larger buffer size (512 vs 65536), which is probably much more efficient and they achieve in just 2 syscalls what takes uutils 152 syscalls.

I was wrong here below is the corrected version (or click here to expand)

There is a bit of an interesting twist, which is that uutils is much faster one some large files. Here, I made the test file 1GB:

❯ time ./target/release/coreutils shred test
________________________________________________________
Executed in    8.72 secs    fish           external
   usr time    2.98 secs    1.02 millis    2.98 secs
   sys time    4.64 secs    0.14 millis    4.64 secs
❯ time shred test
________________________________________________________
Executed in   69.59 secs    fish           external
   usr time    2.13 secs  447.00 micros    2.13 secs
   sys time    0.90 secs   61.00 micros    0.90 secs

So maybe the small buffer size just works better? I'm not sure what other difference could have such a large impact? Maybe uutils is using a different faster source of randomness?

I made a big mistake in the benchmarks above, which is that they were operating on the same file, if I create new files each time (with truncate) I get:

• GNU: ~5 seconds
• uutils with a small buffer: 10 - 20 seconds
• uutils with a large buffer: 3.5 - 53 seconds

So I guess the conclusion is that IO is hard to measure 😄

[sylvestre]

Nice, I wonder if there are some tools to detect these different behaviors

[AbhinavMir]

@sylvestre, something like gprof?