FileStream benchmarks #1633

adamsitnik · 2021-01-18T17:46:10Z

Main changes:

add benchmark that copies one file into another (existing benchmark was copying from file to Stream.Null)
add Open&Close benchmark
add Lock&Unlcok benchmark
add Seek (forward and backward) benchmarks
give better names to differentiate user buffer (passed to stream) and internal stream buffer (passed by Stream to OS)
add more file sizes (1kB, 1MB, 100MB) where it makes sense

cc @stephentoub

…ire file content was read

…ream.Null)

src/benchmarks/micro/libraries/System.IO.FileSystem/Perf.FileStream.cs

danmoseley · 2021-01-19T01:31:00Z

src/benchmarks/micro/libraries/System.IO.FileSystem/Perf.FileStream.cs

-
-        [Params(DefaultBufferSize / 8, 200000)]
-        public int BufferSize;
+        private const int OneKiloByte = 1_000;


May be interesting to review "real world" values passed to BufferSize
https://grep.app/search?q=new%20FileStream.%2A0x&regexp=true&filter[lang][0]=C%23
https://grep.app/search?q=new%20FileStream.%2A%5Cd%7B4%2C%7D&regexp=true&filter[lang][0]=C%23
https://grep.app/search?q=new%20FileStream.%2Asize%3A%5Cs%3F.%5B%5E4%5D%5Cd%7B3%2C%7D&regexp=true&filter[lang][0]=C%23

0x1000 / 4096 dominate (redundantly)
I also see 0x1_000 , 2000, 1024, 512 << 10, 64 * 1024

stephentoub · 2021-01-20T22:12:22Z

src/benchmarks/micro/libraries/System.IO.FileSystem/Perf.FileStream.cs

-            using (FileStream reader = new FileStream(_filePath, FileMode.Open, FileAccess.Read, FileShare.Read, DefaultBufferSize, FileOptions.Asynchronous))
+            byte[] userBuffer = _userBuffers[userBufferSize];
+            long bytesRead = 0;
+            using (FileStream fileStream = new FileStream(_sourceFilePaths[fileSize], FileMode.Open, FileAccess.Read, FileShare.Read, FourKiloBytes, options))


We also end up measuring the open/close time. That's ok?

I would like this benchmark to answer the question "How long does it take to read X bytes from a file?". Since opening (and closing) the file is mandatory to read it, I think that it's OK to include it here.

src/benchmarks/micro/libraries/System.IO.FileSystem/Perf.FileStream.cs

…WriteAsync that accepts a Memory

…s already test CancellationTokens

adamsitnik · 2021-01-27T13:43:55Z

@jozkee @carlossanlop the CI failures are unrelated, PTAL

jozkee

I wonder if we should measure Flush as well?
That aside, looks great to me!

adamsitnik · 2021-01-28T11:54:22Z

I wonder if we should measure Flush as well?

Great point, I've added benchmark and verified with profiler that most of the time (85%) is spent in Flushing

adamsitnik added 10 commits January 18, 2021 14:30

add OpenClose and OpenCloseAsync benchmarks

469b0ab

use the number of returned bytes to stop the Read benchmarks when ent…

fae0cb4

…ire file content was read

add overloads for Read|Write with CancellationToken

0fdf222

add CopyToFile benchmarks (the exisitng CopyTo benchmark writes to St…

a2555d5

…ream.Null)

rewrite the benchmarks

0241d68

make sure every benchmark has as many files as needed

b193fce

add Lock and Unlock benchmarks

c8b84a3

add Seek benchmarks

bec3b44

fix an important bug

f1bd482

refactor

7ceb186

adamsitnik requested review from carlossanlop and jozkee January 18, 2021 17:46

adamsitnik mentioned this pull request Jan 18, 2021

Introduce FileStreamStrategy as a first step of FileStream rewrite dotnet/runtime#47128

Merged

danmoseley reviewed Jan 19, 2021

View reviewed changes

src/benchmarks/micro/libraries/System.IO.FileSystem/Perf.FileStream.cs Outdated Show resolved Hide resolved

danmoseley reviewed Jan 19, 2021

View reviewed changes

src/benchmarks/micro/libraries/System.IO.FileSystem/Perf.FileStream.cs Outdated Show resolved Hide resolved

danmoseley reviewed Jan 19, 2021

View reviewed changes

stephentoub reviewed Jan 20, 2021

View reviewed changes

adamsitnik added 6 commits January 21, 2021 14:29

address code review feedback: use powers of 2

334ec61

address code review feedback: add benchmarks that test ReadAsync and …

139e806

…WriteAsync that accepts a Memory

remove the "WithCancellationToken" benchmarks as the Memory benchmark…

39f6a6c

…s already test CancellationTokens

remove ReadAsyncArray and add FileOptions.None to async benchmarks

b0cf969

will this fix "Bad file descriptor" we are getting on Ubuntu?

29fe931

make sure the LockUnlock benchmark does not crash on OSX

e037664

jozkee approved these changes Jan 28, 2021

View reviewed changes

address code review feedback and add benchmark for Flush and FlushAsync

59e549c

adamsitnik merged commit 358051e into dotnet:master Feb 2, 2021

adamsitnik deleted the fileStream branch February 2, 2021 10:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FileStream benchmarks #1633

FileStream benchmarks #1633

adamsitnik commented Jan 18, 2021

danmoseley Jan 19, 2021 •

edited

Loading

stephentoub Jan 20, 2021

adamsitnik Jan 21, 2021

adamsitnik commented Jan 27, 2021

jozkee left a comment

adamsitnik commented Jan 28, 2021

FileStream benchmarks #1633

FileStream benchmarks #1633

Conversation

adamsitnik commented Jan 18, 2021

danmoseley Jan 19, 2021 • edited Loading

Choose a reason for hiding this comment

stephentoub Jan 20, 2021

Choose a reason for hiding this comment

adamsitnik Jan 21, 2021

Choose a reason for hiding this comment

adamsitnik commented Jan 27, 2021

jozkee left a comment

Choose a reason for hiding this comment

adamsitnik commented Jan 28, 2021

danmoseley Jan 19, 2021 •

edited

Loading