Unified exceptional/exceptionless non-blocking I/O

[tarcieri]

The main motivation for this change is because JRuby 9000 supports exceptionless non-blocking I/O for TCPSockets but not forSSLSockets. Both accept the :exception => false syntax, so it's harmless to pass it to SSLSockets even though they silently ignore it. Unfortunately, even with :exception => false JRuby 9000 still throws IO::WaitReadable/IO::WaitWritable, so until that's fixed upstream we need to handle it.

This change also adds handling non-blocking reads during writes and writes during reads, which is necessary for SSLSockets to work correctly on all platforms (i.e. an SSL handshake might require performing a read for a write to complete, or vice versa)

I'm pleased with the result in as much as it provides a single approach which covers all cases (#perform_io).

[zanker]

Are we going to keep lowering this until it's 0? :P

[tarcieri]

Seems like the logical conclusion 😜 For what it's worth it was nickel-and-diming me over 58 vs 57.9

[zanker]

I'm fine with the code as-is, however loop vs begin ... rescue retry end the performance is interesting.

require 'benchmark'

ITERATIONS = 1_000_000

Benchmark.bm do |x|
  x.report(:loop) do
    total = 0

    loop do
      begin
        raise RuntimeError, "ABC"
      rescue RuntimeError
      end

      break if (total += 1) >= ITERATIONS
    end
  end

  x.report(:retry) do
    total = 0

    begin
      raise RuntimeError, "ABC"
    rescue RuntimeError
      retry if (total += 1) < ITERATIONS
    end
  end
end

On MRI, the performance of loop vs retry is totally irrelevant.

       user     system      total        real
loop  2.030000   0.090000   2.120000 (  2.118746)
retry  2.060000   0.060000   2.120000 (  2.134159)

       user     system      total        real
loop  1.820000   0.070000   1.890000 (  1.893491)
retry  1.810000   0.060000   1.870000 (  1.863721)

on JRuby 1.7.19, with iterations reduced to 50,000 (because 1,000,000 takes forever)

       user     system      total        real
loop  6.080000   0.080000   6.160000 (  5.104000)
retry  4.310000   0.020000   4.330000 (  4.240000)

And on JRuby 9.0.0.0

       user     system      total        real
loop  7.240000   0.110000   7.350000 (  5.535063)
retry  4.190000   0.020000   4.210000 (  4.122280)

So when exceptionless NIO isn't available, HttpRb will get about 30% slower. That's not really great, but you should only be having to loop 0 - 1 times so I'm not sure how much we care?

[tarcieri]

Given the body of the loop is performing I/O, and that the number of iterations should typically be low, and also that this will only impact older versions of JRuby (while fixing compatibility for JRuby 9000), I guess I'm not too worried.

[tarcieri]

[headius]

The exception-based non-blocking IO logic on JRuby does not generate a backtrace, which greatly reduces the cost. The benchmark would be more indicative of this performance by providing a blank backtrace, as in:

raise RuntimeException, "ABC", []

With this change, JRuby performance with ITERATIONS = 1_000_000 is faster than MRI on my machine, and retry is a win:

JRUBY:
       user     system      total        real
loop  2.730000   0.110000   2.840000 (  1.482137)
retry  1.240000   0.020000   1.260000 (  1.067515)
       user     system      total        real
loop  1.340000   0.000000   1.340000 (  1.147885)
retry  1.060000   0.010000   1.070000 (  0.935158)
       user     system      total        real
loop  1.160000   0.000000   1.160000 (  1.090444)
retry  0.910000   0.010000   0.920000 (  0.903934)

MRI 2.2.2:
       user     system      total        real
loop  1.510000   0.010000   1.520000 (  1.514647)
retry  1.480000   0.000000   1.480000 (  1.488237)
       user     system      total        real
loop  1.660000   0.000000   1.660000 (  1.671367)
retry  1.550000   0.010000   1.560000 (  1.555238)
       user     system      total        real
loop  1.660000   0.000000   1.660000 (  1.666488)
retry  1.510000   0.010000   1.520000 (  1.523886)

[headius]

I should point out that both loop and retry in the JRuby case (with fixed benchmark) are taking in the neighborhod of 1µs per iteration. It's very unlikely that will overshadow any IO happening, as @tarcieri pointed out.

[zanker]

@headius Huh super cool, thanks for looking into this! Was the issue that it wasn't being reran multiple times and JIT wasn't being used, or is there a performance diff in gathering a backtrace for loop vs retry?

[headius]

Backtrace-gathering on JVM is orders of magnitude slower than in MRI, due to it having to stitch together interpreter vs JIT vs JIT+inlining, among other things.

For exceptions that are frequently used for flow control, like EAGAIN, JRuby omits the backtrace and only inserts a message saying how to turn it on again. The big difference in my run of the benchmark was calling the three-argument form of Kernel#raise that takes a backtrace array, which is more equivalent to our making EAGAIN not gather a proper backtrace.

[headius]

Oh, and to clarify: the retry form is faster because it's a direct jump back to the begin rather than loop leaving the block and re-entering it (all the overhead of "activating" the block for each iteration). It's a small difference here, but if there's a more complex loop block with lots of call state (like nested blocks) it could be a bigger hit.

[zanker]

Interesting, I never actually knew EGAIN has backtraces skipped, nice trick. Thanks for explaining all this!