Performance Improvement Ideas

[wied03]

A lot of time gets spent loading the example groups and examples that will be tested.

1. Precompilation/minification
2. Closure compilation?
3. Other profiling
4. When specific examples are focused on, see if they can be filtered out BEFORE opal compilation
5. Per Remove Kernel#caller patches #36 and Compiler/Line Number Metadata #57, reduce the exception throwing just to get stack traces

[jgaskins]

Posting my initial thoughts here. Keep in mind that some of this is speculation and isn't a reason to at least give it a shot:

Precompilation/minification

I don't think minification improves performance for anything but transfer over the wire. If it's loading from localhost, this is very little time (as in microseconds to transfer a few hundred KB). Precompilation would definitely be great, though. If the Opal compiler doesn't have to process it and only needs to compile the app, that would help a lot unless the Opal::Builder instance is reused between spec runs, in which case the compiled RSpec will be cached.

Closure compilation?

AFAIK, most JS VMs will not compile a function at all until it is executed. I'd need to run more benchmarks to be sure about it, though.

Other profiling

I can see what I can do about this. I've got a couple ideas, but just to clarify before I spend too much time on it, opal-rspec does use the actual RSpec gem code, right? We may be limited in performance by RSpec internals if so.

When specific examples are focused on, see if they can be filtered out BEFORE opal compilation

This would probably be pretty cool, but I wonder if the effort would be worth it. We could check the benefit by precompiling a spec run with and without unfocused specs (that is, literally remove unfocused specs from the spec code) and timing the compilation time. Then run each of those and check the runtimes.

My hypothesis is that they would differ only slightly and that the RSpec load time would dwarf it no matter what we do.

reduce the exception throwing just to get stack traces

This might have a pretty big performance impact for failing specs. Exceptions are probably the worst-performing thing you can do in a JS VM. Generating an exception, throwing it, and catching it all cause deoptimizations — which is bad enough on its own, but the process of unwinding the stack and generating that stack trace is also slow inside the VM itself.

JRuby (at least pre-9k) suffers from this same problem and for almost the same reason. MRI has internal optimizations for generating stack traces (so you can call caller from anywhere, not just in exceptions), but most other VMs for any language don't.

[wied03]

@jgaskins - Thanks for the info. I'll close this now the the main next step will be to get rid of the metadata exception throws to get trace info.

[wied03]

@jgaskins - Just so you know, the exception throwing I mentioned above used to happen for every example, not just failing ones. It was being used to give RSpec metadata about where each example is (RSpec normally uses #caller on MRI). On master now, this has been removed and there is no location data for examples until they fail, at which point there is a stack trace.

[wied03]

@jgaskins - https://github.com/karma-runner/karma-closure is doing number 4. In my case, I'd have to do it with a webpack/opal loader because I decided getting into the guts of Karma with https://github.com/wied03/karma-opal-rspec/ was a losing proposition.