Comment on bloom implementation

[kirawi]

In a recent YouTube video on Bevy, a commenter says:

Bloom did not have a bad reputation because it was overdone, It had a bad reputation because it was done wrong! Interestingly enough Bevy is also doing it wrong which is hilarious. They will likely be getting a PR from me soon if i find time.

Believe it or not bloom occurs in real life, in camera lenses and, especially, in our eyes. Bloom is critical in producing a realistic image and properly conveying bright colors. A better name for bloom would be something like "lens scattering" actually. Bloom is created by the lens medium (glass or our eyes) slightly dispersing the image, scattering a little bit of light around before it gets to the sensor or the light receptors. Combined with filmic tonemapping (which desaturates bright colors) bloom helps convey the colors of bright materials by creating a halo around them, which on darker backgrounds retains the saturation of the color.

Obviosuly bloom becomes essentially useless when applied in a non-HDR context since we are blurring an already tone-mapped image and have no access to the real colors. In real life that would be analogous to our eyes seeing a perfectly sharp image and our brains blurring it afterwards. Very dumb. Develepers might try to compensate for this by using tricks such as a threshold which "extracts brightest regions", but in reality that only makes the final result worse. On top of that, back in the day the image was also blurred in a way that was not faithful to how light scatters in materials in real life doubling down or the unattractive unrealistic look.

Now the funny thing is, Bevy seems to have made 2 out of 3 possible mistakes they could make: they applied bloom after the tone-mapping step which is the same as applying bloom in a non-HDR context, and they have a threshold slider that is not set to 0 by default. The only advantage they seem to have over the games from the 90s is that at least they used a modern scattering algorithm. I really hope i can find time to fix it but I've been very busy lately

The user was hesitant to open an issue as they weren't certain of their understanding of the bloom implementation, but I believe it's important to at least have it out there.

[alice-i-cecile]

Ping @bevyengine/rendering-team :) Thanks for surfacing this, I suspect we'd be open to debate on all of those points.

[IceSentry]

This user seems to be very confident about an issue that doesn't exist. Bloom is applied before tonemapping in bevy. https://github.com/bevyengine/bevy/blob/main/crates/bevy_core_pipeline/src/bloom/mod.rs#L80

It's very possible there's other things that are wrong with it, but this is not one of them.

[IceSentry]

With that said, there are still issues with the bloom implementation, see #6636 for example

[cart]

Yeah that was a hilariously wrong ego trip.

[superdump]

Even if it was wrong about where we apply tonemapping and the tone of the comment could have been more constructive, it was informative and some improvements are being made to our tonemapping and bloom implementations as a consequence, so I appreciate the feedback.

[CptPotato]

I agree that bevy's bloom doesn't look that great to me - no offense to any of the authors.

I think this is mostly because of the tonemapping choice. Currently tonemapping is applied to luminance only, which I think is unusual.
I'd personally switch to tonemapping each channel individually. It's a rather trivial change and I think that it will result in a subjectively nicer look (desaturation and fade to white as brightness increases). This will probably also result in the look that the YouTube commenter envisioned.

Here's a gradient comparison between tonemapping the luminance only (top) and tonemapping RGB individually (bottom) as brightness increases (by pow(2.0, x)):

screenshot

---

On the other hand, the YouTube comment comes over as particularly entitled, even if it was not meant this way.

I disagree that bloom after tonemapping (or on SDR content) is a mistake in general. Also, as IceSentry mentioned, bevy already does this correctly according to the comment. It usually looks better in HDR scenarios, I agree. But even if it doesn't accurately model the real world, that doesn't mean that it can't look good and be of value artistically.
Calling it "obviously useless" seems very shallow, only accounting for the "generic realistic 3D triple-A look".

The same thing goes for the threshold value. Even modern games (Doom for example) use threshold values to "contain" the bloom effect a bit more, and allow for exaggerated highlights without blurring dim areas too much. Again, not physically accurate but that doesn't invalidate its usefulness when applied with care.

[CptPotato]

Currently tonemapping is applied to luminance only, which I think is unusual. I'd personally switch to tonemapping each channel individually. [...] and I think that it will result in a subjectively nicer look.

To elaborate on this, especially with highly saturated colors the current implementation fails to soften highlights and produces rather harsh colors.

Here's something closer to the bloom screenshot from the initial PR:

screenshot

Left: luminance tonemapping
Right: RGB tonemapping (identical tonemapping curve)

This is not actually in bevy, I used shadertoy and bevy's tonemapping code here to illustrate this

[superdump]

I think the majority was still your code. Mostly the plumbing changed. Thanks for following up with this information! More tonemapping options / a closer look at tonemapping was intended after all this stuff was merged. Similarly more and a closer look at upscaling.

[superdump]

Looks like @JMS55 is already changing the tonemapping to ACES filmic and tweaking some other things.

[StarLederer]

the YouTube comment comes over as particularly entitled

Sorry 🥲. Writing clearly is hard as it is, making it sound nice makes it even harder

I disagree that bloom after tonemapping (or on SDR content) is a mistake in general

You have a point, it is possible to pull off and it can be a stylistic choice. In my comment i was focusing on the physically inspired interpretation of bloom.

Calling it "obviously useless" seems very shallow

I'm trying to be shallow and technical. We are writing code, not a poem. But i guess that does make me sound rude (this sentence included).

modern games use threshold values to "contain" the bloom effect a bit more, and allow for exaggerated highlights without blurring dim areas too much

I have an idea how this can be achieved in a physically-inspired way. I describe it as "deblooming" in #6703 and will suggest a similar option to #6677

[CptPotato]

Thanks for the clarification and for collaborating!

The "deblooming" approach sounds interesting, I haven't come across something like that, yet.
Also, I think the adjusted blending from #6703 results in a nicer falloff than the current implementation. It might be worth exploring that in the existing PR aswell.

[pkupper]

I think it makes sense to give developers artistic control beyond what is realistic while keeping the defaults as physically accurate as possible. The issue with a non-zero threshold besides not being physically accurate is that it can make the bloom effect very scene dependent making it hard to pick good defaults.

Similar to how the goal of PBR is to make materials look consistent under different lighting conditions, I'd say it makes sense to apply the same approach to the bloom effect which attempts to emulate a real physical phenomenon.

Therefore I would argue to change the default bloom settings to have a default threshold of zero and a linear response.

Another issue with Bevy's current bloom Implementation is that it is not energy preserving. It increases the overall brightness of the image due to using additive blending which is not physically accurate. A common approach that addresses this seems to be to linearly interpolate between the original image and the bloom output depending on the intensity.

There is a great video that outlines what a completely realistic bloom effect would involve, depending on the shape of the camera aperture. While probably not feasible in realtime it gives a good theoretical overview on how bloom works:
https://youtu.be/QWqb5Gewbx8

Another good resource is this guest article on learnopengl.com which addresses the threshold and energy conservation topics:
https://learnopengl.com/Guest-Articles/2022/Phys.-Based-Bloom
Note that the bloom effect from the original bloom tutorial on learnopengl.com, while being a great learning resource for OpenGL, is very unrealistic.

[superdump]

Thanks for the links.

[JMS55]

Working on a fix to these issues: #6677

[StarLederer]

Hi (i'm the Youtube user)

[kirawi]

@alice-i-cecile @cart The recent comment should be removed.

[StarLederer]

Now it looks like my comment is the recent one 😟

[kirawi]

Haha, I'm guessing they already removed the spam one.