Vulkan: Weird behavior with emissive materials when VoxelGI and ReflectionProbes is set as Interior

[jcostello]

Godot version

v4.0.dev

System information

Mx Linux (Debian Based)

Issue description

When adding VoxelGI + ReflectionProbes and this last one is set as Interior, weird lighting issue happens.

No Interior

Internal

Steps to reproduce

Use minimal reproduction project and set RefProbes as internal.

Minimal reproduction project

GiPlusRefs.zip

[clayjohn]

I think I know what is going on! I noticed odd blending behaviour in the reflection probe code and this confirms my suspicion.

When reflection probes are used they blend between the reflection probe colour and the current specular light colour (i.e. reflection). When set to exterior, the blend happens based on how close the pixel is to the center of the probe, but when using interior the specular value is just thrown away!

This means with interior probes all other forms of specular GI get overwritten. Clearly we need to figure different blending behaviour.

I will update with links to the relevant code when I am back in front of a computer.

[Calinou]

I will update with links to the relevant code when I am back in front of a computer.

This is where ReflectionProbes are drawn:

Shared function across Vulkan Clustered + Vulkan Mobile

godot/servers/rendering/renderer_rd/shaders/forward_clustered/scene_forward_clustered.glsl

Lines 1307 to 1374 in 7c7a86f

	{ // process reflections
	vec4 reflection_accum = vec4(0.0, 0.0, 0.0, 0.0);
	vec4 ambient_accum = vec4(0.0, 0.0, 0.0, 0.0);
	uint cluster_reflection_offset = cluster_offset + implementation_data.cluster_type_size * 3;
	uint item_min;
	uint item_max;
	uint item_from;
	uint item_to;
	cluster_get_item_range(cluster_reflection_offset + implementation_data.max_cluster_element_count_div_32 + cluster_z, item_min, item_max, item_from, item_to);
	#ifdef USE_SUBGROUPS
	item_from = subgroupBroadcastFirst(subgroupMin(item_from));
	item_to = subgroupBroadcastFirst(subgroupMax(item_to));
	#endif
	#ifdef LIGHT_ANISOTROPY_USED
	// https://google.github.io/filament/Filament.html#lighting/imagebasedlights/anisotropy
	vec3 anisotropic_direction = anisotropy >= 0.0 ? binormal : tangent;
	vec3 anisotropic_tangent = cross(anisotropic_direction, view);
	vec3 anisotropic_normal = cross(anisotropic_tangent, anisotropic_direction);
	vec3 bent_normal = normalize(mix(normal, anisotropic_normal, abs(anisotropy) * clamp(5.0 * roughness, 0.0, 1.0)));
	#else
	vec3 bent_normal = normal;
	#endif
	vec3 ref_vec = normalize(reflect(-view, bent_normal));
	ref_vec = mix(ref_vec, bent_normal, roughness * roughness);
	for (uint i = item_from; i < item_to; i++) {
	uint mask = cluster_buffer.data[cluster_reflection_offset + i];
	mask &= cluster_get_range_clip_mask(i, item_min, item_max);
	#ifdef USE_SUBGROUPS
	uint merged_mask = subgroupBroadcastFirst(subgroupOr(mask));
	#else
	uint merged_mask = mask;
	#endif
	while (merged_mask != 0) {
	uint bit = findMSB(merged_mask);
	merged_mask &= ~(1 << bit);
	#ifdef USE_SUBGROUPS
	if (((1 << bit) & mask) == 0) { //do not process if not originally here
	continue;
	}
	#endif
	uint reflection_index = 32 * i + bit;
	if (!bool(reflections.data[reflection_index].mask & instances.data[instance_index].layer_mask)) {
	continue; //not masked
	}
	reflection_process(reflection_index, vertex, ref_vec, normal, roughness, ambient_light, specular_light, ambient_accum, reflection_accum);
	}
	}
	if (reflection_accum.a > 0.0) {
	specular_light = reflection_accum.rgb / reflection_accum.a;
	}
	#if !defined(USE_LIGHTMAP)
	if (ambient_accum.a > 0.0) {
	ambient_light = ambient_accum.rgb / ambient_accum.a;
	}
	#endif
	}

Vulkan Clustered

godot/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl

Lines 878 to 956 in 7c7a86f

	void reflection_process(uint ref_index, vec3 vertex, vec3 ref_vec, vec3 normal, float roughness, vec3 ambient_light, vec3 specular_light, inout vec4 ambient_accum, inout vec4 reflection_accum) {
	vec3 box_extents = reflections.data[ref_index].box_extents;
	vec3 local_pos = (reflections.data[ref_index].local_matrix * vec4(vertex, 1.0)).xyz;
	if (any(greaterThan(abs(local_pos), box_extents))) { //out of the reflection box
	return;
	}
	vec3 inner_pos = abs(local_pos / box_extents);
	float blend = max(inner_pos.x, max(inner_pos.y, inner_pos.z));
	//make blend more rounded
	blend = mix(length(inner_pos), blend, blend);
	blend *= blend;
	blend = max(0.0, 1.0 - blend);
	if (reflections.data[ref_index].intensity > 0.0) { // compute reflection
	vec3 local_ref_vec = (reflections.data[ref_index].local_matrix * vec4(ref_vec, 0.0)).xyz;
	if (reflections.data[ref_index].box_project) { //box project
	vec3 nrdir = normalize(local_ref_vec);
	vec3 rbmax = (box_extents - local_pos) / nrdir;
	vec3 rbmin = (-box_extents - local_pos) / nrdir;
	vec3 rbminmax = mix(rbmin, rbmax, greaterThan(nrdir, vec3(0.0, 0.0, 0.0)));
	float fa = min(min(rbminmax.x, rbminmax.y), rbminmax.z);
	vec3 posonbox = local_pos + nrdir * fa;
	local_ref_vec = posonbox - reflections.data[ref_index].box_offset;
	}
	vec4 reflection;
	reflection.rgb = textureLod(samplerCubeArray(reflection_atlas, material_samplers[SAMPLER_LINEAR_WITH_MIPMAPS_CLAMP]), vec4(local_ref_vec, reflections.data[ref_index].index), roughness * MAX_ROUGHNESS_LOD).rgb * sc_luminance_multiplier;
	reflection.rgb *= reflections.data[ref_index].exposure_normalization;
	if (reflections.data[ref_index].exterior) {
	reflection.rgb = mix(specular_light, reflection.rgb, blend);
	}
	reflection.rgb *= reflections.data[ref_index].intensity; //intensity
	reflection.a = blend;
	reflection.rgb *= reflection.a;
	reflection_accum += reflection;
	}
	switch (reflections.data[ref_index].ambient_mode) {
	case REFLECTION_AMBIENT_DISABLED: {
	//do nothing
	} break;
	case REFLECTION_AMBIENT_ENVIRONMENT: {
	//do nothing
	vec3 local_amb_vec = (reflections.data[ref_index].local_matrix * vec4(normal, 0.0)).xyz;
	vec4 ambient_out;
	ambient_out.rgb = textureLod(samplerCubeArray(reflection_atlas, material_samplers[SAMPLER_LINEAR_WITH_MIPMAPS_CLAMP]), vec4(local_amb_vec, reflections.data[ref_index].index), MAX_ROUGHNESS_LOD).rgb;
	ambient_out.rgb *= reflections.data[ref_index].exposure_normalization;
	ambient_out.a = blend;
	if (reflections.data[ref_index].exterior) {
	ambient_out.rgb = mix(ambient_light, ambient_out.rgb, blend);
	}
	ambient_out.rgb *= ambient_out.a;
	ambient_accum += ambient_out;
	} break;
	case REFLECTION_AMBIENT_COLOR: {
	vec4 ambient_out;
	ambient_out.a = blend;
	ambient_out.rgb = reflections.data[ref_index].ambient;
	if (reflections.data[ref_index].exterior) {
	ambient_out.rgb = mix(ambient_light, ambient_out.rgb, blend);
	}
	ambient_out.rgb *= ambient_out.a;
	ambient_accum += ambient_out;
	} break;
	}
	}

Vulkan Mobile

godot/servers/rendering/renderer_rd/shaders/forward_mobile/scene_forward_mobile.glsl

Lines 1058 to 1100 in 7c7a86f

	if (!sc_disable_reflection_probes) { //Reflection probes
	vec4 reflection_accum = vec4(0.0, 0.0, 0.0, 0.0);
	vec4 ambient_accum = vec4(0.0, 0.0, 0.0, 0.0);
	uint reflection_indices = draw_call.reflection_probes.x;
	#ifdef LIGHT_ANISOTROPY_USED
	// https://google.github.io/filament/Filament.html#lighting/imagebasedlights/anisotropy
	vec3 anisotropic_direction = anisotropy >= 0.0 ? binormal : tangent;
	vec3 anisotropic_tangent = cross(anisotropic_direction, view);
	vec3 anisotropic_normal = cross(anisotropic_tangent, anisotropic_direction);
	vec3 bent_normal = normalize(mix(normal, anisotropic_normal, abs(anisotropy) * clamp(5.0 * roughness, 0.0, 1.0)));
	#else
	vec3 bent_normal = normal;
	#endif
	vec3 ref_vec = normalize(reflect(-view, bent_normal));
	ref_vec = mix(ref_vec, bent_normal, roughness * roughness);
	for (uint i = 0; i < 8; i++) {
	uint reflection_index = reflection_indices & 0xFF;
	if (i == 4) {
	reflection_indices = draw_call.reflection_probes.y;
	} else {
	reflection_indices = reflection_indices >> 8;
	}
	if (reflection_index == 0xFF) {
	break;
	}
	reflection_process(reflection_index, vertex, ref_vec, bent_normal, roughness, ambient_light, specular_light, ambient_accum, reflection_accum);
	}
	if (reflection_accum.a > 0.0) {
	specular_light = reflection_accum.rgb / reflection_accum.a;
	}
	#if !defined(USE_LIGHTMAP)
	if (ambient_accum.a > 0.0) {
	ambient_light = ambient_accum.rgb / ambient_accum.a;
	}
	#endif
	} //Reflection probes

The exterior flag is used in the shaders, and is the opposite of interior. Fixing this would also be a good opportunity to flip it in the shader so that it's consistent with the scripting API.

[clayjohn]

Updated MRP for RC2:
GiPlusRefsRC2.zip

Here are the problematic lines:

godot/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl

Lines 920 to 923 in 29f670b

	if (reflections.data[ref_index].exterior) {
	reflection.rgb = mix(specular_light, reflection.rgb, blend);
	}

specular_light contains all the previously calculated specular color. Since Reflection Probes are the last form of GI added, specular_light contains the specular light from the sky, from lightmaps, from VoxelGI, and from SDFGI. When set to interior, all those specular light sources are ignored.

In the MRP this totally breaks the scene because the reflection probe is poorly placed to include the emissive sphere in its reflection.

We need a better blending procedure that can intelligently blend between reflection probes and other light sources. The interior flag is really only meant to exclude light from the sky. I don't think it should exclude all other light sources.

For most cases I expect that leaving the interior flag unchecked will be good enough when using ReflectionProbes with other forms of GI