RENDERERGL2: Strange shadows on skyboxes when looking through glass shaders

[kai-li-wop]

One very strange thing is that the skybox gets shadows, but only when looking through a glass shader. Since WoP works a lot with glass shaders, I think this affects the user experience a lot when using the OpenGL2 renderer. Screenshots are take from PadShip but it also occurs in many other maps like CybJourney. Maybe @zturtleman could have a look because this should also appear with ioquake3. See example shader how it is used in PadShip to create the glass effect.

textures/pad_pirate/pglass04
{
	qer_editorimage textures/pad_pirate/pglass04
	surfaceparm trans
	cull none
	surfaceparm nolightmap
	qer_trans 	0.5

	{
		map textures/pad_pirate/pglass04
		blendFunc GL_ONE GL_ONE
		rgbGen identity
	}
	{
		map $lightmap
		rgbGen identity
		blendFunc filter
	}
}

[zturtleman]

I looked into this a month ago and I don't know what's going on with the shadows. Rotating the player left/right rotates the shadows in front of you like a clock. Wrong shadow projection/axis maybe? I don't really know how the shadow mapping works.

However it seems like all the glass shaders have a lightmap stage and surfaceParm nolightmap. I think nolightmap forces lightmapIndex to LIGHTMAP_BY_VERTEX which uses white image for lightmap? The opengl2 renderer could probably disable sun shadows on shaders with surfaceParm nolightmap (or possibly negative lightmapIndex) to fix the issue. I haven't got round to specifically testing and committing a solution to ioquake3 though.

[zturtleman]

I fixed the issue in ioquake3 by changing the shadow mapping instead of my previous suggested workaround to disable it on these shaders that happen to request no lightmap. ioquake/ioq3#680

[mgerhardy]

Thanks a lot - cherry-picked into world of padman with 8eaf5e5

[kai-li-wop]

@zturtleman: It seems not to be fixed. The shadows appear different but are still there. See picture taken in wop_journey. Same in wop_padship. Not sure, did we miss something to pick from ioquake3 @mgerhardy?

[mgerhardy]

Nope - everything is picked - just verified again. Where is it in wop_padship? The one from the previous screenshot was fixed for me.

UPDATE: still visible for me, too - just not in the same way and the same window I've checked before.

[zturtleman]

Using using the latest wop code, I don't see the shadow in wop_journeyCTL. I think I checked all the windowed but what is the location from viewpos console command?

To debug this, change this line to if (1) { to display the depth and screen-space shadow image on the HUD. If wop_padship has a second ship in the sky in the shadow image like the first image here, it's the same problem.

worldofpadman/code/renderergl2/tr_backend.c

Lines 1536 to 1542 in e172431

	if (0) {
	ivec4_t dstBoxDepth;
	VectorSet4(dstBoxDepth, 256, glConfig.vidHeight - 256, 256, 256);
	FBO_BlitFromTexture(tr.renderDepthImage, NULL, NULL, dstFbo, dstBoxDepth, NULL, NULL, 0);
	VectorSet4(dstBoxDepth, 512, glConfig.vidHeight - 256, 256, 256);
	FBO_BlitFromTexture(tr.screenShadowImage, NULL, NULL, dstFbo, dstBoxDepth, NULL, NULL, 0);
	}

I guessing the epsilon need to be increased. Maybe try if ( depth >= 1.0 - DEPTH_MAX_ERROR * 2 ).

worldofpadman/code/renderergl2/glsl/shadowmask_fp.glsl

Line 106 in 8eaf5e5

if ( depth >= 1.0 - DEPTH_MAX_ERROR )

I just used the existing DEPTH_MAX_ERROR constant in the file as an epsilon as it worked for me. I'm using AMD graphic card on Linux. Nvidia or different driver maybe have different precision and need a larger epsilon. (or I'm looking at the wrong window.)

[kai-li-wop]

I added some screenshots with the viewpos coordinates. I hope this is suitable. I am running WoP on Windows 10 with some old AMD Radeon.

GPU - AMD Radeon R9 200 Series
VRAM - 8192 MB - GDDR5 1375 MHz
Memory Clock - 1375 MHz
Core Clock - 1030 MHz
Total Memory Bandwidth - 352 GByte/s
Memory Bit Rate - 5.50 Gbps
OpenGL® API Version - 4.6
OpenCL™ API Version - 2.0

Looks like there is a second ship. ;)

[kai-li-wop]

Looks like if ( depth >= 1.0 - DEPTH_MAX_ERROR * 2 ) doesn't change anything. I am not sure if I do something wrong while compiling.

[zturtleman]

Okay, I'm definitely not seeing the incorrect shadows in wop_journeyCTL and seems to be the same issue with shadow image in the screenshots.

Times 2 was a guess, I don't have anything to properly base it on. Try if ( depth >= 1.0 - 0.0000001 ) and if it doesn't work try removing a 0 in 0.0000001 until it works. (Want the most 0s that works or it might remove shadows on far geometry.)

[kai-li-wop]

if ( depth >= 1.0 - 0.0001 ) works on wop_padship, when looking through the window (1st screenshot). But when I float (noclip) through the window far enough away, it's coming back but not visible for the normal player (2nd screenshot). But this is still not working on wop_journeyCTL. Needs further testing.

[kai-li-wop]

if ( depth >= 1.0 - 0.001 ) (one zero less) also removes the issue on wop_journeyCTL but also removes the possibility to see brushwork and skybox while looking through two glass window shaders. Compare left and right part of the window on the screenshot. But the issue is gone on wop_padship for sure now. Also no shadow when floating far outside the window, so 0.001 it is then.

[kai-li-wop]

I need to correct myself. The issues are not connected. It is also not possible to see the skybox/brushwork while looking through 2 glass window shaders with current code if ( depth >= 1.0 - DEPTH_MAX_ERROR ). I guess due to the blend function of the shader the brightness simply adds up to plain white.

[zturtleman]

Okay, it seems reasonable to me. Before I merge ioquake/ioq3#684, are you done testing?

I changed it from if ( depth >= 1.0 - 0.001 ) to if ( depth >= 0.999 ). It should be mathematically similar, floating point can round different. I don't imagine it would cause a problem but if could briefly check again it's okay I would appreciate it. (I should of thought of this before, sorry.)

[kai-li-wop]

if ( depth >= 0.999 ) just works fine on my side, no ghost shadows visible. Maybe @mgerhardy and @3aTmE could approve this fix on their machines?

And thank you for taking care of this problem. This was the most visible drawback of the openGL2 renderer in WoP.

[kai-li-wop]

Should be fixed with 63ab631