Vulkan: Normal map mipmaps suffer from artifacts when VRAM compression is used

[abaky]

Godot version

v4.0.alpha2.official [79077e6]

System information

Windows 10

Issue description

Material seems to very quickly and aggressively switch to lover res version of roughness texture, as a result you get pixelate roughness

matTest.mp4

In the video, I move the camera from super close up to 30 to 50 meters away from mesh plane.

Steps to reproduce

Create ORMMaterail3D or StandardMaterial3D and put texture with packed AO, roughness and metalness in appropriate slots. Assign that material to mesh instance and up close it looks ok, but as soon as you move slightly away from mesh instance, the lover res version kicks in and pixelated effect starts to appear.

Minimal reproduction project

No response

[Calinou]

@abaky Please upload a minimal reproduction project to make this easier to troubleshoot.

Also:

• Does this occur if you set the texture compression on the ORM map to Lossless instead of VRAM Compressed?
• Does this occur if you disable Screen Space Roughness Limiter in the Project Settings?

[abaky]

@Calinou changing compression on ORM texture to Lossless or/and disabling Screen Space Roughness Limiter does not seem to make a difference.

[abaky]

Minimal reproduction project

[Calinou]

It seems that #49981 wasn't fully resolved after all (or it's a different issue).
This is actually linked to the normal map's compression mode, not the ORM map's:

Normal map set to VRAM Compressed

Normal map set to Lossless

If you disable mipmap generation, those artifacts are no longer present but the texture will look grainy at a distance. The normal map's original VRAM-compressed texture looks good otherwise.

For the normal map, Godot uses RGTC (red-green texture compression) as the blue channel is discarded and reconstructed in the shader:

This is expected behavior, but I don't know why the quality is so low. Increasing Lossy Quality to 1 and reimporting doesn't improve the VRAM-compressed normal map quality. cc @fire

In the meantime, you can set the albedo and ORM textures to VRAM Compressed and it still looks good. If you wish to reduce memory usage or speed up loading times, use the Size Limit property to downsize the normal map on import.

[abaky]

Only changing compression from VRAM Compressed to VRAM Uncompressed already fixes the visual for me, I don't have to disable mipmaps.

[fire]

I'm at my computer now.

Try BPTC VRAM compression. (BC7 compression).

[abaky]

In the project settings I enabled Bptc and disabled S3Tc, reimported normal map using VRAM Compressed and the artifacts are gone. If I leave S3Tc on then it still looks pixelated. When first enabling Bptc I got this error if that means enaything

[fire]

Here are some screenshots of my results.

[abaky]

@fire Ok, enabling Bptc in settings and enabling Bptc Ldr on imported normal texture and reimporting that texture, have fixed the visual pixilation for me.

[fire]

Are you still affected by this issue?

[abaky]

Sorry for late replay

Yes, in v4.0.alpha4.official [f470979] I made a fresh project and the normal map still produces artifacts.

This is all set to default import vise.

After I enable BptcLdr and reimport textures the problems is gone.

[ModularNucleus]

I believe the issue I have is the same as this issue (or at least related).

The issue is visible banding ("stairstepping") in normal maps when imported with S3TC/RGTC (DXT5/BC5?) compression. Artifacts that to my albeit limited knowledge should be avoidable (see below). As I understand it, banding is a common issue with normal maps exported to 8-bit per channel images such as 8-bit PNGs. As Godot does not support 16-bit PNGs, a workaround is exporting to e.g. 16/32-bit OpenEXR and then dithering down to 8-bit PNGs. The dithering noise masks the banding. When importing to Godot this should then allow slighty noisy, but almost banding-free normal maps.
See e.g.: https://polycount.com/discussion/148303/of-bit-depths-banding-and-normal-maps

However, in my experiments I have been unable to achieve this when using S3TC compression. For reference, my authoring process is as follows:

• Blender: 1) Create bake image as 32bit float OpenEXR. 2) Bake normal map (from Multires) to image. 3) Change image "Color Space" to sRGB. 4) Export image as 32bit float, sRGB OpenEXR.
• GIMP: 1) Open image. 2) Reduce image precision to 8-bit and dither - e.g. with Floyd-Steinberg. 3) Export image as 8-bit PNG
• Godot: Import PNG with S3TC VRAM compression -> visible banding when using the normal map in StandardMaterial3D

As discussed, enabling BPTC compression and importing with BPTC Ldr enabled (BC7 compression) fixes the banding issue:

S3TC/RGTC (DXT5/BC5) - I assume this corresponds to Godot's image class' format Enum "FORMAT_RGTC_RG = 21"?

BPTC (BC7) - I assume this corresponds to Godot's image class' format Enum "FORMAT_BPTC_RGBA = 22"?

Using BPTC / BC7 for importing normal maps is a viable solution, then. However, I still have a few caveats:

• Compression time is increased with BC7
• As I read it, according to e.g. Sebastian Aaltonen, BC5 should be able to compete in quality with BC7 for normal maps in most cases: https://twitter.com/SebAaltonen/status/978736887627223040

My knowledge of normal maps and texture compression is very limited, but the question is whether Godot's RGTC / DXT5 compression is working as it should when compressing normal maps? I have included a simple MRP where the normal map has been imported with both S3TC and BPTC (Ldr enabled):
Normal map compression.zip

[Calinou]

I can still reproduce this with this MRP on 4.1.dev 9b9bb41 after removing the .godot/ folder and reimporting all resources:

@ModularNucleus Godot currently doesn't dither images on import if the source has a higher precision. This could be added, but it'd need to be an import option you can disable as it's sometimes undesired.

That said, with the MRP linked above, this particular issue is unrelated. It doesn't happen because the normal map is lacking precision; it happens because the VRAM compression is altering the texture too much. RGTC compression should avoid this, but using standard DXT RGB(A) compression on a normal map will make it look broken like it does here.

The normal map in Godot is using RGTC compression:

Maybe it's internally not configuring things correctly, converting a RGB-compressed VRAM texture into a RG one.

[AlexHCC]

I get the same visual artifact regardless of texture compression or mipmaps, and I haven't found a way to fix it.

Here is a forum post I've found online that describes the same issue.

I think this is a major issue since it breaks all normal maps, and the only way to fix it is to remove them.
The bug persists in 4.1 betas.

[fire]

Are we using sufficiently high quality vram compression for normal maps? Like BPTC or ASTC?

[clayjohn]

Current best workaround is to change compression to Basis Universal instead of VRAM compressed. Both options ultimately use the same compression format (DXT5_RA_AS_RG), but using Basis Universal outputs correct results.

[Saul2022]

Current best workaround is to change compression to Basis Universal instead of VRAM compressed. I can confirm that in 4.1 rc2, BasisU gives better results than vram compression deleting most white artifacts when you are far but worse than lossy.

[npip99]

I'd like to add here that this doesn't appear to be specific to Normal map mipmaps. This pixelation issue occurs on every Sampler2D.

Currently, heightmaps usage in procedural geometry vertex shaders are completely broken when using compressed VRAM, it destroys the procedural geometry with the artifacts as vertices alternate between up / down / up / down / up due to the dithering pattern, rather than staying smooth. Heightmaps PNGs are often huge and smooth, and should lend themselves to compression very well.

[Calinou]

I'd like to add here that this doesn't appear to be specific to Normal map mipmaps. This pixelation issue occurs on every Sampler2D.

This is expected with VRAM compression, which can't render smooth gradients by design (it works with 4×4 macroblocks, sometimes larger with ASTC). Use lossless compression when you need smooth heightmaps instead, or for pixel art textures in general.

[npip99]

Use lossless compression when you need smooth heightmaps instead, or for pixel art textures in general.

I'll note for anyone else that might be on this thread (In response to lossless compression):

VRAM Compression with "High Quality" (BPTC) looks virtually identical to the original image (Lossless), and uses 33.3% of the VRAM. Checking with the color picker, the RGBs appear unmanipulated.

"low quality" (DXT5) uses 16.6% of the original VRAM, but has significant artifacts as seen above; these artifacts are visible even when zoomed out: Using a half-resolution image with BPTC appears to be better if smoothness is important, as the pixelated images from DXT5 are very noisy and can cause aliasing.

When "normal map" is set to "enable", both low quality (DXT5_RG_AS_RA) and "High Quality" (BPTC) use 33.33% of the VRAM of a lossless image, so "High Quality" appears to come at no cost.

[@ https://docs.godotengine.org/en/stable/tutorials/assets_pipeline/importing_images.html#detect-3d Under the section "Detect 3D", perhaps "High Quality" should be emphasized more as an option. It saves a lot of VRAM and doesn't have the same jarring artifacts that DXT5 does.]

[npip99]

I believe this issue specifically has to do with the RA_AS_RG transformation, involving mipmap generation.

A huge percentage of the mipmap becomes set to "1.0", for no clear reason (I'll call this 1.0 area the "glitched" area). The glitches happen in a pixelated format, but the pixels are much larger than the underlying resolution (You can see the higher resolution by the tiny pixels in the un-glitched regions). In specific, it's observed that the glitched pixels are 4x4 "true" pixels (Compression block?).

Another property is that only the "x" coordinate of the normalmap is glitched. The "y" coordinate is unaffected and generates correctly.

---

The shader code to generate the two images is here (Other than this shader code, it's just using the project posted by the OP):

shader_type spatial;
render_mode blend_mix,depth_draw_opaque,cull_back,diffuse_burley,specular_schlick_ggx;

uniform sampler2D texture_normal : filter_nearest_mipmap,repeat_enable;

void fragment() {
	vec2 base_uv = UV;
	ROUGHNESS = 1.0;
	METALLIC = 0.0;
	vec3 normal_map = textureLod(texture_normal,base_uv,3.0).rgb;
	normal_map = 2.0*normal_map-1.0;
	normal_map.z = sqrt(max(0.0, 1.0 - dot(normal_map.xy, normal_map.xy)));
	
	bool SHOW_GLITCH = true;
	if (SHOW_GLITCH) {
		ALBEDO = vec3(clamp((20.0*normal_map.x+1.0)/2.0, 0, 1), 0, 0);
	} else {
		ALBEDO = vec3(clamp((20.0*normal_map.y+1.0)/2.0, 0, 1), 0, 0);
	}
}

---

I think the issue would be along the lines of

• https://github.com/godotengine/godot/blob/master/drivers/gles3/storage/texture_storage.cpp#L590C15-L590C35
• https://github.com/godotengine/godot/blob/master/core/io/image.cpp#L3890C13-L3890C35
  • Does this need to also set w[i+1] = w[i+3] for the mipmaps too? Not sure.

[fire]

Now that astc and bptc are in, we could default to "high quality" for normal maps since there is significant loss in the current dxt5 and etc2 usages.

Previously mobile devices didn't have astc support with Godot Engine.

[Calinou]

Now that astc and bptc are in, we could default to "high quality" for normal maps since there is significant loss in the current dxt5 and etc2 usages.

Previously mobile devices didn't have astc support with Godot Engine.

Defaulting to high quality would slow iteration a lot while you're still working on assets, so I don't think it's a good idea. Even if we have Betsy integration to speed up BPTC compression, it doesn't support ASTC and won't work when using the Compatibility rendering method.

Ideally, we should perform high-quality import only before exporting a project so that iteration times are not affected.

[BlueCube3310]

I may be wrong about this, but DXT5 compression differs from RGTC_RG as with the latter the red and green channels are interpreted the same way as DXT5's alpha channel, which results in significantly higher quality at the cost of data size (16 bytes per block), while DXT5 compresses the RGB channels in a similar way to DXT1 (5:6:5 quantized endpoints), so the quality is generally lower.

After analyzing the texture compression code, it seems that Godot converts the RG normal map into an RGBA texture (G to Alpha, G and B set to 0), and compresses it to the DXT5 format, which results in visible artifacts due to the algorithm's lack of precision as the R channel gets quantized heavily.

Edit:
Quantization artifacts can be seen on the DXT5 version:

DXT5 RA-As-RG	RGTC RG

[npip99]

Just current status on this issue as a result of the most recent PR (Woot! Checked it out and it fixes it nicely):

I'm not sure if that PR should close this issue, because FORMAT_DXT5_RA_AS_RG is still in the codebase and has broken mipmaps. Unless the plan is to remove FORMAT_DXT5_RA_AS_RG (Perhaps we still keep this issue open until that's done?).

I don't believe this issue is because of quantization artifacts, I think somewhere the implementation of FORMAT_DXT5_RA_AS_RG has a bug (Gave some LOC regions that might be where it is?).

Reasoning:

• In the example I showed above a very high percentage of the image just becomes set to "1.0"; it's not merely low quality, but completely incorrect, over half the image gets set to exactly "1.0" even if the original image is dark / close to "0.0". Additionally, it only happens at mipmap levels, so it's not an issue of RA_AS_RG's ability to hold the data itself, but rather specifically an issue related to generating mipmaps. It's exactly the 4x4 sections that are set to "1.0" that come out as the ugly pixelations in the wood example, the other 4x4 sections are fine.

---

Maybe we rename this issue to "FORMAT_DXT5_RA_AS_RG garbles R-channel in mipmaps", since the issue is no longer Normal Map's default mipmap setting.

Currently https://docs.godotengine.org/en/stable/classes/class_image.html#enum-image-format has no info on RA_AS_RG, would be clean to mention "R channel is garbled" / "Deprecated" / or what the plan is etc.

[BlueCube3310]

Additionally, it only happens at mipmap levels, so it's not an issue of RA_AS_RG's ability to hold the data itself

It still happens to images without mipmaps, just less noticeably:

VRAM Compressed	Basis Universal

FORMAT_DXT5_RA_AS_RG is still in the codebase and has broken mipmaps. Unless the plan is to remove FORMAT_DXT5_RA_AS_RG

BasisUniversal still uses the same format (DXT5_RA-As-RG), yet it doesn't exhibit the artifacts. The conversion from RG to RA is handled the same way there as it is with etcpak.
The artifacting also happens with VRAM Compressed ETC2_RA_As_RG, so the issue is likely due to etcpak lacking accuracy in favor of fast compression speed.