Fixes #2815: Non-linear bias corrupts SurfaceNormals when requested through AirSim

[LSBOSS]

This fix forces usage of linear gamma for RenderTargets utilized for capturing SurfaceNormals and Segmentations, resulting in correct colors being output. (See #2815 for discussion about what is considered being correct)

As a bonus, the post process material for extracting world normals and packing them into a RGB texture has been marked as unlit for some eventual speed ups and also because the buffer visualization material of Unreal Editor is doing the very same :)

[rajat2004]

Having direct int values makes it more difficult to understand, might be better to use enum fields such as ImageType::Scene etc.

[LSBOSS]

Makes absolute sense - I just bluntly copied what was there before :) Let me fix this in the next commit to this pull request.

[LSBOSS]

DONE 👍

[rajat2004]

So does forcing Linear Gamma Space change the values in segmentation image like you mentioned in the issue?
(I have no idea what this does though 😅 )

[LSBOSS]

It absolutely does. Now the values of the segmentation output correspond to those of mentioned seg_rgbs.txt.

The ForeLinearGamma flag basically tells UE4 to not apply any transformation of input colors (which UE4 is usally doing) when running through gamma correction. The background is that UE4 internally stores color information in sRGB space wherever it can for a better color reproduction. That's a non linear space that, if displayed on your monitor directly, seems to be too dark for the human eye. Here come's the gamma correction into action and is "correcting" things back to normal.

That's all fine for ImageType::Scene but not for textures not in sRGB space and where exact colors are important. E.g. Segmentation and SurfaceNormals.

[madratman]

Thanks for another good PR.

Here's a segmentation test. Use this settings.json

{
    "SettingsVersion": 1.2,
    "SimMode": "ComputerVision",
    "Vehicles": {
        "MyVehicle": {
            "VehicleType": "ComputerVision",
            "Cameras": {
                "my_camera": {
                    "CaptureSettings": [
                        {
                            "ImageType": 5,
                            "FOV_Degrees": 90,
                            "Width": 1024,
                            "Height": 1024
                        }
                    ],
                    "X": 0.0,
                    "Y": 0.0,
                    "Z": 0.0,
                    "Pitch": 0.0,
                    "Roll": 0.0,
                    "Yaw": 0.0
                }
            }
        }
    }
}

with the following:

import airsim
import numpy as np

client = airsim.VehicleClient(timeout_value = 7200)
client.confirmConnection()

responses = client.simGetImages([airsim.ImageRequest("my_camera", airsim.ImageType.Segmentation, False, False)])

img1d = np.fromstring(responses[0].image_data_uint8, dtype=np.uint8)
img_rgb = img1d.reshape(responses[0].height, responses[0].width, 3)
print(np.unique(img_rgb.reshape(-1, img_rgb.shape[-1]), axis=0)) 

With PR, we get the following BGR values:

[[ 17 194 164]
 [103 216  44]
 [121   9 150]
 [205  72  54]
 [225  35  73]]

With 1.3.1 binaries, we get:

[[ 73 226 210]
 [170 237 115]
 [183  53 202]
 [232 145 127]
 [241 104 146]]

[madratman]

/azp run microsoft.AirSim

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).