Incorrect FOV in simGetCameraInfo API

[xyxu1024]

Bug report

• AirSim Version/#commit: 1.4.0
• UE/Unity version: 4.26
• autopilot version: None
• OS Version: Windows

What's the issue you encountered?

My AirSim setting.json is here

{
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,
  "SimMode": "Car",
  "CameraDefaults": {
    "CaptureSettings": [
      {
        "ImageType": 0,
        "Width": 1920,
        "Height": 1080,
        "FOV_Degrees": 150,
        "AutoExposureSpeed": 100,
        "MotionBlurAmount": 0
      }
    ]
  }
}

All image sensor will give me a RGB picture of fov 150:

But when I call the API simGetCameraInfo, it always says fov is 90:

    for ii in range(5):
        camera_info = client.simGetCameraInfo(str(ii))
        responses = client.simGetImages([airsim.ImageRequest(str(ii), 0, False, False)])
        print(camera_info)

These are result:

<CameraInfo> {   'fov': 90.0,
    'pose': <Pose> {   'orientation': <Quaternionr> {   'w_val': 1.0,
    'x_val': 3.0929157219361514e-05,
    'y_val': 1.4432319271964755e-13,
    'z_val': 4.666250497109559e-09},
    'position': <Vector3r> {   'x_val': 2.0,
    'y_val': 6.207146361703053e-05,
    'z_val': -0.7654522061347961}},
    'proj_mat': <ProjectionMatrix> {   'matrix': [   [   0.0,
                      0.2679492235183716,
                      0.0,
                      0.0],
                  [   0.0,
                      0.0,
                      -0.47635418176651,
                      0.0],
                  [   0.0,
                      0.0,
                      0.0,
                      10.0],
                  [   -1.0,
                      0.0,
                      0.0,
                      0.0]]}}
<CameraInfo> {   'fov': 90.0,
    'pose': <Pose> {   'orientation': <Quaternionr> {   'w_val': 1.0,
    'x_val': 3.0929157219361514e-05,
    'y_val': 1.4432319271964755e-13,
    'z_val': 4.666250497109559e-09},
    'position': <Vector3r> {   'x_val': 2.0,
    'y_val': 0.12506206333637238,
    'z_val': -0.7654444575309753}},
    'proj_mat': <ProjectionMatrix> {   'matrix': [   [   0.0,
                      0.2679492235183716,
                      0.0,
                      0.0],
                  [   0.0,
                      0.0,
                      -0.47635418176651,
                      0.0],
                  [   0.0,
                      0.0,
                      0.0,
                      10.0],
                  [   -1.0,
                      0.0,
                      0.0,
                      0.0]]}}
<CameraInfo> {   'fov': 90.0,
    'pose': <Pose> {   'orientation': <Quaternionr> {   'w_val': 1.0,
    'x_val': 3.0929157219361514e-05,
    'y_val': 1.4432319271964755e-13,
    'z_val': 4.666250497109559e-09},
    'position': <Vector3r> {   'x_val': 2.0,
    'y_val': -0.12493792921304703,
    'z_val': -0.7654598951339722}},
    'proj_mat': <ProjectionMatrix> {   'matrix': [   [   0.0,
                      0.2679492235183716,
                      0.0,
                      0.0],
                  [   0.0,
                      0.0,
                      -0.47635418176651,
                      0.0],
                  [   0.0,
                      0.0,
                      0.0,
                      10.0],
                  [   -1.0,
                      0.0,
                      0.0,
                      0.0]]}}
<CameraInfo> {   'fov': 90.0,
    'pose': <Pose> {   'orientation': <Quaternionr> {   'w_val': 1.0,
    'x_val': 3.0929157219361514e-05,
    'y_val': 1.4432319271964755e-13,
    'z_val': 4.666250497109559e-09},
    'position': <Vector3r> {   'x_val': -6.442479616453056e-08,
    'y_val': -0.2499224841594696,
    'z_val': -1.015467643737793}},
    'proj_mat': <ProjectionMatrix> {   'matrix': [   [   0.0,
                      0.2679492235183716,
                      0.0,
                      0.0],
                  [   0.0,
                      0.0,
                      -0.47635418176651,
                      0.0],
                  [   0.0,
                      0.0,
                      0.0,
                      10.0],
                  [   -1.0,
                      0.0,
                      0.0,
                      0.0]]}}
<CameraInfo> {   'fov': 90.0,
    'pose': <Pose> {   'orientation': <Quaternionr> {   'w_val': 1.7881393432617188e-07,
    'x_val': -5.5305621193824095e-12,
    'y_val': -3.0929142667446285e-05,
    'z_val': 0.9999999403953552},
    'position': <Vector3r> {   'x_val': -2.0,
    'y_val': 6.203413067851216e-05,
    'z_val': -0.7654522061347961}},
    'proj_mat': <ProjectionMatrix> {   'matrix': [   [   0.0,
                      0.2679492235183716,
                      0.0,
                      0.0],
                  [   0.0,
                      0.0,
                      -0.47635418176651,
                      0.0],
                  [   0.0,
                      0.0,
                      0.0,
                      10.0],
                  [   -1.0,
                      0.0,
                      0.0,
                      0.0]]}}

I think the setting.json is efficient, but the API simGetCameraInfo returns an incorrect fov info

Settings

{
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,
  "SimMode": "Car",
  "CameraDefaults": {
    "CaptureSettings": [
      {
        "ImageType": 0,
        "Width": 1920,
        "Height": 1080,
        "FOV_Degrees": 150,
        "AutoExposureSpeed": 100,
        "MotionBlurAmount": 0
      }
    ]
  }
}

How can the issue be reproduced?

Include full error message in text form

What's better than filing an issue? Filing a pull request :).

[rajat2004]

Could you test #3278, it should fix the problem for settings as well

[xyxu1024]

In fact, I have seen this commit. Yes, I modify my local airsim code, and it test ok for THAT issue. According to #3278, the API simGetCameraInfo will give a correct response once I use API simSetCameraFov to set FOV

But this is a different issue, I set FoV using setting.json instead of API simSetCameraFov

[rajat2004]

@xyxu1024 You're correct, the linked PR doesn't solve the issue, sorry for that. I had a look at the code, the main problem here is that the settings allows different FoVs for different image types, which raises the question as to what should be the reported FoV in this case

[xyxu1024]

Futher more testing, the fov info returned from the APIsimGetCameraInfo is from the key field "ImageType: -1"setting.json

  "CameraDefaults": {
    "CaptureSettings": [
		{
          "ImageType": -1,
          "FOV_Degrees": 150
        }
    ]
}

[rajat2004]

So should this be classified as a bug to be fixed or a "feature" 😅. Anyways, a note regarding this should definitely be added to the docs

[zimmy87]

I think the camera FOV being based on the value set with ImageType=-1 is the intended behavior here. There is a note in the docs here that specifies that the ImageType value -1 is a special value used for setting the camera's parameters (as opposed to just the capture)