Fix actor name conflict for multiple ComputerVision vehicles #4109

rajat2004 · 2021-10-23T14:04:46Z

About

Fixes the name conflict by including the vehicle name

How Has This Been Tested?

With the below settings from #4092

{
    "SettingsVersion": 1.2,
    "SimMode": "ComputerVision",
    "Vehicles": {
        "Cam1": {
            "VehicleType": "ComputerVision",
            "X": 26,
            "Y": -1,
            "Z": -10
        },
        "Cam2": {
            "VehicleType": "ComputerVision",
            "X": 26,
            "Y": -2,
            "Z": -20
        }
    }
}

Screenshots (if appropriate):

Octavian-Zhang · 2021-10-24T09:50:47Z

Hi, @rajat2004, this fix can though survive the test setting above, it will crash Unreal when using the simAddVehicle() API to spawn "computervision" pawn during runtime. Please add the following dummy program to the test case also, along with the settings.json file above:

STRICT_MODE_OFF
#ifndef RPCLIB_MSGPACK
#define RPCLIB_MSGPACK clmdep_msgpack
#endif // !RPCLIB_MSGPACK
STRICT_MODE_ON

#include "api/RpcLibClientBase.hpp"

int main()
{
    using namespace msr::airlib;
    msr::airlib::RpcLibClientBase client;

    client.confirmConnection();
    client.simAddVehicle("runtimeCamera", "computervision", Pose(), ""); // Unreal Crash

    return 0;
}

Octavian-Zhang · 2021-10-24T10:23:08Z

Hi, @rajat2004, this fix can though resolve pawn name conflict, the x, y, z initial position settings from the settings.json file above is not respected. Please add the following dummy program to the test case also, along with the settings.json file above:

STRICT_MODE_OFF
#ifndef RPCLIB_MSGPACK
#define RPCLIB_MSGPACK clmdep_msgpack
#endif // !RPCLIB_MSGPACK
STRICT_MODE_ON

#include "api/RpcLibClientBase.hpp"

int main()
{
    using namespace msr::airlib;
    msr::airlib::RpcLibClientBase client;

    client.confirmConnection();
    auto PawnList = client.listVehicles();

    for (size_t i = 0; i < PawnList.size(); i++) {
        auto pawnpose = client.simGetVehiclePose(PawnList[i]);
        std::cout << PawnList[i] + " Attitude:" << "\t"
                  << pawnpose.orientation.coeffs().x() << " "
                  << pawnpose.orientation.coeffs().y() << " "
                  << pawnpose.orientation.coeffs().z() << " "
                  << pawnpose.orientation.coeffs().w() << std::endl;
        std::cout << PawnList[i] + " Position:"  << "\t"
                  << pawnpose.position.x() << " "
                  << pawnpose.position.y() << " "
                  << pawnpose.position.z() << std::endl;
    }

    return 0;
}

See the printed orientation&position inquiries differs from the JSON setting.

rajat2004 · 2021-10-24T11:35:44Z

I tried simAddVehicle but am getting a different error than what you mentioned in #4092 (comment), was the project built correctly with this change? And it works correctly for the multi-vehicle settings right?

terminating with uncaught exception of type std::runtime_error: Multiple reset() calls detected without call to update()
Signal 6 caught.
Malloc Size=65538 LargeMemoryPoolOffset=65554 
CommonUnixCrashHandler: Signal=6
Malloc Size=35086 LargeMemoryPoolOffset=100670 
Malloc Size=50986 LargeMemoryPoolOffset=151674 
Malloc Size=35086 LargeMemoryPoolOffset=186782 
Malloc Size=50986 LargeMemoryPoolOffset=237786 
Malloc Size=35086 LargeMemoryPoolOffset=272894 
Malloc Size=51038 LargeMemoryPoolOffset=323950 
[2021.10.24-11.18.06:375][365]LogCore: === Critical error: ===
Unhandled Exception: SIGABRT: abort() called

[2021.10.24-11.18.06:375][365]LogCore: Fatal error!

0x00007f5f2854efb7 libc.so.6!gsignal(+0xc7)
0x00007f5f28550921 libc.so.6!abort(+0x140)
0x00007f5f327a207b libUE4Editor-InputCore.so!abort_message()
0x00007f5f3278826e libUE4Editor-InputCore.so!demangling_terminate_handler()()
0x00007f5f32787fb3 libUE4Editor-InputCore.so!std::__terminate(void (*)())()
0x00007f5f327a3de6 libUE4Editor-InputCore.so!__cxxabiv1::failed_throw(__cxxabiv1::__cxa_exception*)()
0x00007f5f327a3d7f libUE4Editor-InputCore.so!__cxa_throw(+0x6e)
0x00007f5e69231e15 libUE4Editor-AirSim.so!msr::airlib::UpdatableObject::failResetUpdateOrdering(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) [/home/rajat/AirSim/Unreal/Environments/Blocks/Plugins/AirSim/Source/AirLib/include/common/UpdatableObject.hpp:103]
0x00007f5e6922732c libUE4Editor-AirSim.so!msr::airlib::UpdatableObject::reset() [/home/rajat/AirSim/Unreal/Environments/Blocks/Plugins/AirSim/Source/AirLib/include/common/UpdatableObject.hpp:42]
0x00007f5e691f7eee libUE4Editor-AirSim.so!ASimModeBase::createVehicleAtRuntime(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, msr::airlib::VectorMathT<Eigen::Matrix<float, 3, 1, 0, 3, 1>, Eigen::Quaternion<float, 2>, float>::Pose const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) [/home/rajat/AirSim/Unreal/Environments/Blocks/Plugins/AirSim/Source/SimMode/SimModeBase.cpp:662]

This is happening with both CV and Car vehicles, so not something specific to this. I'll work on this separately.

As for the incorrect pose, that's another issue which needs looking into 😞

Octavian-Zhang · 2021-10-24T12:12:13Z

I tried simAddVehicle but am getting a different error than what you mentioned in #4092 (comment), was the project built correctly with this change? And it works correctly for the multi-vehicle settings right?

terminating with uncaught exception of type std::runtime_error: Multiple reset() calls detected without call to update()
Signal 6 caught.
Malloc Size=65538 LargeMemoryPoolOffset=65554 
CommonUnixCrashHandler: Signal=6
Malloc Size=35086 LargeMemoryPoolOffset=100670 
Malloc Size=50986 LargeMemoryPoolOffset=151674 
Malloc Size=35086 LargeMemoryPoolOffset=186782 
Malloc Size=50986 LargeMemoryPoolOffset=237786 
Malloc Size=35086 LargeMemoryPoolOffset=272894 
Malloc Size=51038 LargeMemoryPoolOffset=323950 
[2021.10.24-11.18.06:375][365]LogCore: === Critical error: ===
Unhandled Exception: SIGABRT: abort() called

[2021.10.24-11.18.06:375][365]LogCore: Fatal error!

0x00007f5f2854efb7 libc.so.6!gsignal(+0xc7)
0x00007f5f28550921 libc.so.6!abort(+0x140)
0x00007f5f327a207b libUE4Editor-InputCore.so!abort_message()
0x00007f5f3278826e libUE4Editor-InputCore.so!demangling_terminate_handler()()
0x00007f5f32787fb3 libUE4Editor-InputCore.so!std::__terminate(void (*)())()
0x00007f5f327a3de6 libUE4Editor-InputCore.so!__cxxabiv1::failed_throw(__cxxabiv1::__cxa_exception*)()
0x00007f5f327a3d7f libUE4Editor-InputCore.so!__cxa_throw(+0x6e)
0x00007f5e69231e15 libUE4Editor-AirSim.so!msr::airlib::UpdatableObject::failResetUpdateOrdering(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) [/home/rajat/AirSim/Unreal/Environments/Blocks/Plugins/AirSim/Source/AirLib/include/common/UpdatableObject.hpp:103]
0x00007f5e6922732c libUE4Editor-AirSim.so!msr::airlib::UpdatableObject::reset() [/home/rajat/AirSim/Unreal/Environments/Blocks/Plugins/AirSim/Source/AirLib/include/common/UpdatableObject.hpp:42]
0x00007f5e691f7eee libUE4Editor-AirSim.so!ASimModeBase::createVehicleAtRuntime(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, msr::airlib::VectorMathT<Eigen::Matrix<float, 3, 1, 0, 3, 1>, Eigen::Quaternion<float, 2>, float>::Pose const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) [/home/rajat/AirSim/Unreal/Environments/Blocks/Plugins/AirSim/Source/SimMode/SimModeBase.cpp:662]

This is happening with both CV and Car vehicles, so not something specific to this. I'll work on this separately.

As for the incorrect pose, that's another issue which needs looking into 😞

Hi, @rajat2004, the error message from [#4092 (comment)] is actually from f1ccd2d109073e2b16506a5ed31240ffad92d815, microsoft/AirSim:master, not the one that we are talking about in this pull request. Actually my error message with simAddVehicle() using 2a983990dc1d5587265424e5ef26d2f9682d01da from rajat2004/AirSim:fix-multi-cv-vehicles isn't any simpler:

EpicAccountId:d5db57bff9294ce5ab1dbf0a38dc8144

Unhandled Exception: 0xe06d7363

KERNELBASE
VCRUNTIME140
UE4Editor_AirSim!msr::airlib::UpdatableObject::failResetUpdateOrdering() [C:\Users\Oscar\Documents\Unreal Projects\LandscapeMountains\Plugins\AirSim\Source\AirLib\include\common\UpdatableObject.hpp:103]
UE4Editor_AirSim!msr::airlib::UpdatableObject::reset() [D:\git\uavSwarm\AirSim\AirLib\include\common\UpdatableObject.hpp:42]
UE4Editor_AirSim!ASimModeBase::createVehicleAtRuntime() [C:\Users\Oscar\Documents\Unreal Projects\LandscapeMountains\Plugins\AirSim\Source\SimMode\SimModeBase.cpp:663]
UE4Editor_AirSim!UE4Function_Private::TFunctionRefCaller<<lambda_0371f3cd4faef594610de2b1a7002952>,void __cdecl(void)>::Call() [C:\Program Files\Epic Games\UE_4.27\Engine\Source\Runtime\Core\Public\Templates\Function.h:548]
UE4Editor_AirSim!TGraphTask<TFunctionGraphTaskImpl<void __cdecl(void),0> >::ExecuteTask() [C:\Program Files\Epic Games\UE_4.27\Engine\Source\Runtime\Core\Public\Async\TaskGraphInterfaces.h:886]
UE4Editor_Core
UE4Editor_Core
UE4Editor_Core
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_UnrealEd
UE4Editor_UnrealEd
UE4Editor
UE4Editor
UE4Editor
UE4Editor
UE4Editor
UE4Editor
kernel32
ntdll

Using Visual Studio "DebugGame Editor" mode, I pinpointed the error to be thrown here:

rajat2004 · 2021-10-24T14:10:12Z

This is strange, the vehicle pose is getting updated correctly when using it with simSetVehiclePose(), maybe issue with the initial pose set through the settings?

Settings, script and output

Test with Car, Multirotor and CV mode by modifying settings

{
    "SettingsVersion": 1.2,
    "SimMode": "ComputerVision",
    "Vehicles": {
        "Cam1": {
            "VehicleType": "ComputerVision",
            "X": 0,
            "Y": -5,
            "Z": 0
        },
        "Cam2": {
            "VehicleType": "ComputerVision",
            "X": 0,
            "Y": 5,
            "Z": 0
        }
    }
}

import airsim

if __name__ == "__main__":
    cl = airsim.VehicleClient()
    cl.confirmConnection()

    vehicles = cl.listVehicles()

    for vehicle in vehicles:
        pose = cl.simGetVehiclePose(vehicle)
        print(f"{vehicle}: {pose}")

        pose.position.x_val += 10
        cl.simSetVehiclePose(pose, True, vehicle)

        print(f"New pose of {vehicle}: {cl.simGetVehiclePose(vehicle)}")

Output:

Connected!
Client Ver:1 (Min Req: 1), Server Ver:1 (Min Req: 1)

Cam1: <Pose> {   'orientation': <Quaternionr> {   'w_val': 1.0,
    'x_val': -0.0,
    'y_val': 0.0,
    'z_val': 0.0},
    'position': <Vector3r> {   'x_val': 0.0,
    'y_val': 0.0,
    'z_val': -0.0}}
New pose of Cam1: <Pose> {   'orientation': <Quaternionr> {   'w_val': 1.0,
    'x_val': -0.0,
    'y_val': 0.0,
    'z_val': 0.0},
    'position': <Vector3r> {   'x_val': 10.0,
    'y_val': 0.0,
    'z_val': -0.0}}
Cam2: <Pose> {   'orientation': <Quaternionr> {   'w_val': 1.0,
    'x_val': -0.0,
    'y_val': 0.0,
    'z_val': 0.0},
    'position': <Vector3r> {   'x_val': 0.0,
    'y_val': 0.0,
    'z_val': -0.0}}
New pose of Cam2: <Pose> {   'orientation': <Quaternionr> {   'w_val': 1.0,
    'x_val': -0.0,
    'y_val': 0.0,
    'z_val': 0.0},
    'position': <Vector3r> {   'x_val': 10.0,
    'y_val': 0.0,

rajat2004 · 2021-10-27T17:45:42Z

I still need to figure out the best to fix this, but if you're stuck at this issue and need to use runtime vehicle creation with CV mode, you can comment out this line (Note that this will break the API for multirotors, hence need to think on the best way to fix it)

AirSim/Unreal/Plugins/AirSim/Source/SimMode/SimModeBase.cpp

Line 662 in d7e4e9e

vehicle_sim_api->reset();

Octavian-Zhang · 2021-10-27T23:11:45Z

Thanks 🙏, @rajat2004. 👍 Maybe you can take a moment to peek into #4111, I suspect it's also interelated.

Prefix camera names with vehicle name

zimmy87 · 2021-12-08T07:35:54Z

Tested this locally and it works for me, so I am moving ahead with merging. Thanks for the contribution @rajat2004!

rajat2004 · 2021-12-08T07:48:55Z

Thanks @zimmy87! You're on fire the last few days with all the PRs🔥

rajat2004 · 2021-12-12T06:59:28Z

@Octavian-Zhang would be great if you could try out #4203 and see if it fixes the CV simAddVehicle issue

rajat2004 mentioned this pull request Oct 23, 2021

Spawning multiple "ComputerVision" pawns results in actor name conflict. #4092

Closed

rajat2004 mentioned this pull request Oct 24, 2021

Feasability of removing SimMode #4110

Open

Fix actor name conflict for multiple ComputerVision vehicles

cee7b52

Prefix camera names with vehicle name

rajat2004 force-pushed the fix-multi-cv-vehicles branch from 2a98399 to cee7b52 Compare December 5, 2021 13:54

zimmy87 merged commit 4266b89 into microsoft:master Dec 8, 2021

rajat2004 deleted the fix-multi-cv-vehicles branch December 8, 2021 09:45

rajat2004 mentioned this pull request Dec 12, 2021

Fix simAddVehicle for Car vehicle #4203

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix actor name conflict for multiple ComputerVision vehicles #4109

Fix actor name conflict for multiple ComputerVision vehicles #4109

rajat2004 commented Oct 23, 2021

Octavian-Zhang commented Oct 24, 2021

Octavian-Zhang commented Oct 24, 2021 •

edited

Loading

rajat2004 commented Oct 24, 2021 •

edited

Loading

Octavian-Zhang commented Oct 24, 2021

rajat2004 commented Oct 24, 2021

rajat2004 commented Oct 27, 2021

Octavian-Zhang commented Oct 27, 2021

zimmy87 commented Dec 8, 2021

rajat2004 commented Dec 8, 2021

rajat2004 commented Dec 12, 2021

Fix actor name conflict for multiple ComputerVision vehicles #4109

Fix actor name conflict for multiple ComputerVision vehicles #4109

Conversation

rajat2004 commented Oct 23, 2021

About

How Has This Been Tested?

Screenshots (if appropriate):

Octavian-Zhang commented Oct 24, 2021

Octavian-Zhang commented Oct 24, 2021 • edited Loading

rajat2004 commented Oct 24, 2021 • edited Loading

Octavian-Zhang commented Oct 24, 2021

rajat2004 commented Oct 24, 2021

rajat2004 commented Oct 27, 2021

Octavian-Zhang commented Oct 27, 2021

zimmy87 commented Dec 8, 2021

rajat2004 commented Dec 8, 2021

rajat2004 commented Dec 12, 2021

Octavian-Zhang commented Oct 24, 2021 •

edited

Loading

rajat2004 commented Oct 24, 2021 •

edited

Loading