Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid XML input file test fixes #810

Merged
merged 3 commits into from
Feb 24, 2022
Merged

Invalid XML input file test fixes #810

merged 3 commits into from
Feb 24, 2022

Conversation

ptheywood
Copy link
Member

Closes #808

  • Uses a EXPECT_EXIT in the google test, this emits a warning.
  • SKips the python test, as cannot trap the C/C++ call to exit.

Also renames the test suite this test belongs to as reccommended by google test

This does generate a warning during google test execution due to the use of threading in CUDA applications.
Can't find a way to handle this via pytest, if the exit were a pytthon sys.exit() it would be testable
@ptheywood
Copy link
Member Author

Linux release, seatbelts=OFF tests:

[==========] Running 871 tests from 77 test suites.
[----------] Global test environment set-up.
[----------] 1 test from TestSimulationDeathTest
[ RUN      ] TestSimulationDeathTest.ArgParse_inputfile_short

[WARNING] /home/ptheywood/code/flamegpu/FLAMEGPU2/build/_deps/googletest-src/googletest/src/gtest-death-test.cc:1121:: Death tests use fork(), which is unsafe particularly in a threaded context. For this test, Google Test detected 4 threads. See https://github.com/google/googletest/blob/master/docs/advanced.md#death-tests-and-threads for more explanation and suggested solutions, especially if this is the last message you see before your test times out.
[       OK ] TestSimulationDeathTest.ArgParse_inputfile_short (8 ms)
[----------] 1 test from TestSimulationDeathTest (8 ms total)

...

[==========] 871 tests from 77 test suites ran. (339195 ms total)
[  PASSED  ] 871 tests.

  YOU HAVE 145 DISABLED TESTS

Pytest showing the skip

python3 -m pytest ../tests/swig/python/gpu/test_cuda_simulation.py
========================================================== test session starts ===========================================================
platform linux -- Python 3.9.5, pytest-7.0.1, pluggy-1.0.0
rootdir: /home/ptheywood/code/flamegpu/FLAMEGPU2
collected 19 items                                                                                                                       

../tests/swig/python/gpu/test_cuda_simulation.py .....s.............                                                               [100%]

===================================================== 18 passed, 1 skipped in 6.14s ======================================================

@ptheywood ptheywood requested review from mondus and Robadob and removed request for mondus February 23, 2022 17:58
@ptheywood ptheywood added this to the v2.0.0-alpha.3 milestone Feb 23, 2022
@Robadob
Copy link
Member

Robadob commented Feb 23, 2022

Will take a look tomorrow if I get time to build tests

@Robadob
Copy link
Member

Robadob commented Feb 24, 2022

Python tests ran to completion (but had some failures, despite USE_GLM off):

=============================================== short test summary info ===============================================
FAILED gpu/test_cuda_ensemble.py::TestCUDAEnsemble::test_setStepLog - AttributeError: 'SwigPyObject' object has no at...
FAILED io/test_logging.py::LoggingTest::test_CUDAEnsembleSimulate - AttributeError: 'SwigPyObject' object has no attr...
FAILED io/test_logging.py::LoggingTest::test_CUDASimulationSimulate - AttributeError: 'SwigPyObject' object has no at...
FAILED io/test_logging.py::LoggingTest::test_CUDASimulationStep - AttributeError: 'SwigPyObject' object has no attrib...
FAILED io/test_logging_exceptions.py::LoggingExceptionTest::test_AgentLogFrameExceptions - AttributeError: 'SwigPyObj...
FAILED io/test_logging_exceptions.py::LoggingExceptionTest::test_LogFrameExceptions - AttributeError: 'SwigPyObject' ...
FAILED io/test_logging_exceptions.py::LoggingExceptionTest::test_LoggerSupportedFileType - TypeError: exportLog() mis...
FAILED runtime/test_host_environment.py::HostEnvironmentTest::test_exception_property_read_only - pyflamegpu.pyflameg...
FAILED runtime/messaging/test_brute_force.py::TestMessage_BruteForce::test_Optional2 - AssertionError: assert -192 ==...
================================ 9 failed, 544 passed, 5 skipped in 1582.41s (0:26:22) ================================

C tests had 1 failure, which I believe is the test you modified. Will investigate:

[----------] Global test environment tear-down
[==========] 991 tests from 79 test suites ran. (597022 ms total)
[  PASSED  ] 990 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] TestSimulationDeathTest.ArgParse_inputfile_short

 1 FAILED TEST
  YOU HAVE 25 DISABLED TESTS

@Robadob
Copy link
Member

Robadob commented Feb 24, 2022

This is the actual failure details.

[ RUN      ] TestSimulationDeathTest.ArgParse_inputfile_short
C:\Users\Robadob\fgpu2\build\_deps\googletest-src\googletest\src/gtest-port.cc(865): error: Failed
Syntax error at index 0 in simple regular expression "[.]*": '[' is unsupported.
C:\Users\Robadob\fgpu2\build\_deps\googletest-src\googletest\src/gtest-port.cc(865): error: Failed
Syntax error at index 2 in simple regular expression "[.]*": ']' is unsupported.
Running main() from c:\users\robadob\fgpu2\tests\helpers\main.cu
c:\users\robadob\fgpu2\tests\test_cases\gpu\test_cuda_simulation.cu(107): error: Death test: c.initialise(sizeof(argv) / sizeof(char*), argv)
    Result: died but not with expected error.
  Expected: contains regular expression "[.]*"
Actual msg:
[  DEATH   ] Loading input file 'I_DO_NOT_EXIST.xml' failed!
[  DEATH   ] Detail: C:\Users\Robadob\fgpu2\src\flamegpu\io\XMLStateReader.cpp(90): TinyXML error: File could not be opened.
[  DEATH   ]  Error code: 3
[  FAILED  ] TestSimulationDeathTest.ArgParse_inputfile_short (2988 ms)

Copy link
Member

@Robadob Robadob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current C test doesn't work on Windows.

tests/test_cases/gpu/test_cuda_simulation.cu Outdated Show resolved Hide resolved
@ptheywood ptheywood merged commit 178c932 into master Feb 24, 2022
@ptheywood ptheywood deleted the death-tests branch February 24, 2022 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

exit() called within the test suite
2 participants