Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Various fixes to make more Windows CI tests pass #2821

Merged
merged 1 commit into from
Jan 12, 2021

Conversation

lgritz
Copy link
Collaborator

@lgritz lgritz commented Jan 8, 2021

  • Try to address the OpenEXR thread pool hanging when we call exit().

    • Strive to remove all exit() calls after any file reads happen,
      in oiiotool, iconvert, and idiff.

    • This does not stop all hanging.

    • Temp workaround: for the Windows CI tests, use ctest commands to
      allow each test to timeout 4 times before giving up. To be clear,
      this does not solve the problem, it only masks it. But it helps
      us "pass" those tests, exposing any other problems occurring with
      them, and we will return later to find the source of any residual
      timeouts.

  • Update ref output for several tests where the differ slightly under Windows.

  • Make runtest.py convert text output line endings cr-lf -> lf before
    comparing to reference output. (This eliminates unnecessary diffs.)

  • Update many tests' run.py file to use double quotes instead of single
    quotes for arguments containing some characters -- looks like the Windows
    shell invoked by Python system() does not handle the single quotes quite
    the same way as Unix shells.

  • Helper: ArgParse: add abort() method, lets you stop command line parsing.

  • Helper: Filesystem::generic_filepath. Similar to C++17
    std::filesystem::path::generic_string(), but for a filepath sent as
    string_view.

    While I was there, I noticed that a few Filesystem functions that I
    declared as noexcept actually call things that can throw, so I put a
    few try/catch in stratgic spots. Nobody had reported a problem with
    this, and maybe it would be fine, but it's the right thing to do.

  • oiiotool: when printing the command line after errors, genericize
    identifiable filenames so they render the same on Windows and POSIX,
    and strip directory name from the oiiotool filepath. This makes
    error output comparison more equal between Windows and POSIX, making
    several more tests pass.

In total, this seems to get us down to failing only 7 tests (not counting
spurious timeouts that cause failures).

* Try to address the OpenEXR thread pool hanging when we call exit().

    * Strive to remove all exit() calls after any file reads happen,
      in oiiotool, iconvert, and idiff.

    * This does not stop all hanging.

    * Temp workaround: for the Windows CI tests, use ctest commands to
      allow each test to timeout 3 times before giving up. To be clear,
      this does not solve the problem, it only masks it. But it helps
      us "pass" those tests, exposing any other problems occurring with
      them, and we will return later to find the source of any residual
      timeouts.

* Update ref output for several tests where the differ slightly under Windows.

* Make runtest.py convert text output line endings cr-lf -> lf before
  comparing to reference output. (This eliminates unnecessary diffs.)

* Update many tests' run.py file to use double quotes instead of single
  quotes for arguments containing some characters -- looks like the Windows
  shell invoked by Python system() does not handle the single quotes quite
  the same way as Unix shells.

* oiiotool: when printing the command line after errors, genericize
  identifiable filenames so they render the same on Windows and POSIX,
  and strip directory name from the oiiotool filepath. This makes
  error output comparison more equal between Windows and POSIX, making
  several more tests pass.

In total, this seems to get us down to failing only 7 tests (not counting
spurious timeouts that cause failures).
@lgritz lgritz merged commit a5dd972 into AcademySoftwareFoundation:master Jan 12, 2021
@lgritz lgritz deleted the lg-noexit branch January 12, 2021 18:29
lgritz added a commit to lgritz/OpenImageIO that referenced this pull request Jan 13, 2021
…dation#2821)

* Try to address the OpenEXR thread pool hanging when we call exit().

    * Strive to remove all exit() calls after any file reads happen,
      in oiiotool, iconvert, and idiff.

    * This DOES NOT stop all hanging. It's still a bit of a mystery whether
      there is anything we can do to fully prevent it without a fix on the
      OpenEXR side.

    * Temp workaround: for the Windows CI tests, use ctest commands to
      allow each test to timeout 3 times before giving up. To be clear,
      this does not solve the problem, it only masks it. But it helps
      us "pass" those tests, exposing any other problems occurring with
      them, and we will return later to find the source of any residual
      timeouts.

* Update ref output for several tests where the differ slightly under Windows.

* Make runtest.py convert text output line endings cr-lf -> lf before
  comparing to reference output. (This eliminates unnecessary diffs.)

* Update many tests' run.py file to use double quotes instead of single
  quotes for arguments containing some characters -- looks like the Windows
  shell invoked by Python system() does not handle the single quotes quite
  the same way as Unix shells.

* oiiotool: when printing the command line after errors, genericize
  identifiable filenames so they render the same on Windows and POSIX,
  and strip directory name from the oiiotool filepath. This makes
  error output comparison more equal between Windows and POSIX, making
  several more tests pass.

In total, this seems to get us down to failing only 7 tests (not counting
spurious timeouts that cause failures).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant