Controlled shutdown of IlmThread pool in all apps in which we use it.

We know that there are some corner cases on Windows when it can be
unsafe to exit because the Ilm thread pool (which is used implicitly
by OpenEXR) hangs. The exact conditions that cause this are not well
understood.

Some time back, in PR 3582, we added an explicit shutdown in oiiotool
right before we exit, to alleviate the problem. But this doesn't help
for other apps like idiff, nor for apps that use libOpenImageIO and
aren't aware of the problem, so wouldn't know to do this.

This patch changes to a different strategy: In set_exr_threads(),
which will always be called any time we use openexr files, use
`std::atexit()` to ensure that when the program terminates -- any
program using OpenImageIO in cases when openexr was used -- we set the
IlmThread global thread pool size to 0 to shut it down cleanly.
Hopefully, that will cover even more cases where this OpenEXR bug
might be tickled.

src/oiiotool/oiiotool.cpp

@@ -19,7 +19,6 @@
 
 #include "oiiotool.h"
 
-#include <OpenEXR/IlmThreadPool.h>
 #include <OpenEXR/ImfTimeCode.h>
 #include <OpenImageIO/Imath.h>
 
@@ -7413,11 +7412,5 @@ main(int argc, char* argv[])
     ot.image_stack.clear();
     ot.image_labels.clear();
 
-    // Force the OpenEXR threadpool to shutdown because their destruction
-    // might cause us to hang on Windows when it tries to communicate with
-    // threads that would have already been terminated without releasing any
-    // held mutexes.
-    IlmThread::ThreadPool::globalThreadPool().setNumThreads(0);
-
     return ot.return_value;
 }

src/openexr.imageio/exroutput.cpp

@@ -14,6 +14,7 @@
 #include <OpenImageIO/Imath.h>
 #include <OpenImageIO/platform.h>
 
+#include <OpenEXR/IlmThreadPool.h>
 #include <OpenEXR/ImfChannelList.h>
 #include <OpenEXR/ImfEnvmap.h>
 #include <OpenEXR/ImfOutputFile.h>
@@ -290,6 +291,20 @@ set_exr_threads()
         exr_threads = oiio_threads;
         Imf::setGlobalThreadCount(exr_threads);
     }
+
+    // If we're ever in this function, which we would be any time we use
+    // openexr threads, also proactively ensure that we exit the application,
+    // we force the OpenEXR threadpool to shut down because their destruction
+    // might cause us to hang on Windows when it tries to communicate with
+    // threads that would have already been terminated without releasing any
+    // held mutexes.
+    static std::once_flag set_atexit_once;
+    std::call_once(set_atexit_once, []() {
+        std::atexit([]() {
+            print("EXITING and setting ilmthreads = 0\n");
+            IlmThread::ThreadPool::globalThreadPool().setNumThreads(0);
+        });
+    });
 }
 
 }  // namespace pvt