macOS: Multiprocessor sampling stops unexpectedly on macOS Catalina if seaborn was imported

[ivlis]

Sampling stops with the following error in the console:

libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Couldn't close file

Seems to be related with this issue:
matplotlib/matplotlib#15410

I understand that this is an external package issue, but it makes PyMC3 practically unusable.

Please provide any additional information below.

Versions and main components

• PyMC3 Version: 3.8
• Theano Version: 1.0.4
• Python Version: 3.7
• Operating system: osX Catalina (only Catalina is affected, Mojave is OK)
• How did you install PyMC3: conda

[michaelosthege]

Can you provide a minimal example to reproduce the problem?

And how did you reach the conclusion that it could be related to matplotlib? The error message is rather generic. Do you have a trace?

[rosgori]

@ivlis Look at this https://discourse.mc-stan.org/t/issues-with-pystan-multiprocessing-and-subsequently-the-pystan3-pre-release-on-mac-os-catalina/12798

[ivlis]

@rosgori I tried to manually edit the file in my matplotlib installation, it seems to have no effect.

@michaelosthege

Here is a minimal example which demonstrates the problem:

import seaborn as sns
import pymc3 as pm
import theano.tensor as tt
import numpy as np
import theano
theano.config.gcc.cxxflags = "-Wno-c++11-narrowing"


features = np.random.rand(4000, 10, 5)
obs_n = [10]*4000
obs_k = [1]*4000

with pm.Model() as model:
    θ = pm.Normal('θ_0', 0, 1, shape=(10, 5))
    a = pm.Deterministic('a', t t.tensordot(θ, features,  axes=[[0,1], [1,2]]))

    y_hat = pm.Binomial('y_hat', p=pm.math.invlogit(a), n=obs_n, observed=obs_k)

with model:
    trace = pm.sample(target_accept=0.95, tune=1000, draws=200, init='adapt_diag')

Sampling does not start

$ python example.py
Only 200 samples in chain.
Auto-assigning NUTS sampler...
Initializing NUTS using adapt_diag...
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [θ_0]
Sampling 4 chains, 0 divergences:   0%|                                                                                                            | 0/4800 [00:00<?, ?draws/s]

Removing import seaborn solves the problem.

[homofortis]

I had the same problem on Ubuntu. Followed @ivlis advise and removed seaborn.

[AlexAndorra]

We seem to have find a fix to this weird, vexing issue by setting the multiprocessing start method to "forkserver" on MacOS 🎉 This will be shipped with PyMC 3.9
Closed by #3919