-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Callback for divergences #838
Comments
Is this a feature of PyMC? Can you link the resources (even if they are outdated) EDIT: Is it this CC @AlfredoJF |
Hi @wd60622 This is one of the examples I found in the PyMC docs https://www.pymc.io/projects/examples/en/2022.12.0/howto/sampling_callback.html def my_callback(trace, draw):
if len(trace) >= 100:
raise KeyboardInterrupt()
with model:
trace = pm.sample(tune=0, draws=500, callback=my_callback, chains=1)
print(len(trace)) And also this old response from class DivergentEarlyStopping:
def __init__(self):
self.count = 0
def __call__(self, trace, draw):
if draw.tuning:
return
self.count += int(draw.stats[0]['diverging'])
if self.count > 10:
raise KeyboardInterrupt()
callback = DivergentEarlyStopping()
with pm.Model():
y = pm.Normal('y', mu=0., sigma=3.)
x = pm.Normal('x', mu=0., sigma=pm.math.exp(y/2), shape=9)
trace = pm.sample(1000, callback=callback) I tried both a part of the MMM class and in the fit method as params and **kwargs. If I add sampler_config= {"progressbar": True,
"callback": my_callback
} Error:
|
That seems like a model builder bug then. Good catch! We will likely have to write a customer encoder or have your class work with this. You can currently work around this by using kwargs at fit rather than storing this in sampler_config. i.e. mmm = MMM(
...,
# Only what is json serial
sampler_config={...},
)
mmm.fit(X, y, callback=my_callback) Or overriding the class NewMMM(MMM):
def set_idata_attrs(self, idata):
# Previous code except for handle sampler_config The first solution will likely be easier. Give that one a try and let me know how that goes |
Any thoughts on this @juanitorduz |
@wd60622 Using the first option only works for
|
Can you share the code? If numpyro sampler doesnt support callback function, thatd be pymc functionality |
Hi @wd60622 Here is the code that worked for me using pymc sampler. class DivergentEarlyStopping:
# only for pymc sampler (for now)
def __init__(self, max_divergences):
self.count = 0
self.max_divergences = max_divergences
def __call__(self, trace, draw):
if draw.tuning:
return
self.count += int(draw.stats[0]['diverging'])
if self.count > self.max_divergences:
raise RuntimeError(f"My RuntimeError: Early stopping activated! {self.count} > {self.max_divergences}")
callback = DivergentEarlyStopping(10)
mmm = MMM(
...,
)
mmm.fit(X, y, callback=callback, nuts_sampler="pymc") |
It seems |
Discussed in #837
Originally posted by AlfredoJF July 17, 2024
Hi,
Is it possible to implement callbacks in pymc-marketing? If so, how can I create a callback to monitor the number of divergences?
I'd like to use this as an early stopping when divergences are greater than 10 but the resources for PyMC seemed outdated
Thanks!
The text was updated successfully, but these errors were encountered: