SMC-ABC add distance, refactor and update notebook #3996

aloctavodia · 2020-07-03T22:03:10Z

adds the wasserstein and energy distance functions
Refactors API, the distance, sum_stats and epsilon arguments are now passed pm.Simulator istead of pm.sample_smc, which I think is a more natural place.
Add random method to the pm.Simulator.
Add option to save the simulated data, as data is generated as part of the sampler it makes sense to have the option to save it, instead of computing it later (defaults to False).
Improves LaTeX representation
Finally, this also updates the SMC-ABC notebook and clarifies a few details on how to define and run an ABC model.

review-notebook-app · 2020-07-03T22:03:16Z

Check out this pull request on

Review Jupyter notebook visual diffs & provide feedback on notebooks.

Powered by ReviewNB

junpenglao

LGTM

junpenglao · 2020-07-04T06:56:54Z

pymc3/distributions/simulator.py

+    def gaussian_kernel(self, obs_data, sim_data):
+        return np.sum(-0.5 * ((obs_data - sim_data) / self.epsilon) ** 2)

-        raise NotImplementedError("Not implemented yet")
+    # we are assuming obs_data and sim_data are already sorted!
+    def wasserstein(self, obs_data, sim_data):
+        return np.mean(np.abs((obs_data - sim_data) / self.epsilon))
+
+    # we are assuming obs_data and sim_data are already sorted!
+    def energy(self, obs_data, sim_data):
+        return 1.4142 * np.mean(((obs_data - sim_data) / self.epsilon) ** 2) ** 0.5


recommend isolating it into separate functions instead of methods of a single class

junpenglao · 2020-07-06T14:19:52Z

pymc3/distributions/simulator.py

+            self.distance = gaussian_kernel
+        elif distance == "wasserstein":
+            self.distance = wasserstein
+            sum_stat = "sort"


Can we do a assertion here or a warning that sum_stat need to be "sort" for wasserstein and energy?

junpenglao · 2020-07-06T14:24:02Z

pymc3/distributions/simulator.py

-        raise NotImplementedError("Not implemented yet")
+        params = draw_values([*self.params], point=point, size=size)
+        if size is None:
+            return self.function(*params)


if self.function is a theano function it will return a tensor - maybe we can push the draw_values call to the end and return that?

self.function is intended to be a Python function not a Theano one.

junpenglao · 2020-07-06T14:26:00Z

pymc3/distributions/simulator.py

        """
        This class stores a function defined by the user in python language.

        function: function
            Simulation function defined by the user.
        params: list
            Parameters passed to function.
+        distance: str or callable
+            Distance functions. Available options are "gaussian_kernel" (default), "wasserstein",
+            "energy" or a user defined function


Suggested change

"energy" or a user defined function

"energy" or a user defined function that takes epsilon (a scaler), and (the summary statistics of) observed_data, and simulated_data as input.

…ing wassertein and energy distances

codecov · 2020-07-07T15:25:47Z

Codecov Report

Merging #3996 into master will increase coverage by 0.11%.
The diff coverage is 90.36%.

@@            Coverage Diff             @@
##           master    #3996      +/-   ##
==========================================
+ Coverage   86.65%   86.77%   +0.11%     
==========================================
  Files          88       88              
  Lines       14090    14137      +47     
==========================================
+ Hits        12210    12267      +57     
+ Misses       1880     1870      -10

Impacted Files	Coverage Δ
pymc3/distributions/simulator.py	`86.66% <85.71%> (+28.77%)`	⬆️
pymc3/smc/sample_smc.py	`87.50% <88.88%> (-0.66%)`	⬇️
pymc3/smc/smc.py	`99.48% <100.00%> (+5.73%)`	⬆️

* Update GP NBs to use standard notebook style (pymc-devs#3978) * update gp-latent nb to use arviz * rerun, run black * rerun after fixes from comments * rerun black * rewrite radon notebook using ArviZ and xarray (pymc-devs#3963) * rewrite radon notebook using ArviZ and xarray Roughly half notebook has been updated * add comments on xarray usage * rewrite 2n half of notebook * minor fix * rerun notebook and minor changes * rerun notebook on pymc3.9.2 and ArviZ 0.9.0 * remove unused import * add change to release notes * SMC: refactor, speed-up and run multiple chains in parallel for diagnostics (pymc-devs#3981) * first attempt to vectorize smc kernel * add ess, remove multiprocessing * run multiple chains * remove unused imports * add more info to report * minor fix * test log * fix type_num error * remove unused imports update BF notebook * update notebook with diagnostics * update notebooks * update notebook * update notebook * Honor discard_tuned_samples during KeyboardInterrupt (pymc-devs#3785) * Honor discard_tuned_samples during KeyboardInterrupt * Do not compute convergence checks without samples * Add time values as sampler stats for NUTS (pymc-devs#3986) * Add time values as sampler stats for NUTS * Use float time counters for nuts stats * Add timing sampler stats to release notes * Improve doc of time related sampler stats Co-authored-by: Alexandre ANDORRA <andorra.alexandre@gmail.com> Co-authored-by: Alexandre ANDORRA <andorra.alexandre@gmail.com> * Drop support for py3.6 (pymc-devs#3992) * Drop support for py3.6 * Update RELEASE-NOTES.md Co-authored-by: Colin <ColCarroll@users.noreply.github.com> Co-authored-by: Colin <ColCarroll@users.noreply.github.com> * Fix Mixture distribution mode computation and logp dimensions Closes pymc-devs#3994. * Add more info to divergence warnings (pymc-devs#3990) * Add more info to divergence warnings * Add dataclasses as requirement for py3.6 * Fix tests for extra divergence info * Remove py3.6 requirements * follow-up of py36 drop (pymc-devs#3998) * Revert "Drop support for py3.6 (pymc-devs#3992)" This reverts commit 1bf867e. * Update README.rst * Update setup.py * Update requirements.txt * Update requirements.txt Co-authored-by: Adrian Seyboldt <aseyboldt@users.noreply.github.com> * Show pickling issues in notebook on windows (pymc-devs#3991) * Merge close remote connection * Manually pickle step method in multiprocess sampling * Fix tests for extra divergence info * Add test for remote process crash * Better formatting in test_parallel_sampling Co-authored-by: Junpeng Lao <junpenglao@gmail.com> * Use mp_ctx forkserver on MacOS * Add test for pickle with dill Co-authored-by: Junpeng Lao <junpenglao@gmail.com> * Fix keep_size for arviz structures. (pymc-devs#4006) * Fix posterior pred. sampling keep_size w/ arviz input. Previously posterior predictive sampling functions did not properly handle the `keep_size` keyword argument when getting an xarray Dataset as parameter. Also extended these functions to accept InferenceData object as input. * Reformatting. * Check type errors. Make errors consistent across sample_posterior_predictive and fast_sample_posterior_predictive, and add 2 tests. * Add changelog entry. Co-authored-by: Robert P. Goldman <rpgoldman@sift.net> * SMC-ABC add distance, refactor and update notebook (pymc-devs#3996) * update notebook * move dist functions out of simulator class * fix docstring * add warning and test for automatic selection of sort sum_stat when using wassertein and energy distances * update release notes * fix typo * add sim_data test * update and add tests * update and add tests * add docs for interpretation of length scales in periodic kernel (pymc-devs#3989) * fix the expression of periodic kernel * revert change and add doc * FIXUP: add suggested doc string * FIXUP: revertchanges in .gitignore * Fix Matplotlib type error for tests (pymc-devs#4023) * Fix for issue 4022. Check for support for `warn` argument in `matplotlib.use()` call. Drop it if it causes an error. * Alternative fix. * Switch from pm.DensityDist to pm.Potential to describe the likelihood in MLDA notebooks and script examples. This is done because of the bug described in arviz-devs/arviz#1279. The commit also changes a few parameters in the MLDA .py example to match the ones in the equivalent notebook. * Remove Dirichlet distribution type restrictions (pymc-devs#4000) * Remove Dirichlet distribution type restrictions Closes pymc-devs#3999. * Add missing Dirichlet shape parameters to tests * Remove Dirichlet positive concentration parameter constructor tests This test can't be performed in the constructor if we're allowing Theano-type distribution parameters. * Add a hack to statically infer Dirichlet argument shapes Co-authored-by: Brandon T. Willard <brandonwillard@users.noreply.github.com> Co-authored-by: Bill Engels <w.j.engels@gmail.com> Co-authored-by: Oriol Abril-Pla <oriol.abril.pla@gmail.com> Co-authored-by: Osvaldo Martin <aloctavodia@gmail.com> Co-authored-by: Adrian Seyboldt <aseyboldt@users.noreply.github.com> Co-authored-by: Alexandre ANDORRA <andorra.alexandre@gmail.com> Co-authored-by: Colin <ColCarroll@users.noreply.github.com> Co-authored-by: Brandon T. Willard <brandonwillard@users.noreply.github.com> Co-authored-by: Junpeng Lao <junpenglao@gmail.com> Co-authored-by: rpgoldman <rpgoldman@goldman-tribe.org> Co-authored-by: Robert P. Goldman <rpgoldman@sift.net> Co-authored-by: Tirth Patel <tirthasheshpatel@gmail.com> Co-authored-by: Brandon T. Willard <971601+brandonwillard@users.noreply.github.com>

update notebook

2fb0380

aloctavodia changed the title ~~update notebook~~ SMC-ABC add distance, refactor and update notebook Jul 3, 2020

junpenglao approved these changes Jul 4, 2020

View reviewed changes

aloctavodia added 2 commits July 6, 2020 09:39

move dist functions out of simulator class

fe30260

fix docstring

019b2df

junpenglao reviewed Jul 6, 2020

View reviewed changes

add warning and test for automatic selection of sort sum_stat when us…

bbbbbd0

…ing wassertein and energy distances

aloctavodia added 6 commits July 10, 2020 11:48

update release notes

9ec5cc7

Merge branch 'master' into abc_ref

ca60571

fix typo

88ec8e1

add sim_data test

3e42ca8

update and add tests

1aabed8

update and add tests

f2976cd

aloctavodia merged commit 9e8975f into pymc-devs:master Jul 16, 2020

aloctavodia deleted the abc_ref branch July 16, 2020 13:07

kyleabeauchamp added this to the 3.9.3 milestone Jul 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SMC-ABC add distance, refactor and update notebook #3996

SMC-ABC add distance, refactor and update notebook #3996

aloctavodia commented Jul 3, 2020

review-notebook-app bot commented Jul 3, 2020

junpenglao left a comment

junpenglao Jul 4, 2020

junpenglao Jul 6, 2020

junpenglao Jul 6, 2020

aloctavodia Jul 7, 2020

junpenglao Jul 6, 2020

codecov bot commented Jul 7, 2020 •

edited

Loading

	"energy" or a user defined function
	"energy" or a user defined function that takes epsilon (a scaler), and (the summary statistics of) observed_data, and simulated_data as input.

SMC-ABC add distance, refactor and update notebook #3996

SMC-ABC add distance, refactor and update notebook #3996

Conversation

aloctavodia commented Jul 3, 2020

review-notebook-app bot commented Jul 3, 2020

junpenglao left a comment

Choose a reason for hiding this comment

junpenglao Jul 4, 2020

Choose a reason for hiding this comment

junpenglao Jul 6, 2020

Choose a reason for hiding this comment

junpenglao Jul 6, 2020

Choose a reason for hiding this comment

aloctavodia Jul 7, 2020

Choose a reason for hiding this comment

junpenglao Jul 6, 2020

Choose a reason for hiding this comment

codecov bot commented Jul 7, 2020 • edited Loading

Codecov Report

codecov bot commented Jul 7, 2020 •

edited

Loading