pymc-testing

Name: pymc-testing
Author: pymc-labs

117installs10forks

How to install pymc-testing?

pymc-labs/python-analytics-skills/pymc-testing

$npx -y skills add pymc-labs/python-analytics-skills --skill pymc-testing

Installs into the current project.

Summary

Load when writing or modifying pytest tests that touch pymc.Model, pm.sample, or any PyMC model code. Covers pymc.testing.mock_sample, pytest fixtures for Bayesian models, and the distinction between fast structure-only tests (mocking) and slow posterior inference tests. Triggers

Files · 1

View on GitHub

SKILL.md

1	# PyMC Testing
2
3	PyMC provides testing utilities to speed up test suites by mocking MCMC sampling with prior predictive sampling. This is useful for checking model structure without running expensive inference.
4
5	## Mock Sampling vs Real Sampling
6
7	\| Aspect \| Mock Sampling \| Real Sampling \|
8	\|--------\|---------------\|---------------\|
9	\| Speed \| Fast (seconds) \| Slow (minutes) \|
10	\| Use case \| Model structure, downstream code \| Posterior values, convergence \|
11	\| Output \| `prior`, `prior_predictive` \| Full `posterior`, `sample_stats`, warmup groups \|
12	\| Divergences \| Mocked (configurable) \| Real diagnostics \|
13
14	Use mocking when: Testing model specification, CI/CD pipelines, plotting code, API integration, serialization.
15
16	Use real sampling when: Checking posterior values, ESS/r_hat diagnostics, LOO-CV, model comparison. See [pymc-modeling skill](../pymc-modeling/SKILL.md) for real inference.
17
18	## PyMC Testing Utilities
19
20	See: https://www.pymc.io/projects/docs/en/latest/api/testing.html
21
22	### mock_sample
23
24	Replaces `pm.sample()` with prior predictive sampling:
25
26	```python
27	from functools import partial
28	import numpy as np
29	import pymc as pm
30	from pymc.testing import mock_sample
31
32	# Basic usage - replaces pm.sample
33	pm.sample = mock_sample
34
35	with pm.Model() as model:
36	pm.Normal("x", 0, 1)
37	idata = pm.sample() # Uses prior predictive, not MCMC
38	```
39
40	### mock_sample_setup_and_teardown
41
42	Pytest fixture helper for setup/tear-down:
43
44	```python
45	# conftest.py
46	import pytest
47	from pymc.testing import mock_sample_setup_and_teardown
48
49	mock_pymc_sample = pytest.fixture(scope="function")(mock_sample_setup_and_teardown)
50
51	# test_model.py
52	def test_model_runs(mock_pymc_sample):
53	with pm.Model() as model:
54	pm.Normal("x", 0, 1)
55	idata = pm.sample()
56	assert "x" in idata["posterior"]
57	```
58
59	A production-ready example from pymc-marketing:
60
61	- conftest.py: https://github.com/pymc-labs/pymc-marketing/blob/main/tests/conftest.py
62	- Also configures pytest markers for slow tests with `--run-slow` / `--only-slow` CLI options
63
64	### Distribution testing helpers
65
66	`pymc.testing` also exposes helpers for validating custom distributions against their analytic log-CDFs. In PyMC 6, both forms are available:
67
68	```python
69	from pymc.testing import check_logcdf, check_logccdf
70
71	check_logcdf(MyDist, domain, params) # log CDF
72	check_logccdf(MyDist, domain, params) # log complementary CDF (log survival)
73	```
74
75	## Mocking Sample Stats
76
77	By default, no sample_stats are created. Pass a dictionary to mock specific stats:
78
79	```python
80	from functools import partial
81	import numpy as np
82	import pymc as pm
83	from pymc.testing import mock_sample
84
85	def mock_diverging(size):
86	return np.zeros(size, dtype=int)
87
88	def mock_tree_depth(size):
89	return np.random.choice(range(2, 10), size=size)
90
91	mock_sample_with_stats = partial(
92	mock_sample,
93	sample_stats={
94	"diverging": mock_diverging,
95	"tree_depth": mock_tree_depth,
96	},
97	)
98
99	pm.sample = mock_sample_with_stats
100	```
101
102	Example from [pymc-marketing](https://github.com/pymc-labs/pymc-marketing/blob/main/scripts/run_notebooks/injected.py):
103
104	```python
105	from functools import partial
106	import numpy as np
107	import pymc as pm
108	import pymc.testing
109
110	def mock_diverging(size):
111	return np.zeros(size, dtype=int)
112
113	pm.sample = partial(
114	pymc.testing.mock_sample,
115	sample_stats={"diverging": mock_diverging},
116	)
117	pm.HalfFlat = pm.HalfNormal
118	pm.Flat = pm.Normal
119	```
120
121	## What Gets Mocked
122
123	The fixture automatically replaces:
124	- `pm.Flat` → `pm.Normal`
125	- `pm.HalfFlat` → `pm.HalfNormal`
126
127	This ensures prior predictive sampling works without invalid starting values.
128
129	## DataTree Structure Comparison
130
131	Mock sampling output (from `mock_sample`):
132	- `posterior` (derived from prior predictive)
133	- `observed_data`
134
135	Note: `mock_sample` uses prior predictive internally but returns it as `posterior` to mimic the `pm.sample()` API. By default there is no `prior`, `prior_predictive`, `posterior_predictive`, or `sample_stats` group. However, you can pass a `sample_stats` dictionary to mock specific stats (see Mocking Sample Stats section).
136
137	Real sampling output (from `pm.sample`):
138	- `posterior`
139	- `sample_stats`
140	- `observed_data`
141
142	Note: `posterior_predictive` is NOT included by default - you must call `pm.sample_posterior_predictive(idata, model=model)` separately. Warmup groups are sampler-dependent (nutpie includes them, default NUTS does not).
143
144	Gotcha: Code that expects `posterior_predictive`, warmup groups, or sample_stats will fail with mock sampling. Different samplers p

People who liked this also viewed

6 picks

Type

pymc-testing

117installs10forks

View on GitHub

How to install pymc-testing?

pymc-labs/python-analytics-skills/pymc-testing

$npx -y skills add pymc-labs/python-analytics-skills --skill pymc-testing

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# PyMC Testing
2
3	PyMC provides testing utilities to speed up test suites by mocking MCMC sampling with prior predictive sampling. This is useful for checking model structure without running expensive inference.
4
5	## Mock Sampling vs Real Sampling
6
7	\| Aspect \| Mock Sampling \| Real Sampling \|
8	\|--------\|---------------\|---------------\|
9	\| Speed \| Fast (seconds) \| Slow (minutes) \|
10	\| Use case \| Model structure, downstream code \| Posterior values, convergence \|
11	\| Output \| `prior`, `prior_predictive` \| Full `posterior`, `sample_stats`, warmup groups \|
12	\| Divergences \| Mocked (configurable) \| Real diagnostics \|
13
14	Use mocking when: Testing model specification, CI/CD pipelines, plotting code, API integration, serialization.
15
16	Use real sampling when: Checking posterior values, ESS/r_hat diagnostics, LOO-CV, model comparison. See [pymc-modeling skill](../pymc-modeling/SKILL.md) for real inference.
17
18	## PyMC Testing Utilities
19
20	See: https://www.pymc.io/projects/docs/en/latest/api/testing.html
21
22	### mock_sample
23
24	Replaces `pm.sample()` with prior predictive sampling:
25
26	```python
27	from functools import partial
28	import numpy as np
29	import pymc as pm
30	from pymc.testing import mock_sample
31
32	# Basic usage - replaces pm.sample
33	pm.sample = mock_sample
34
35	with pm.Model() as model:
36	pm.Normal("x", 0, 1)
37	idata = pm.sample() # Uses prior predictive, not MCMC
38	```
39
40	### mock_sample_setup_and_teardown
41
42	Pytest fixture helper for setup/tear-down:
43
44	```python
45	# conftest.py
46	import pytest
47	from pymc.testing import mock_sample_setup_and_teardown
48
49	mock_pymc_sample = pytest.fixture(scope="function")(mock_sample_setup_and_teardown)
50
51	# test_model.py
52	def test_model_runs(mock_pymc_sample):
53	with pm.Model() as model:
54	pm.Normal("x", 0, 1)
55	idata = pm.sample()
56	assert "x" in idata["posterior"]
57	```
58
59	A production-ready example from pymc-marketing:
60
61	- conftest.py: https://github.com/pymc-labs/pymc-marketing/blob/main/tests/conftest.py
62	- Also configures pytest markers for slow tests with `--run-slow` / `--only-slow` CLI options
63
64	### Distribution testing helpers
65
66	`pymc.testing` also exposes helpers for validating custom distributions against their analytic log-CDFs. In PyMC 6, both forms are available:
67
68	```python
69	from pymc.testing import check_logcdf, check_logccdf
70
71	check_logcdf(MyDist, domain, params) # log CDF
72	check_logccdf(MyDist, domain, params) # log complementary CDF (log survival)
73	```
74
75	## Mocking Sample Stats
76
77	By default, no sample_stats are created. Pass a dictionary to mock specific stats:
78
79	```python
80	from functools import partial
81	import numpy as np
82	import pymc as pm
83	from pymc.testing import mock_sample
84
85	def mock_diverging(size):
86	return np.zeros(size, dtype=int)
87
88	def mock_tree_depth(size):
89	return np.random.choice(range(2, 10), size=size)
90
91	mock_sample_with_stats = partial(
92	mock_sample,
93	sample_stats={
94	"diverging": mock_diverging,
95	"tree_depth": mock_tree_depth,
96	},
97	)
98
99	pm.sample = mock_sample_with_stats
100	```
101
102	Example from [pymc-marketing](https://github.com/pymc-labs/pymc-marketing/blob/main/scripts/run_notebooks/injected.py):
103
104	```python
105	from functools import partial
106	import numpy as np
107	import pymc as pm
108	import pymc.testing
109
110	def mock_diverging(size):
111	return np.zeros(size, dtype=int)
112
113	pm.sample = partial(
114	pymc.testing.mock_sample,
115	sample_stats={"diverging": mock_diverging},
116	)
117	pm.HalfFlat = pm.HalfNormal
118	pm.Flat = pm.Normal
119	```
120
121	## What Gets Mocked
122
123	The fixture automatically replaces:
124	- `pm.Flat` → `pm.Normal`
125	- `pm.HalfFlat` → `pm.HalfNormal`
126
127	This ensures prior predictive sampling works without invalid starting values.
128
129	## DataTree Structure Comparison
130
131	Mock sampling output (from `mock_sample`):
132	- `posterior` (derived from prior predictive)
133	- `observed_data`
134
135	Note: `mock_sample` uses prior predictive internally but returns it as `posterior` to mimic the `pm.sample()` API. By default there is no `prior`, `prior_predictive`, `posterior_predictive`, or `sample_stats` group. However, you can pass a `sample_stats` dictionary to mock specific stats (see Mocking Sample Stats section).
136
137	Real sampling output (from `pm.sample`):
138	- `posterior`
139	- `sample_stats`
140	- `observed_data`
141
142	Note: `posterior_predictive` is NOT included by default - you must call `pm.sample_posterior_predictive(idata, model=model)` separately. Warmup groups are sampler-dependent (nutpie includes them, default NUTS does not).
143
144	Gotcha: Code that expects `posterior_predictive`, warmup groups, or sample_stats will fail with mock sampling. Different samplers p

People who liked this also viewed

6 picks

Type