Paired Model — Bayesian Bootstrap¶
Nonparametric paired A/B test using Rubin's Bayesian bootstrap: a
Dirichlet(1, …, 1) "prior" over the empirical distribution of paired
differences yields a full posterior on Δ = p_A − p_B without any
parametric likelihood or prior elicitation. Decision making is routed
through the ROPE / posterior-mass framework only — Savage–Dickey BFs do
not apply because there is no parametric prior on Δ to evaluate at
the null.
bayes_paired_bootstrap
¶
Bayesian-bootstrap paired-proportions test.
Nonparametric paired A/B test using Rubin's (1981) Bayesian bootstrap.
For paired binary observations (y_A_i, y_B_i) we form the per-pair
differences D_i = y_A_i - y_B_i ∈ {-1, 0, +1} and place a flat
Dirichlet(1, …, 1) "prior" over the simplex of weights on the empirical
distribution. Each posterior draw of the average treatment effect is
with α = 1 the standard noninformative choice.
The procedure produces a full posterior on Δ = p_A − p_B without
any parametric likelihood and without any latent δ_A on the logit
scale. Decisions are driven by the three quantities that are
well-defined directly under the BB posterior:
- Posterior of null —
P(Δ ∈ ROPE | data), exposed via :meth:PairedBayesPropTestBB.rope_testasROPEResult.pct_in_rope. - Posterior of superiority —
P(p_A > p_B | data), exposed viamodel.summary.p_A_greater_B. - ROPE decision — composite ROPE call returning the full
:class:
ROPEResult.
Things deliberately not exposed by this class:
savage_dickey_test— the BB has no parametric prior onΔto evaluate at the null. Use one of the parametric paired classes (Laplace or Pólya–Gamma) if you need a point-null BF.posterior_probability_H0— for the parametric classes this is a Bayes-factor-style conversion fromBF_01toP(H_0 | data); under the BB the same quantity is justROPEResult.pct_in_roperead off the posterior directly. Adding a thin wrapper would force the user to commit to a prior onH_0that has no role in the BB posterior itself, and any default flat-prior choice would be reparametrisation-non-invariant (Lindley–Jeffreys).
Use cases where this is the right tool:
- Sample size is large enough that a nonparametric posterior is trustworthy (≳ 100 paired observations).
- The user wants to sidestep prior elicitation entirely.
- The user wants robustness against model misspecification of the underlying paired logistic likelihood.
PairedBayesPropTestBB(seed=0, n_samples=20000, rope_epsilon=0.02, dirichlet_alpha=1.0, threshold=0.5, verbose=False)
¶
Bases: BaseBayesPropTest
Bayesian-bootstrap paired A/B test for binary outcomes.
Nonparametric counterpart of
:class:bayesprop.resources.bayes_paired_laplace.PairedBayesPropTest
and
:class:bayesprop.resources.bayes_paired_pg.PairedBayesPropTestPG.
Generative model (Rubin, 1981)::
D_i = y_A_i - y_B_i ∈ {-1, 0, +1}
w ~ Dirichlet(α · 1_n) (α = 1 = standard BB prior)
Δ = Σ_i w_i · D_i ∈ [-1, +1]
The class deliberately omits savage_dickey_test because the
Bayesian bootstrap has no parametric prior on Δ to evaluate at
the null. All decisions are routed through ROPE / posterior mass.
Attributes:
| Name | Type | Description |
|---|---|---|
n_samples |
int
|
Number of Bayesian-bootstrap posterior draws. |
seed |
int
|
Random seed for reproducibility. |
rope_epsilon |
float
|
Half-width of the default ROPE (default 0.02 = 2 pp). |
dirichlet_alpha |
float
|
Concentration of the Dirichlet weights (default 1.0 = standard noninformative BB). |
y_A_obs |
ndarray | None
|
Observed binary outcomes for arm A (set by :meth: |
y_B_obs |
ndarray | None
|
Observed binary outcomes for arm B (set by :meth: |
delta_samples |
ndarray | None
|
Posterior draws of |
summary |
PairedSummary | None
|
:class: |
trace_summary |
DataFrame | None
|
|
Initialise configuration.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
seed
|
int
|
Random seed. |
0
|
n_samples
|
int
|
Number of Bayesian-bootstrap posterior draws. |
20000
|
rope_epsilon
|
float
|
Half-width of the default ROPE on |
0.02
|
dirichlet_alpha
|
float
|
Concentration of the Dirichlet weights.
The standard Bayesian bootstrap uses |
1.0
|
threshold
|
float
|
Cutoff used to binarise continuous inputs in
|
0.5
|
verbose
|
bool
|
If |
False
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in bayesprop/resources/bayes_paired_bootstrap.py
__repr__()
¶
Return an informative string representation.
Source code in bayesprop/resources/bayes_paired_bootstrap.py
fit(y_A_obs, y_B_obs)
¶
Draw the Bayesian-bootstrap posterior on Δ = p_A − p_B.
Vectorised: a single rng.dirichlet call produces all weight
vectors at once, then W @ D gives every posterior draw in one
matmul. For large n_samples × n the Dirichlet draws are
chunked to keep peak memory bounded (~400 MB).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
y_A_obs
|
ArrayLike
|
Observed scores for arm A — either binary |
required |
y_B_obs
|
ArrayLike
|
Observed scores for arm B — same conventions.
Length |
required |
Returns:
| Type | Description |
|---|---|
PairedBayesPropTestBB
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes mismatch or values are outside |
Source code in bayesprop/resources/bayes_paired_bootstrap.py
175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 | |
rope_test(rope=None, ci_mass=0.95)
¶
ROPE analysis on the Bayesian-bootstrap posterior of Δ.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rope
|
tuple[float, float] | None
|
|
None
|
ci_mass
|
float
|
Credible-interval mass (default 95%). |
0.95
|
Returns:
| Name | Type | Description |
|---|---|---|
Populated |
ROPEResult
|
class: |
Source code in bayesprop/resources/bayes_paired_bootstrap.py
decide()
¶
Run the ROPE-based composite decision.
The Bayesian-bootstrap class deliberately ships only one decision sub-result — the ROPE analysis — because the three quantities of interest are already directly available from the BB posterior:
- Posterior of null,
P(Δ ∈ ROPE | data)⇒rope_test().pct_in_rope(also reachable on the returned :class:HypothesisDecisionviadecide().rope.pct_in_rope). - Posterior of superiority,
P(p_A > p_B | data)⇒model.summary.p_A_greater_B. - ROPE decision (reject / accept / undecided) ⇒
rope_test().decision.
The bayes_factor and posterior_null sub-fields of the
returned :class:HypothesisDecision are always None for
this class — the BB has no parametric prior on Δ, so a
Savage–Dickey BF and a prior-weighted P(H_0 | data) are
both undefined. Adding a thin Bayes-factor-style wrapper on
top of the ROPE mass would require committing to a prior on
H_0 that has no role in the BB posterior itself.
Returns:
| Type | Description |
|---|---|
HypothesisDecision
|
class: |
HypothesisDecision
|
populated; |
Source code in bayesprop/resources/bayes_paired_bootstrap.py
plot_posterior(rope=None, bins=80, figsize=(9, 5), ax=None)
¶
Histogram of the posterior on Δ with the 95 % CI and ROPE.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rope
|
tuple[float, float] | None
|
|
None
|
bins
|
int
|
Histogram bin count. |
80
|
figsize
|
tuple[float, float]
|
Figure size if a new figure is created. |
(9, 5)
|
ax
|
Any
|
Existing axes to draw on (creates a new figure if None). |
None
|
Returns:
| Type | Description |
|---|---|
Any
|
The matplotlib axes used for plotting. |
Source code in bayesprop/resources/bayes_paired_bootstrap.py
plot_posteriors(**kwargs)
¶
Overlaid KDE posteriors of θ_A and θ_B (probability scale).
Each posterior draw uses the Bayesian-bootstrap Dirichlet weights applied to the per-arm binary outcomes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
**kwargs
|
Any
|
Accepts |
{}
|
Source code in bayesprop/resources/bayes_paired_bootstrap.py
plot_posterior_delta(color='#9C27B0', **kwargs)
¶
KDE posterior density of Δ = θ_A − θ_B (probability scale) with 95% CI.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
color
|
str
|
Colour for the density curve and fill. |
'#9C27B0'
|
**kwargs
|
Any
|
Accepts |
{}
|
Source code in bayesprop/resources/bayes_paired_bootstrap.py
print_summary()
¶
Print a human-readable summary of the fitted model.