The OpenLift model
OpenLift estimates the outcome in a treated geography as a function of matched control-market outcomes and calendar effects. The model is trained only on the pre-treatment period, when neither the test nor controls should contain the intervention.
μt = α + Xtβ + dow[t]
Here, yt is the observed outcome in the test market, Xt contains control-market outcomes, α is an intercept, β contains control coefficients, and dow[t] represents day-of-week effects. Optional holiday and weather covariates can be included where supported.
OpenLift uses Student-t observation noise for robustness. Compared with a normal likelihood, the heavier tails reduce the influence of occasional large residuals without pretending outliers do not exist. Shrinkage priors on control coefficients regularise the model when candidate controls overlap in the signal they explain.
Learning the untreated relationship
The pre-period teaches the model how the test market typically moves with its controls. This is why market selection and a sufficiently representative history matter. If the pre-period relationship is unstable, a sophisticated posterior will still describe the wrong process.
- Build candidate controls. Use similarity metrics and business exclusions to identify plausible untreated markets.
- Fit on pre-treatment observations. Estimate the posterior for the intercept, control weights, seasonal effects, and noise.
- Run diagnostics. Check sampling quality, posterior predictive fit, residual structure, and sensitivity to market choice.
- Freeze the model. The treated outcome must not teach the counterfactual how to explain away the intervention.
The post-period test outcome is used to calculate lift, not to refit the untreated relationship.
Projecting the counterfactual
After pre-period fitting, posterior samples of the model parameters are combined with observed post-period control outcomes. Each sample produces a plausible untreated path for the test market. Together those paths form the posterior predictive distribution of the counterfactual.
This is more informative than a point forecast. A narrow posterior implies the controls and noise model constrain the untreated path strongly; a wide posterior indicates that several counterfactual outcomes remain plausible. The uncertainty should travel into the lift estimate and the decision.
Calculating posterior lift
For every posterior sample and post-period timestamp, OpenLift subtracts the predicted counterfactual from the observed test outcome. It then aggregates the sample paths to report absolute lift, percentage lift, probability of positive lift, and a highest-density interval.
Percentage lifts = Σ Liftt,s / Σ ŷt,s
The reported lift is a posterior summary, not a platform attribution number. A posterior mean of +10% does not mean the effect is known to be exactly 10%. The interval and probability describe what the observed data and assumptions support.
Turning a distribution into evidence
OpenLift combines posterior probability, interval width, optional match quality, and data-quality warnings into evidence categories. These categories are decision aids rather than guarantees.
| SIGNAL | INTERPRETATION | DECISION USE |
|---|---|---|
| High P(lift > 0), narrow interval | Effect is consistently positive across posterior samples | Supports scale if economics also work |
| Positive mean, interval crosses zero | Direction may be positive but evidence is incomplete | Hold or design a stronger retest |
| Near-zero mean, narrow interval | Large positive effects are unlikely under the model | Cut or reallocate, subject to decision threshold |
| Wide interval | Test or counterfactual is insufficiently precise | Improve design before making a strong claim |
When spend is available, posterior incremental outcomes can also produce distributions for incremental CAC, ROAS, and profit. Margin and lifetime-value inputs are assumptions and should be varied in sensitivity analysis.
Diagnostics and honest limitations
Bayesian inference exposes parameter and predictive uncertainty, but it does not verify the causal assumptions by itself. Review posterior predictive fit, pre-period residuals, sampling diagnostics, coefficient stability, and sensitivity to control-market inclusion.
The model can fail when control markets are contaminated, the relationship changes after treatment for unrelated reasons, outcome measurement breaks, or overlapping interventions affect the test geography. Linear control relationships and the current seasonal structure may not capture every data-generating process.
A responsible report therefore includes the estimated effect, interval, probability, model diagnostics, design limitations, and the business action those signals justify. Transparency is part of the measurement—not an appendix.