Source

Statistical method & guardrails — found from sitemap — Monetate

Checked for Monetate on 1 Oct 2026

Technical details

type: page
http status: 200
content hash: sha256:2e302731e9ccb12b5ec47ef99311d6001f867cc07e0514f1e940ff5f7b2fcd2e
permission: robots_ok
screenshot: Screenshot on file (internal exhibit, not published)

Cited by

Confidence levels Report an error

“Monetate reports confidence at three confidence levels: 90%, 95%, and 99%.”
Confidence definition Report an error

“Confidence is the measurement of the likelihood that the observed change in a KPI did not occur by random chance.”
Confidence factors Report an error

“Confidence is influenced by a number of factors, including the difference in lift between experiment and control groups and the sample size.”
Confidence declaration scope Report an error

“For all metrics and KPIs, Monetate declares confidence at 90%, 95%, and 99%.”
Confidence novelty effect Report an error

“However, the novelty of what you test may diminish over time. When this occurs, the behavior of the experiment begins to trend towards that of the control group and statistical confidence drops.”
Confidence time to significance Report an error

“The time to reach statistical confidence, or the time needed to determine if the current performance difference is statistically significant at the desired confidence level, is based on a number of variables, including sample size and the magnitude of difference between the performance of the experiment and control groups.”