Statistical method & guardrails — found from sitemap — Convert Experiences

Cited by

Facts read from this source

• Srm always on Report an error

  “Sample Ratio Mismatch (SRM) Detection ALWAYS ON Runs automatically when visitor counts exist, before any launch decision can be trusted.”

• Modes Report an error

  “Test Planning Use this before launch. Enter your local baseline and business-relevant MDE to calculate visitors per variation and estimated duration. Test Analysis Use this after data arrives. Enter observed results to evaluate the effect, uncertainty, statistical evidence, and always-on SRM check.”

• Metrics Report an error

  “Conversion rate Use for binary outcomes such as signups, purchases, or clicks.”

• Rpv variance upload Report an error

  “Revenue per visitor Use when revenue impact matters more than CVR alone. Upload visitor-level values so the plan reflects observed variance.”

• Methods Report an error

  “Frequentist Commit to the fixed sample before launch and do not stop at the first significant read. Sequential Plan against the same target MDE, then use always-valid evidence when you need to monitor a live conversion, revenue, or products test. Bayesian Use the fixed-horizon sample as a stability reference, then evaluate conversion probability, expected loss, and the credible interval.”

• Frequentist fixed sample Report an error

  “Fixed sample size with classical hypothesis testing. Use this when you can commit before launch and avoid early checks.”

• Sequential peeking Report an error

  “Supports peeking without invalidating the p-value. Use this when you need to monitor a live test.”

• Bayesian outputs Report an error

  “Provides probability of winning and expected loss. Use this when probability language will help stakeholders act.”

• Corrections Report an error

  “Bonferroni and Sidak corrections adjust the threshold when you compare more than one variant.”

• Continuous CSV upload Report an error

  “For revenue and products, upload baseline CSV data so the sample size reflects observed variance instead of a generic assumption.”

• Outputs Report an error

  “Enter your baseline rate, MDE, confidence, and power to get the recommended visitors per variation, test duration, and an MDE curve.”

• Inputs Report an error

  “The calculator sizes your test from baseline, target MDE, confidence, power, tails, variants, and correction method.”

• Duration inputs Report an error

  “Daily Traffic Total visitors per day to the page or flow being tested - not your whole site. Match the time period you used for your baseline rate.”

• Trial Report an error

  “Start 15-day free trial”

• Trial conditions Report an error

  “No credit card · Flicker-free · Carbon-neutral EU servers”

• Privacy controls Report an error

  “GDPR privacy controls”

• Stats claims Report an error

  “40,000+ sites optimized 15 yrs of experimentation”

• Mde guidance Report an error

  “Halving your MDE roughly quadruples the visitors you'll need.”

• Error definition Report an error

  “A Type I error is a false positive and is controlled by the significance threshold. A Type II error is a false negative and is controlled by statistical power at the target MDE.”

2 facts read from this page are not shown because they could not be confirmed on the page as captured.