Source
Statistical method & guardrails — found from sitemap — Varify.io
Checked for Varify.io on 1 Oct 2026
- Page
- https://varify.io/en/userdocumentation/significance-calculation/
- Checked
- 1 Oct 2026, 10:53 UTC
- How we may use it
- Public page, crawling permitted
Technical details
- type
- page
- http status
- 200
- content hash
- sha256:9066129efed1fc793eddb86780152cc6514b64ce58727d28ae89a21056df7102
- permission
- robots_ok
- screenshot
- Screenshot on file (internal exhibit, not published)
Cited by
Facts read from this source
-
Default method Report an error
“By default, Varify uses a statistical frequentist method to evaluate test results.”
-
Significance threshold Report an error
“If this is greater than 95 %, the result is displayed as significant in the tool.”
-
One sided tests Report an error
“A one-sided chi-square test is used for binomial targets (e.g. click rate, conversion rate). For sales or value metrics (e.g. average order value, revenue per visitor), a one-sided Student t-test is used.”
-
Method scope Report an error
“The method is set centrally at the account level and then applies to all experiments in that account—it cannot be selected on a per-experiment basis.”
-
Two sided pro plan Report an error
“In Pro Plan, this account setting can alternatively be switched to a two-tailed frequentist method—the change then affects all experiments in the account.”
-
Bayesian method Report an error
“The Bayesian method is also available in the Pro Plan.”
-
Bayesian output Report an error
“it does not calculate p-values, but a probability that a variant is better than the original.”
-
Bayesian runtime forecast Report an error
“With the Bayesian method, Varify displays a runtime forecast that estimates when 95 % significance is likely to be reached.”
-
Aa test recommendations Report an error
“Duration: at least 10 days At least 500 conversions per variant Add a maximum of 3 targets, with a focus on the main KPI”
-
Significance calculator Report an error
“For this purpose Varify.io provides a significance calculator.”
-
Method config location Report an error
“The method can be configured under „Advanced Tracking Setup.“.”
-
False positive risk Report an error
“if a test runs for a very short time or many metrics are evaluated at the same time, the chance of a false positive increases”
Scores citing this record
- The CRO Consultant Statistical method & guardrails
- The Data Protection Officer Statistical method & guardrails
- The E-Commerce Manager Statistical method & guardrails
- The Growth Lead Statistical method & guardrails
- The Product Engineer Statistical method & guardrails
- The Skeptic Statistical method & guardrails