Experiment types & delivery
How this is scored
What can be tested and where: client-side changes through an editor, server-side and feature experiments through SDKs, multivariate and multi-page tests, and personalisation — judged on what the documentation shows rather than on the feature grid.
0 — Not an experimentation tool; the product cannot split traffic or serve variants (analytics, heatmaps or session recording only).
3 — Client-side A/B tests built in a visual editor, with a traffic split and targeting by URL or device.
5 — A/B, split-URL and multivariate tests through a visual or code editor, audience targeting on behaviour and attributes, and either rule-based personalisation or server-side experiments through a documented SDK.
8 — Client-side and server-side experiments through documented SDKs for common languages, multi-page tests, feature-flag experiments sharing one audience and metric model with web tests, mutually exclusive experiment groups, holdouts, and personalisation that can itself be tested against a control.
10 — One experimentation programme across every surface: web, app, server and edge from the same platform, experiment interactions managed, a documented experiment lifecycle from hypothesis to archived result, and a library of past results a team can search.
The CRO Consultant
For clients past editor-only testing, the platform documents A/B, ABn, split and full and fractional multivariate tests plus multi-armed bandits, with both a Visual Editor and a code-level Advanced Editor, and server-side experimentation is worked through end to end for Shopify feature flags. It stops short of a full programme because the server-side story rests on that one integration plus homepage mentions of REST/JSON APIs and mobile apps; we found no public information on SDKs for common languages, holdouts or mutually exclusive experiment groups. 1 5 6 9