Experiment types & delivery
How this is scored
What can be tested and where: client-side changes through an editor, server-side and feature experiments through SDKs, multivariate and multi-page tests, and personalisation — judged on what the documentation shows rather than on the feature grid.
0 — Simple A/B split of one page element through a visual editor; no server-side option, no targeting beyond URL.
3 — Client-side A/B and split-URL tests with basic audience targeting, and no SDK or server-side delivery.
5 — Client-side and server-side experiments through documented SDKs for common languages, multivariate and multi-page tests, audience targeting on behaviour and attributes, and rule-based personalisation.
8 — Feature-flag-based experiments sharing one audience and metric model with web tests, mutually exclusive experiment groups, holdouts, edge or CDN delivery, and personalisation that can itself be tested against a control.
10 — One experimentation programme across every surface: web, app, server and edge from the same platform, experiment interactions managed, a documented experiment lifecycle from hypothesis to archived result, and a library of past results a team can search.
The Data Protection Officer
Documentation covers the visual and advanced editors, multivariate and split tests, multi-armed bandits and targets, plus a documented server-side route — but the server-side evidence on the documentation pages is a Shopify-specific integration with feature flags, and we found no public information on SDKs for common languages, mutually exclusive experiment groups, holdouts or edge delivery. The claim of experimentation everywhere with REST and JSON APIs appears on the homepage rather than in documentation. 1 5 6