How it works

Five ideas explain the whole system.

The config file is the source of truth#

splithub.config.ts declares goals, analytics properties and experiments. It is committed, reviewed and diffed like the rest of the code. The CLI edits it (create, start, pause, finish, ship, analytics create), the SDK reads it, and in cloud mode push uploads a normalized copy so the dashboard and the API can read the same thing.

The dashboard never edits the file. Experiments it creates are "cloud-only" until npx splithub pull writes them into the file; experiments the file owns are read-only in the dashboard except for pausing and finishing. Live overrides (status, traffic, weights, value payloads) sit on top of the config. The browser SDK applies them from a first visit on: it waits briefly for them when nothing is cached and refreshes them in the background on later pages. It never changes a variant a page has shown.

Apps are the exception. A config built into an app binary only changes with a store release, so app experiments start and change in the dashboard, and installed apps pick them up on their next launch. Native iOS and Android apps have no config file at all. See Apps.

Properties and experiments#

An analytics property is a stream of pageviews and events, like a site in a web analytics tool: visitors, visits, pages, sources, events, goal conversions, funnels and retention. It has no variants and no verdicts; its reports are counts. Declare it under properties in the config, or make it in the dashboard; ids look like sh-x1h78w. Events go only to a property named explicitly: property on the provider or client, data-property on sh.js.

An experiment splits visitors between variants and compares one goal. Both share one install, one visitor id and one track() call: with a property on the provider, each track() sends one event row and each page one page row, and the experiments whose goal is that event or page count them from those rows. Nothing extra is sent or billed. See Analytics.

Assignment is a local hash#

A visitor gets a random id in the sh_vid cookie (first party, set by the SDK or the Next.js middleware). Assignment uses MurmurHash3 with the visitor id, experiment key and seed, then selects from the cumulative variant weights. A separate hash controls entry into the traffic share; a holdout and layers, when you use them, are checked before that. Given the same config and targeting context, browser, edge and server agree. Bucketing is local; fetching cloud overrides or hosted script config may require a request. Crawlers see the control and are never counted.

Consequences: changing seed reshuffles every visitor (never do it on a running experiment); changing traffic does not; a visitor keeps the same variant across pages and sessions in the same browser, while the id and experiment settings remain unchanged.

Exposures, events and goals#

An exposure is "this visitor saw this experiment in this variant". The SDK records it when the variant is actually rendered (<Experiment> mounts, useValue is read, the redirect fires).

An event is anything your app sends with track(name, props?). Any name is recorded; nothing to declare first. A goal is an event marked as one: in goals in the config, with the goal switch on an event in the dashboard, or simply by naming the event as an experiment's goal (Video Played becomes the goal video-played). A page visit (pageview with a path) or a click on a selector can be a goal too.

In SplitHub Cloud attribution is decided at ingest:

• a visitor counts once per experiment, in the variant of their first exposure; a returning visitor is not stored or billed again;
• a goal converts the experiments that measure it when it arrives at or after that exposure; earlier goals count for nothing;
• units are unique visitors, not hits; at most 50 of one goal per visitor per day are kept;
• goals from visitors who never saw an experiment do not reach any experiment (an analytics property still records the event).

Self-hosted stores apply the same rule when they read. In the cloud, exposures of running experiments, pageviews and custom events are billed; goal rows never are. Exposures for paused, finished or draft experiments are dropped at the edge and not stored.

Self-hosted or cloud#

	Self-hosted	SplitHub Cloud
Where events go	your database through a Store (SQLite, D1, Postgres, MySQL, custom)	https://go.splithub.io
Config line	storage: { mode: 'self', endpoint: '/api/splithub' }	storage: { mode: 'cloud', key: 'pk_…' }
Experiment results	raw numbers per variant: visitors, conversions, rate, revenue	verdicts, p-values, stars, guardrails, factor effects, SRM, breakdowns
Analytics	raw counts: pageviews, visitors, visits, events by name	reports: pages, sources, channels, funnels, paths, retention, realtime
Where you read it	CLI and MCP against your app's endpoint	CLI, MCP, dashboard, email
Live overrides	no (edit the file)	yes
Price	free, unlimited	50,000 visitors (exposures, pageviews and events) per project per month free, then packs or Pro

The SDK, the CLI and the MCP server are the same in both modes. The analysis (verdicts, stars, guardrails, factor effects, breakdown comparisons, AI reviews) runs only in SplitHub Cloud; a self-hosted endpoint answers with analysis: { available: false } next to its numbers. To switch, update storage and configure the destination: a handler and database for self-hosting, or a project, keys and config push for cloud.

Vocabulary#

• property: an analytics stream of pageviews and events, read as reports.
• experiment: one question with 2+ variants; kind is value, component, redirect or mvt.
• control: the first declared variant, what exists today.
• weight: relative share of traffic per variant (50/50, 1/1, 90/10).
• traffic: share of visitors who enter the experiment at all (0.5 = half; the rest see the control and record no exposure).
• conversion: a visitor who reached the goal, counted once.
• verdict: the one-line conclusion of the statistics, with its status code.
• SRM: sample ratio mismatch, the split you got differs from the one you configured; the setup is broken.
• guardrail: a goal that must not get worse; a variant that hurts one is never the winner.