# Alerts, digest, AI reviews

Three ways to be told instead of checking, for experiments and analytics properties alike. All three live on the Alerts tab and the AI tab of a cloud project and need Pro.

## Alert rules

A rule is a scope, a condition with thresholds, and actions. Experiment rules watch one experiment or every running one; analytics rules watch one property. Rules are evaluated every hour and fire at most once per experiment per cooldown window.

### On experiments

| Condition | Fires when |
|---|---|
| `srm` | the split differs from the configured weights (p < 0.001) |
| `winner` | the verdict is `winner_found` (optionally only once every variant has N visitors) |
| `loser` | a variant is at least X% worse than the control with enough sample |
| `exposures` | unique visitors on the experiment reach N |
| `conversions` | conversions reach N |
| `duration` | the experiment has run N days |
| `no_events` | no exposures for N hours while running |

### On properties

| Condition | Fires when |
|---|---|
| `traffic_drop` | visitors in the last 24 hours fell by X% or more against the same 24 hours a week before (default 50%; needs 20+ visitors a week before) |
| `traffic_spike` | visitors in the last 24 hours rose by X% or more against the same 24 hours a week before (default 50%; needs 20+ visitors now) |
| `goal_silent` | a goal of the property had no conversions in the last N hours (default 24) after converting the week before |
| `event_count` | an event, or any event of the property, reaches N, counted from when the rule was made; with repeat, at every multiple of N |

### On the account

| Condition | Fires when |
|---|---|
| `quota` | the project used X% of its monthly allowance |
| `usage` | this month's billed events cross another multiple of N (every 1,000, say) |

Actions: send to channels, and for experiment rules also pause the experiment, finish it, or drop the losing variant (set its weight to 0). A rule can combine them: "SRM → pause and email", "winner → finish and email", "loser after 10k visitors → drop the loser". Analytics and account rules only notify. `event_count` and `usage` are milestones: each one fires once, whatever the cooldown.

Every project starts with sensible experiment rules created **disabled**; enabling one needs Pro. **Test** runs the rule as a dry run and lists what it would hit right now.

From the CLI:

```bash
npx splithub alerts                                   # rules, progress, recent alerts
npx splithub alerts add srm --pause
npx splithub alerts add loser --lift 10 --experiment checkout-steps --drop-loser
npx splithub alerts add traffic_drop --property sh-x1h78w --percent 50
npx splithub alerts add goal_silent --property sh-x1h78w --goal signup --hours 24
npx splithub alerts add event_count --property sh-x1h78w --event Signup --count 1000 --repeat
npx splithub alerts test <id>                         # what would fire now, nothing sent
```

Other flags: `exposures --visitors N`, `conversions --conversions N`, `duration --days N`, `no_events --hours 24`, `quota --percent 80`, `usage --every 1000`, `winner|loser --min-visitors N`, `--experiment <key>` for one experiment, and on any rule `--name`, `--cooldown <hours>`, `--disabled`, `--finish`. `alerts enable|disable|delete <id>` manage a rule. The MCP tool is `splithub_alerts`.

## Channels

Saved once, reused by rules and the digest: an email address, a Slack or Discord webhook, or a webhook URL of your own (JSON POST with the rule, the experiment or property, the message and the actions taken). Add them on the Alerts tab; pick them in a rule's "deliver to" multiselect. **Test** on a channel sends a test message now.

## The digest

A scheduled summary, separate from the rules: on chosen weekdays, weekly or monthly, at a chosen hour in the project's time zone. It has two parts:

- **Experiments**: running experiments with their verdicts, finished ones since the last digest, quota usage, and what to decide next.
- **Analytics**: per property, visitors, pageviews, top sources and pages, goal conversions and the notable events of the period, each against the period before (up to 10 properties; pick which ones or cover them all).

The team's notes of the period close it. Sent to the chosen channels; each member can also switch it off in their profile.

## AI reviews

An analyst's second opinion written by a language model from the experiment setup, aggregate results and available variant screenshots. Raw event rows and visitor ids are not supplied as statistical input:

- **On start**: a setup review, including available variant screenshots.
- **On finish**: when an experiment finishes (dashboard, CLI, API, cap, rule), a review is written and attached to the finish email.
- **Progress checks**: the daily job checks running experiments with at least 100 visitors. It writes a review when the verdict changes from the previous snapshot or a week has passed since the last review, subject to the automatic review cooldown.
- **On demand**: the AI tab's "Review now".

Reviews are stored with the experiment and readable from the dashboard, `npx splithub review [<id>]`, the MCP tool `splithub_review`, and `GET /v1/reviews[/<id>]` with an API key. They are advice; the verdict from the statistics stays the source of truth, and the skills tell agents exactly that.

## Webhook payload

```json
{
  "kind": "traffic_drop",
  "project": "project-id",
  "subject": "[SplitHub] My app: Traffic drop",
  "text": "Alert summary and dashboard link",
  "rule": "Traffic drop",
  "experiment": null,
  "property": "sh-x1h78w",
  "message": "Visitors to \"sh-x1h78w\" fell 62% in the last 24 hours: 410, against 1,080 the same day last week.",
  "actions": [],
  "link": "https://app.splithub.io/…"
}
```

Experiment alerts carry the experiment key in `experiment` and no `property`; verdict alerts add a `voice` line. Requests carry `X-SplitHub-Event` and `X-SplitHub-Delivery` headers, and `X-SplitHub-Signature: t=<unix seconds>,v1=<hex HMAC-SHA256 of "<t>.<body>">` when the channel has a signing secret.