Skip to content

How Experiments Run

Every Dalton experiment is a straight A/B test. Traffic splits evenly between your original page and each variant, the split never moves, and the result is read at the end of a window fixed before launch.

A running experiment counting down to its conclusion

The planned window

Dalton sets a conclusion date before the test starts, based on the traffic the targeted pages get.

WindowGiven when
2 weeksThe pages can send about 5,000 visitors to each variant in that time
4 weeksThey need longer
6 weeksThey cannot reach it. Also the hard ceiling

Windows are always whole weeks, so a run counts the same number of Mondays as Saturdays. For the first three days the plan shows as two weeks and is provisional, until there is enough traffic to project from.

Days only count while the test is collecting. Pausing does not advance the countdown.

Check the window before you launch

The targeting picker estimates time to results before you commit. Widening the targeting is cheaper than waiting. See Targeting Groups.

Why no leader appears early

While a test runs, Leading variant stays empty even when the numbers look convincing. Lift and chance to beat are shown throughout, but no leader is named until the window completes.

Stop a test the moment it looks good and you will stop it on the days it happened to look good. Two exceptions, both using a calculation that stays valid however often it is checked:

  • A clear loser is flagged for early stopping once every variant has 50 conversions and all are decisively behind.
  • A clear win can be called early, but only after it has held for two days.

What Dalton needs before it reports

Figures show as a dash until every variant, including the original, clears both bars:

  • 350 visitors
  • 50 conversions

Conversions are the binding one. A page converting at 0.5% clears 350 visitors on two orders, and one more order then swings the lift by tens of percent.

Calling the result

Chance to beatVerdict
95%+Winner. Deploy it, or conclude it
80-95%Promising. Worth another week if under the six-week ceiling
5-80%No effect
Under 5%The original won

A test that runs its full window without clearing the data bars is reported as inconclusive. The page is too quiet for that question, and running it longer will not fix it.

While it runs

  • Do not stop a test on a bad day three unless Dalton flagged it.
  • Do not edit the original page mid-test.
  • Pause if something is broken. Changes revert instantly.

Next: Reading Your Results.