Sequential test monitoring

순차 실험 모니터링

Show why an early lead in daily experiment results can mislead.

···
html
<div class="seq"><div class="days">DAY <b id="day">1</b> / 7 <span id="decision">WAIT</span></div><div class="baseline"></div><div class="line a"></div><div class="line b" id="bline"></div><div class="labels">A · CONTROL <span>B · VARIANT</span></div><div class="stop">PLANNED DECISION: DAY 7</div></div>
css
.seq{position:relative;width:min(85%,510px);height:clamp(120px,28vmin,220px);font:800 clamp(10px,max(1.7vmin,1vw),14px)/1 sans-serif}.days{display:flex;gap:5px}.days span{margin-left:auto;color:var(--accent)}.baseline{position:absolute;top:52%;width:100%;height:1px;background:var(--line)}.line{position:absolute;left:0;top:47%;width:100%;height:3px;background:var(--fg)}.line.b{background:var(--accent);transform-origin:left;transition:transform .4s,top .4s}.labels{position:absolute;bottom:24px;left:0;right:0;display:flex;justify-content:space-between}.labels span{color:var(--accent)}.stop{position:absolute;bottom:0;color:var(--muted)}
js
let day=1;const d=document.querySelector('#day'),b=document.querySelector('#bline'),s=document.querySelector('#decision');const levels=[30,34,42,45,46,47,44];function draw(){d.textContent=String(day);b.style.transform='scaleX('+day/7+')';b.style.top=levels[day-1]+'%';s.textContent=day===7?'REVIEW':'WAIT';day=day===7?1:day+1}draw();setInterval(draw,650);

Checking experiment results every day can make a random early gap look like a win. Sequential monitoring requires a prespecified rule for reading interim evidence and stopping.

In the demo, B's early lead disappears on later days, and the decision opens only at the planned endpoint. A fixed-horizon test should not be stopped opportunistically.

A planned sequential method is different from casual peeking. Record the statistical procedure, minimum useful effect, and guardrails before launch.

When to use

Use it when stakeholders want interim decisions; set the stopping rule first.

Open as page ↗