A/B testing

A/B 테스트

Randomly assign two versions to compare their effect on conversion.

···
html
<div class="experiment"><div class="sample">SAMPLE <b id="sample">200</b> / GROUP</div><div class="plot"><div class="variant"><span>A · CONTROL</span><div class="bar"><i class="a"></i><em class="aerr"></em></div><b>8.0%</b></div><div class="variant"><span>B · NEW</span><div class="bar"><i class="b"></i><em class="berr"></em></div><b>9.2%</b></div></div><div class="foot">UNCERTAINTY NARROWS AS SAMPLE GROWS</div></div>
css
.experiment{width:min(88%,580px);font:800 clamp(10px,max(1.7vmin,1vw),14px)/1.2 sans-serif}.sample{text-align:right;color:var(--muted);letter-spacing:.08em}.sample b{color:var(--fg)}.plot{display:flex;align-items:end;justify-content:center;gap:clamp(20px,8vw,100px);height:clamp(95px,28vmin,230px);border-bottom:2px solid var(--fg);margin-top:8px}.variant{text-align:center;width:clamp(68px,15vw,130px)}.variant span{display:block;white-space:nowrap;color:var(--muted);margin-bottom:6px}.variant b{display:block;margin-top:6px;font-size:clamp(13px,3vmin,25px)}.bar{position:relative;height:clamp(65px,20vmin,180px);display:flex;align-items:end}.bar i{display:block;width:100%;height:80%;background:var(--fg)}.bar .b{height:92%;background:var(--accent)}.bar em{position:absolute;left:50%;top:10%;width:2px;height:40%;background:var(--accent-3);animation:uncertainty 4s ease-in-out infinite alternate}.bar em::before,.bar em::after{content:'';position:absolute;width:15px;height:2px;background:var(--accent-3);left:-7px}.bar em::before{top:0}.bar em::after{bottom:0}.bar .berr{top:2%}.foot{text-align:center;color:var(--accent);letter-spacing:.08em;margin-top:clamp(8px,2vmin,18px)}@keyframes uncertainty{from{height:48%}to{height:12%}}
js
const sample=document.querySelector('#sample');let n=200;setInterval(()=>{n=n>=3200?200:n*2;sample.textContent=String(n)},1100);

An A/B test randomly assigns visitors to two groups and changes one element. Sample size and uncertainty matter before attributing any observed difference to that change.

The demo narrows the estimated ranges around A and B conversion rates as the sample grows. Define the success metric, smallest useful effect, and test duration before choosing a winner.

A taller B bar can still be due to chance. Avoid stopping early because an interim result looks promising.

When to use

Use it to test conversion effects of a copy or layout change. Keep returning visitors in the same group.

Open as page ↗