Digital marketing
A/B testing is a way to find out what actually works rather than what feels right to you. The difference is large — and a big surprise to many advertisers.
In brief
- Test one variable at a time.
- You need statistical significance (usually 100+ conversions per version).
- Let the test run for at least 1–2 weeks.
- 90 % of tests produce no significant difference — that is normal.
What to test in Meta advertising
- Creative: image versus image, video versus still
- Headline: benefit-focused versus problem-focused
- CTA — “Buy now” versus “See more”
- Audience: interests A versus B
What to test on a landing page
- Headline: the main message
- Image — the hero visual
- CTA button: wording, colour, position
- Form length: 3 versus 5 fields
Statistical significance
A test result is significant when the confidence level is above 95 %. In practice that means at least 100 conversions per version.
Under 30 conversions per version means you genuinely do not know what works. Do not draw conclusions.
The most common mistakes
- Testing several variables at once — you cannot tell which one mattered
- Stopping too early — a three-day test means nothing
- Declaring a “winner” when the difference is tiny
- Not documenting the tests, so the same test gets run again a month later
Frequently asked questions
How many tests should run at once?
One significant test at a time per page or campaign. More than that splits the traffic.
How long should a test run?
At least a week, preferably 2–4. It depends not only on the number of conversions but on the variation between weekdays and weekends.
What is a good lift in an A/B test?
A 5 % lift is normal, 20 % is rare. If the difference is 100 % or more, check for errors.
Related reading
External sources:
Want practical support?
Book a free 30-minute assessment. We go through your situation and work out the best next step.
Related articles
Digital marketing
Outsourcing digital marketing: what it costs and when it pays off
Outsourcing digital marketing pays off when there is more to do than there is time or skill for, and when the value of one…
8 min
Digital marketing
Marketing measurement: four numbers that tell the truth
Marketing measurement usually falls down on the same thing: people track numbers that look good in a report but say nothing about the business.…
7 min
Digital marketing
Email marketing: the only channel whose audience you own
Email marketing is the only channel whose audience you own. A social algorithm can change overnight and ad prices can rise, but an email…
8 min
Get in touch
Shall we take the first step together?
Tell us briefly where you stand. We reply the same working day, usually within a couple of hours.
- A free 30-minute call, no strings attached
- You get a concrete view of where you stand
- We say plainly if we are not the right fit
- Phone+358 50 469 0039
- Emailmisa@purodigital.fi