Find out everything you need to know about A/B testing your copy. The what, why, how and where.
I hate it when people disagree with me.
Especially when:
- It’s about something I’ve written.
- It’s my boss.
- Said boss doesn’t have a clue.
OK, you got me. I’ve got someone specific in mind. We’ll call her Sara.
She would look at my work and give me a little sympathetic shake of her head and whip out a red pen (oh yeah, she liked to work on paper).
Have you ever gone head-to-head with your boss or client over something small like a CTA?
It’s fist-clenchingly frustrating.
No amount of Body Combat classes are going to release the pressure built up from when you know YOU.ARE.RIGHT.
The best way to prove your point get results for your business/client?
A/B test your copy.
It really is the only true measure of success.
What is A/B testing?
Excellent question.
In simple terms, A/B testing is where two variants go head-to-head in an experiment to see which one is more successful.
Of course, nothing in this life is simple.
To get a valid result, you can’t pluck two ideas from the air. You need a strategy.
The ingredients for a successful A/B test:
- A big enough group of people to be statistically relevant.
- A measurable metric. A conversion is usually a sale, but sophisticated tests might also be looking at repeat visitors or a specific action like signing up to a newsletter or opening an email.
- A rationale behind the choices you make. E.g. for subject lines the first experiment I usually like to do is try the same lines but personalise one with the user’s first name. My rationale is that people instantly recognise their name and are more likely to notice a subject line addressed to them individually.
This last point is less obvious but critical to successful A/B testing.
You’re looking to build your knowledge of what works and what doesn’t in both design and copy.
A/B testing should be a validation (or not) of thoughts and theories you already have about consumer behaviour.
- Test something similar, and you’re unlikely to see a significant difference.
- Test without a rationale, and you won’t be able to scale the change in other areas of your product.
Where can you A/B test copy?
Again it comes down to finding a big enough significance to measure.
A place where absolutely everyone should be A/B testing is in the subject lines of your emails (as long as you’re sending them to enough people).
Most email platforms support such tests, and the metric you’re testing is clear: open rates.
If we go back to these two criteria again: volume and measurable metric, other options arise:
- Call to actions (CTAs – the copy on buttons)
- Supporting copy around CTAs
- Headings
- Elements of social proof and/or authority on landing pages
- Forms
- Removing text (this one’s my favourite)
Why are people resistant to A/B testing?
If A/B testing were easy, far more businesses would be doing it and doing it well.
The biggest block I have seen is lack of tools and expertise in-house.
The big guys, like Booking.com, are able to run 1000+ experiments a day. Yet, it’s been slow to be picked up by most companies.
A few excuses I’ve heard for this:
- Expense: It costs a lot to build your own A/B testing capabilities.
- Experience: Many businesses don’t understand how to A/B test effectively and why it’s so important.
- Power: The people making the decisions are reluctant to let go of what’s a rather fun thing to do, boss people around. (OK, I haven’t heard this one, so much as witnessed it to be true.)
The counter argument for A/B testing:
- Expense: A/B testing done right shows you where the money is. When you can do more of the stuff that works, your business grows.
- Experience: The benefit of our gig economy, means you can hire expert consultants who’ve worked across numerous projects for different companies and have a deep experience. We know what works.
- Power: Good bosses know that when their employees are empowered, they do their best work. Enjoy the power that comes from running a thriving business where people work hard because they believe in what they do.
Creating perfect work in an imperfect world
If your business/client doesn’t have A/B testing capabilities yet, you’re limited to what you can do. But that doesn’t mean you can’t test.
Here are other, more subjective ways of testing your words.
- Poll other writers in your social media groups on your top choices.
- Reach out to your connections or ideal customers and ask what they think.
- Be consistent with what you already have in your product.
- Launch first, ask for feedback and be open for feedback.
- Look at what the big guys are doing. (So this one is a little tricky because you don’t know if you’re in the winning or losing variant. But if you use a successful product repeatedly and see the same copy over time chances are it’s been proven to work.)
As a UX writer you’re not in control of the experimentation culture at your company/client.
However, understanding A/B testing and advocating for it holds you in good stead for when:
- That dream job comes along.
- The rest of the world catches on.
It won’t be long.
Are you new to this UX writing thing? it can be overwhelming. That’s why I think you’ll love my free UX Writer’s Guide. It’s 10 things I wish I’d known at the start of my career. Check it out.
