Testing.
Most marketers and copywriters know the importance of it.
They pay lip service to it.
But so few do it consistently.
In many ways, testing was a lot easier to understand (and explain) before the advent of the Internet.
Thanks to powerful software and the ability to manipulate countless factors on a web page, the terms multivariate and taguchi became all the rage a few years ago. And lots of people got caught up in the bells and whistles.
These are great but it’s pointless for marketers to dive into them if they haven’t mastered the basic A/B split test.
Here’s what I tell clients who are new to testing.
1) Remember the K.I.S.S. principle and…
2) Test big factors.
So, what are the big factors to test?
There are plenty, but it’s hard to get bigger than the headline.
The legendary adman, John Caples, evangelized about the importance of headlines and influenced the thinking of every major ad writer who’s come after him.
(Later this week, I’ll have a series of space ads he not only helped create but was the front man in.)
The direct mail package above for the red wine extract, resveratrol, has been mailing strong for over five years.
It’s one of the true Energizer Bunnies of direct mail with no signs of quitting.
If you want to really understand the nuances of what makes a winner tick, this is worth checking out.
And only one significant factor has changed in over five years — the headline.
From:
How The French Live 42 Percent Longer Than Everyone Else
To:
How The French Live Longer Than Everyone Else
Why is the second headline the new control?
That’s the kind of question that helps you really understand how to harmonize with your market.
And the fastest way to get there is by testing.
Next post: testing appeals in your ads — something Caples was fanatical about.
Some more wisdom on the importance of headlines, by the late Gary Halbert:
“How important are headlines? I’m glad you asked. Some pundits say the headline accounts for 80% of the success of the ad. I, myself, have re-headlined ads and increased their pull by 475%. I have a client who pays me $195,000 per year to write headlines. Headlines are where I spend more creative effort than any other aspect of my work.”
What about 42 Reasons the French Live Longer than Anyone Else?
Or for that matter use
33 reasons
21 reasons
101 reasons
Yes, I agree with you about testing headlines. Caples was my biggest influence regarding the significance of the headline. 80-90% of your ad relied the power of the headline to get your copy read.
I was disappointed that Gary Halbert was impressed with Michel Fortin because of his ability to test everything online. For example Michel from what I learned tested small things that did not SCREAM!
Come on colors? Frivilous things. Michel has not revealed anything new.
Eugene Schwartz, Caples,& the other masters rule. Gary Benc… is a toned down version of Eugene Schwartz. Gary Halbert had developed a tendency to use alot of exclamation points & hype. Don’t get me wrong I learned alot from Gary but he didn’t always practice what he preached.
Hey Darren,
This magalog would be worthy of an entire lecture/dissection.
The “42% longer” was a very precise way to amplify — and partially prove the claim — that the French do live longer in spite of smoking like chimneys and chomping down brie and baguettes.
The new control may just be easier to wrap your eyes. Design also does a lot in this piece.
Hi Mark,
I’m not opposed to testing little things but I’m a steak eater. The pickles, relish and accoutrements are fine but you don’t need them to survive.
Thanks Jean-Paul,
Testing used to be difficult and take a bit of leg work and statistics skills. Now it’s stupid simple. Google Website Optimizer is a good choice (and free), but the easiest I’ve tried is Visual Website Optimizer. You can do A/B and MVT tests and create alternative versions of pages just as easily as you can create variations on a Word document – if not easier.
@Mark:
I’m sorry you were disappointed.
But just to make sure, I did talk about testing “big” things first. Then to move on to testing smaller things once you’ve decided on your control. My intent was to teach that one should not discount the smaller things, too.
(Offline, that’s a different story, obviously. But online, it’s possible with software.)
For example, a question I’m often asked is what to test first. I always tell people to focus on the AIDA formula.
Test changes in how to grab attention (like the headlines, pictures, layout, lead and deck copy, etc), then interest (the appeal, story, promise, etc), then desire (the benefits, risk reversal, testimonials, etc), and then action (the offer itself, the price, the ordering process, etc).
In fact, the ordering process is often one of the biggest stumbling blocs in online conversions. I recently wrote a blog post describing that you should first test removing bottlenecks before moving on to conversion boosters…
http://www.michelfortin.com/are-bottlenecks-clogging-your-sales/
Nevertheless, my initial intent was to not focus on the little things but to keep them in mind. Sometimes, it’s the little things can offer dramatic bumps in response. Often, it’s the incremental boosts that can become significant.
Sure, some of the bumps are small. But it’s easy now with software like Google Web Optimizer to test multiple “small” things. A small increase in conversion is not significant, but add 10, 20, or 50 small increments in boosts, then you can double or triple your conversion rate — or more.
Take a look at some of these “small” tests from these websites…
WhichTestWon.com
ABTests.com
MarketingExperiments.com
You’ll notice a small 25% increase here, a %12 increase there, etc. But after a while, you add them all up and they can become %100, even %1,000, increases.
A final note.
I’ve discovered that some the best and biggest jumps in conversions are not in testing small things or even in testing big things, for that matter. But in testing things with a higher level of disparity between them.
In other words, it’s not about A vs. B (and whether “A” and “B” or small or significant), but how wildly different “A” and “B” are. The more wildly different A and B are, oftentimes the biggest boosts in response you get.
Of course, take it everything I’ve said with a grain of salt.
Hope this clarifies.
Hi Michel,
Thanks for taking the time to clarify.
“A small increase in conversion is not significant, but add 10, 20, or 50 small increments in boosts, then you can double or triple your conversion rate — or more.”
Indeed, the additive effect of small conversion increases is magical, just like compound interest.
My main point was it’s hard for neophytes to apprehend this if they haven’t felt the power of testing headlines and appeals first.
“My main point was it’s hard for neophytes to apprehend this if they haven’t felt the power of testing headlines and appeals first.”
Agreed 100%.
Mark, I appreciate your balls in coming out the way you have but your comments both right and wrong, depending on the situation, and in absence of context is just useless information that you cannot take action on.
If you have a crappy website making you a measly 10k a year, yes, why in the world would you bother with testing little things? what a waste of your time,… 5% increase = $500 per year… i agree you are much better focusing on big things, like headlines, pscyhological elements, proof elements you name it, … where yo ucan see 100% – 500%+ bumps… or just do something else that pays better!
On the other hand, if I’m making $1,000,000 a year through my site/email e.t.c , and I take 60 minutes out of may day to send an email testing variations of a colour and it gives me a ‘crappy’ 5% return, … i’ll take $50k in pure profits extra for 120 minutes work…. who wouldn’t?
I just don’t get why you wouldn’t bother doing the test, especially online, and especially when you can even segment a small amount of traffic to your new test (keeping most on the control so you don’t lose sales if you theory is a dud).
On top of this, what michel saying is true (i’ve never ready any of his stuff btw so can’t comment.. but hope to one day) … you can also acheive outsized returns by doing lots of little test…
Real life example: A client of mine does 300k per month.. I told him to make a ‘small change’… chane background colour from green to white on a box and change the position of one of his order buttons… an exra 11.8k in orders per month on average (been 4 months so far and so far so good) … and that one was for free… lol
All in context my friend.
Ash
p.s. yes i did get a bonus on top of my fees.. $1,000 and a free lunch 🙁 …. gotta love clients!
p.s. it was meant to read
“i’ll take $50k in pure profits extra for 60 minutes work”
and also forgot to add:
Bencivenag rocks.. he has rocked my world and made me good money.. and will continue to do so…
@Mark: “Gary Bencivenga is a toned down version of Eugene Schwartz??”
Hard to think of two writers with more divergent styles.