An A to B Guide on Split Testing

Share on facebook
Share on twitter
Share on linkedin
Share on pinterest

I used to hand out flyers to gain more customers. I personalized these to promote myself as the area rep of a company. Most would glance at and pocket my flyer. Others would return it after a look. Some would ask questions and promise to call (Few actually did). I thought my ad copy and headline was compelling enough, but it wasn’t working like I thought.

Over time, I’d make small alterations to see if these would improve conversions. Usually it would be the headline or the color of the font, or the font itself. I made changes based on feedback. I distributed my original version in one location and a modified one at another.

Soon, I discovered which flyer worked better. However, it didn’t end there. After a couple of weeks, I designed another improved version, to see which one of those worked better. I continued doing this through my entire career at the company. 

A/B Testing In A Nutshell

I just described to you what A/B testing is. Basically, it’s a process of comparing two versions of a promotional tool to find out which one is better. Honestly, I didn’t know back then I was conducting an A/B test.

A/B testing, or split testing, is not new. However, it became popular because of the ability of the internet. The principles behind making small adjustments to flyers also apply to online marketing. When you hear about A/B testing now, it’s more about comparing versions of a blog web page, ad, app, e-mail, or landing page.

An A/B test will help you discover if your online marketing assets can perform better. However, what to test and interpreting this data can be complex, which is why we break it down for you here.

The single most important practice in split testing is also the easiest … only test ONE SINGLE ELEMENT at a time.

If you create two completely different versions of a form, and the second one performs better … you won’t know why it performs better … was it your headline? Your image? The font? You just won’t know.

To run a split test, you basically create two versions of your marketing piece, with only one element changed between them. Using software, you show these versions to the same audience … one version will be shown 50% of the time to your audience, and the other version the other 50% of the time.

The first version is called the Control, or Version A. The second version, the one with the element change, is called the Challenger, or Version B.

After a specified time, with a specified amount of traffic, you then analyze and compare which one performed better.

You also define what this “better” is … more clicks per 100 visitors, lower bounce rate, more sales, higher sign-ups, etc … define what is a winner depending on what you are trying to accomplish with your marketing campaign.

Why Split Test?

Why even bother split testing? Think of it this way:

You spend $100 per day in advertising. Each sale nets you $150. With a conversion rate of 1%, you make $50 … that is, you spend $100 and make $150, so you are left over with $50.

What if you increase your conversion rate to 1.1% through split testing? It doesn’t seem like much … you will make $55 per day instead of $50.

However, you then decide to take this slightly better converting ad and do another split test. The winner also converts better, and you make $10 more per day. Keep testing, and you can quickly double your profits on an ad campaign!

Of course, the test may fail, and the original does better. That’s okay as well, because now you know what works and what doesn’t.

With little steps, you optimize your promotion to perform better and better, stretching your advertising budget further and making slightly better margins.

You can have many visitors come to your property but very few conversions. Instead of throwing your hands up and saying you failed, with split testing you can find out if all you need to do is tweak an element.

Likewise, A/B testing lets you know what resonates with your target audience. Sometimes, it’s not what you think. Unless you test, you’ll never find out what it is.  

With my flyers, re-wording my headline did the trick. However, with online options, you have so much more data at your fingertips to fine-tune and optimize anything and everything.

Study Your Analytics Data

Split testing means gathering data to make an informed decision as to which marketing copy and elements within works better. This data can be gathered from a variety of analytics programs, such as Google Analytics, Simple Analytics and Matomo.  

Analysis of your data can reveal a lot about your online visitors. You can learn more about their behavior and where they click (and even where they move their mouse). You’ll also discover their age, gender, location and the device they use while accessing your site.

Your data will also determine which of your online assets needs improvement.

Maybe, it will tell you to tailor-fit your web pages for smartphone users. Or, your content might not be persuasive enough to encourage your visitors to click on your CTA (Call to Action) button as often as you like. Your analytics will tell you all of this.   

When NOT to Split Test?

Before we really dive in, take a moment to check if you really need A/B testing. It might not be useful for you for these reasons:

Your online presence or marketing is complementary.

This means that you still rely on phone calls, referrals or visits to your physical shop for conversions or sales.

An example is a dental clinic. Their website can convince me to have my teeth examined soon. A CTA button such as “Click here for 50% off on extractions” will not help. Unless my teeth hurt, there’s no urgency to make an appointment. But in case they do, I have that dentist’s number, but in this instance it’s hard to test for what the best CTA wording will be.    

You don’t have enough data.

You need a sampling size of conversions in the hundreds, at least, to do a test and make an informed decision as to which copy won. A thousand visitors within a month is preferable. A/B testing isn’t made for smaller numbers.

However, there are other ways you can evaluate your online assets besides split testing. I suggest the following:

Add a survey to your landing or webpage. A pop-up can ask visitors to answer some questions. Some examples:

  • Did our site provide the information you were searching for?
  • Was it easy to navigate around our page?
  • In what ways can we improve your experience in our site?

(Helpful Tip: People prefer “yes” or “no” questions)

Make phone calls. Call your customers and invite them to visit your website if they haven’t done so. Ask them for feedback, and offer a small incentive for doing so, such as a coupon or entry into a contest. Thank them for their continued support.

What Should You Test For?  

The biggest challenge is to decide what you should actually test for. The layout? The image? The message? The color of the button?

Before you even get this far, you need to know who you are delivering your marketing message to. To do this, you need to first do market research to find your “ideal customer” who is most receptive to your message.

Once you have an idea of who your target audience is, you then create your marketing material catered to them. Knowing your market means a more effective marketing message in the first place, as you “know” what your audience desires … or you at least think you do!

Even after all your marketing research, what you think works the best might not actually be what works the best. This is where split testing comes in. Here are the elements you should test:

Your Headline

The first thing visitors read on your web or landing page is your headline. Is it interesting? Does it catch their attention? Or, do your visitors exit immediately after reading your headline?

E-mails have subject lines instead of headlines. Again, does it persuade a recipient to click and continue reading?

When you split test your headlines, only change one word or a phrase … if you change the entire headline and see a positive difference, that’s great! However, you won’t know what part of the headline your audience better related to … the emotional trigger? A certain word? A certain phrasing?

Your Visuals

This can be your corporate colors. I know you have to use your own colors to establish your identity. According to studies, colors evoke different emotions. These can subconsciously influence your readers. Do your colors induce positive or negative feelings? If it’s the latter, limit corporate colors to just your logo.  

In truth, color preference is subjective. Color combinations on your webpage, however, can hurt the eyes. This does not automatically spell doom for your webpage. There are other elements like your content to consider. But, be prudent with your combination choices.

Sometimes a jarring color is attention-grabbing, and is all it takes to increase the effectiveness of your ad. You won’t know this unless you test.    

Photos and Videos

Is your image worth a thousand words? Does it convey your message effectively? Test your online asset by replacing your photo with another. Or, substitute it with a video instead. This might be the better pick for endorsements and testimonials.

If you have a video channel, an infographic might also increase conversions.

(Helpful Tip: Ads or webpages with images of happy faces score better)

Layout and design

If you are testing your landing page, where is your CTA button or link located? Is it prominent on your page? If you can’t place it in the center, top or bottom, the right side is your next best option. Why? Seventy to ninety percent of all people are right-handed. This is backed up with research – right-handed people always favor their right side.

(Helpful Tip: If you have a store, display your slow-selling items on the right.)

Test by moving the other elements of your page around. Don’t change your content. Just re-arrange. Use more circular and oval text boxes. Our brains are more drawn to those.   

Main Content

Just how much of your content should you revise? Your analytics can help you out. It can track how far visitors are scrolling.  Do they read a quarter, half, three-fourths or the entirety of your page? You’ll get actual numbers from your analytics.

Tracking results give you an idea which part of your page needs improvement. I suggest using bullet points if most of your visitors leave before reaching the halfway mark. Stick to the main features if you’re describing a product. If your visitors need to see your complete specs, then list them all. 

Highlight your competitive edge or what makes you different. A competitor can offer the same product or service at the same price.  But, if you can provide an extra benefit like an extended warranty, then this is probably your main selling point. Put it in bold for emphasis.

Again, you have a better understanding of your product and market. Ask yourself, aside from a discount, what can persuade my audience to act now?      

Call To Action Statement

Let’s talk more about your CTA. Is it persuasive enough? Do you replace just one word or everything? A word change may be enough to increase conversions.

By the way, don’t limit yourself to a one or two-worded CTA like “learn more.” It can be a phrase or a short sentence like “Yes, send me more information.”

The internet is full of CTA examples. Take a look and maybe these can trigger a “eureka moment.”   


I will often read user reviews before I purchase or sign-up for anything. Expect your audience to do the same. Save them time by posting a video of testimonials from your current customers. Why a video? It’s more effective than images.

Now if you already have a video, review it. Maybe, your testimonials don’t sound enthusiastic. If this is the case, make a new video or replace with photos instead.

Useful A/B Testing Tools

In order to serve a balanced 50/50 of your two versions to visitors, you’ll need a testing tool. These three listed below are recognized as the top split testing tools out there, though there are many others. (Let me know if you recommend one.)


Crazyegg are well known for their heat maps, where they track a user’s mouse movements to let you know where on your page they mainly look and click. They also have a full-featured A/B split testing tool as well. Prices start at $24/month.


VWO has long been the workhorse of split testing campaigns. They offer standard split testing, multivariate testing (that is, you can test several elements at once, and the software will detect the best element per section in your tests) and URL testing as well (you can split your traffic between different URLs. Prices start at $99/month.

 Google Optimize

This Google split testing tool is built upon Google Analytics (which you must use.) It is easy to get started, free, and includes personalization options, split testing and multivariate testing.

The road to more conversions doesn’t end with one test. Improvement is a continuing process. Don’t stop after getting it right the first time. Keep optimizing and increasing the effectiveness and profit of your marketing!

Sign Up for Great Content

Enter your email address in order to receive more great content from us to help grow your online business!

Like this article? Please share!

Share on facebook
Share on twitter
Share on linkedin
Share on pinterest

Leave a Reply

Your email address will not be published. Required fields are marked *

More Articles to Supercharge Your Digital Success ...