The best marketing team are well known for fine-tuning their emails a lot. Their emails looks perfect when you receive one of them. But what you might not know, itâs that they literally have tested every single word in the email.
A/B testing is the process of trying different combinations of an email to determine and pick the one that performs the best - for examples the one that gives the best metrics (opens, clicks, replies, etc.)
Unfortunately, Gmail doesnât provide a tool out of the box to send an A/B tests with their platform. But thanks to Google Sheets and a Gmail mail merge solution, you can easily start an A/B tests using your Gmail account. Letâs see how to do it.
Letâs prepare your AB test
To send your first A/B tests from a Gmail or Google Workspace (formerly G Suite), you need to start from⊠a Google Sheets. As I said in the introduction, Gmail doesnât provide any marketing tools. But a myriad of third-party mail merge tools, such as Mailmeteor, have filled the gap to power up your Gmail account.
Here are the steps to follow:
- Open a new Google Sheets or copy our demo spreadsheet
-
Add your mailing list in the column A. You can copy/paste (or import) your entire mailing list or just a subset of it. The A/B test will run on all the recipients that youâve put into the spreadsheet
- Create a new column entitled âGroup (A/B)â
-
Now split your recipients list in two groups. One half will be part of the group A and the other part will be the group B. To do so, just add A or B in the relevant cells.
Your spreadsheet is almost ready! That was quick ;)
What do you want to A/B test?
Now, it totally depends on the A/B test you want to perform. Sometimes you might want to try out a different subject, sometimes tweak a sentence, sometimes a whole paragraph or the entire email.
To have a great A/B test, I recommend you to restrict your test to the minimum. A/B tests are more effective if you change a (very) small part of your email. For example the subject or a sentence. If you A/B test more than that, it generally means that your test is too large and you wonât learn much from it.
For this tutorial, letâs see how to run an A/B test on the subject line. Then we will see how to run an A/B test on a sentence.
How to A/B test the emailâs subject?
Testing the emailâs subject is probably the most common A/B tests in email marketing. The emailâs subject is the first part of your email that recipients will interact with. And you get only one chance to make a first good impression.
To fine tune the subject of your emails, follow these steps:
- Open your mailing list in Google Sheets and make sure that recipients are divided into two groups (see above).
-
Create a new column in your spreadsheet and enter âsubjectâ in the first cell of that column.
-
Now, in the second cell of that column, copy/paste the following formulae:
=IF(B2="A";"My e-mail subject";"My other email subject")What this formulae does is simply checking if the recipient from the row #2 is in the group A, by checking the content of the cell B2. If so, the subject will be âMy e-mail subjectâ. If the recipient is not part of group A, the subject will be âMy other e-mail subjectâ.
Youâll have to update this formulae with your own subjects. Also, make sure that the first part of the formulae, the condition =IF(B2=, is correct. For example if in your spreadsheet youâve used the column C to split your recipients in two groups, youâll have to update the formulae as follows =IF(C2=
- Update the formulae with your own subjects to A/B test, then, replicate the formulae all across the mailing list (as shown in the GIF above).
-
Now, open Mailmeteor by going to the âAdd-onsâ menu, then âMailmeteorâ > âOpen Mailmeteorâ. If you donât have it yet, click here to install it.
-
In Mailmeteor, create a new template and in the subjectâs field, enter the following . This tells Mailmeteor that the subject is a variable and that it should use, for each recipient, the content of the column entitled âsubjectâ.
- Once your template is ready, click on âSaveâ, then preview it. Whenever you are ready hit send!
Using this simple technique you can try out different subjects and see which one performs the better. If you have a Mailmeteor Premium account, and you have enable the tracking, Mailmeteor will show you which group has the best open-rate and click-rate. Making it really easy to decide which subject you should use.
How to A/B test your emailâs body?
Like I said earlier, itâs a best practice to A/B test only a small part of your e-mail. In the following, we will run a test by changing one sentence of a paragraph.
While you can apply the same principles exposed here to change multiple sentences or even send totally different emails to each of your recipient, itâs best to test very small changes. Otherwise you wonât know precisely what makes one version better than the other.
To fine tune a part of your emailâs body, follow these steps:
- Open your mailing list in Google Sheets and make sure that recipients are divided into two groups (see above).
-
Create a new column in your spreadsheet and enter the word âalternativeâ in the first cell of that column.
-
Now, in the second cell of that column, enter the following formulae:
=IF(B2="A";"Word/sentence for the group A";"Other word/sentence for the group B")What this formulae does is simply checking if the recipient from the row #2 is in the group A, by checking the content of the cell B2. If so, the emailâs body will use âWord/sentence for the group Aâ. If the recipient is not part of group A, the subject will be âOther word/sentence for the group Bâ.
Youâll have to update this formulae with the word/sentence that you want to AB test. Also, make sure that the first part of the formulae, the condition =IF(B2=, is correct. For example if, in your spreadsheet, youâve used the column C to split your recipients in two groups, youâll have to update the formulae as follows =IF(C2=
-
Once youâve updated the formulae, replicate it all across the column.
-
Then, open Mailmeteor by going to the âAdd-onsâ menu, then âMailmeteorâ > âOpen Mailmeteorâ. If you donât have it yet, click here to install it.
-
In Mailmeteor, create a new template and in the emailâs body, enter the merge field where you want to test the two alternatives. This tells Mailmeteor that you are using variables and that it should use, for each recipient, the content of the column entitled âalternativeâ.
- Once your template is ready, preview it, then hit send!
Of course, you combine multiples merge fields (aka variables) in the same email. For example, you could use the recipientâs first name (if you have it). You could also use the same technique to run an A/B test on multiple parts of your email. But thatâs not something I would recommend as itâs a best practice to narrow your test to the minimum as Iâve explained earlier.
Test your AB significance
When your AB test is completed, itâs now time to validate your results by calcultating its significance. The significance will let you know if the copy change impacted your results positively and by how much.
In this example, we sent 2000 emails in total, split into 2 groups of 1000 recipients.
In the first group A, 90 recipients engaged with our email by clicking one of its link, thatâs a 9% conversion rate. In the second group B, there were 120 clicks, thatâs a 12% conversion rate.
Now itâs not enough to say that 12% is better than 9%. It actually depends on your AB test group sizes. Calculating the significance tells us that indeed the test B converted 34% better, which is statistically significant. In others words, you should definitely go for the test B copy!
There are various AB test calculators out there. We can recommend the free and open source AB test significance calculator made my Matt Mazur or the simpler AB test calculator from Neil Patel.
Letâs wrap it up
In this tutorial, you have learned how to A/B test your emailing campaign using your Gmail account. Youâve seen that itâs really easy to do so, using both Google Sheets and a mail merge solution such as Mailmeteor.
Two more words on Mailmeteor: You can send up to 75 emails per day, using the free version of Mailmeteor, which lets you run an A/B test on two groups of ~40 people in each. Plenty enough for a simple test.
If you need to run your test on more recipients, grab a premium account to send up to 2000 emails/day (one test per day with two groups of 1000 recipients each). Mailmeteor Premium also comes with tracking feature to easily compute metrics from your campaigns, such as the open rate or click rate of each group.
And youâre not limited to test email with plain text. Mailmeteor supports HTML email, which make it the tool of choice for all email marketers.
This guide was written by Corentin Brossault, Mailmeteor's co-founder. Mailmeteor is a simple & privacy-focused emailing software. Trusted by millions of users worldwide, it is often considered as the best Gmail mail merge to send mass personalized emails. Give us a try and let us know what you think!