Zach Heller — November 30, 2018 — November 30, 2018
Subject line testing is a crucial part of any email marketer’s role. It is singularly one of the simplest and most effective ways to improve the performance of an email marketing campaign.
But saying it is simple is one thing. So today I wanted to actually take the time to lay out the do’s and don’ts of subject line testing so that every brand, organization and marketing team can succeed in their quest to find better subject lines.
What is Subject Line Testing
Essentially, subject line testing is the process by which you seek to improve the open rate on your emails by delivering the same email to different audiences – one of which sees one subject line, and the second group sees the other. In this way, subject line testing is a very specific example of an A/B Test.
The reason that brands of all types run subject line tests is that it is one of the best ways to improve the overall performance of an email marketing campaign. Because the subject line is one of the first considerations for an email recipient – one of the first things they see along with the sender once an email lands in their inbox – it is one of the most important elements of any email.
Often, a subject line alone will mean the difference between a recipient choosing to open the email or not. So by testing multiple subject lines, we aim to improve the open rates over time.
How to Set Up a Subject Line Test
There are a number of different ways to set up a subject line test:
- You can test the entire list. If you are testing an entire list, you simply send 50% of them subject line A and the other 50% subject line B. These results will help you plan for future emails.
- You can test just part of a list. You might choose to only test part of the list, so that you can use the results of the test to send the winning subject line to the rest of the names on the list. For example, you might take 20% of your list and split it into two groups (10% each) for the test. Then wait 24 hours to determine the winner before sending the remaining 80%.
- You can test programmatic emails. If you have an automated email campaign, one that triggers based on user actions and often includes more than one email in a sequence, you can build tests into your program. Send half of your users down one path and half down another, and run subject line tests at each step in the process. However, here you must remember that the sequence matters, meaning the results of one test at the beginning could impact results of emails that come later.
How to Judge the Results of a Subject Line Test
Judging the results of a subject line test should be quite easy. Remember from the information above that the goal of any subject line test is to improve the open rate of that email. And so it is the open rate that we must measure.
Compare the open rate of subject line A against the open rate of subject line B, and you should get your answer. The email with the higher open rate is the winner.
A few additional things to consider:
- Allow enough time between the email send and the results analysis. Not everyone will open the email right away. A good rule of thumb is to wait at least 24 hours to declare a winner.
- Statistical significance matters in any A/B test. A result is considered statistically significant if the confidence level is at least 90-95%, meaning you can confidently conclude that the subject line was responsible for the difference in open rates (rather than chance). This is a great free tool to measure statistical significance.
- It can be tempting to look at metrics other than the open rate, such as the click through rate. But the subject line will have less impact there. If the open rate on one version is significantly higher, but the click through rate on that version is significantly lower, you might consider what other factors are leading to those results.
What Types of Subject Lines Should You Test
As a general rule, you should test all of your subject lines. There is no surefire way to know which subject lines will perform best for an individual email. And testing is the only way to determine that with any certainty.
But just like anything else that you do, some measure of priority must be determined. Which emails are the most important? How would an improvement in the open rate of each email impact the business at large?
Once you have determined which emails to test, there are some best practices you can aim to incorporate into your new subject lines:
- Personalization – include the person’s name or other details in the subject line
- Urgency – convey a sense of urgency around offers
- Questions
- Punctuation
- Less is More – limit the length of your subject line
- Capitalization – try different ways of incorporating capitalization
- Active Words
- Positive language vs. negative language
You can check out this tool from SubjectLine.com, which calculates a score based on a very specific formula for subject line effectiveness. But remember that that is just a guide and just because a subject line does not score well doesn’t mean that it won’t perform well with your lists.
Digital & Social Articles on Business 2 Community
(69)
