The Key to Subject Line Success: Testing, Not Guessing

A 67% lift in open rates and an 87% increase in revenue. This is just one example of the level of improvements FulcrumTech achieves for clients, thanks to subject line testing and optimization. In this case, the client was a major U.S. publishing company. Subject lines are what drive recipients to the top of the sales funnel in email campaigns; yet, many organizations spend little time and effort crafting or testing them. Here we show you the impact that subject line testing can have on email campaign performance, as well as offer some practical advice about how to achieve subject line success.

Subject Line Testing Success Stories

A great example of subject line testing success was demonstrated by President Obama’s reelection email campaign. The Obama fundraising emails brought in most of the $690 million that the campaign was able to raise online. As revealed by Obama staffers in a Bloomberg Businessweek article, the online fundraising appeals were the product of extensive A/B testing of the subject lines, as well as testing of the content and formatting of the messages.

A team of analysts tested subject lines by sending emails to small groups of campaign supporters. Then, based on the donations those emails brought in, the team projected and compared how much the emails would generate if sent to the entire Obama campaign email list. Of the hundreds of subject lines tested, two examples of the most successful — which each brought in millions of dollars — may surprise you: "Hey" and "I will be outspent."

The following are some other examples of email subject line tests that show the kind of impact testing can have on your bottom line:

• MarketingExperiments ran a subject line test to demonstrate just how powerful testing and optimization can be. They asked for ideas from blog readers to promote their Optimization Summit 2012 and tested six different subject line treatments. The winning subject line — "Do your landing pages pass this test?" — generated a 125.9% higher unique click-through rate and drove more than twice as many visitors to the landing page compared to the subject line that performed the worst. Click here to see the test details and all of the subject lines evaluated.
• MarketingSherpa Blog highlighted a study done by AWeber that tested clarity versus creativity in email subject lines. Sent to a list of more than 45,000 subscribers, the test was run across 20 subject lines. The clear subject lines were the hands-down winner in all channels measured, generating 1,107% more comments, 617% more website traffic, 366% more email subscriptions, 331% more Facebook likes, and 315% more tweets.
• In another MarketingSherpa Blog case study, seven subject lines that were personalized had a 5.13% higher average open rate and a 17.36% higher click-through rate compared to the same subject line without personalization.
• MailerMailer’s 2012 Email Marketing Metrics Report analyzed subject lines in over 1.2 billion email messages and found that short subject lines (fewer than 15 characters) had the highest open rate (15.2%) compared to the longest subject lines (more than 51 characters) with the lowest open rate (10.4%). The highest click rates (4%), however, were achieved with subject lines between 28 and 39 characters, with the lowest click rates (1.8%) for subject lines with more than 51 characters.

What Should You Test in Subject Lines?

As demonstrated by the above test examples, you can test subject line length, personalization, and wording. When it comes to wording, be sure to take a look at what words are trending in relation to your email message. For example, Google’s AdWords Keyword Tool is one way to find words and phrases related to your subject line topic, as well as to see how frequently they are searched for online.

Plus, you can also test upper-case letters, punctuation, and symbols, which marketers often incorporate in subject lines to capture their subscribers’ attention. An Experian Marketing Services’ 2012 study of symbols in subject lines found that symbols resulted in as much as a 15% lift compared with regular promotional email subject lines. But will your target audience think that the subject lines with symbols or upper-case letters look like spam? Only with testing will you know for sure how they’ll respond.

How Can You Test Subject Lines?

Some email service providers have A/B split testing tools that marketers find useful in comparing subject lines. You can also set up your own A/B split tests, which we talked about in a previous NewsLever feature. Ensuring that you have statistically significant sample sizes is critical to getting accurate test results. Online calculators, such as the MaCor Sample Size Calculator, can help you determine the right sample sizes. In addition, this Silverpop blog provides some great tips for setting up the right size A/B split test groups for testing subject lines.

What Criteria Should You Use to Test Subject Lines?

As the subject line test examples cited in this article demonstrate, open, click-through, and conversion rates are commonly used to determine the relative success of subject lines. In addition, a click-to-open rate (the number of unique clicks divided by the number of unique opens) indicates whether a subject line captured more qualified people — the higher the click-to-open rate the better. Similarly, a higher conversion rate also shows a subject line’s ability to bring in more qualified subscribers.

When planning your email campaign strategy, be sure you understand what can be achieved by testing and optimizing subject lines. You can use FulcrumTech’s proprietary financial modeling tool — ROI Goalsetter® — to do that by creating your model and seeing the impact of various levels of open-rate improvements.

Then, once you know your measurable goal of a specific open rate, you can set up a subject line test — one of the biggest levers you have for improving open rates.

To get the best results from each of your email campaigns, you need a great strategy and continual improvement to deliver against your campaign goals. Subject lines are an integral part of that strategy and improvement. Need help in driving consistently strong results from your email campaigns? Email us or give us a call at 215-489-9336 and see what FulcrumTech can do for your organization’s email-marketing program!