SUBJECT LINE TESTING - Anatomy of an Email Inbox 3/16/2017

The subject line in your email is one of the main reasons people decide to click and engage with your content.  It can make the difference between great or mediocre email performance.

Email service providers (ESP) understand this and many of them build this functionality into their systems.

I have referenced one of my favorite best practices sites, behave.org, for testing before.  See some of their subject line testing results for ideas on things you can test.  Marketers are seeing small and large impacts from these tests, but done well, some have seen more than double the open rates.  So, what does that mean in raw numbers and potential revenue?  Look at the simple example below.  This spreadsheet allows you to input your own metrics and see the revenue impact possible with a subject line test.  Want this spreadsheet?  Request Now.

So how do you do subject line testing?  As I said above, most of the large ESPs have this functionality already built in.  We will use MailChimp as an example to show the process steps.

1.  Create a New Campaign

2.  From the drop down, pick A/B Testing as your campaign type

3.  Pick your list, either an existing one or create a new one

4.  Select subject line as the type of test you would like to do (here and in other ESPs you can select how many subject lines to test, however just be careful on doing too many at once unless you have a very large list since the more you split it, the more difficult to obtain valid results).  MailChimp actually recommends a minimum of 5,000 per test cell

5. Select the percentage split of how many of the recipients will receive each version of the subject line.  This will depend on how many subject lines you have, so for two split 50/50, for 3, 33/33/33, etc. 

6.  Select how long before you declare a winner.  You can select hours or days.  Your selection will depend on the size of your list.  If your list is small, you may want to leave a longer time to allow enough results to come in to have a valid sample

7.  Select the metric you are measuring.  For subject line testing, it will typically be the open rate, but you could select something else if you have full funnel metrics

8.  ALWAYS remember to personalize with the first name if you have it unless this is your test

9.  Enter in your test subject lines, from and what metrics you want to track

10. Select and populate your email content and design

11. Test and send by following the prompts

12. Review your results and add to your ongoing learning

While this is only one walk-through in how to do a subject line test, other systems may work differently including the ability to embed a subject line test as part of normal sends.  This typically works by allowing you to set 10-20% of your list to run through the test in the first wave of sending, then after reading those results, continue with the full list send based on the winning subject line.

Even if your system doesn't have robust, built in subject line testing, you can do it yourself by creating two campaigns with everything the same except the subject line.  While this is a bit more work and can be a bit problematic from a sending standpoint, it still provides you the ability to test and learn how subject lines can improve your bottom line.

Today's Tips:

-  Include subject line testing as a normal part of your email program

-  As always, use personalization

-  Read results, accumulate your learnings and continue to improve!

TESTING - Anatomy of an Email Inbox - Nov 8, 2016

It is finally here - voting day!  I'm sure most of us are glad to be at this point and moving onward, whatever that brings.

How does that relate to our topic of testing?  It is early in the day, but I took a look at my inbox this morning to see if anyone was leveraging the day and in fact some are.  Of 54 emails that I received since midnight, 6 had some element of the election theme in them:

Builders Designs subject line:  "The Polls are Open for cyber Monday!"
Fandango:  "Election Day, Movie Style"
IAB Smart Brief:  "How tech companies are getting involved on Election Day."
Groupon:  "Elect to save on Tech, Home and More Doorbusters."
World Market:  "We voted. And the winners are..."
Kohls:  "Uniting all sides behind our $10 off."

When we think of testing, often subject lines are easy to test and can have dramatic impact and the ESPs do a good job generally of facilitating this.  So, first question is:  Do you test EVERY single subject line when you deploy emails?  You have to think based on the above that at least a few of the people sending these are trying to be relevant.  Perhaps not specific to their brand and products, but certainly to what people are likely to be thinking about today.  Were these subject lines conceived of and tested with the deployment?  Did they win or was I just part of the test cell?  How effective were these?  All questions we cannot answer specific to these inbox emails, but certainly can and should for ourselves.

And beyond subject lines, what other elements of emails should we be testing?  A lot of these answers depend on you, your product, your strategy, brand and goals.  The point is there is a lot to learn while deploying your emails and you should be actively implementing a detailed testing plan to insure you continually improve and drive incremental results.

One thing I like to do as well is keep a log of testing results that not only build and pass on that history but allow you as a marketer to demonstrate the impact of your efforts on driving revenue. Consider if you could tie every test to your KPIs and show that via testing you drove incremental results for your business?  Summarize these over the course of a month, quarter and year, and oftentimes you will find you have a compelling story.  While there will definitely be wins and some losses, the act of driving this accountability insures the wins will far outnumber the losses.

So how do you figure out a testing strategy and what to test?  One website I highly recommend for ideas and insights is whichtestwon (now called behave.org).  Especially fun to sign up for their emails where they send you a test and let you vote, then tell you the results.  And a bonus, in addition to tons of ideas for email testing, there are ones for virtually all your other executions as well.

Here's some general categories of items to think about testing related to email marketing:

- Certainly subject lines as we noted above and go beyond just the words.  Simple things like punctuation, capitalization, placement of words, use of emojis and more can have a dramatic impact.
- Body length - I did a test for a client while I was President of .Com marketing.  The test had generally the same content but the body copy of one was much longer than the other.  When we read the results, we found that while the short copy had more opens, the long copy had more engagement which was what we really wanted.  Publishers Clearing House is the king of this kind of long copy engagement with their mailers.  That's why there are multiple pages, stickers and things to go through in the envelope.  Their testing clearly showed more engagement when they provide more opportunities to do so.  This ultimately leads to more entries.  We saw those same type of results with our direct mail while I was at Sears Home Services.
- Content type - there are many options here - should you include benefits, pricing, offers and deals, imagery, what kind of imagery, how much, headlines...again, simple things can make a difference.  At Stow, we tested extensively and one thing we found was that simply putting people in images increased performance.  Pricing is another area of contention oftentimes in business and when and how you should use and embrace it.  Testing of course will tell you, but generally my experience has been that sharing pricing too early in the process can reduce your opportunity for engagement.
- Specificity - there are some good tests on behave.org that address this.  Should you be more or less specific and when would one work better than another?  Should you provide detailed steps or general guides?  How should you present those specifics:  list, within a box, in bold or another color, highlighted...lots of tests to conceive to impact your KPIs.
- Timing of emails - this is one of the topics that I discussed in a past blog and one of my pet peeves.  The reality is timing and effectiveness are definitely tied together, but I wonder how many marketers even test timing.  If I look at all the endless, irrelevant, SOUR emails I get in my inbox, I think this is a big opportunity for all of us to improve.

I could go on with more, but the point is, TEST!

Today's Tips for Better Testing:

- Get great ideas from behave.org - one of my favorite emails that I love to read.
-  Have an ongoing testing program for emails (and other areas of marketing).
-  Set the appropriate KPIs for each test with a hypothesis and expected results.
-  Accumulate your testing performance and showcase to leadership your wins and the impact you are having on the business.  This makes it very easy come budget season as well to obtain the funding for those testing tools and perhaps new ones.
- ALWAYS test your subject lines.
-  Leverage the built in testing tools that your ESP provides as well as Google's testing capabilities or other paid providers.  I especially like Optimizely as a tool that is easy to use and allows for fast learning

Change your culture to be one of continuous testing where failure and learning are embraced as an important part of growth and improvement.  Leverage the great tools out there to learn:  Adobe Target, Optimizely, Google and your ESPs.  Take the time to deliver a better experience to your prospect and customers. They will reward you with higher loyalty and bottom line results and you will not be a SOUR emailer!