Email marketing guide: Step 7 Test, learn and refine to increase email effectiveness

This section, we look at some of the more advanced techniques to get more from your email marketing. We will cover:

  • Improving email delivery.
  • Tracking email campaigns.
  • Improving email marketing for mobile devices.
  • Using A/B split testing for email optimisation.

Improving email delivery

Q. Have our email delivery techniques been reviewed?

Email delivery arguably isn’t the most interesting part of email marketing for marketers. But since it can directly affect your results it’s worthwhile knowing the right questions to ask

What is it? Spam
There are differences in law per country about what is legitimate email. Unsolicited marketing emails are often also referred to as spam. But spam is these days defined in the eye of the beholder and not only by law, classifying unwanted, too frequent or too sales oriented emails as “spammy” even if they gave permission to receive them
How do ISPs identify spam?

In the case of the ISPs and webmail services, they define spam as emails that users don’t want. They filter email based on sender IP address reputation and increasingly domain reputation. A reputation score is defined from a variety of factors, including:

  • High bounce rates. This results from poor list hygiene. Often with limited ability or inter- est to receive and process bounces from botnets, bouncing addresses stay on spam lists.
  • High spam complaints. Users clicking the junk and report as spam button in their email client.
  • Spam traps. These email addresses that should not be on your list. The ISPs create these and watch for unsolicited email to the addresses. They most often get onto your list if you purchase data. As old email addresses may be converted by ISPs into traps, poor hygiene and data management can also result in spam traps.
  • Very low engagement. Low open rates and a high number of deletes without reading. No user will reply in response to a spam email.
  • Campaigns sent across hundreds if not thousands of IP addresses. The IP address- es rapidly change and come and go as spammers use a botnet of ‘zombie computers’. The email volume from each zombie IP address does not have smooth flow. It shows high peaks of activity.
  • Incomplete setup of technical criteria. These include correct DNS, SPF, DKIM and DMARC email authentication19 settings that show that your email software and IP ad- 7 dresses are allowed to send on your domains’ behalf.
  • Sending from a dynamic IP address. For example, those allocated to home ADSL con- nections.
  • Send volume and consistency. Because spammers don’t tend to have a consistent send frequency, ISPs look at how frequent you send your messages. If you suddenly start to send more emails than usual, this might damage your email reputation. In particular this is important when starting with a new email service provider or IP address. As Andy Thorpe explains20; ISPs monitor new IP addresses more closely and you will have to build up a reputation or warm up your IP-address.

So, to not be treated as spam by the ISPs means to not look like spam with the above char- acteristics!

In the early days ISPs used content to filter spam, words such as Viagra were obvious choices. These techniques were crude and easily circumvented. The ISPs are clever people with large resources and have been moving closer and closer to measuring the metrics that are fundamental characteristics of the spammers’ business model. This is shrewd as it is something that spammers can’t fake.

So, what can we do to avoid being identified as a spammer? The factors today that are important to inbox placement are low bounce rates, low complaint rates, high engagement, correct technical setup, content validity and consistent email volume flow.

Assessing your curent delivery ?
Q. Has our email delivery optimisation been reviewed?

The delivery rate as provided by email marketing tools is the percentage of emails that didn’t bounce. If you are emailing at least weekly to each email address on your customer base and you have good list hygiene practices in place then your delivery rate should be over 98 percent, that is less than two percent of emails sent are bounced.

What is it? Email marketing Bounce rate
Bounces are emails that could not be delivered and in turn return a bounce message. Commonly they are categorised into hard bounces and soft bounces. Hard bounces are bounces due to a permanent error; whereas soft bounces mean a temporary error.

As an example, a hard bounce will occur if someone terminated their email account, whereas a soft bounce example could be if the person’s inbox is full. ISPs may also send a soft bounce as an initial warning that an email is seen as spam.

But delivery rate isn’t the same as inbox placement. This is why the statistics defined by the the Email Experience Council call it Email Accepted Rate21, not delivered rate. ReturnPath have recently reported an average inbox placement as low 88 percent, even when the mailer has a good IP reputation as measured by a SenderScore of over 91.

You can review your Senderscore at

If your delivered rate is lower than 90 percent, then your next step is to review and change list hygiene best practice and reduce your bounce rate before moving on to the guidance in this post.

Deliverability, as opposed to delivery, is about more than just removing the bounces. It’s about arriving to the inbox. Before the rise of spam and aggressive spam filters to counter it, just knowing an email had not bounced was sufficient to be confident it was delivered to the inbox. Spam forced ISPs into additional filtering and the difference between deliverability and delivery arose.

An ISP will do one of four things for emails not bounced:

  • Place in the inbox.
  • Place in the junk folder.
  • Place it in another user specified or promotions folder.
  • Throw the email away.

Unlike bounce issues, whereby the ISP tells you that you’ve not reached the inbox, there is often no notification to senders by ISPs if you are put in junk or deleted.

The challenge then, is given the ISPs won’t tell you, how do you monitor if your emails are placed in the inbox?

There are two main methods:
  1. Delivery confirmation seed addresses. Delivery confirmation seed addresses are simply email accounts you create with the ISPs and include on your mailing list. Once the campaign has been sent, log into each of the seed accounts you created and check if the email arrived to the inbox. The inference is that if the seed address email got to the inbox you can expect the rest of the campaigns emails for that ISP to have reached the inbox. This is generally a sound assumption.
  2. Campaign metric changes. Your open, click, bounce and complaint rates also provide insight to inbox deliverability and alert you to a potential change in deliverability. It’s not as simple as just looking at your average open or click rate and if it drops concluding there is a deliverability problem. There are so many other factors that could cause this that such a conclusion is nonsensical.

To use these campaign metrics you need to look at how these vary and trend per ISP like Gmail or The key is knowing what is normal for your campaigns and looking at trends. Here the ratio from Hotmail to Yahoo is 1.82 (4.04/2.21). If this ratio changes significantly it implies the deliverability has changed between Hotmail and Yahoo. So ask your email service provider to do an analysis of this ratio.

Best Practice Tip 22 Review delivery variation by ISP
Email delivery will vary by platform such as Gmail, Hotmail, etc. If you see a much lower delivery rate, then it may indicate a problem with delivery.

Tim Watson recommends keeping spam complaint rates below 0.2 percent to ensure a good reputation and the lower the better. However, dropping spam complaint rates can be a sign of trouble too. If your open, click and spam rates all drop together it indicates you are not reaching the inbox.

The spam complaint rates are dropping simply because no one is seeing your email and thus will no longer complain about it! Two more tools to consider are blocklists and (formerly Hotmail) SNDS. All ISP filtering includes use of block lists. These are lists of IP addresses which are considered to be sending spam.

ISPs hold their own private lists as well as using the public lists. Use a service that will alert you if your IP address appears on a public block list. If you are using a good ESP they should be doing this for you.

Assess engagement ‘beyond the click’ through web analytics

Q. Has the effectiveness of email ‘beyond the click’ been reviewed?

For many years, it was difficult to track engagement of your visitors after they click through on your email through to the site. This is important, because great open and click through rates, don’t necessarily mean great conversions and we also want the email channel to receive proper attribution of the revenues generated. In more email systems you can tag the conversion page such as a sales or subscription thank you page oftentimes even fully automated.

Best Practice Tip 23 Integrate web analytics campaign tracking
You should define standard campaign tracking codes for different email campaigns to enable you to review engagement against conversion to sale or other goals, otherwise emails will be recorded within your analytics system as direct visits. If your site is a transactional site with a clear sales conversion goal then success in conversion will be shown by e-commerce tracking.

With the widespread adoption of web analytics and in particular Google Analytics for tracking pay per click AdWords campaigns, many email marketing vendors have made it easier for you to track after the click by automatically integrating Google Analytics campaign tracking into the hyperlinks in your email templates.

Integration with other advanced web analytics systems such as Omniture and Coremetrics is also possible.

For Email marketing the parameters (* are optional) I recommend are:

  • utm_medium – medium used for marketing, i.e. email.
  • utm_campaign – campaign name, e.g. EnewsNovember.
  • utm_source – this is usually the media owner, but for email marketing can be used to specify the source of email list, e.g. HouseList or the name of external list providers/ newsletter ads.
  • * utm_term – in AdWords used to identify the keyword used to trigger the ad, can be used in email marketing to identify individual links (optional), e.g. Offer1, can be based on click text summary.
  • * utm_content – used to track an individual or segments response (optional), this could be based on any field in database, e.g. user-id, user email, etc.

If your site is not transactional, then you should define other conversion goals for engagement with the site. In both cases you can define an actual or nominal monetary value for conversion. In Google Analytics this will enable you to compare campaign effectiveness according to $Index value or total value.

Best Practice Tip 24 Review web analytics bounce rates
Compare landing page bounce rates (for the site not hard bounces for the email campaign) for evaluating the relevance of the landing page content. If you find the bounce rate for the landing page is relatively high, and conversion rates low, you can be sure you are directing your visitor to a less relevant page from your email. You can also compare landing pages on duration, number of pages subsequently viewed and conversion to goals pre-defined in the analytics system.

Improving email marketing for mobile devices

Q. Has our use of email on mobile devices been reviewed?

The reading of emails on mobile devices and particularly on smartphones and tablets is an important part of email marketing with the concept of design first for mobile now a trend. Data from the Litmus shows the huge impact of emails being read on mobile devices24 like iPhone, iPad and Google Android.

Android stats are likely understated with respect to iOS devices, due to iOS loading images by default and the Android blocking by default, leading to blocking of open tracking. Even when taking this into account the biggest device for mobile opens is the iPhone.

We’ve seen the industry trends, you can check to see how important mobile email marketing is for you. The majority of brands now have mobile opens making up between 25 percent and 75 percent of their total opens25 open rates below 15 percent are an exception.

Best Practice Tip 25 Define the target platforms for mobile email marketing
You should decide which platforms your emails should work on and then take steps to evaluate how these effectively should work.

To find your mobile usage levels take a look in your Google Analytics at Audience, Mobile Devices. You’ll be able to see your mobile audience and their devices to focus on. You can also set up an Advanced Segment for email visits only to see the relative importance of mobile devices. This B2B example shows that although iOS devices are dominant, some Android devices are increasingly important.

To better understand the behaviour of visitors on your site referred from email marketing you can create an advanced segment for email visitors once you have set up tracking.

Best Practice Tip 26 Apply Advanced Segments for Email in Google Analytics

The Advanced segment should be based on a medium of ‘email’ if that is how you have labelled your links. This can help you see the devices people who click through to your site specifically from your emails.

You select the advanced segment using the down arrow at the top left above the reports in Google Analytics, then choose “Create New Segment” and set the medium to “Email”.

According to research by Custora26, email marketing is driving 26.7% of sales on mobile and 23.1% of the sales were via tablet devices, compared to the 20.9% of all sales on desktop.

Mobile email experience reviewed and conversions optimized

Has the effectiveness of email marketing on mobile devices been reviewed?

The most common approaches are now to design a skinny email that gives good user experience on desktop and mobile or to use full responsive email design. Fortunately Inter- net-enabled phones now include browsers and email clients that are based on the WebKit rendering engine, which has good HTML and CSS support. This includes the iPhone, Android devices, webOS.

What is it? Responsive email design
With responsive email design, we use @media queries to code and designing the email. Responsive design allows the email to adapt to the device and screen size used at the time of viewing the email. The goal is to improve the user experience and increase conversions.

Using responsive design allows the best user experience to be presented for any device. Using responsive design allows an email to automatically adapt the device being used to view the email, allowing changes such as:

  • Hide content
  • Change font sizes
  • Change the email width
  • Resize images
  • Re-stack rows into columns

However, responsive design often also increases the cost of the template and ongoing campaigning. While some Email Marketing software offers standard mobile compatible templates, it often needs extra coding and design effort. So it may not be worth it for everyone. An alternative approach is to provide a skinny design.

Some guidelines to consider when creating mobile first thinking emails:

  • Use pre-header text. This is shown in the iPhone inbox view.
  • Reduce the use of large branding headers and banner images in the area above the fold.
  • Use larger fonts or increase font size, 12pt for the body and 24pt for headings.
  • Use even shorter and chunked content. Already good advice for email in general and even more important for mobile.
  • Only include images that add to the message, drop anything purely artistic.
  • Space out links so they are easy to touch without getting the wrong link.
  • Use buttons for calls to action, a 22 x 44 pixel minimum as this is the size of your touch area.
  • Favour single column design or side by side when displaying many products.
  • Keep the HTML size and image sizes down, readers may be on a slow 3G connection.
  • Place calls to action on images – there is no equivalent of the hand icon when ‘hovering over’ on a touch interface.
  • Drop content, removing calls to action, text and design elements that aren’t central to the email objective.

Leave a Reply

Your email address will not be published. Required fields are marked *