Deliverability can often be the biggest barrier between you and your email ROI goals. Many factors may prevent email from reaching the intended inbox, and it can be tough to translate the receiving network requirements into actionable marketing strategies.
Watch Email Deliverability Superstars Carmi Lopez-Jones and Steve Sonnenberg for their webinar, Cracking the Inbox Code: Bounce and Performance Reporting.
You'll learn how to:
- Decipher elusive bounce codes
- Set up bounce reporting in Marketo Engage
- Create custom email performance reporting by network
In this tips and tricks webinar, we’ll start by walking through the benefits of ISP reporting as well as how to determine which ISPs, or networks, you should prioritize based on the makeup of your lead database. Once we have that information, we’ll customize the Marketo Email Performance report to reflect the most relevant ISPs, hosting platforms, and filtering companies in an effort to better monitor those networks individually. From there, Carmi will proceed into bounce management and reporting.
Before we dig into building reports by ISP….let’s start with some context.
The first thing to note when discussing Deliverability reporting is that the term “Delivered” does not necessarily mean delivered to the inbox. When an email is reported as being delivered that simply means that the receiving server has accepted the message at the gateway.
After the message is accepted, the receiving network can process it in a number of different ways that are not visible to an Email Service or Marketing Automation provider like Marketo. The email could be delivered directly to the recipient’s inbox or junk folder, it could be reviewed and tested by the receiving networks filters, or it could potentially be quarantined by the receiving network's security.
With all that in mind, it’s fair to say that….
…Deliverability is HARD!
Marketo’s partner 250ok reports a positive trend of improvement, with a 4% increase in inbox placement and a 5% decrease in missing emails between EOY 2016 and 2017.
Now while 250ok reports some positive trends in email delivery and deliverability, at the end of the day they are still reporting that only about 87% of commercial mail is delivered
Between missing mail and mail in the spam folder only about 77—78% of the mail that is sent and delivered actually makes it to the inbox!
In our last webinar, you heard from Tifaine Highly that inbox placement is hard, especially on platforms such as Google. One of the issues that we hear from clients most often is difficulty delivering to the Inbox at Gmail and Google Apps hosted business domains.
But Google isn’t the only platform that should be monitored…
A savvy marketer should always understand the list distribution of their lead database so you can customize your send strategy to the individual networks that are most important to your business. This is necessary in today's email universe because each ISP, each hosting platform, and each filtering company have their own measures of a senders’ email reputation and they all treat reputation uniquely. Google for instance places a large emphasis on the reputation of domains where other networks like Microsofts’ Office365 and Hotmail platforms still emphasize IP reputation more.
Some will filter heavily based on how engaged a senders audience is while others may weigh complaints as more of a defining factor of a senders reputation. Many smaller ISPs and b2b platforms could be more sensitive to large email volume spikes whereas larger global ISPs with greater resources may not be as sensitive to large volumes of mail all at one time.
Understanding where you’re sending to and the reputational nuances of each network is crucial to maximizing your inbox delivery potential. Now you may be asking yourself; “how do I figure out what my list distribution is?” If you target consumers, it could be as simple as loading your list of email addresses into a spreadsheet and running counts based on the domains of the individual addresses. If your audience is primarily business domains, the task is slightly more complex. Since b2b domain names tend to all be custom, it’s difficult to know what platform is hosting a domain just by looking at it.
There are tools however that can be used to analyze your database of email addresses and report back on the platforms where those emails are hosted. One path is to analyze the Mail Exchange or MX records of each domain in your file. That's a lot of work. Another option that automates that function is the ‘seedlist optimizer’ within Marketo’s Email Deliverability tools. The seedlist optimizer will take a file of your email subscribers, analyze it, and then reporting back on how many domains are hosted at each provider, detail the breakdown of the domains by provider and the portion of your list that's represented at each provider. You can then use that information to customize your email performance reports to reflect those distributions and then consistently report by ISP….. (next slide)
Now let’s walk through that process. To do so, let’s open up our instance of Marketo. (start at main landing page) (*******navigate away from powerpoint into web browser to show the process… talk track continues on next slide******)
To retrieve a sample of our list, let’s navigate to the Lead Database (show navigation to lead database). Create a new smart list will pull all the leads that this instance has sent to in the last 90 days.
Now, the timeframe you choose to use to pull your list should be relevant to the leads that you regularly email. This will give you the truest depiction of which ISPs and hosting platforms you regularly mail to.
Let’s export the smartlist.
Marketo will export this list into a CSV format. We’re going to open this in excel….. delete any data that is not an email address….. and then convert the file into a txt format which is the necessary format to load the file into the seedlist optimizer. One thing to note is that if you’d prefer not to load the actual full email addresses into the tool you can delete everything to the left of the @ symbol and simply load in the domains to be analyzed.
Now that we have our TXT file of email addresses, or domains depending on which option you chose, let’s navigate to the Email Deliverability tools.
To get to the seedlist optimizer you’ll hover over the inbox tile and click on “seedlist optimizer”.
On the right side of the screen, click on “upload list”
A popup window will appear for us to load our list into. Click “choose file” and navigate to your .txt file, then select it. Before clicking “upload list” let’s name our list. We’ll call this “Sent Leads_90 Days”…. Click “upload list”. Once the pop up window disappears you’ll see a status of “processing” for your list. Keep in mind that the amount of time the tool takes to process your list will depend on how large of a file you uploaded. To see our list distribution we’ll click on “view results” once the file has finished processing.
The results will be broken out by Region, Hosting Providers, and Filtering Companies as you see here. Each platform will show the quantity of recipients at that network along with what percentage they account for in the total file. If you want to see the individual domains at each platform then navigate to the platform and click on the “domains” button on the right side of the screen.
The domains are going to be listed in order of greatest to least based on the quantity of addresses at that platform. From this point we have one of two options. We could take note of the largest domains at the individual platforms which will give us the data necessary to customize the email performance reports based off domains. The second option would be to export the list and apply segmentations to the lead data. We'll review both options within your Marketo instance but let's first walk through editing the email performance report.
Let’s take this information and navigate back to the Marketo Instance to customize our reporting. (***switch browser tabs to go to Marketo***talk track continued on next slide)
Once you’re back in Marketo, navigate to “analytics” to report at the instance or workspace level. (***click on analytics***)
Click on the standard “email performance” report. (***click report***)
Navigate to your smartlist for this report. (***click smartlist***)
Once you’re in your smartlist you’ll want to filter by the “email address” field. Drag the field into your filter and change your filtering option from “is” to “contains”. (***display this onscreen***)
In your value dropdown you can enter one or multiple domains one line at a time. When doing so simply type the @ symbol followed by your domain. (***display this onscreen***) Including the @ symbol makes it easier for the search engine to query the database so make sure that’s not omitted.
If you’re attempting to report against a global domain that may have multiple country extensions, like .ca, .co.uk, etc., then it will be simplest to not include any extension so the query will look just for the domain name.
Once you have your smart list set, save and name your report. Let’s name this report “email performance – Gmail” (***display this onscreen). Now the report will be available for you to regularly monitor your deliverability for Gmail. (next slide)
So here’s what the standard email performance report looks like
Total audience 311K
AVG Delivery rate: 99.2%
AVG Open Rate is 22.9%
AVG Click to Open Rate is 1.5%
However, when we use our filtered reporting a different pattern emerges.
Of the total 311K, the lion’s share is Gmail with more than 198K Gmail subscribers
Gmail’s average Open Rate is 33.9%
And CTOR average is 4.1%
So, Gmail is performing better than the average overall.
For OATH, the volume is 73K
Delivery at 99.7%
OR at 15.6%
CTOR at 6.2%
The Hotmail Audience is quite a bit smaller at 44K
Delivery is 99.9%
Like OATH, the OR average is less than ½ of Gmail
And CTOR averages is better at 5.7% compared to 4% at Gmail
So Hotmail is a smaller audience, but the open rates are less than ½ that than at Gmail and 6 points less than the average overall.
Thanks for all the insight, Steve!
Now that we know how to evaluate the complexion of our audience, and which networks or ISPs are most important to us, let’s talk about Bounce Reporting….one of the KEY indicators of successful delivery (or failure!)
Hi everyone. My name is Carmi and I’m happy to chat about Bounces, one of my favorite topics. Let’s start with the age old question…..”How should I manage bounces?” or “Doesn’t my email provider do that for me?” Well, like most deliverability dilemmas, the answer is it depends!” Let me break it down.
In general, there are two options for email delivery. It will either be Accepted at the gateway, or Rejected.
Seems pretty straightforward, right? Well, kind of…..
Bounce codes can be kind of intimidating….but here’s the thing.. They are PURE internet gold! I mean it. I could spend days immersed in bounce codes, because they help piece together the puzzle. Each of them holds the key to the reputational drivers that are impacting your email program success.
We look for things like high rates of invalids, evidence of blocks or throttling due to complaints, spam filtering, and the dreaded blacklisting. By interpreting the logs holistically, rather than just one domain or receiving network at a time, a pattern emerges.
What you see on screen right now is a sampling of the types of HARD BOUNCE error codes that Marketo receives back when you deploy mail. You’ll notice there are some commonalities….and some differences. You’ll see things like
No such user
Recipient not found
Address rejected
These area all indicators that the address is invalid. You’ll likely also see errors like “The content of this message looks like spam” or “Your IP is temporarily deferred due to user complaints.” Those are indicators that your mail is rejected due to a reputational issue. Some will contain a URL to a website with more details on the bounce, some will provide little more than “policy block” with no additional insight as to which policy was violated.
If that isn’t confusing enough, soft bounces tend to be a little bit fuzzier and can include things like network congestion, throttling, mailbox full, and even things like “couldn’t connect at this time”
Here’s where it gets tricky.
Click
This error “No MXs for this domain could be reached” COULD be just a typical temporary issue. Or, it could be a sign that the domain doesn’t exist.
Here’s why….during the SMTP conversation, we will attempt to connect to the address repeatedly for up to 36 hours. At that point, if you see this error, it means we just weren’t able to reach the recipient domain. Sometimes that’s an internet traffic jam….but if you see this pattern repeatedly over the course of several mailings, you’ll want to ensure that the domain is configured to receive email. In Marketo, you have control over soft bounce handling logic, so its important to listen for those that are chronically soft bouncing with no signs of email life, and pruning those over time. Because if they are repeatedly soft bouncing, especially with an error like “couldn’t connect to an MX,” there’s a good shot that domain doesn’t exist.
Is your head spinning yet from all those bounce codes? Mine too. Good news! Marketo makes it easier by translating these errors into categories that we can better understand….because as you saw, each network can reply with pretty much anything they want. While they tend to have similar bounce codes, sometimes you need an expert to know what to do.
So, in Marketo there are 2 types of Hard Bounces and 3 types of soft Bounces.
Let’s start with Hard bounces
Category 1 are spam blocks. This means your mail was deemed spam by the receiving network. This could be due to a blacklisting or content filter or even a block of your sending IP address or domain.
In Marketo, we’ll set these to email suspend for 24 hours and you can try them again at a future state. Spam blocks are generally temporary in nature, some will resolve themselves within a few hours; but sometimes they may linger and may require tender care and feeding to resolve.
Soft Bounces on the other hand are most often temporary issues, like DNS issues, timeouts, deferrals. We’re going to retry those for you for up to 36 hours before returning a SOFT BOUNCE error. So, if you see soft bounces exceeding 3-5%, it’s time to revisit your soft bounce handling logic.
So, listen….some ESPs will manage soft bounces with a standard “3 strikes you’re out” policy, where any email addresses that bounce more than 3 times or some other arbitrary value, are in a hold pattern or coded invalid. Marketo empowers You to manage your bounce logic in accordance with your own mail cadence and frequency. It’s often misunderstood, but a really solid feature.
Now…The bounce detail for Hard Bounces is stored in the activity log; however, it can be tricky to export the details for further review with out a little elbow grease. Let’s get to it.
I’m going to walk you through the 5 steps to create a bounce directory in Marketo. I’ll show you how to extract the information needed to create a directory of leads bouncing emails and how to make the list automatically update. This will also create counters for bounces and successful deliveries after bounces happen.
Step 1: You’re going to create 6 custom fields
Read the slide
Quickie refresher on how to create a new custom field:
Go to Admin panel
Click on Field Managemeent
Click on New Custom Field
Repeat the process so that you have
You’ll need 6 custom fields:
Two DateTime fields
Name the DateTime fields “Bounce Date” and “Email Delivered After Bounce”
Two Score fields
Name the Score fields “Email Bounces” and “Deliveries After Bounces”
Two String fields
Name the String fields “Email Bounce Details” and “Email Bounce Category”
Step 2) Create a program to house everything
READ SLIDE
Our program type is Default
Step 3, READ SLIDE
Step 4) READ SLIDE
Ok for the first Smart Campaign, you’ll want to ensure you’re including both “email bounces” and “email bounces soft” as triggers
The flow will have 5 steps:
Flow Step 1: Change Data Value
Attribute: “Bounce Date”
New Value: {{system.dateTime}}
Flow Step 2: Change Data Value.
Attribute: “Email Bounce Category”
New Value: {{trigger.category}}
Flow Step 3: Change Data Value
Attribute: “Email Bounce Details”
New Value: {{trigger.details}}
Flow Step 4: Change Score
Score Name: “Email Bounces”
Change: +1
Flow Step 5: Add to List
List Name: “Active Bounce List”
Be sure to set your campaign to allow leads to go through EVERY time.
The Smart List of the 2nd campaign will need a trigger for “Email is Delivered” set to “is any” and also a filter of “Member of List” looking just for leads that belong to your Static List. This way, the campaign will only apply to leads who have had an email bounce but have then had an email successfully delivered afterwards.
You’ll have 3 flow steps:
log when the email was delivered;
add a point to the "Deliveries After Bounce" Score Field and
remove the lead from the "Active Bounce List".
Be sure to set your campaign to allow leads to go through EVERY time.
So, now we need to create a custom View using the custom fields you’ve created.
Once the list has the necessary details, it can be exported with those columns included so that you can work with it. You can identify emerging trends and issues with your deliverability by looking for common themes among the bounce details.
The benefits to monitoring your bounces are a plenty
Knowledge is power
You need visibility to identify issues early, to get ahead of them, and proactively manage your sender reputation
It can also provide insight into whether something YOU’RE doing is impacting your deliverability or if you’re being impacted by others in the shared environment.
We thank you so much for joining us today and hope we’ve left you empowered to build your own email performance reporting by network and bounce reporting.
These are two of our best tips and tricks to better monitor and manage your delivery and deliverability.
We’ll leave you with some resources on the next couple of slides. Friendly reminder, the deck will be shared with you after the webinar.
We appreciate your partnership and your feedback. Please do share your thoughts with us. Meanwhile, what questions do you have for us, Lola?
Change slides.
For Steve
Q: It seems like there are a couple of ways to review my email performance reporting in Marketo by ISP. The first one you showed is using filters in the Analytics tool, the other is using segmentation. What are the pros/cons of each?
A: Using the ‘filter’ option is an easy way if you’re focused on specific networks that are limited to a handful of domains. Like Microsoft may include Hotmail.com; live.com; msn.com. However, if your audience is mostly B2B, segmentations can be more useful to speed up the view when you have more than a handful of domains that map into a network. An example would be Google Apps hosted mail, which may have hundreds or more domains that could slow down the reporting speed.
For Steve
Q: I've isolated my problem to be Office 365. Now what? How do I improve my delivery? Can you call them?
A: Great question. Microsoft is an interesting network to troubleshoot. Often times we see delivery failures tied to elevated complaint rates and low engagement. In order to get to the root cause of the issue, we'd recommend you start by looking at Microsoft’s Reputation tool. It’s called SNDS and that data may help you to determine if you're triggering high complaints or hitting traps. You should also look at the response rates for your audience at the O365 domains. If they are low, you may need to set aside your non-responders and attempt to re-engage them ONLY after you've improved your inbox rates. We can definitely support you with that effort. Generally that takes the form of a consulting engagement to ferret out the root cause of the issue, help you make program changes and attempt re-engagement once you’ve achieved better inbox rates.
For Carmi
Q: You said Marketo doesn’t remove invalid domains….is that right?
A: Not exactly. Marketo has a GREAT feature that allows YOU to customize your soft bounce logic to accommodate your own sending cadence and frequency. It’s less arbitrary than a “3 strikes your out” rule. In the case of an invalid domain, the BOUNCE response is a soft bounce that says “couldn’t connect to an MX at this time.” The shorthand version is it’s important to configure a policy that will listen for those bounces that are chronic and prune them over time, in accordance with your sending logic.
For Carmi
Q: Are their resources to help me solve my deliverability problem?
A: Absolutely! We have a wealth of deliverability content in the Community as well as a team of Deliverability experts whose charter is to help customers tackle the trickiest of deliverability issues.