(Editor’s Note: This blog article was updated on March 06, 2024)
Are you a Sender (a marketer) using an Email Service Provider (ESP), or do you want to learn who is responsible for blocking malicious traffic, such as spam, malware, and other email attacks? Let’s understand who blocks email and how spam filtering and reporting work.
What is an Email Service Provider?
An Email Service Provider is a company that provides email marketing or bulk email service infrastructure to send marketing messages. An ESP’s service will include email templating and statistics tools to measure recipient engagement. Additionally, they will provide automated handling for bounced emails and recipients who unsubscribe from Sender email campaigns. Companies that are ESPs include Constant Contact, Hubspot, Mailchimp, SparkPost, and many others.
Who are the Mailbox Providers?
Conversely, to ESPs, Mailbox Providers provide infrastructure to consumers and businesses to receive messages and send personal emails. Mailbox Providers include Apple, Google’s Gmail, Yahoo, Telcos, and Mailbox Providers around the globe, as well as several security companies like Barracuda, Cisco, Fortinet, McAfee, Mimecast, SonicWall, Vade Secure, and others.
Mailbox providers are typically referred to as “receivers.” The individual receiving an email is generally called a “recipient.”
Who is responsible for blocking spam?
Receivers block inbound spam.
Mailbox Providers frequently use various techniques to block spam to stop spam, phishing, malware, and other harmful content – many of which come from compromised systems, open relays, proxies, and other vulnerabilities.
Are ESPs responsible for blocking emails?
Email service providers (ESPs) often also block email but may not do it enough. There is also a common misconception that only the email list owner (Sender) is responsible for reputation. Unfortunately, the Senders get ESPs in trouble if the ESPs are not proactive in preventing abuse on their platform. If an ESP blocks you, don’t get upset with them; they act in all their customers’ and your best interests. Ultimately, they are assuring not only your inbox delivery but everyone else.
Two things to be aware of are
- Reputation is gained or lost at Mail Providers with each email you (and your ESP) send. List hygiene helps everyone. So, blocking early by an ESP is far better for you than you unknowingly sending lots of emails that trash your domain (and brand) than having a regular Mailbox Provider bulk folder for you because that can be a complex problem unwind.
- Some blocklists (not Abusix) will also block groups of IP addresses due to a continuing problem on a single IP address, thus catching other unrelated Senders. So, blocking early by an ESP is far better than having one of these blocklists escalate their blocking.
In the end, professional ESPs do not allow abuse on their platform because they care about you as a long-term customer and because it causes deliverability issues for other customers. So, it is better to work with your ESP and follow their guidance than to disregard their deliverability guidance.
What is email spam filtering?
Spam filters are a stack of decisions made by a receiving mail server; they aren’t one thing. They include three (sometimes more) primary filtering stages.
- In the connection where IP addresses are checked (TCP Connection), Abusix blocks more than 99% of all spam on this level.
- The SMTP session where domains and message continuity are validated
- Content filtering where URLs and attachments are checked.
How do email spam filters work?
Let’s dive into how spam filtering works in more detail. A TCP connection with the recipient’s email server is the first thing that happens when an email is sent.
The TCP connection is where most spam and malicious emails are filtered from email streams by rejecting IP addresses that are misconfigured, infected, or observed recently sending emails to spam traps. Suppose the receiver uses a full-featured blocklist, like the Abusix Mail Intelligence Combined IP List, for their DNSBL. In that case, they will reject better than 99% of spam, phishing, and malware at this first stage, so IP blocklisting is a critical component of spam protection.
If an IP address is blocklisted, you need to look at your ESP to solve the problem at their end unless you are on a dedicated server if you are on a dedicated server, as the Sender.
During the SMTP session, several additional checks are made.
- A full-featured DNS blocklist (DNSBL) like Abusix Mail Intelligence with a domain list will be checked for domains associated with spam. A reputable blocklist provider won’t block an ESP’s IP address for spam during the TCP connection, but they ounce/block individual Sender domains for malicious activity during the SMTP session.
- The receiving server will also check rDNS (Reverse DNS) to ensure your ESP’s mail server is configured correctly. While this is rarely a problem at professional ESPs, it is a common problem with self-hosted mail servers in general. If the email fails the Domain blocklist check or the RDNS check, or there is no recipient, the email will be bounced back to the server that sent the email.
- The return (email) address’s domain and sending IP address are checked in the SPF (Sender Policy Framework) DNS text record to ensure that the domain and IP address claimed in the envelope of the email (used when an email gets bounced), are authorized to send on behalf of the domain owner.
- A second email security authentication check called DKIM (Domain Keys Identified Mail) is a second email security verification check. This authentication protocol further reinforces your identity and detects if messages are modified in transit and man-in-the-middle attacks, often used for phishing attacks. If the email fails SPF or DKIM checks, the receiver will send the failed email to the address the Sender includes in their DMARC record, a DNS text record. The email of record for DMARC is critical since the trigger is either where the Sender’s identity has been forged (aka spoofed) or tampered with by a man-in-the-middle attack.
- The final check in this part of the receipt process ensures that each recipient’s address is valid, not for a closed or non-existent account. Generally, these are also bounced, but the server may accept all corporate email destinations. This is important, as you should remove all non-opens after a while so that the security provider protecting the corporate destination does not measure your reputation by the number of emails you send to non-existent addresses.
Whew! That’s a lot of checks to ensure the email originates from a legitimate Sender. They are all necessary since email remains a consumer or business’s most vulnerable security weakness.
Failures on the domain level, except for the RDNS check, are the responsibility of the Sender (domain owner), not the ESP. But that said, good ESPs always have well-staffed deliverability teams to guide you.
- If you are on a domain DNSBL, the ESP may suggest you remove all non-opens or reconfirm your list.
- If SPF, DKIM, or DMARC need to be fixed, the ESP will point you to a DMARC provider.
- Suppose the Mailbox Provider is spam foldering all your emails, and they don’t bounce (sometimes called the silent bounce). In that case, the problem is that either your domain or short URL is associated with spam, too many non-existent users, low interaction with your emails, or a high complaint rate.
Regardless, all these issues originate with the Sender (you) and not your ESP. Fixing a domain block requires good list hygiene, which is up to you, the Sender.
The good news is that any reputable ESP will block Senders from sending emails to Mailbox Providers if your SPF record or DKIM configuration is broken. If they are broken, the Sender is a risk not only to the recipient but also to the recipient in fake emails that appear to come from the Sender (e.g., spoofing).
ESPs during the sending process also ensure Senders do not exceed a bounce threshold and keep them from doing more damage to themselves than they intend.
Did the ESP and Sender make it through the gauntlet of basic checks?
Once the Sender’s message has been accepted during the SMTP process, the connection will be closed, and the email and all its attachments will be transferred to the content filter. The content filter or parts of the content filter, like the anti-virus filter, are called anti-spam software because the software is added to a mail server, often an open-source platform like PostFix or Exim.
The content filter also contains several steps.
- First, it will check the Sender’s message body for various things.
- Then, the filter will check if the content is similar to other spam, like ensuring the Sender is not a stranded Nigerian prince (otherwise known as a 419 scam) or other types of common spam.
- The content filter will check URLs against a full-featured DNSBL like Abusix for domains, short URLs, and document storage URLs (like Google Docs, OneDrive, Dropbox, Box, and others) often associated with spam and phishing. This check may also use Cyren and VadeSecure to check URLs.
- An AntiVirus (A/V) scan checks the attachment. The AV scan is essential to protect against malware and enforce additional actions for specific files that might change settings. AV scans are critical to protecting against zero-day attacks from Senders with no history or those with a low Sender reputation.
Finally, Mailbox Providers also log all hard bounces by legitimate Senders and suspected spam email Senders. So even though the Sener might have made it past the TCP connection, SMTP session, and content checks, bounce rates may be used to determine the ESPs and Sender’s email-sending reputation. Too many hard bounces will trigger the placement of all legitimate Sender’s emails in the spam filter.
If you want to read more about spam protection, look at our article “The more filters, the better.”
DNSBLs and different types of spam filters
For inbound email, Mailbox Providers are using the following Abusix DNSBLs. Sometimes, lists are used more than once, as different parts of the message may be checked at other times. While some configurations are similar, some are different. If you are an ESP, we suggest that if they are filtering with these DNSBLs, you should, too!
- TCP Connection
- SMTP Session
- Abusix’s Domain List
- Abusix’s Newly Observed Domain List
- Abusix’s Spam Blocklist (a component of Combined, checking all headers) ****
- Abusix’s Exploit Blocklist (a component of Combined, checking all headers)
- Content Filter
- Abusix’s Domain List
- Abusix’s Newly Observed Domain List
- Abusix’s Short Hash (Short URLs found in spam)
- Abusix’s Disk Hash (Drive Storage URLs found in spam)
Email Spam Reporting
How does it work?
When recipients receive messages they do not want, they can click the “This is spam” button. Once a recipient does this, the email and subsequent emails from that Sender are placed in that user’s spam folder.
In some cases, the Mailbox Providers also operate a feedback loop (FBL), which allows them to return complaints in the MARF (Mail Abuse Reporting Format). In the past, to receive FBL messages, an ESP or Direct Sender had to be subscribed to a specific FBL service. However, moving forward, ESPs will need to follow RFC 9477 and either add a DKIM-aligned CFBL-Address: header to all emails sent or implement the Internet-Draft, which requires additional DNS TXT records to be configured. You can do both for maximum compatibility, as this will not cause any issues. When requesting FBLs, the Mailbox Provider expects the trusted email service provider to unsubscribe each recipient from the Sender lists.
For more about FBLs, see the article “Mastering Complaint Feedback Loops, A guide for Email Senders”
What you should know about email deliverability in 2024
When discussing deliverability, we know what this is about – every marketer likes to get their message delivered to the inbox. But, from a blocklist provider’s perspective, there are some that marketers should know to keep them off of blocklists. So here are some tips:
- Keep in mind that if an email is unexpected, it will likely end up in the spam folder. This ensures deliverability and builds your relationship with a subscriber from the beginning of the subscription process.
- Make sure the reputation for your domain is your own. Do not let scammers use your domain and damage your reputation.
- As a Sender, certified by a third party, like the Certified Sender Alliance (CSA) Sparkpost a Message Bird Company or Validity. Sender accreditation services verify that email Senders follow best practice email practices. Remember that you’re sending to a real person, NOT a mailbox.
- Hard bounce from an invalid, closed, or non-existent email address; these emails will never be successfully delivered. Therefore, permanently remove them immediately from your lists.
- Legally renting or purchasing lists of people who have agreed to email communications or scraping websites for email addresses is a fast way to build a contact list. Still, it is a horrible and illegal idea.
- If you change your domain, tell your subscribers in advance, or many subscribers will click on this as spam.
- Constantly monitor your ESP’s IP addresses, domain reputation, and key performance indicators (KPIs) for your email, like bounce rates, unsubscribe rates, and unengaged recipient rates.
- Expired email addresses can become unknown users and cause bounces, increasing the likelihood that your email won’t reach the recipient’s inboxes but will be placed in the spam folder. Remember, expired email addresses can become spam traps.
Always remember that responsible and legitimate email marketers regularly read up on email laws, email standards, Mailbox Provider behavior, and spam filter technology. Staying in the know will help ensure you, the Sender, always follow best practices.
You can also watch our video on YouTube titled “How to Stay Off Email Blocklists.“
