Blog , 10 May 2021

Abusix Mail Intelligence – Short URL Hash List

Steve Freegard,

Sr. Product Owner Abusix Intelligence

Welcome to Part 7 of our series of blog posts dissecting each of the datasets available as part of Abusix Mail Intelligence. This time we’re looking at our Short URL Hash list

How the Short URL Hash List is being built:

This list is completely automated and is being built from messages hitting our main trap network. For frequently abused domains (e.g. those commonly used for spam, phishing and malware) or for free domains, we look for these domains in our entire trap network.

What is this Short URL Hash List designed for:

Short URLs are a major problem. You have absolutely no idea where they are going to take you prior to clicking them and the services themselves are absolutely useless at taking abusive links down. We can’t list the shortener domain itself as that would cause lots of false-positives.

The spammers, phishers and malware authors have turned to URL shorteners for these reasons and because it makes it much harder to filter the messages.   

To determine where a short link points, you have to make an external web connection to that shortener service.  This doesn’t scale if you handle anything more than a handful of messages per minute, so it’s difficult for a spam filter to do this in real-time.

We created the Short URL hash list to provide immediately actionable intelligence that scales to hundreds of messages per second.  As soon as your spam filter sees what looks like a short URL, it can check to see if we’ve seen that short URL in spam.

Abusix Mail Intelligence is the only commercial mail reputation provider to produce this data, however because we were the first to offer this, it will take time for the spam filter vendors to add support for it. At the time of writing only rspamd supports these lookups.

Reasons for being listed on the Short URL Hash List:

Common reasons for being listed in the Short URL Hash List:

  • Spam, phishing and malware.
  • Compromised websites are abused to host redirectors, phishing, or malware pages, and shortened links to these sites are then used.

Hope that is useful.

Until next time – stay safe.

Steve

Share
Linkedin Icon Twitter Icon Facebook Icon E-mal Icon
Two computer monitors showing AbuseHQ

Start Your Free Trial

Let's protect your mail servers with a reliable blocklist! 14-day free trial! No credit card details needed!

Get started

Products & Tools

Type

Topic

Get in touch

Talk to us

The quickest way to get in touch with the team is via our online chat feature at the bottom right of this page. Alternatively, feel free to email us at [email protected] or send us a message via our form.

Is your IP blocked?
To get that resolved, please use our lookup-service and follow the instructions in order to delist your IP/domain.