Manually evaluating the effectiveness of a DNSBL or blocklist can be difficult, time-consuming, and error-prone. You need to have a proper understanding of how your SMTP server works.

Here are some questions you’d need to have an answer to if you want to run a manual comparison:

• Does my SMTP server query DNSBLs in parallel or serially?

• Does it take the first positive result, or does it wait until all of the lists have been checked? 
• How are the DNSBL results logged?
• Can the DNSBL results be easily parsed from the logs?

To make this whole process easier, we’ve written a comparison tool that you can use to evaluate our Abusix Mail Intelligence blocklist and compare it against any other DNSBL to see what sort of results you would see if you were to use it. 

How does the comparison tool work:

The Abusix Mail Intelligence comparison tool works by querying both the Abusix Mail Intelligence blocklist and the comparison list of any other DNSBL in parallel and records the results from both lists.

Because Abusix Mail Intelligence also includes a welcome list (previously known as whitelist), we query this too, so we can report the number of potential false positives seen on the comparison list.

The Abusix Mail Intelligence comparison tool doesn’t require any installation, as it is provided as a Linux binary. All you need is your API/query key which you can get from your dashboard in our portal, where you would need to create a free account. 

Then it can be run on a Linux host that is able to query the DNS namespace of the list you wish to compare Abusix Mail Intelligence with.

The comparison tool only works with IPs, it does not support domains or hash lists. The reason for this is because domains and hash lists are much harder to compare as there are lots of different ways that these lists work and how they need to be queried. It would make it error-prone to set up and difficult to do a fair comparison.

How to access the comparison tool:

To access the comparison tool, you’ll need to login into your Abusix account or create a new one. Navigate to the “Email Protection” section on the left navigation.

Click “Not sure if our blocklist works best for you?” and then download one of the scripts and follow the instructions within our documentation page.

How to run the comparison tool:

There are two ways that you can run the comparison. One is in batch mode and the other one is real-time mode.

Batch mode:

In batch mode, you extract a list of IP addresses from recent log files (no older than two days) and then make this list unique (you can include the unique counts too).

Then you run the comparison tool across this list of addresses. It will query both lists at once and output a CSV file that contains the IP, count of hits (if provided), reverse DNS, welcome list flag, DNSWL flag, result(s) from Abusix Mail Intelligence and results from the comparison list along with a statistical summary of the results.

The output CSV file allows you to load the results into a spreadsheet which you can then use the “Auto Filter” function to filter the results to show the unique hits between Abusix Mail Intelligence and the comparison list and to be able to look for any false positives in either.

Batch mode is very fast as it can use aggregated counts or a deduplicated list of IPs, so even very large sites can get results within a few hours. However, it does not give you a good indication of what would happen if you were to run the same data in real-time as it won’t take listing latency into consideration (e.g. which listed a given IP first and what effect that might have).

Real-time mode:

To address these shortcomings we created the “real-time” mode. This allows you to run the comparison tool in real-time against a stream of IP addresses. 

This stream of addresses will need to come from a centralized log of some sort and the connecting IP address extracted from each connection and output, one per line and written to the UNIX standard output, this is then piped to the comparison tool which then reads and queries them.

In this mode, the statistics are output every 10 seconds, but to save considerable disk space, the CSV output will only contain IPs queried and found to be listed in the compared list but not in Abusix Mail Intelligence.

Realtime mode will run continuously until it is terminated with a Ctrl+C at which point the summary will be written to the screen and the CSV file will be closed.

So, real-time mode or batch mode?

Real-time mode is more resource-intensive and requires centralized logging, whilst batch mode doesn’t provide a perfect indication of the results as the lookups are being done after the event. 

Generally, we’d recommend running the comparison tool within your evaluation phase. If you are currently trialing Abusix Mail Intelligence to see if our suite of blocklists is for you, we recommend doing the comparison on a weekday, as the weekends always mean much lower genuine messages. The comparison only needs to be run once, so it’s not taking a lot of your time. If needed, our support team is happy to assist and give you more guidance on how to analyze the data.