At FireLogic, it’s a fact of life that we have to diagnose and fix issues with traditional mail servers on a fairly frequent basis. While we actively push most of our clients to move their email servers up to the cloud into stable, secure environments like Office 365, there are still those who choose to host their systems internally. These are the systems we tend to have the most issues with, as many of these platforms are aging systems like Exchange 2003 and 2007 which are nearing their EOL (End of Life) soon or in the no-so-distant future.
Luckily, there are some great tools available at our disposal which make troubleshooting email flow or connection problems a lot easier to make sense of. Keep in mind that all three of these tools I will go over can be used for cloud email platforms (Office 365, Google Apps, etc) or on-premise systems (Exchange 2007, Exchange 2013, Groupwise, etc) alike. And one tool in specific made by Microsoft which I cover below not only handles email problems, but also helps troubleshoot Outlook issues, Lync issues, and much more.
Let’s have a look at the short list of email tools we use at FireLogic to get down to the bottom of SMTP nightmares.
MXToolbox.com SMTP Diagnostics Tool
This is probably the first tool I go to anytime we are having email flow or server connection problems with Exchange systems onsite. I’m not saying cloud platforms don’t have issues, but in general, email flow problems are quantified with onsite servers because of all the links in the email flow chain. These include:
- The servers themselves. Exchange 2007 and older email platforms (in my experience) are not managed properly according to best practices, which leads to issues storage quota overloads, spam overloads, and other configuration problems that can destroy email flow and quality of service.
- Internet Service Providers. It goes without saying that ISPs treat internal mail servers with a raised eyebrow. Port 25 outgoing email blocks are fairly common these days, and many of them like Comcast go a step further by throttling or disabling all outbound email flow if you are suspected of having malware outbreaks at your office that are bombarding outsiders with spam.
- Spam Blacklist/Whitelist providers. A nasty fact of life for anyone that’s been on the receiving end of such a shunning is that many larger companies employ the use of spam filtering blacklists from the likes of Spamhaus to tell their internal systems whether to accept or reject email from other parties. Getting onto one of these lists can stop all mail from hitting specific outside parties that rely on these lists. It’s a nasty black hole to get into, but we clean these up for clients at least a few times a year.
- Incompatibilities between servers. Another possible, but less common, problem is that of two email servers that just can’t communicate with one another properly when exchanging mail. Cloud providers like Office 365 very rarely have this issue, but I do see this dealing with Exchange 2003 quite frequently these days. Especially when not configured properly, older Exchange systems trying to pass mail to rare species like GroupWise or Lotus systems can run into a bevy of problems, usually related to message coding or SMTP communication bugs.
As you can see, the number of kinks that can hit a delivery chain for email are numerous. This is why the MXToolbox.com SMTP Diagnostics test can really help you get a 10000ft view of your issues at a glance. All you need to run the test is your SMTP server address (in IP or domain name format) and you the tool will run a few important checks against your server in question.
Here is an example of the kind of readout you will get on your test:
For example, the above test I ran on a client server (an Exchange 2003 system being taken down in a few days) has two issues right now. First, it is suffering from an SMTP Reverse DNS mismatch which will affect mail flow on receiving servers that knock this item into their calculations of spam confidence levels. You can read up on SCLs from this excellent post on Microsoft’s TechNet repository, which goes over the specifics of how Microsoft’s systems attach Spam Confidence Level scores to each email that passes through an Exchange server.
Another problem the test brought to light is the fact that my SMTP transaction time on this server is quite bad. I know for a fact that this is due to old hardware, little memory, and also old Exchange software at this point, and since we are moving this client to Office 365, this will be a non-issue shortly. But if your email system is staying put and is seeing consistent 5 second+ times, you may need to look at what the deeper issue at hand likely is. While most legitimate email systems are willing to wait to pass along messages, not all servers are as lenient when it comes to waiting on your slow system.
This SMTP Diagnostics tool is probably the first one in my war chest which is used in situations where email flow is slow or broken between servers. It is not a black and white test by any means, in that its checks merely provide you with a guiding baseline of how your system is functioning and what direction you should be looking for resolution. This tool alone will not pinpoint specific problems for you to fix, but it’s great at giving you a quick thumbs or thumbs down on your system’s health and functionality.
The MXToolbox.com website also has a few other useful tools for IT pros. Their blacklist check tool is fairly decent at letting you know if your server has been blacklisted, and the tool which gave them their namesake, an MX lookup check, is probably the one which I use their site most often for. Validating your MX server records is extremely useful when migrating from on-premise servers to Office 365, something which I described at length in a previous article.
Microsoft Message Analyzer
Whoever says Microsoft doesn’t know how to make great tools, needs to have a look at the Microsoft Message Analyzer. It’s a fully browser based tool like the above one from MXToolbox, and it provides access to a host of excellent utilities like an Outlook connection troubleshooter, Lync troubleshooter, and much more. If you use anything in the Microsoft communications ecosystem including Outlook, Exchange, Lync, or OCS, you NEED to keep this website in your short list of items to check out when problems arise. It’s that good.
Getting into the Message Analyzer requires one extra click over to the last tab on this page, in which you are given an area to copy/paste your message header from any email in question. A message header is the gobbly gook code that explains everything that happened behind the scenes with any email ever sent. Who was the sending server, who was the receiving server, how much time it took for a message to pass between servers — etc etc. It’s the in-depth logbook of what happened to an email before you got it, and it helps troubleshoot all kinds of issues with email flow.
Indiana University put out a great how-to article on viewing headers of an email in almost any email client out there including Outlook Web App for Office 365 (which is what I use). And for Google Apps users, Google’s own help pages have information on how to view their headers.
Below is a sample copy/paste screen from a message header I copied from my own work inbox:
And after running the analysis, here is how the information of the results are displayed:
The most important aspects which could help in troubleshooting include the Submitting Host area that displayers respective receiving servers, and their associated transaction time on the right hand side under Delay. You can see which part of the email chain is giving the most slowdown in getting the message to its intended endpoint. In the above example email from Lenovo to myself, you can see that the message took a few hops between systems that Lenovo uses for email sending (likely for mass email sending) and then it comes into the Microsoft Office 365 datacenters, jumping from the first line of defence which is Exchange Online Protection spam/virus filtering and then passed over to the actual internal servers which host our company’s domain email accounts.
We can also find the Spam Confidence Level of any message analyzed, what language the message was encoded in, and what country sent the email originally. Using the timestamps of the oldest stamp and the newest stamp, you can calculate with ease how much time it took for the message to get to its intended endpoint. In this case, we had a 6 second transaction time from start to end, which is very respectable in the email industry. Slower systems that we work with, like Exchange 2003 boxes and GoDaddy email systems, the transaction times are upwards of 10 seconds plus, or worse. It all depends.
While yes, you could glean all of the same info from a given email without using the Microsoft Message Analyzer, the information presented in the form this tool spits out is understandable for any email admin or IT pro. Simply reading the header of an email ad-hoc in all plain text form is difficult and usually reserved for the best email IT pros out there. I can do it, but I choose not to, since this tool makes it easy to follow for myself – and also for my clients when I have to share results information.
Port 25 Solutions Email Verification Test
This is by far the easiest tool of them all to use. That’s because it requires no more effort than sending out a blank email to the following email address:
Be sure you do this from the domain experiencing receiving or sending issues, as sending this from a fully functional domain will not tell you anything about issues going on with a problematic domain!
The results are sent back to the email address used to send the test email, and they look something like this:
This test is rather basic compared to what the other services above provide, but it hones in on a few key areas related to your domain’s setup and utilization of spam fighting technologies like DKIM and SPF, to name a few.
However, keep in mind that other systems out there may have very strict spam fighting policies in place. A lot of email providers are starting to enable policies which require an SPF test to come in with a “pass” result on received email. And while it’s less common, providers may start enforcing DKIM checks as well on incoming messages.
The results of the Port 25 Email Verification test gives you a clear readout of how other systems may be interpreting email from your domain. It lists out all of the relevant checks that are done like SPF, DKIM, SenderID, DomainKeys, etc and also tells you what kind of result your message would have gotten in a regular correspondence situation.
On my above example of a message coming from my work email at FireLogic to Port 25’s testing tool service, I was able to find out that SPF and SenderID checks are coming in with all green “pass” marks, but DKIM and DomainKeys checks are only showing as “neutral.”
I know for a fact that the reason my DKIM and DomainKeys checks are coming in with sub-par results is because Office 365 doesn’t yet support either of those technologies. It has been stated numerous times in the Office 365 forums on posts like this by Microsoft support reps. It is claimed that support for DKIM will be coming, but we are not sure when yet. As soon as it is released, we will surely implement it for our own needs to help fight spam even better.
And again, while every receiving system doesn’t have the same anti-spam rules in place, you should probably use this in situations where you are getting email bouncebacks from third party servers which claim your message is on a blacklist or is violating other anti-spam rules. I’ve used it a few times in the last couple of months for Exchange 2003 systems that had a hard time getting mail to specific outside parties, and it was very helpful.
I’m hoping these three tools I described provide some insight as to the utilities our company uses in troubleshooting email flow problems, and how you can look into some of these problems on your own using the very same services. They are all free, work well, and have gotten us out of sticky situations in numerous instances. They all come highly recommended by FireLogic, so enjoy!