On May 16th we started experiencing mail flow issues on our unsupported Exchange 2007 environment hosted on Server 2003 OS’s. Mail was coming in over SMTP and being delivered to users as normal, however internal emails, and outgoing emails appeared to be going into $null. I could not see them in any Exchange logs, there was no trace of what was happening to them.
After some investigating, another engineer found two ports were blocked, 135 and 445 or RPC and MS Directory Services. The Windows firewall on all machines is disabled by default so we double checked that and moved on. First stop, our antivirus, we disabled and then completely uninstalled on what servers still had it running, wait how was AV missing on these servers? Next we checked our other fancy AI machine learning super advanced security application, Cylance, that was missing as well… hmm…
The workday had ended and myself and the other engineer were at home for the evening, I was VPN’ed in, determined to find what was blocking this traffic. I had seen a few event logs referencing IPsec and a GUID but I hadn’t thought much on those, I did a quick google search on IPsec rules and Server 2003, loaded up the IPsec Management Console and noticed there was a new rule created, last modified a few hours earlier, the rule was simply called ‘win’. Upon entering the rule I noticed there was a deny list, and sure enough, ports 135 and 445 were listed, unassigned the win IPsec rule, and mailflow started up again. I checked the 3 other 2003 servers, same rule, unassigned, mailflow issues resolved.
Also during investigation, in event logs, I found an MSI trying to install when the other admins were logging in. js.mykings.top was the name, and I found this in registry, scheduled tasks, and startup.