Rogue SMTP on My Network: The Night My Grafana Dashboard Made My Heart Stop

A routine look at my Grafana DMARC dashboard turned into a full-blown rogue-SMTP mystery. DKIM fails, rising outbound mail… hacked? Docker container gone rogue? The truth was far stranger. Here’s how I hunted it down, built an SMTP honeypot, and solved it.

Rogue SMTP on My Network: The Night My Grafana Dashboard Made My Heart Stop
Photo by Mason B. / Unsplash

A tale of DMARC doom, Docker demons, and a Ghost that wouldn’t stop emailing.

If you want to test your stress tolerance, forget bungee jumping or stock trading. Just self-host your email server… and then open your Grafana DMARC dashboard one morning.

That’s exactly how my day started.

I’m sipping coffee, feeling very pleased with myself because my on-prem monitoring—Grafana, ElasticSearch, the usual orchestra of blinking lights—is running beautifully.

Then Grafana lights up with yellow where it oughts to be green:

Outgoing emails: SPF PASS… DKIM FAIL. Upwards trend.
Multiple domains sending DMARC aggregate reports.

Wait… what?

SPF pass?
DKIM fail?
Outgoing emails… I didn’t send?

Congratulations. You are now in the Premier League of Sysadmin Panic™.


🚨 Chapter 1: The Freakout — “Have I Been Hacked?”

The DMARC dashboard is showing what looks suspiciously like outbound spam. And not just any spam—the kind that passes SPF but fails DKIM. The kind that still technically comes from me.

For context:
I run my own mail server. Tight setup. DKIM signed. SPF rock-solid. DMARC strict. The kind of configuration that would make Email Police nod in approval.

So if something is sending mail from my IP without DKIM signatures…
It’s not my mail server.
Therefore: something else is sending mail.

And that something else might be a compromised container, rogue script, hijacked process, or—God forbid—some malware zombie running in my stack.

Or maybe I’m just tired and hallucinating.
Nope. DMARC reports from half the internet don’t hallucinate.


📡 Chapter 2: Into the Firewall — Mikrotik, Show Me Your Sins

So I go into Mikrotik Firewall, open the NAT/Filter rules, and start digging.

We add rules.
We add logs.
We add logs for the logs, because this is how you cope when panic mode activates.

Finally, we catch something:

SMTP traffic from my host IP.

But this is where it gets interesting:
my mail server is clean. Its logs show nothing unusual, no sudden bursts of outgoing messages, nothing out of the ordinary.

But that IP…
That IP is also where a dozen Docker containers live. Ghost blogging engine, n8n automations, Supabase, ntfy, tiny one-off webhooks… basically a nerd’s garden of little services humming along.

So something on that machine is levering port 25 directly.


🔍 Chapter 3: watch-rogue-emails.sh — Automating Panic, One Netstat at a Time

When something in your network starts sending mail you didn’t authorize, panic is inevitable—but in my case, panic comes with a bash script.

I needed visibility.
I needed answers.
I needed something that would sit quietly in the corner, watching every packet like a medieval scribe documenting the plague.

So I wrote:

watch-rogue-emails.sh

A tiny shell script with a big job:
watch every TCP connection on ports 25, 465, and 587 and tell me exactly which process spawned it.

The script does three things brilliantly:

  1. Continuously runs netstat, filtering for SMTP portsnetstat -ntp | grep -E ':(25|465|587)\b'
  2. Logs every new connection, timestamped, into my Synology storage
  3. Extracts PIDs from the netstat output, checks /proc/<pid>/cmdline, and prints what process actually opened the socket

Every time there was a new SMTP connection, the script would write:

===== 2025-11-18 13:37:42 =====
tcp 0 0 192.168.x.x:49832 18.65.x.x:25 ESTABLISHED pid=1234/smtpd

If a process had touched port 25, this script would catch it in the act, pants down, SMTP packet in hand.

At least… that was the plan.

Because after running the watcher for hours, I got a strange result:

Nothing except my legitimate mail server processes.
No other host-level processes opening port 25.
No PIDs.
No smoking gun.

Which led to a chilling realization:

If the host isn’t opening connections… the container is opening them directly via Docker’s NAT stack.

Meaning the packets never came through a traditional host-level socket.
Meaning netstat never saw them.
Meaning my nice, polite, bash script was staring at the OS while the real culprit was quietly slipping packets through Docker’s internal networking like a ninja mailing postcards.

It wasn’t a dead end—far from it.
It was the clue that changed the whole investigation.

If the host didn’t log it, but the router did, the only remaining explanation was:

A Docker container was sending mail straight out to the internet.

The watcher didn’t find the culprit.
But it narrowed the suspects.

And the hunt continued.


📦 Chapter 4: Building an SMTP Honeypot — Because If You Can't Find the Emails, Catch Them

I needed proof.

I needed to see what was actually being sent.

So I built something quick and slightly ridiculous:

Python SMTP Sinkhole Service™

A tiny script that:

  • Pretends to be an SMTP relay
  • Accepts all incoming mail
  • Logs everything
  • Stores every message for forensic analysis
  • Doesn’t actually forward anything
    (because I’m trying NOT to become a spammer)

Then I changed my firewall DNAT with a masquarading rule:
every outgoing SMTP connection gets quietly redirected into my Python honeypot.

If something tries to send mail… it’ll end up talking to my trap instead.

And oh boy… something talked.


📨 Chapter 5: The Discovery — When You Blame Hackers but It Was Actually Ghost

I open the honeypot logs.

I see the messages.

The From headers.
The To addresses.
The email templates.

And I see it:

“Please confirm your email address to complete your sign-up.”
— Ghost CMS

I just stared at it for a moment, a cocktail of relief and annoyance swirling gently in my bloodstream.

It wasn’t malware.
It wasn’t a botnet.
It wasn’t an intruder who breached my stack.

It was my Ghost website.

Or more precisely:
Bots spamming the Sign-Up feature on Ghost, providing tons of fake email addresses.

Ghost was diligently trying to send confirmation emails…
using its own internal SMTP client, completely bypassing my proper DKIM-configured mail server.

Thus:

  • SPF: PASS (because the IP matches my domain)
  • DKIM: FAIL (because Ghost isn’t signing mail)
  • DMARC: ANGRY

My MX reputation?
Rapidly circling the drain.


🛠 Chapter 6: Fixing the Chaos

Two fixes:

1. Fix Ghost’s SMTP config

I pointed Ghost to my actual mail server instead of letting it send mail directly on port 25.

Now emails are signed with DKIM and pass DMARC.
No more rouge spam-looking junk.

2. Disable Ghost Member Sign-ups

Because honestly?
I can’t be bothered.
I don’t want random bots turning my domain into an accidental newsletter machine.

Sign-ups disabled.
Problem solved.
MX reputation saved.
Sanity restored.


🎬 Chapter 7: Epilogue — Lessons Learned

What I thought was a hack turned out to be a CMS feature being abused by bored internet bots.

But I learned a few things along the way:

  • DMARC reports are priceless — boring until they aren’t.
  • Self-hosting email is a lifestyle — equal parts pride and pain.
  • Docker container monitoring is sneaky — outgoing traffic doesn’t always go through the host.
  • Honeypots are amazing — nothing beats seeing the raw SMTP payload from your suspect.
  • Ghost needs love — or at least a correct SMTP configuration.
  • Bots will sign up for anything if it has a text box.

And most importantly:

Never assume you're hacked until you’ve proven you’re stupid.

On this occasion, I wasn’t hacked.
Just haunted by a Ghost with an overenthusiastic send button.