DMARC, DKIM, and SPF oh my!
Fri Jul 28, 2023 3:53pm

Just hopefully finished a battle with the mailing list server failing outbound email's DKIM check due to:
dkim=neutral (body hash did not verify)

That means if my SPF (what servers I marked as allowed to send emails for my domain) also failed for some reason, DMARC would fail which would either move the messages directly to spam or outright rejected on the recipients mailbox. Then, on top of that, now I'm getting sent to spam which hurts the domain score overall regardless of these email spam checks.

I was able to find the root cause of my DKIM failure. That error message means that my email body was modified by one of the servers from when I mailed it to when it was received.

It turns out, it was getting modified (most likely) by my host SMTP server because the email body didn't match up to RFC 3676 standard which states:
If faced
   with such a word which exceeds 78 characters (but less than 998
   characters, the [SMTP] limit on line length), the agent SHOULD send
   the word as is and exceed the 78-character limit on line length.

   A generating agent SHOULD:

   o  Ensure all lines (fixed and flowed) are 78 characters or fewer in
      length, counting any trailing space as well as a space added as
      stuffing, but not counting the CRLF, unless a word by itself
      exceeds 78 characters.

   o  Trim spaces before user-inserted hard line breaks.

So, simple enough right? I just have to make sure the lines of my body don't exceed 78 characters. So I do that and it works 'somewhat'. If the message body is HTML, I had to add extra checks not to introduce line breaks at the start of tags (ex: '<a' ). So I added that as well and things looked good again.

That is... until I started messing with unicode characters in the subject or body. This completely blew up my hacky line splitting because unicode chars are 2bytes instead of one. So I ended up accidentally splitting the unicode character in half sometimes.

After fighting a bit to get the unicode characters to display right, I started failing the DKIM check again because now the body hash wasn't matching again.

*Banging head on wall*

Long story short, I ended up switching from using the Email::Simple module which lets you create the raw email data to a much more friendly Email::Stuffer module. This automatically fixes the line breaks, escapes characters correctly, and sets the appropriate header values. Very similar to what I'm using in the Java code.

While I eventually probably would have gotten it all working correctly. It wasn't worth re-inventing the wheel as possibly an octagon with a bunch of rough edges and shaky craftsmanship.

Who knew email stuff was so complicated!

    "Forces act when not restrained" - Puckdropper