New plugin for bounce validation

[lnedry]

I wrote a plugin that I would like to contribute to Haraka. It adds a unique header to outbound emails which the bounce plugin can then use to verify that a bounce is legitimate. The header is an MD5 hash of the From, Date, and Message-ID headers plus a secret phrase.
https://github.com/lnedry/haraka-plugin-null_hash

If we can get this added to Haraka, I will create a PR for the bounce plugin to check for this header.

I've been using this plugin in my production servers for about six months without any issues.

[DoobleD]

Interesting addition, thanks! We have been fighting fake bounces as well with custom plugins of our own.

[msimerson]

I like this, it seems like a good idea.

About the header name, why did you settle on X-Null-Hash?

Im mostly asking because RFC 6648 wants to deprecate using X- headers. I'm thinking something more like X-Haraka-Null-Hash could be a better netizen choice.

[msimerson]

Another question: could this be useful apart from bounce validation? If not, why not add it as a feature to the bounce plugin?

[lnedry]

My plan was to add it to the bounce plugin.
Do you have some ideas about how this might be useful apart from bounce validation?

[msimerson]

My plan was to add it to the bounce plugin.

👍

Do you have some ideas about how this might be useful apart from bounce validation?

I didn't, until you asked. The known-senders plugin uses FCrDNS, SPF, DKIM, and TLS names as forms of domain authentication. A cryptographically secure hash that a haraka instance adds could be another. To avoid replay attacks, including a timestamp as you've done is needed. But the Date header timestamp would be invalidated in every type of reply except a bounce, so that's a limitation. There's likely some additional use, but I'd say ignore all that for now. We don't want perfect to stand in the way of better, and this certainly improves on bounce validation.

[analogic]

I had the same problem. My idea was to create a plugin header_storage that can store any data in an encrypted header (aes-256-cbc). It is useful for authenticating the email source, or storing internal metadata like counters to prevent internal loops.
Also, please don't use MD5, it's been broken for a long time...

[lnedry]

What do you think about using HMAC-SHA256 instead?

[analogic]

that should be ok by today's standards...

[msimerson]

At the risk if bike shedding, does anyone else have thoughts about the name Null Hash? (If nobody else pipes up, then I defer to @lnedry's choice) The term null hash conveys nothing at all to me. If I were reading through email headers, I'd have no clue what a null hash might be used for. A web search doesn't help. One argument FOR it, is that it seems unique, so it might end up pointing back at the Haraka bounce plugin. But then, why not just give it a name like X-Haraka-Bounce-*, which at least conveys what is originating it. RTFM can do the rest of the work.

Potential Names

• X-Null-Hash
• X-Haraka-Null-Hash
• X-Haraka-Bounce-Hash
• X-Haraka-Bounce-Validation-Hash
• X-Haraka-Bounce-Validation
• X-Haraka-BV
• X-Haraka-BV-Hash

I include BV only because I've seen it used in RFC conversations on email working groups, so there is a precedence for it.

[lnedry]

FWIW, I prefer adding it to the existing bounce plugin.

How would that work? Is there an easy way to detect if an email is inbound or outbound?

[msimerson]

Doesn't it work exactly the same way within the bounce plugin as when it's a separate plugin (eg, running on hook_data_post)? The only difference might be that in your plugin, you name the function hook_data_post, but when integrated with the bounce plugin, you give it a better non-special name such as add_bounce_validation_hash and then register that function with a line like this in the register() function:

  this.register_hook('data_post', 'add_bounce_validation_hash')

[lnedry]

I was planning to update the bounce plugin to check for a validation header.

If you are suggesting merging these two plugins, my initial concern is that they serve different purposes and operate on different email flows. Merging may lead to unnecessary complexity and performance issues, as each plugin has code that isn't necessary for both inbound and outbound processing. It seems that it would be simpler to keep them as separate plugins but I'm open to suggestions.

Of course, I may be overlooking something obvious.

[msimerson]

my initial concern is that they serve different purposes and operate on different email flows. Merging may lead to unnecessary complexity and performance issues, as each plugin has code that isn't necessary for both inbound and outbound processing.

This is really only a packaging issue. One hook will be adding the hash, another hook will be checking for it, right? Whether those hooks exist in one file (plugin) or two files (plugins) makes no difference at all to Haraka. Whether the config settings exist in one file or two makes little difference to Haraka. Since The Computer[TM] doesn't really care, we then optimize for the humans.

Is it easier to reason about, maintain, and document the dependencies if all of it is in the bounce plugin, or if it's spread across more? Haraka initially had many tiny little plugins that did one tiny thing on some aspect of the SMTP connection. Prime examples were the access and helo-* plugins. They were all related but understanding how they fit together and which one you wanted, and how that one interacted with the other ones was devilishly difficult. Combining those many itty bitty plugins into "this plugin has all the fcrdns/bounce/ACL stuff" changed almost nothing at all from Haraka's perspective, but it sure made it much easier for humans to find (discoverability) and reason (write/update/maintain) about.

[DoobleD]

I agree with @msimerson. To me it would make more sense to have both hooks added to the existing bounce plugin. Both are actions taken to improve how bounces are managed, and they are very much linked to each other. Having them sit on separate plugins would make it harder to follow.

[lnedry]

Thanks for all of the suggestions! I'm working on a PR for the bounce plugin, merging my code.

[lnedry]

If the secret phrase is set to the default, should I log a warning? And maybe skip adding the verification header?

[msimerson]

Don't have a default secret. Instead have a secret setting with instructions for generating a secure secret.

[lnedry]

Is there an easy way to detect if an email is inbound or outbound? I don't want to add a hash header to an inbound email. Nor do I want to validate an outbound mail.

Is checking connection.relaying good enough?

[msimerson]

Kinda.

Classically, a Haraka email is considered outbound if the connection has relaying set. That's a bit naive though, because messages sent from a local sender to a local recipient would have relaying set and most folks wouldn't think of those as "outbound." In practice, I tend to think of inbound emails as messages that I have specific routing information for, versus everything else that is routed by MX.

[lnedry]

Just to clarify, if relaying is set, then I need to add a verification header. If it is not set, and the message is a bounce, then I need to check for a verification header in the body of the message.

[msimerson]

Correct.

[lnedry]

I'm almost done with merging my code with the bounce plugin but I've found a few things that are questionable. Instead of creating a bounce PR for all of my changes, I'll first make a couple small PRs to make sure these minor fixes are acceptable.

I guess at this point we can close this discussion.