Hi Stephen! Thanks a lot for your previous mail. It was pretty much everything that I required to start.

I had a doubt regarding the signing in ARC-Seal that I hope you could clear.

According to the draft, here <https://tools.ietf.org/html/draft-andersen-arc-00#section-5.2>, the ARC-Seal header is constructed from the referenced message signatures given by the "k" tag. I didn't understand this part clearly. Do we need to specify the headers (i.e. the ARC-Message signature and ARC-authentication results) that we want to seal in the "k" tag just how we would use the "h" tag in the DKIM signature? (The usage of the "k" tag isn't clear to me.)

Thanks!

On Wed, Feb 10, 2016 at 3:19 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:

Hi, Aditya!

I've had a chance to read your mail and review the I-D (Internet Draft) and relevant RFCs a bit, and can now make a few comments.

First, your understanding is a little bit shallow. You should get yourself a "canvas" and draw a detailed flowchart of what's going on here. I don't mean to be harsh. There's an important (especially to you! ;-) principle here: you're volunteering for *too* *much* work. Another point is that you're not giving the I-D authors enough credit for caring about you and the end users. Homework: how do these principles apply in the comments below? :-)

Aditya Divekar writes:

...

When we use mailman, the mailing list service adds an extra phrase in the subject - [Mailman-Developers] and an extra footer in the mail giving links about the FAQ, archives and the security policy. This alters the original subject and the body of the mail that the sender sent in the first place. According to my knowledge, this is what might cause the mail to be rejected by yahoo, aol, or other p=reject policy domains.

Close enough for our purposes.

• If you get interested in doing more mail authentication stuff you'll need the more accurate story.

...

Thus implementing ARC would involve including the ARC authentication result header, the signature and the seal in every mail that Mailman receives before it forwards it in the mailing list.

Yes.

...

This would probably involve using the pydkim, gs.dmarc and pyspf libraries for verification before we generate the ARC authentication results.

Here's where you start doing too much work.

• You can (for GSoC) assume that the original authentication results should already be available.

  • For the common MTAs used with Mailman, the Authentication-Results (A-R) header should be available, it should have been added by *your* MTA (otherwise it was added by someone you shouldn't trust!), and you can detect that. (This may not be verifiable -- need to check, and it's another case to deal with later.) The ARC-Authentication-Results is a *copy* of this header (see 5.1.3 in the I-D). (This is the "DRY" principle at work.)

  • The A-R header already contains SPF and DKIM results, and maybe DMARC (that's cheap to check, though). Thus for verification you don't need to do any work! (First draft -- pragmatically, some Mailman sites may not implement A-R, and as mentioned you may not be able to trust A-R. That's why we're doing this anyway -- it's really an MTA function, but some sites won't implement.)

  • "For GSoC" is important -- as an extension it's desirable that you handle the case where they're *not* available. But that comes later.

• But you do absolutely need to sign things (ARC-Message-Signature, ARC-Seal) with new signatures, and that uses DKIM (modified somewhat, don't know yet if you'll need to modify package code).

Having implemented that much, you now know a lot about how the DKIM module (eg, pydkim) works.

• Learning DKIM verification is probably easier (the relevant header field should tell the module everything about what it needs to do!)

• SPF and DMARC are different modules, but again, you've learned a lot.

In general, sequence your plans to learn things that will make later work easier. (This tends to happen naturally as you do the work, but you can often buy 10%-25% by planning it in advance.)

...

As a starter I think I should understand how the dkim,dmarc and spf authentication processes are coded. could you tell me how to find existing code where I can read and understand how the authentication methods are implemented?

On PyPI. It's all open source (at least these modules will surely be -- IETF people are real sticklers for open source). If you need/want VCS checkouts, you can get them later. Here's complete (enough) list.

DKIM

dkimpy by Scott Kitterman = frequent DMARC/IETF poster, that's a +1 pydkim by Greg Hewgill = dunno authres by Scott Kitterman et al (A-R header up to RFC 7001, current is RFC 7601, may need work?), same author is good sign (he cares, also multiple modules in same author's style are easier to understand, and if you're lucky, they'll already have compatible APIs) BTW, you missed this one. :-) gs.dmarc by Michael JasonSmith = dunno emailprotectionslib by Alex DeFreese = dunno

DMARC

• same list as DKIM, basically

SPF

pypolicyd-spf by Scott Kitterman pyspf by Stuart Gathman = dunno sikwan.spfcheck by Francois Vanderkelen = dunno hydrate-spf by James Pearson = dunno python-slimta-spf by Ian Good = dunno pysrs by Stuart Gathman = second package, +0.5

• most of the DKIM packages

Where I wrote "dunno", I don't know the author offhand. I'd need to Google a bit to decide whether I trust him. In several cases, my guess is that they are not protocol hackers, but rather implementing a messaging system that may or may not have email at its center. Need extra attention to auditing accuracy of protocol implementation unless they're verified to be IETF contributors.

In any case, code needs to be reviewed for quality (accuracy of implementation -- high-quality email posts are no guarantee :-) -- and coding technique) and style (if a modified version is needed, you might need to distribute with Mailman).

Gotta go, but that should get you started.

Steve

Hi Stephen! Thanks a lot for your previous mail. It was pretty much everything that I required to start.

I had a doubt regarding the signing in ARC-Seal that I hope you could clear.

According to the draft, here <https://tools.ietf.org/html/draft-andersen-arc-00#section-5.2>, the ARC-Seal header is constructed from the referenced message signatures given by the "k" tag. I didn't understand this part clearly. Do we need to specify the headers (i.e. the ARC-Message signature and ARC-authentication results) that we want to seal in the "k" tag just how we would use the "h" tag in the DKIM signature? (The usage of the "k" tag isn't clear to me.)

Thanks!

On Wed, Feb 10, 2016 at 3:19 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:

Hi, Aditya!

I've had a chance to read your mail and review the I-D (Internet Draft) and relevant RFCs a bit, and can now make a few comments.

First, your understanding is a little bit shallow. You should get yourself a "canvas" and draw a detailed flowchart of what's going on here. I don't mean to be harsh. There's an important (especially to you! ;-) principle here: you're volunteering for *too* *much* work. Another point is that you're not giving the I-D authors enough credit for caring about you and the end users. Homework: how do these principles apply in the comments below? :-)

Aditya Divekar writes:

...

When we use mailman, the mailing list service adds an extra phrase in the subject - [Mailman-Developers] and an extra footer in the mail giving links about the FAQ, archives and the security policy. This alters the original subject and the body of the mail that the sender sent in the first place. According to my knowledge, this is what might cause the mail to be rejected by yahoo, aol, or other p=reject policy domains.

Close enough for our purposes.

• If you get interested in doing more mail authentication stuff you'll need the more accurate story.

...

Thus implementing ARC would involve including the ARC authentication result header, the signature and the seal in every mail that Mailman receives before it forwards it in the mailing list.

Yes.

...

This would probably involve using the pydkim, gs.dmarc and pyspf libraries for verification before we generate the ARC authentication results.

Here's where you start doing too much work.

• You can (for GSoC) assume that the original authentication results should already be available.

  • For the common MTAs used with Mailman, the Authentication-Results (A-R) header should be available, it should have been added by *your* MTA (otherwise it was added by someone you shouldn't trust!), and you can detect that. (This may not be verifiable -- need to check, and it's another case to deal with later.) The ARC-Authentication-Results is a *copy* of this header (see 5.1.3 in the I-D). (This is the "DRY" principle at work.)

  • The A-R header already contains SPF and DKIM results, and maybe DMARC (that's cheap to check, though). Thus for verification you don't need to do any work! (First draft -- pragmatically, some Mailman sites may not implement A-R, and as mentioned you may not be able to trust A-R. That's why we're doing this anyway -- it's really an MTA function, but some sites won't implement.)

  • "For GSoC" is important -- as an extension it's desirable that you handle the case where they're *not* available. But that comes later.

• But you do absolutely need to sign things (ARC-Message-Signature, ARC-Seal) with new signatures, and that uses DKIM (modified somewhat, don't know yet if you'll need to modify package code).

Having implemented that much, you now know a lot about how the DKIM module (eg, pydkim) works.

• Learning DKIM verification is probably easier (the relevant header field should tell the module everything about what it needs to do!)

• SPF and DMARC are different modules, but again, you've learned a lot.

In general, sequence your plans to learn things that will make later work easier. (This tends to happen naturally as you do the work, but you can often buy 10%-25% by planning it in advance.)

...

As a starter I think I should understand how the dkim,dmarc and spf authentication processes are coded. could you tell me how to find existing code where I can read and understand how the authentication methods are implemented?

On PyPI. It's all open source (at least these modules will surely be -- IETF people are real sticklers for open source). If you need/want VCS checkouts, you can get them later. Here's complete (enough) list.

DKIM

dkimpy by Scott Kitterman = frequent DMARC/IETF poster, that's a +1 pydkim by Greg Hewgill = dunno authres by Scott Kitterman et al (A-R header up to RFC 7001, current is RFC 7601, may need work?), same author is good sign (he cares, also multiple modules in same author's style are easier to understand, and if you're lucky, they'll already have compatible APIs) BTW, you missed this one. :-) gs.dmarc by Michael JasonSmith = dunno emailprotectionslib by Alex DeFreese = dunno

DMARC

• same list as DKIM, basically

SPF

pypolicyd-spf by Scott Kitterman pyspf by Stuart Gathman = dunno sikwan.spfcheck by Francois Vanderkelen = dunno hydrate-spf by James Pearson = dunno python-slimta-spf by Ian Good = dunno pysrs by Stuart Gathman = second package, +0.5

• most of the DKIM packages

Where I wrote "dunno", I don't know the author offhand. I'd need to Google a bit to decide whether I trust him. In several cases, my guess is that they are not protocol hackers, but rather implementing a messaging system that may or may not have email at its center. Need extra attention to auditing accuracy of protocol implementation unless they're verified to be IETF contributors.

In any case, code needs to be reviewed for quality (accuracy of implementation -- high-quality email posts are no guarantee :-) -- and coding technique) and style (if a modified version is needed, you might need to distribute with Mailman).

Gotta go, but that should get you started.

Steve