HI Stephen, great to get in touch with you.

On Fri, Dec 1, 2017 at 2:21 AM, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

Gene Shuman writes:

...

Over the past few months I've developed an patch for Mailman 3, <https://gitlab.com/mailman/mailman/merge_requests/252> giving it ARC <https://tools.ietf.org/html/draft-ietf-dmarc-arc-protocol-09> email authentication capabilities. The work is (99%)completed, and it just needs somebody to do a final review of the ARC functionality itself so I can implement any requested changes.

So at this point we have two complete ARC patches, yours and Aditya Divekar's from last year's GSoC which I mentored, and I'm pretty sure somebody else is working on it. I think somebody else mentioned a patch, too.

So I know very well the other competing ARC PR that you're referring to. I investigated it closely before I started own my work on ARC for Mailman. It's pretty clear to me, and probably anybody else who takes a closer look at it that it has basically 0 chance of getting merged into Mailman.

There are two major issues at hand. One, that it is based of of a somewhat sloppy custom fork of dkimpy, that would never have had a chance to make its way into the main library itself. My ARC code on the other hand is already merged into that library and released. There is still a minor patch waiting to be merged(just pathological edge cases), which has been approved and is just waiting on the maintainer of the library to have a bit of time to complete the merge & release. I've been in close discussion with the maintainer(Scott Kitterman) through this entire process, and it's on track for merging as soon as he can spare some time, which my understanding will be not too long from now. My best guess will be some time in January at the latest, but I can reach out to him and see if he can confirm this.

The second point is that the original PR is basically dead. The code is not active, has not been tested in any interops and will certainly not validation reasonably against the reference ARC test suite, as the specification itself has changed significantly since that PR was created(and for multiple other reasons). It is my personally recommendation that that PR should just be discarded. (Sincere apologies to the author if you're on this mailing list, no offense intended)

However I'm curious about the other patch you mentioned. If this is the one from ValiMail, this is also me. I don't work for them any more, but I am now contracting with them to get this ARC code which has been basically ready for 6-8+ months reviewed & merged. If there is another competing developmental effort for ARC underway, I strongly recommend that they halt their efforts. My patch is complete, has passed a first review stage, and has been validated extensively, technically. There is quite literally no reason at all I can imagine for anybody else to be working on this, as, pending feedback, It's basically done. But if you are aware of another attempt at ARC in Mailman, I would *very* much appreciate it if you point me at it and its developers.

...

I had been working with Barry Warsaw,

I wish he'd mentioned it....

...

who had done a preliminary stylistic review, and subsequently, I've eliminated all raised issues. The branch is rebased as of last week, and passes all of qa, except one instance, which is something I'd like to discuss with somebody. (It fails a style test for line lengths in a test file which contains a bunch of cryptographic hashes).

I don't understand. Python offers several ways to deal with this, such as

sha_values = ["0123456789012345678901234567890123456789"]
foo(sha_value[0])

or if they're really long, adjacent strings will concatenate:

foo("01234567890123456789"
    "01234567890123456789")

I would think either would be fine, depending on your preferred style. Abhilash Raj (who just received the baton from Barry) would have the final say.

I entirely understand this. But these types of solutions just seem a bit awkward to me. Compensating for line lengths this way, just to satisfy a stylistic requirement, I personally believe would reduce the quality of the code. I understand the reason for line length limits just fine. But to split a 500 character cryptographic hex string into segments just to appease to static analyzer seem like it imposes a reduction of code quality that entirely misses the spirit of the underlying rule.

An argument could then be made to just put these crypto strings into files. But there are maybe ~15 violations of this nature. And I'd guess we don't think we want to clutter the file system up with ~15+ random, very specific crypto files. For fragility reason if no other.

But I'm 100% behind whatever the community thinks is best practice here. I personally advocate for simply turning off the line length restriction in the 2 relevant test files. After that I would advocate concatenating shorter strings, which wouldn't be too bad. After that moving those strings to files, which I do kind of think is a bad idea, but whatever. Honestly any of these are fine, and would probably in totally take me no more than an hour or so to implement. So whatever approach you guys prefer, I have no problem with, and will do so ASAP. Just let me know what is preferred.

I guess the main question I have at this point is whether you've

participated in any of the interoperability testing that the ARC developers have done.

Unequivocally, yes. I personally wrote the ARC test suite. I worked directly with Kurt Anderson and several others on elucidating all of the corner cases of the ARC RFC to make sure the test suite(an the spec itself) was bullet proof, and AFAIK, it is now accepted as the industry standard reference by which to validate all ARC implementations. I worked with Google to make sure their ARC implementation passed the test suite. I contributed significantly, and directly worked with the author of OpenARC, an ARC milter written in C. As of maybe a few weeks ago, it now completely validates against the test suite & as such is considered production ready. And as I'd mentioned, I solely wrote the ARC implementation in dkimpy, which my Mailman patch directly relies on, and of course made sure it was 100% ARC specification compliant.

I'll take a look at your MR and see if there are any features that differ across the implementations. Right now I'm buried until the 13th for sure, and more likely until Christmas. I definitely will be able to work on it during the holiday vacation, though.

Fantastic. To be perfectly honest, I've probably personally written more ARC code than anybody else in the world. I've outlined several proposals for conveying the cryptographic integrity of this software to the Mailman community, in a previous email. If you have any thoughts on the various approaches, please chime in. My personal hope is to have a fully qualified/validated/etc PR ready to merge into Mailman by the end of next January(a date I made up but which seems feasible). I have the bandwidth and can help out in any and all ways in the mean time. And Abhilash Raj has also said that he is going to try to find some time help finish this process as well.

Steve

Thank you greatly for your response, and all of your time in general. Regards, =Gene

-- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull@sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN

HI Stephen, great to get in touch with you.

On Fri, Dec 1, 2017 at 2:21 AM, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

Gene Shuman writes:

...

Over the past few months I've developed an patch for Mailman 3, <https://gitlab.com/mailman/mailman/merge_requests/252> giving it ARC <https://tools.ietf.org/html/draft-ietf-dmarc-arc-protocol-09> email authentication capabilities. The work is (99%)completed, and it just needs somebody to do a final review of the ARC functionality itself so I can implement any requested changes.

So at this point we have two complete ARC patches, yours and Aditya Divekar's from last year's GSoC which I mentored, and I'm pretty sure somebody else is working on it. I think somebody else mentioned a patch, too.

So I know very well the other competing ARC PR that you're referring to. I investigated it closely before I started own my work on ARC for Mailman. It's pretty clear to me, and probably anybody else who takes a closer look at it that it has basically 0 chance of getting merged into Mailman.

There are two major issues at hand. One, that it is based of of a somewhat sloppy custom fork of dkimpy, that would never have had a chance to make its way into the main library itself. My ARC code on the other hand is already merged into that library and released. There is still a minor patch waiting to be merged(just pathological edge cases), which has been approved and is just waiting on the maintainer of the library to have a bit of time to complete the merge & release. I've been in close discussion with the maintainer(Scott Kitterman) through this entire process, and it's on track for merging as soon as he can spare some time, which my understanding will be not too long from now. My best guess will be some time in January at the latest, but I can reach out to him and see if he can confirm this.

The second point is that the original PR is basically dead. The code is not active, has not been tested in any interops and will certainly not validation reasonably against the reference ARC test suite, as the specification itself has changed significantly since that PR was created(and for multiple other reasons). It is my personally recommendation that that PR should just be discarded. (Sincere apologies to the author if you're on this mailing list, no offense intended)

However I'm curious about the other patch you mentioned. If this is the one from ValiMail, this is also me. I don't work for them any more, but I am now contracting with them to get this ARC code which has been basically ready for 6-8+ months reviewed & merged. If there is another competing developmental effort for ARC underway, I strongly recommend that they halt their efforts. My patch is complete, has passed a first review stage, and has been validated extensively, technically. There is quite literally no reason at all I can imagine for anybody else to be working on this, as, pending feedback, It's basically done. But if you are aware of another attempt at ARC in Mailman, I would *very* much appreciate it if you point me at it and its developers.

...

I had been working with Barry Warsaw,

I wish he'd mentioned it....

...

who had done a preliminary stylistic review, and subsequently, I've eliminated all raised issues. The branch is rebased as of last week, and passes all of qa, except one instance, which is something I'd like to discuss with somebody. (It fails a style test for line lengths in a test file which contains a bunch of cryptographic hashes).

I don't understand. Python offers several ways to deal with this, such as

sha_values = ["0123456789012345678901234567890123456789"]
foo(sha_value[0])

or if they're really long, adjacent strings will concatenate:

foo("01234567890123456789"
    "01234567890123456789")

I would think either would be fine, depending on your preferred style. Abhilash Raj (who just received the baton from Barry) would have the final say.

I entirely understand this. But these types of solutions just seem a bit awkward to me. Compensating for line lengths this way, just to satisfy a stylistic requirement, I personally believe would reduce the quality of the code. I understand the reason for line length limits just fine. But to split a 500 character cryptographic hex string into segments just to appease to static analyzer seem like it imposes a reduction of code quality that entirely misses the spirit of the underlying rule.

An argument could then be made to just put these crypto strings into files. But there are maybe ~15 violations of this nature. And I'd guess we don't think we want to clutter the file system up with ~15+ random, very specific crypto files. For fragility reason if no other.

But I'm 100% behind whatever the community thinks is best practice here. I personally advocate for simply turning off the line length restriction in the 2 relevant test files. After that I would advocate concatenating shorter strings, which wouldn't be too bad. After that moving those strings to files, which I do kind of think is a bad idea, but whatever. Honestly any of these are fine, and would probably in totally take me no more than an hour or so to implement. So whatever approach you guys prefer, I have no problem with, and will do so ASAP. Just let me know what is preferred.

I guess the main question I have at this point is whether you've

participated in any of the interoperability testing that the ARC developers have done.

Unequivocally, yes. I personally wrote the ARC test suite. I worked directly with Kurt Anderson and several others on elucidating all of the corner cases of the ARC RFC to make sure the test suite(an the spec itself) was bullet proof, and AFAIK, it is now accepted as the industry standard reference by which to validate all ARC implementations. I worked with Google to make sure their ARC implementation passed the test suite. I contributed significantly, and directly worked with the author of OpenARC, an ARC milter written in C. As of maybe a few weeks ago, it now completely validates against the test suite & as such is considered production ready. And as I'd mentioned, I solely wrote the ARC implementation in dkimpy, which my Mailman patch directly relies on, and of course made sure it was 100% ARC specification compliant.

I'll take a look at your MR and see if there are any features that differ across the implementations. Right now I'm buried until the 13th for sure, and more likely until Christmas. I definitely will be able to work on it during the holiday vacation, though.

Fantastic. To be perfectly honest, I've probably personally written more ARC code than anybody else in the world. I've outlined several proposals for conveying the cryptographic integrity of this software to the Mailman community, in a previous email. If you have any thoughts on the various approaches, please chime in. My personal hope is to have a fully qualified/validated/etc PR ready to merge into Mailman by the end of next January(a date I made up but which seems feasible). I have the bandwidth and can help out in any and all ways in the mean time. And Abhilash Raj has also said that he is going to try to find some time help finish this process as well.

Steve

Thank you greatly for your response, and all of your time in general. Regards, =Gene

-- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull@sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN