[Mailman-Developers] ARC PR for Mailman 3

[Gene Shuman]

HI Stephen, great to get in touch with you.



On Fri, Dec 1, 2017 at 2:21 AM, Stephen J. Turnbull <
turnbull.stephen.fw at u.tsukuba.ac.jp> wrote:

> Gene Shuman writes:
>
>  > Over the past few months I've developed an patch for Mailman 3,
>  > <https://gitlab.com/mailman/mailman/merge_requests/252> giving it ARC
>  > <https://tools.ietf.org/html/draft-ietf-dmarc-arc-protocol-09> email
>  > authentication capabilities.  The work is (99%)completed, and it just
> needs
>  > somebody to do a final review of the ARC functionality itself so I can
>  > implement any requested changes.
>
> So at this point we have two complete ARC patches, yours and Aditya
> Divekar's from last year's GSoC which I mentored, and I'm pretty sure
> somebody else is working on it.  I think somebody else mentioned a
> patch, too.
>

So I know very well the other competing ARC PR that you're referring to.  I
investigated it closely before I started own my work on ARC for Mailman.
It's pretty clear to me, and probably anybody else who takes a closer look
at it that it has basically 0 chance of getting merged into Mailman.

There are two major issues at hand.  One, that it is based of of a somewhat
sloppy custom fork of dkimpy, that would never have had a chance to make
its way into the main library itself.  My ARC code on the other hand is
already merged into that library and released.  There is still a minor
patch waiting to be merged(just pathological edge cases), which has been
approved and is just waiting on the maintainer of the library to have a bit
of time to complete the merge & release.  I've been in close discussion
with the maintainer(Scott Kitterman) through this entire process, and it's
on track for merging as soon as he can spare some time, which my
understanding will be not too long from now.  My best guess will be some
time in January at the latest, but I can reach out to him and see if he can
confirm this.

The second point is that the original PR is basically dead.  The code is
not active, has not been tested in any interops and will certainly not
validation reasonably against the reference ARC test suite, as the
specification itself has changed significantly since that PR was
created(and for multiple other reasons).   It is my personally
recommendation that that PR should just be discarded.  (Sincere apologies
to the author if you're on this mailing list, no offense intended)

However I'm curious about the other patch you mentioned.  If this is the
one from ValiMail, this is also me.  I don't work for them any more, but I
am now contracting with them to get this ARC code which has been basically
ready for 6-8+ months reviewed & merged.  If there is another competing
developmental effort for ARC underway, I strongly recommend that they halt
their efforts.  My patch is complete, has passed a first review stage, and
has been validated extensively, technically.  There is quite literally no
reason at all I can imagine for anybody else to be working on this, as,
pending feedback, It's basically done.  But if you are aware of another
attempt at ARC in Mailman, I would *very* much appreciate it if you point
me at it and its developers.


>
>  > I had been working with Barry Warsaw,
>
> I wish he'd mentioned it....
>
>  > who had done a preliminary stylistic review, and subsequently, I've
>  > eliminated all raised issues.  The branch is rebased as of last
>  > week, and passes all of qa, except one instance, which is something
>  > I'd like to discuss with somebody. (It fails a style test for line
>  > lengths in a test file which contains a bunch of cryptographic
>  > hashes).
>
> I don't understand.  Python offers several ways to deal with this,
> such as
>
>     sha_values = ["0123456789012345678901234567890123456789"]
>     foo(sha_value[0])
>
> or if they're really long, adjacent strings will concatenate:
>
>     foo("01234567890123456789"
>         "01234567890123456789")
>
> I would think either would be fine, depending on your preferred
> style.  Abhilash Raj (who just received the baton from Barry) would
> have the final say.
>

I entirely understand this.  But these types of solutions just seem a bit
awkward to me. Compensating for line lengths this way, just to satisfy a
stylistic requirement, I personally believe would reduce the quality of the
code.  I understand the reason for line length limits just fine.  But to
split a 500 character cryptographic hex string into segments just to
appease to static analyzer seem like it imposes a reduction of code quality
that entirely misses the spirit of the underlying rule.

An argument could then be made to just put these crypto strings into
files.  But there are maybe ~15 violations of this nature.  And I'd guess
we don't think we want to clutter the file system up with ~15+ random, very
specific crypto files. For fragility reason if no other.

But I'm 100% behind whatever the community thinks is best practice here.  I
personally advocate for simply turning off the line length restriction in
the 2 relevant test files.  After that I would advocate concatenating
shorter strings, which wouldn't be too bad.  After that moving those
strings to files, which I do kind of think is a bad idea, but whatever.
Honestly any of these are fine, and would probably in totally take me no
more than an hour or so to implement.  So whatever approach you guys
prefer, I have no problem with, and will do so ASAP.  Just let me know what
is preferred.

I guess the main question I have at this point is whether you've
> participated in any of the interoperability testing that the ARC
> developers have done.
>

Unequivocally, yes.  I personally wrote the ARC test suite.  I worked
directly with Kurt Anderson and several others on elucidating all of the
corner cases of the ARC RFC to make sure the test suite(an the spec itself)
was bullet proof, and AFAIK, it is now accepted as the industry standard
reference by which to validate all ARC implementations.  I worked with
Google to make sure their ARC implementation passed the test suite.  I
contributed significantly, and directly worked with the author of OpenARC,
an ARC milter written in C.  As of maybe a few weeks ago, it now completely
validates against the test suite & as such is considered production ready.
And as I'd mentioned, I solely wrote the ARC implementation in dkimpy,
which my Mailman patch directly relies on, and of course made sure it was
100% ARC specification compliant.


> I'll take a look at your MR and see if there are any features that
> differ across the implementations.  Right now I'm buried until the
> 13th for sure, and more likely until Christmas.  I definitely will be
> able to work on it during the holiday vacation, though.
>

Fantastic.  To be perfectly honest, I've probably personally written more
ARC code than anybody else in the world.  I've outlined several proposals
for conveying the cryptographic integrity of this software to the Mailman
community, in a previous email.  If you have any thoughts on the various
approaches, please chime in.   My personal hope is to have a fully
qualified/validated/etc PR ready to merge into Mailman by the end of next
January(a date I made up but which seems feasible).  I have the bandwidth
and can help out in any and all ways in the mean time.  And Abhilash Raj
has also said that he is going to try to find some time help finish this
process as well.

Steve
>

Thank you greatly for your response, and all of your time in general.
Regards,
=Gene


>
>
> --
> Associate Professor              Division of Policy and Planning Science
> http://turnbull/sk.tsukuba.ac.jp/     Faculty of Systems and Information
> Email: turnbull at sk.tsukuba.ac.jp                   University of Tsukuba
> Tel: 029-853-5175                 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN
>