Hi all,
I'm writing again about the ARC PR that I've been working on that I'd like to get reviewed & merged. I've been in touch with Abhilash Raj about this and he's graciously agreed to put some energy into it when he has time. But in general, I've put some thought into it, and have come up with a general plan to streamline this review, to make it as simple as possible for Abhilash or anybody else with the interest or bandwidth to help.
First the are 4 major components of the review:
- Barry Warsaw did a preliminary review which focused mostly on the code syntax & organization, and I've correspondently updated the code. So first point is verifying this, which should be easy.
- Next is verifying that the way in which I've approached this problem is appropriately in line with how mailman is abstractly structured. Also easy.
- Next again is verifyomg that the documentation is sufficient from a users(or developers) perspective. I've recently updated the documentation, and believe it to be sufficient. To test this, somebody with possibly marginal familiarity with ARC needs to just try to setup a Mailman server with ARC functionality. This is closely related to the next point.
- Finally is verifying that the code behaves cryptographically correctly
This last point is clearly what most imperatively needs to be right, and I'll devote a bit of this message towards it.
Honestly, I don't think it should be that difficult for somebody to convince themselves Mailman is handling ARC correctly, even knowing little about it. Depending on the level of paranoia here(any of which are justifiable,) there are several routes to going about this. But first an overview of how this PR implements ARC.
This may be a bit long winded, so hopefully bear with me. So this PR does ARC signing & validation via a library of mine, authheaders <https://pypi.python.org/pypi/authheaders/0.9.2>, which basically aggregates & passes along email authentication & signing functions to other libraries. It's quite simple & its easy to see that it's not modifying the results its getting from the other libraries. In particular, with respect to ARC, its using a fork of the library dkimpy <https://launchpad.net/dkimpy>. The fork has passed review, and afaik will be merged & released any day now. The fork brings the dkimpy library's ARC implementation(which I wrote) from 99% compliance with the ARC test suite <https://github.com/ValiMail/arc_test_suite>(which I also authored), to 100%. So it's absolutely demonstrably clear that the library that my ARC Mailman PR is using is cryptographically correct. The test suite itself has also been validated from several other directions, so at this point is authoritative.
In particular, I've worked with the engineer at Google responsible for their ARC implementation, Brandon Long, on this, and I have also worked in depth with the major authors of the ARC rfc to clarify the protocol, illuminate edge cases, and bring the document to completion. But in particular, wrt Google, I've personally verified that their ARC implementation is 100% compliant with the spec & the test suite. And their ARC implementation is live and is validating & signing 100% of their email traffic atm.
So here is my simple proposal for validating the cryptographic validity of my PR. Set up a Mailman server with ARC configured properly. Add some google email addresses to the outbound list, send a bunch of emails, and inspect the headers to see that ARC appropriately verifies. Then take the same Mailman server and turn of ARC and see that the messages fail. As a further step you can set up a nested sequence of Mailman servers, with some or all or none of them ARC enabled, and then check the results. I personally think it is within reason to trust Google as an authority here, as I've personally followed their development, and know that their implementation is 100% compliant.
As an additional level of security here, we can combine this with direct validation fo dkimpy. I can directly show somebody how to test dkimpy against the ARC test suite themselves. I don't think this is necessary, as I've done it hundreds of times, but in 30m I can help somebody check this for themselves personally
Another thing to note. There are tests added to the Mailman test suite that directly test the crypto. They aren't comprehensive, but they are there & they pass.
So to be summarize, I'm perfectly confident that that the ARC implementation in Mailman is rock solid, as I've personally written the intermediary layer, the actual ARC implementation in dkimpy, the ARC test suite, and have been a non-trivial contributor to the specification itself.
And with any of the above steps necessary for this validation, I'm available to help. What ever I can do to make this as painless as possible, I'm available for.
Now to the next level of paranoia, which is the bottom level, and would prove beyond a shadow of a doubt that Mailman's ARC implementation is absolutely cryptographically compliant. First, a bit about the ARC test suite. It's a bunch of yaml files that specifies various messages, edge cases, etc, and what the output should be. It's designed abstractly so that you can test it against any system which is ARC capable, by just creating a system of translation layers between the test suite & the software being tested. It's been tested against dkimpy in such a manner. It's also been tested against googles ARC implementation, OpenARC, and others. It's quite general.
It would be possible to directly run the test suite against Mailman. This would be a bit awkward, as its a mail server, as opposed to a library, but it could be done. And this literally would be bullet proof evidence that the system is behaving cryptographically correctly. However the translation layer may be a pain to write. But I haven't put too much thought into that.
I personally think that the argument that the library, dkimpy, which the ARC PR is currently using passes test-suite validation is strong enough. But I'm an expert wrt ARC, and I understand that argument via claims of expertise aren't really all that valid.
So the bottom line is this. Either I, or somebody else can wire up some ARC enabled Mailman servers, point them at Google, and check that things check out, using Google's ARC implementation as a reference. And possibly to demonstrate running the ARC test suite against dkimpy.
The other option is making a harness to directly run the ARC test suite against mailman. This is doable, and would be a hassle, but is not out of the question. I could likely personally do this, and would happily do so.
Thank you to anybody who has read this far. Please let me know your thoughts. And don't hesitate with any questions about ARC or DMARC, or other email authentication protocols. I'm a near encyclopedic resource about these things at this point, and am happy to share my knowledge.
Looking forward to working with guys!
Regards, =Gene