Hi all,
So I think my work inside mailman for signing and verification is almost complete, here are a few things that I have now working and it would be very nice if some of you can have a look at the code and let me know if the implementation is correct(the logical and the pythonic way).
There is a 'signature rule'1 that can verify signature from the users whose public key is stored in 'var/gpg' directory insider 'pubring.gpg'. This rule also verifies that the email has only two parts one of which must be 'application/pgp-signature'.
The 'signmessage handler'2 signs the message while preserving the sender's signature. The structure of the message for now is a multipart/alternative with each alternative part having one signature(one from list and another from sender). (I have into my todo what Daniel suggested previously3 to have two signatures in one pgp-signature part)
A 'gpg'4 utility which does all the crypto part from signing to verification, generation of list's key, importing key from data(will be used if we allow user's to copy paste their public key data insider postorius), importing key from a public keyserver(default is set as 'pgp.mit.edu' on random basis, please suggest something which you think can be a better default).
In line 81 I am passing an empty string to the comment for the key but still the key generated is still having the default comment 'Generated by gpg.py'. Any idea why? Is it because the string I am passing is a null string and it should have a non-null string to set as comment?
- There are few other changes like adding the signature rule in default-posting-chain chain and signmessage handler in default-posing-pipeline. Also I have added a new config variable as 'gpg_dir' whose default value is set to '$VAR_DIR/gpg'. There are tests written for almost all the modules above, some more tests maybe required to be added to then though)
I am thinking to setup a working installation of this code once I find a place to do that, so that we can find bugs more easily.
Moving on my plan includes adding the APIs for management of keys and options for list-admin and users in postorius and mm-client. If anyone has ever given a thought before about what all things should be there please help me with it.
Abhilash Raj
Hi Abhilash--
I haven't looked at the code much yet, but this is a pretty exciting report! I'm glad to hear everything you've done.
On 08/28/2013 09:37 PM, Abhilash Raj wrote:
- There is a 'signature rule'[1] that can verify signature from the users whose public key is stored in 'var/gpg' directory insider 'pubring.gpg'. This rule also verifies that the email has only two parts one of which must be 'application/pgp-signature'.
i'm not sure about this last constraint. It's certainly possible to have a multipart MIME message with a deeper structure that has more parts if you're looking at the leaves of the MIME tree. the constraint should be: main content-type should be multipart/signed; that part should have exactly two immediate children; the last child should be application/pgp-signature. The first child could itself be multipart/mixed or multipart/alternative.
- The 'signmessage handler'[2] signs the message while preserving the sender's signature. The structure of the message for now is a multipart/alternative with each alternative part having one signature(one from list and another from sender). (I have into my todo what Daniel suggested previously[3] to have two signatures in one pgp-signature part)
have you tested what messages with this structure look like when viewed with any MUA that is PGP/MIME-aware? do you have any examples you could publish, so other people can test other MUAs?
- A 'gpg'[4] utility which does all the crypto part from signing to verification, generation of list's key, importing key from data(will be used if we allow user's to copy paste their public key data insider postorius), importing key from a public keyserver(default is set as 'pgp.mit.edu' on random basis, please suggest something which you think can be a better default).
please do not use pgp.mit.edu [0], it is out of date and poorly-maintained. Your best bet for reliable, redundant service is to use ha.pool.sks-keyservers.net. You can read more about this high-availability DNS round-robin pool on the sks-keyservers website [1].
[0] https://we.riseup.net/debian/openpgp-best-practices#dont-use-pgp-mit-edu [1] https://sks-keyservers.net/overview-of-pools.php
In line 81 I am passing an empty string to the comment for the key but still the key generated is still having the default comment 'Generated by gpg.py'. Any idea why? Is it because the string I am passing is a null string and it should have a non-null string to set as comment?
Take a look at the gnupg module (gnupg.py), which you're relying on, around. around line 976 in my version is the definition of GPG.gen_key_input, which skips "empty" strings:
983 if str(val).strip(): # skip empty strings
984 parms[key] = valunfortunately, the rest of that function definition appears to require a comment, inserting one by default if no comment is supplied by the user. I consider this a bug in gnupg.py, we should get it fixed there.
I've just opened http://bugs.debian.org/721296 to try to get them fixed in debian, but they should probably be forwarded upstream, but upstream's bugtracker [2] appears to require a google account in order to open a ticket, which i do not have or want. If you have a google account, feel free to forward them upstream.
[2] https://code.google.com/p/python-gnupg/issues/list
In the meantime, i recommend just hand-crafting the input for gpg --batch --gen-key instead of using gnupg.GPG.gen_key_input() at all. If you're not sure about the format, i recommend reading /usr/share/doc/gnupg/DETAILS.gz on any debian or debian-derived system, or DETAILS in the gpg source [3].
[3] git://git.gnupg.org/gnupg
I am thinking to setup a working installation of this code once I find a place to do that, so that we can find bugs more easily.
that would be great!
Hope these notes are helpful,
--dkg
Daniel Kahn Gillmor writes:
On 08/28/2013 09:37 PM, Abhilash Raj wrote:
- There is a 'signature rule'[1] that can verify signature from the users whose public key is stored in 'var/gpg' directory insider 'pubring.gpg'. This rule also verifies that the email has only two parts one of which must be 'application/pgp-signature'.
i'm not sure about this last constraint. It's certainly possible to have a multipart MIME message with a deeper structure that has more parts if you're looking at the leaves of the MIME tree. the constraint should be: main content-type should be multipart/signed; that part should have exactly two immediate children; the last child should be application/pgp-signature. The first child could itself be multipart/mixed or multipart/alternative.
The last time I looked (~10 days ago), that was the implementation: look only at the message-level Content-Type, ensure it's multipart/signed, check that there are exactly two parts and that the second is "application/pgp-signature".
Note that nested multipart parts are problematic. You can't strip unwanted parts from them (many lists strip .exes, others text/html, and so on) without breaking the original signature. We need to think carefully about what policies to support regarding signed multiparts. I would suggest that the initial default policy should be
(1) verify the signature, if not DISCARD (2) if valid AND key belongs to explicitly allowed poster (I think probably nobody wants open-post signed lists, but what do I know? ;-) AND signed part is multipart, REJECT with "multipart not allowed for technical reasons" as reason (3) if valid AND key belongs to explicitly allowed poster, APPROVE (4) otherwise HOLD
while we're figuring out a sane UI for configuring more complex policies.
have you tested what messages with this structure look like when viewed with any MUA that is PGP/MIME-aware?
+1
do you have any examples you could publish, so other people can test other MUAs?
+1
please do not use pgp.mit.edu [0], it is out of date and poorly-maintained. Your best bet for reliable, redundant service is to use ha.pool.sks-keyservers.net. You can read more about this high-availability DNS round-robin pool on the sks-keyservers website [1].
Thanks for the suggestion!
On 08/30/2013 12:56 AM, Stephen J. Turnbull wrote:
The last time I looked (~10 days ago), that was the implementation: look only at the message-level Content-Type, ensure it's multipart/signed, check that there are exactly two parts and that the second is "application/pgp-signature".
hum, what if the first part is an .exe of type application/octet-stream? :P
Note that nested multipart parts are problematic. You can't strip unwanted parts from them (many lists strip .exes, others text/html, and so on) without breaking the original signature. We need to think carefully about what policies to support regarding signed multiparts. I would suggest that the initial default policy should be
(1) verify the signature, if not DISCARD (2) if valid AND key belongs to explicitly allowed poster (I think probably nobody wants open-post signed lists, but what do I know? ;-) AND signed part is multipart, REJECT with "multipart not allowed for technical reasons" as reason
Doesn't this seem a little overbroad? what if the message is two inline parts, one text/plain and one text/x-diff (e.g. the common case of someone sending a patch to a project mailing list ? I understand wanting to REJECT if any of the internal parts need to be stripped, but if none of them need to be stripped, it seems like REJECT is too heavy-handed.
Or is the knowledge about what might need to be stripped not available at that stage? If the stripping happens later, can we mark a message as "REJECT if you need to strip" somehow, and get the part-stripping code to respect that marking and REJECT it there?
sorry for the ignorance, just exploring the options.
--dkgDaniel Kahn Gillmor writes:
On 08/30/2013 12:56 AM, Stephen J. Turnbull wrote:
The last time I looked (~10 days ago), that was the implementation: look only at the message-level Content-Type, ensure it's multipart/signed, check that there are exactly two parts and that the second is "application/pgp-signature".
hum, what if the first part is an .exe of type application/octet-stream? :P
Windows users get another reason to switch? ;-)
That should be handled by content filtering. *However*, content filtering needs to be taught to throw away the signature too if a signed part gets filtered.
Doesn't this seem a little overbroad? what if the message is two inline parts, one text/plain and one text/x-diff (e.g. the common case of someone sending a patch to a project mailing list ? I understand wanting to REJECT if any of the internal parts need to be stripped, but if none of them need to be stripped, it seems like REJECT is too heavy-handed.
Sure, but there's a GSoC scope issue here. Principle Number One has to be "Better Safe than Sorry", and this is easy for Abhilash to implement safely. It has to be an option, of course, for admins willing to risk junk getting through (eg, because their users don't use Outlook [Express]).
Or is the knowledge about what might need to be stripped not available at that stage? If the stripping happens later, can we mark a message as "REJECT if you need to strip" somehow, and get the part-stripping code to respect that marking and REJECT it there?
I doubt that Abhilash knows, and I know *I* don't. :-)
My guess is that signature check in principle is going to be a Rule, while part-stripping is a Handler. I don't think Handlers can REJECT, Rules make that decision, and they come first, before the Handlers.
So I see the process as involving two Rules. The first one checks the multipart/signed message itself. If invalid (either a broken signature or signer is not allowed to post), DISCARD (the risk of a spoof is too great).
The second one checks the structure of the message. Presumably code can be reused from elsewhere to do this with more accuracy as you suggest, but the easy check is to disallow multiparts nested in multipart/signed.
My proposal at this point is to accept an easy and safe (== restrictive) implementation for the purposes of GSoC and early beta testing. I don't mean to suggest it's at all satisfactory for a public release of this feature (although we might all be surprised and find out that it's fine for many purposes!)
Steve
participants (3)
-
Abhilash Raj -
Daniel Kahn Gillmor -
Stephen J. Turnbull