Hi,
I will write and publish a patch which integrates PGP signature validation and re-encryption of encrypted posts to mailman. Specs are:
- A post will be distributed only if the PGP signature on the post is from one of the list members.
- For sending encrypted email, a list member encrypts to the public key of the list. The post will be decrypted and re-encrypted to the public keys of all list members.
(Later, the patch will handle RFC 2633 (S/MIME) messages too, next to RFC 2440 (OpenPGP)).
I've taken a look at the NAH6 secure list patch #646989 at http://sourceforge.net/tracker/index.php?func=detail&aid=646989&group_id=103&atid=300103 and at Ben Laurie's patch #645297 at http://sourceforge.net/tracker/index.php?func=detail&aid=645297&group_id=103&atid=300103 , but I believe none of these completely implements the listed requirements (although these will help me implementing, of course). I am asking you to take a look at my plan for implementation. Am I on the right track?
So, the plan:
I think one way to implement it would be to add two modules to GLOBAL_PIPELINE: in front, before SpamDetect, there would be 'PGPCheck'. PGPCheck would check wether the message is encrypted, and, if so, make a temporary decrypted copy in order to verify with which key is was signed. If the message is unencrypted, it would check the signature. It would store this information in new properties of the Mailman Message object.
A second new module in GLOBAL_PIPELINE would be 'PGPRecrypt', to be called after CookHeaders' and before 'ToDigest'. This would, if needed, decrypt the message and reencrypt it to all recipients, and would sign it.
If for instance a list member erroneously signs a post with a wrong public key, and encrypts the message, this message should be handled carefully. I believe the Hold module should be adapted for this. A copy of the original encrypted message should be kept. The message should be decrypted, signed with the listkey, encrypted to the list moderator key, and sent for acknowledgement. If the moderator chooses to deny the message, the poster should get her original message back. (I guess.)
For all PGP handling, I plan to use Frank J. Tobin's GnuPGInterface ( http://py-gnupg.sourceforge.net/ ). I plan to write the patch against current stable Mailman.
Any insight to share on this?
Thanks! Bye,
Joost
-- Joost van Baal http://abramowitz.uvt.nl/ Tilburg University j.e.vanbaal@uvt.nl The Netherlands
Hi Joost,
I will write and publish a patch which integrates PGP signature validation and re-encryption of encrypted posts to mailman. Specs are:
Don't duplicate work which has already been done :-) I made modifications to mailman 2.1.5 for encrypted mailinglists, but I kept quiet for testing. The test installation seems to behave pretty stable now, so this seems to be a goot point making the patch public.
The patch (and a german status of the project) is here: http://medien.informatik.uni-ulm.de/~stefan/gpg-mailman.xhtml
Direct link to the patch:
http://medien.informatik.uni-ulm.de/~stefan/mailman-2.1.5-gpg_2005-02-22.dif...
- A post will be distributed only if the PGP signature on the post is from one of the list members.
Not implemented in my patch. Signatures are recognised but not used as means of authentication for sending to the list.
- For sending encrypted email, a list member encrypts to the public key of the list. The post will be decrypted and re-encrypted to the public keys of all list members.
Done.
(Later, the patch will handle RFC 2633 (S/MIME) messages too, next to RFC 2440 (OpenPGP)).
Modular extension of my patch should be possible.
Features of the patch:
- Web interface for key upload (list key, users' keys)
- Several policies for accepting and delivering mails:
- Incoming mail must not / may / must be encrypted
- Outgoing mails must not / may / must be encrypted
- Information on poster's signature embedded in redistributed message
- Incoming mail formats: Inline-PGP and PGP/MIME
- Outgoing mail formats: Currently PGP/MIME only
Current problems: Inline-PGP has never been standardised. This is awful. Every mailer does it a little bit different (content type of mail, encryption of attachments, treatment of encoding scheme). There is still work to do.
So, the plan:
I think one way to implement it would be to add two modules to GLOBAL_PIPELINE: in front, before SpamDetect, there would be 'PGPCheck'.
At this place, I implemented decryption (and encryption policy enforcement).
A second new module in GLOBAL_PIPELINE would be 'PGPRecrypt', to be called after CookHeaders' and before 'ToDigest'. This would, if needed, decrypt the message and reencrypt it to all recipients, and would sign it.
I found no elegant way for distributing encrypted mails, so I patched a copy of the SMTPDirect handler. Currently, every mail is delivered separately (as done for personalised mails); I didn't figure out at first glance how chunking works... otherwise, a mail might be encrypted to several recipients - increasing its length by several bytes, but reducing the number of forks of gpg processes.
For all PGP handling, I plan to use Frank J. Tobin's GnuPGInterface ( http://py-gnupg.sourceforge.net/ ).
That's what I used. Mind that py-gnupg (and gpgme, too) forks a gpg process for every operation. In large lists, this is pretty costly.
What do you think? The whole signature thing you planned is still missing, and inline-pgp needs some more work.
Stefan.
At 9:01 PM +0100 2005-03-01, Stefan Schlott wrote:
I will write and publish a patch which integrates PGP signature validation and re-encryption of encrypted posts to mailman. Specs are:
Don't duplicate work which has already been done :-) I made modifications to mailman 2.1.5 for encrypted mailinglists, but I kept quiet for testing. The test installation seems to behave pretty stable now, so this seems to be a goot point making the patch public.
Have either of you solved the problem of Mailman not preserving white space in the message body? This kind of thing is death to messages signed or encrypted with PGP, and I have yet to hear of anyone who has addressed this issue.
I didn't figure out atfirst glance how chunking works... otherwise, a mail might be encrypted to several recipients - increasing its length by several bytes, but reducing the number of forks of gpg processes.
You definitely need to encrypt the message to all recipients, then send out one copy. Anything else would be prohibitively expensive in terms of computation time for any reasonable number of recipients.
-- Brad Knowles, brad@stop.mail-abuse.org
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755SAGE member since 1995. See http://www.sage.org/ for more info.
Hi,
Have either of you solved the problem of Mailman not preservingwhite space in the message body? This kind of thing is death to messages signed or encrypted with PGP, and I have yet to hear of anyone who has addressed this issue.
As already mentioned, I didn't cover the sign-only case; I tried to implement an encrypted mailinglist, thus the space-signature-killer was no concern :-)
I didn't figure out atfirst glance how chunking works... otherwise, a mail might be encrypted to several recipients - increasing its length by several bytes, but reducing the number of forks of gpg processes.
You definitely need to encrypt the message to all recipients, thensend out one copy. Anything else would be prohibitively expensive in terms of computation time for any reasonable number of recipients.
This will create huge messages in large MLs. Further, this will reveal all recipients' key ids - something not wanted in anonymous lists. Imho the tradeoff lies somewhere inbetween - encrypt messages to n recipients (yet to be implemented).
Stefan.
Hi,
Stefan: thanks a lot for publishing your patch! Of course, this is _very_ helpful!
On Wed, Mar 02, 2005 at 08:31:56AM +0100, Stefan Schlott wrote: <snip>
I didn't figure out atfirst glance how chunking works... otherwise, a mail might be encrypted to several recipients - increasing its length by several bytes, but reducing the number of forks of gpg processes.
You definitely need to encrypt the message to all recipients, then send out one copy. Anything else would be prohibitively expensive in terms of computation time for any reasonable number of recipients.
This will create huge messages in large MLs. Further, this will reveal all recipients' key ids - something not wanted in anonymous lists. Imho the tradeoff lies somewhere inbetween - encrypt messages to n recipients (yet to be implemented).
I guess gpg's --throw-keyid option is useful here.
Bye,
Joost
-- Joost van Baal http://abramowitz.uvt.nl/ Tilburg University j.e.vanbaal@uvt.nl The Netherlands
Hi,
Stefan: thanks a lot for publishing your patch! Of course, this is _very_ helpful!
Great to hear that :-)
This will create huge messages in large MLs. Further, this will reveal all recipients' key ids - something not wanted in anonymous lists. Imho the tradeoff lies somewhere inbetween - encrypt messages to n recipients (yet to be implemented).
I guess gpg's --throw-keyid option is useful here.
Good point, thanks for the hint.
It would be great if your efforts and the development of my patch could stay consistent, i.e. if you make more modifications to my patches, please let me know. Besides of that, any advice concerning inline-pgp is welcome. A fully pgp-enabled mailman would be a great thing.
Stefan.
At 8:31 AM +0100 2005-03-02, Stefan Schlott wrote:
This will create huge messages in large MLs.
Each additional recipient would be a few extra bytes in the encrypted message. However, since the message is compressed first, unless you've got a list of thousands and thousands of recipients, then odds are that the result would still be smaller in size than a message that was only PGP-signed.
Further, this will revealall recipients' key ids - something not wanted in anonymous lists.
True. A session key would be encrypted to each key id, so the key ids would be visible. However, subscriber information is not too hard to get from Mailman even when it's supposedly limited to being available only to the admin, so I think there may be bigger fish to fry elsewhere.
Imho the tradeoff lies somewhere inbetween - encrypt messages to n recipients (yet to be implemented).
The problem is that encrypting a message is a very CPU-intensive process, and you don't want to figure off thousands and thousands of message encryption processes for every single submission -- you'd DoS yourself to death. You'd have to make n pretty large in order to be able to make this scalable.
-- Brad Knowles, brad@stop.mail-abuse.org
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755SAGE member since 1995. See http://www.sage.org/ for more info.
On Wed, 2005-03-02 at 12:37 +0100, Brad Knowles wrote:
At 8:31 AM +0100 2005-03-02, Stefan Schlott wrote:
Further, this will revealall recipients' key ids - something not wanted in anonymous lists.
True. A session key would be encrypted to each key id, so the key ids would be visible. However, subscriber information is not too hard to get from Mailman even when it's supposedly limited to being available only to the admin, so I think there may be bigger fish to fry elsewhere.
Imho the tradeoff lies somewhere inbetween - encrypt messages to n recipients (yet to be implemented).
The problem is that encrypting a message is a very CPU-intensive process, and you don't want to figure off thousands and thousands of message encryption processes for every single submission -- you'd DoS yourself to death. You'd have to make n pretty large in order to be able to make this scalable.
In theory, you could encrypt the message once with a session key, and then distribute it n times, each time adding the packet which has the session key encrypted with the public key of the recipient. This should cost you very little more in encryption CPU requirements than a message encrypted to n recipients in the normal fashion. The rest of the additional required overhead is basically the same as turning personalisation on for a list.
Not sure how amenable GPG is to doing this sort of hacking, but it sounds plausible to me (obviously too few coffees this morning).
Nigel.-- [ Nigel Metheringham Nigel.Metheringham@InTechnology.co.uk ] [ - Comments in this message are my own and not ITO opinion/policy - ]
Nigel Metheringham wrote:
In theory, you could encrypt the message once with a session key, and then distribute it n times, each time adding the packet which has the session key encrypted with the public key of the recipient.
...which is the very thing making gpg encryption so slow :-) The expensive part is the public key operation, not the symmetric encryption.
Stefan.
On Wed, 2005-03-02 at 13:28 +0100, Stefan Schlott wrote:
Nigel Metheringham wrote:
In theory, you could encrypt the message once with a session key, and then distribute it n times, each time adding the packet which has the session key encrypted with the public key of the recipient.
...which is the very thing making gpg encryption so slow :-) The expensive part is the public key operation, not the symmetric encryption.
True. But you need to do n public key operations for n recipients anyway. All I am changing is whether or not you attach them all to one message, or make n copies of the message each with one public key encrypted session key packet.
Nigel.-- [ Nigel Metheringham Nigel.Metheringham@InTechnology.co.uk ] [ - Comments in this message are my own and not ITO opinion/policy - ]
Brad Knowles wrote:
Further, this will revealall recipients' key ids - something not wanted in anonymous lists.
True. A session key would be encrypted to each key id, so the keyids would be visible. However, subscriber information is not too hard to get from Mailman even when it's supposedly limited to being available only to the admin, so I think there may be bigger fish to fry elsewhere.
Right. And as it has already been pointed out, --throw-keyid should help.
Imho the tradeoff lies somewhere inbetween - encrypt messages to n recipients (yet to be implemented).
The problem is that encrypting a message is a very CPU-intensiveprocess, and you don't want to figure off thousands and thousands of message encryption processes for every single submission -- you'd DoS yourself to death. You'd have to make n pretty large in order to be able to make this scalable.
A quick test gave the following (254 bytes text file as input): lists indeed...
- ASCII-Armored result, 10 recipients: 4211 bytes
- ASCII-Armored result, 20 recipients: 8931 bytes
- ASCII-Armored result, 30 recipients: 14700 bytes
- ASCII-Armored result, 40 recipients: 19242 bytes Every recipient seems to increase file size by approx. 500 bytes. --throw-keyid didn't change file lenghts. Creating chunks larger than 20 or 30 recipients results in really large mails; repeated calls of gpg result in high load. Not good for large
Stefan.
On Mon, Feb 28, 2005 at 06:40:22PM +0100, Joost van Baal wrote:
I will write and publish a patch which integrates PGP signature validation and re-encryption of encrypted posts to mailman. Specs are:
- A post will be distributed only if the PGP signature on the post is from one of the list members.
- For sending encrypted email, a list member encrypts to the public key of the list. The post will be decrypted and re-encrypted to the public keys of all list members.
(Later, the patch will handle RFC 2633 (S/MIME) messages too, next to RFC 2440 (OpenPGP)).
<snip> > So, the plan: <snip> > Any insight to share on this?
Thanks a lot for all feedback.
Thanks to Stefan Schlott's fine work he has done earlier, the first test-able version is available now, from http://www.non-gnu.uvt.nl/pub/mailman/ . For those of you running Debian GNU/Linux, packages are available too (both pre-build and source packages). The patch is still a bit rough on the edges, but could be interesting for developers.
Comments on the code are welcome (but you guessed that :)
The patch is submitted to the patch tracker too: it's Mailman patch #1167696 .
Bye,
Joost
-- Joost van Baal http://abramowitz.uvt.nl/ Tilburg University j.e.vanbaal@uvt.nl The Netherlands
On Tue, Mar 22, 2005 at 05:38:29PM +0100, Joost van Baal wrote:
On Mon, Feb 28, 2005 at 06:40:22PM +0100, Joost van Baal wrote:
I will write and publish a patch which integrates PGP signature validation and re-encryption of encrypted posts to mailman. Specs are:
- A post will be distributed only if the PGP signature on the post is from one of the list members.
- For sending encrypted email, a list member encrypts to the public key of the list. The post will be decrypted and re-encrypted to the public keys of all list members.
(Later, the patch will handle RFC 2633 (S/MIME) messages too, next to RFC 2440 (OpenPGP)).
This patch has been updated to apply to Mailman 2.1.6. It's available from http://non-gnu.uvt.nl/pub/mailman/mailman-2.1.6-ssls_gpg_2005-07-01.patch.gz . It includes prior work from Stefan Schlott. I am running mailman-ssls_2.1.6-0.1_i386.deb now, it is usable. (Although I wouldn't yet recommend to deploy it in production environments!)
The S/MIME functionality is scheduled to be implemented at 2005-10-15.
Comments on the code are still very welcome (but you guessed that :)
More information is available from http://non-gnu.uvt.nl/pub/mailman/ .
Bye,
Joost
-- Joost van Baal http://abramowitz.uvt.nl/ Tilburg University j.e.vanbaal@uvt.nl The Netherlands
participants (4)
-
Brad Knowles -
Joost van Baal -
Nigel Metheringham -
Stefan Schlott