Mailman 3 December 2014 - Mailman-coders

[Merge] lp:~abompard/postorius/bug-1094829 into lp:postorius
by Aurélien Bompard Jan. 17, 2015

Jan. 17, 2015

Aurélien Bompard has proposed merging lp:~abompard/postorius/bug-1094829 into lp:postorius. Requested reviews: Mailman Coders (mailman-coders) For more details, see: https://code.launchpad.net/~abompard/postorius/bug-1094829/+merge/242705 Fix for bug #1094829. Postorius re-uses Django's login view, which also shows error messages when the credentials are invalid. The previous login template is passed to this view, so we can keep the Mozilla Persona button. -- Your team Mailman Coders is requested to review the proposed merge of lp:~abompard/postorius/bug-1094829 into lp:postorius.

3 2

[Merge] lp:~rajeevs1992/postorius/postorius into lp:postorius
by Rajeev S Jan. 9, 2015

Jan. 9, 2015

Rajeev S has proposed merging lp:~rajeevs1992/postorius/postorius into lp:postorius. Requested reviews: Mailman Coders (mailman-coders) For more details, see: https://code.launchpad.net/~rajeevs1992/postorius/postorius/+merge/207826 Added a file upload feature to add Mass subscriptions.Also changed email validator to django builtin validator.(DRY) -- https://code.launchpad.net/~rajeevs1992/postorius/postorius/+merge/207826 Your team Mailman Coders is requested to review the proposed merge of lp:~rajeevs1992/postorius/postorius into lp:postorius.

2 2

[Merge] lp:~barry/mailman/py3 into lp:mailman
by Barry Warsaw Jan. 5, 2015

Jan. 5, 2015

Barry Warsaw has proposed merging lp:~barry/mailman/py3 into lp:mailman. Requested reviews: Mailman Coders (mailman-coders) For more details, see: https://code.launchpad.net/~barry/mailman/py3/+merge/245313 Ports Mailman 3 core to Python 3.4 -- Your team Mailman Coders is requested to review the proposed merge of lp:~barry/mailman/py3 into lp:mailman.

2 1

[Bug 1158721] [NEW] Emails without a text/plain part crash the Approved rule
by Aurélien Bompard Dec. 30, 2014

Dec. 30, 2014

Public bug reported: If an email has no plain text part, the Approved rule will crash with the following message: AttributeError: 'NoneType' object has no attribute 'get_payload' The attached patch fixes the bug and adds a unit test for it. It's been generated from the current bzr branch. ** Affects: mailman Importance: Undecided Status: New ** Patch added: "mailman-notextpart.patch" https://bugs.launchpad.net/bugs/1158721/+attachment/3590081/+files/mailman-… -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1158721 Title: Emails without a text/plain part crash the Approved rule To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1158721/+subscriptions

2 5

[Bug 1400520] [NEW] /addresses/<email>/user REST resource needs documentation
by Barry Warsaw Dec. 30, 2014

Dec. 30, 2014

Public bug reported: In LP: #1312884 some great work was added to provide a user subresource for addresses, which can be used to link, auto-create, unlink and change the user associated with the address. Very good unittests were included in the final branch. However, the new functionality needs to be documented in rest/docs/users.rst. Aurelien, I'm assigning this to you, but if you'd rather I write it, please assign back to me. ** Affects: mailman Importance: High Assignee: Aurélien Bompard (abompard) Status: Triaged ** Tags: mailman3 -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1400520 Title: /addresses/<email>/user REST resource needs documentation To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1400520/+subscriptions

2 4

[Bug 1312884] [NEW] REST: creating a user from an existing address
by Aurélien Bompard Dec. 30, 2014

Dec. 30, 2014

Public bug reported: I'm trying to create a User in Mailman 3 from an existing Address. The existing address is currently unlinked and unverified, and I'd like to create a user and link it to it Here's what Barry suggests : You might want to create the user and link to it from the existing address resource, i.e. the AnAddress class. Right now, you cannot get directly from the address to its linked user, if it has one. Look at _AddressBase._resource_as_dict() in addresses.py. Sketching out an approach: - The address resource would gain a `user` key if there is a linked user. This would be a url to the user resource. If the address is unlinked, then there would be no such key. - AnAddress would gain a /user sub-resource which would either 404 if the address is unlinked or return the linked user's JSON data. - POSTing to /addresses/<email>/user would create a new user and link the address to it, but only if the address is not yet linked. If it's already linked, you'd get a 409 Conflict error. If the new user is created, you'd get 201 Created response. Additional use cases to consider: Q. What if the address is already linked and you want to unlink it? A. DELETE on the /addresses/<email>/user resource Q. What if you want to change the user the address is linked to? A. Currently, it's two operations: DELETE the existing link as above, then see below. The other option is to allow a PUT on the /addresses/<email>/user resource. Q. What if you want to link the unlinked address to an existing user? A. Maybe allow this in the POST to /addresses/<email>/user, i.e. the user would only be created if it doesn't yet exist. If it gets created, you'd get a 201 Created, but if it's just a link operation to an existing user, you'd get a 200 Success response. It's possibly you'd want to include POST boolean that specifies whether creating a new user is okay or not, with the appropriate response codes in the false case. ** Affects: mailman Importance: Undecided Status: New -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1312884 Title: REST: creating a user from an existing address To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1312884/+subscriptions

3 10

[Bug 1170347] [NEW] Digest handler crashes on non-ascii messages with no encoding header
by Aurélien Bompard Dec. 30, 2014

Dec. 30, 2014

Public bug reported: I found an interesting bug in the to_digest handler: when a message is not ascii only but has no encoding defined (which happens in the wild unfortunately), the mailbox will crash when adding the message on line 58 of handlers/to_digest.py. The reason is that due to the mailman-specific email.message.Message modifications, some of the headers are unicode strings. As a consequence, when the mailbox.add code from the standard library serializes the email, it merges unicode strings with bytestrings, which result in an unicode string. If this string contains non-ascii characters, python will raise an UnicodeDecodeError doing so. The solution for that is to call the as_string() method before adding to the mailbox, it will do the serialization properly. I'm attaching the fix (one-liner) and a testcase to reproduce it (much larger). ** Affects: mailman Importance: Undecided Status: New ** Tags: mailman3 ** Patch added: "mailman-digestunicode.patch" https://bugs.launchpad.net/bugs/1170347/+attachment/3647526/+files/mailman-… -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1170347 Title: Digest handler crashes on non-ascii messages with no encoding header To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1170347/+subscriptions

2 6

[Bug 1130957] [NEW] Unicode errors in mailman3
by Aurélien Bompard Dec. 30, 2014

Dec. 30, 2014

Public bug reported: This is with mailman3. I'm getting the following unicode errors: Feb 20 23:07:43 2013 (13496) Uncaught runner exception: Expected unicode, found <type 'str'>: 'foobar(a)example.com' Feb 20 23:07:43 2013 (13496) Traceback (most recent call last): File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 145, in _one_iteration self._process_one_file(msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 228, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/runners/outgoing.py", line 93, in _dispose self._func(mlist, msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/mta/deliver.py", line 93, in deliver refused = agent.deliver(mlist, msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/mta/base.py", line 167, in deliver member = mlist.members.get_member(recipient) File "/srv/dev/mailman/mailman/src/mailman/database/transaction.py", line 78, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/srv/dev/mailman/mailman/src/mailman/model/roster.py", line 111, in get_member Address.email == address, File "/srv/dev/mailman/venv/lib/python2.7/site-packages/storm-0.19-py2.7-linux-x86_64.egg/storm/expr.py", line 412, in __eq__ other = getattr(self, "variable_factory", Variable)(value=other) File "/srv/dev/mailman/venv/lib/python2.7/site-packages/storm-0.19-py2.7-linux-x86_64.egg/storm/variables.py", line 396, in parse_set % (type(value), value)) TypeError: Expected unicode, found <type 'str'>: 'foobar(a)example.com' Feb 20 23:07:43 2013 (13496) SHUNTING: 1361401663.441109+267302c50d71d512974fe5caa7aa0214eab699f6 ----------------- Feb 20 22:20:45 2013 (13337) Uncaught runner exception: decoding Unicode is not supported Feb 20 22:20:45 2013 (13337) Traceback (most recent call last): File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 145, in _one_iteration self._process_one_file(msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 228, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/runners/digest.py", line 344, in _dispose rfc1153_digest.add_message(message, count) File "/srv/dev/mailman/mailman/src/mailman/runners/digest.py", line 269, in add_message payload = unicode(payload, 'us-ascii', 'replace') TypeError: decoding Unicode is not supported Feb 20 22:20:45 2013 (13337) SHUNTING: 1361398845.847068+d12dc2ef44dcca60ade2fc94f6f830bd09a6b912 Feb 20 22:20:52 2013 (13337) Uncaught runner exception: decoding Unicode is not supported Feb 20 22:20:52 2013 (13337) Traceback (most recent call last): File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 145, in _one_iteration self._process_one_file(msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 228, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/runners/digest.py", line 344, in _dispose rfc1153_digest.add_message(message, count) File "/srv/dev/mailman/mailman/src/mailman/runners/digest.py", line 269, in add_message payload = unicode(payload, 'us-ascii', 'replace') TypeError: decoding Unicode is not supported Feb 20 22:20:52 2013 (13337) SHUNTING: 1361398852.901291+bc2f1ab7dad828020238e06815fd2b4adab0fe61 The attached patch fixes both tracebacks. ** Affects: mailman Importance: Undecided Status: New ** Tags: mailman3 ** Patch added: "mailman-unicode.patch" https://bugs.launchpad.net/bugs/1130957/+attachment/3539184/+files/mailman-… -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1130957 Title: Unicode errors in mailman3 To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1130957/+subscriptions

5 12

[Bug 1130696] [NEW] AttributeError: Message instance has no attribute 'sender'
by Aurélien Bompard Dec. 30, 2014

Dec. 30, 2014

Public bug reported: This is with mailman3: Feb 20 14:23:07 2013 (5721) Uncaught runner exception: MIMEMultipart instance has no attribute 'sender' Feb 20 14:23:07 2013 (5721) Traceback (most recent call last): File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 145, in _one_iteration self._process_one_file(msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 218, in _process_one_file elif msg.sender: AttributeError: MIMEMultipart instance has no attribute 'sender' Feb 20 14:23:07 2013 (5721) SHUNTING: 1361370187.943257+1fba5014964bdeae413f5280cd0f4d2cb5f7727b Feb 20 14:23:07 2013 (5721) Uncaught runner exception: Message instance has no attribute 'sender' Feb 20 14:23:07 2013 (5721) Traceback (most recent call last): File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 145, in _one_iteration self._process_one_file(msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 218, in _process_one_file elif msg.sender: AttributeError: Message instance has no attribute 'sender' Feb 20 14:23:08 2013 (5721) SHUNTING: 1361370187.973463+909c4b5e80497ec67e97765281e7ada77a865143 The message seems to be originating from mailman, it's a digest. I'm attaching a very simple patch that handles the case where there is no sender header. ** Affects: mailman Importance: Undecided Status: New ** Tags: easy mailman3 ** Patch added: "mailman-nosender.patch" https://bugs.launchpad.net/bugs/1130696/+attachment/3538395/+files/mailman-… -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1130696 Title: AttributeError: Message instance has no attribute 'sender' To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1130696/+subscriptions

2 4

[Bug 1060951] [NEW] Bug getting i18n'ed attachment filenames (RFC2231)
by Aurélien Bompard Dec. 30, 2014

Dec. 30, 2014

Public bug reported: RFC 2231 allows filenames to have non-ascii characters. The get_filename() method in Python's Message class handles this by calling email.utils.collapse_rfc2231_value() at the end of get_filename. This method returns the filename in Unicode. This fails in Mailman because the mailman.email.message.Message class has a wrapper around get() and __getitem__() to return unicode headers. As a result, the collapse_rfc2231_value() tries to transforms into unicode an already unicode string, and I get the following exception: File "/usr/lib/python2.7/email/utils.py", line 319, in collapse_rfc2231_value return unicode(rawval, charset, errors) TypeError: decoding Unicode is not supported A possible solution to this would be to make Mailman's Message get_filename() method be more than just an exception-catching wrapper, and re-implement the original get_filename() method, inserting a conversion to str before calling collapse_rfc2231_value(). Does this make sense ? Any other idea for a possible solution ? ** Affects: mailman Importance: Undecided Status: New -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1060951 Title: Bug getting i18n'ed attachment filenames (RFC2231) To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1060951/+subscriptions

3 8