Mailman 3 October 2014 - Mailman-coders

[Bug 1312884] [NEW] REST: creating a user from an existing address
by Aurélien Bompard Dec. 30, 2014

Dec. 30, 2014

Public bug reported: I'm trying to create a User in Mailman 3 from an existing Address. The existing address is currently unlinked and unverified, and I'd like to create a user and link it to it Here's what Barry suggests : You might want to create the user and link to it from the existing address resource, i.e. the AnAddress class. Right now, you cannot get directly from the address to its linked user, if it has one. Look at _AddressBase._resource_as_dict() in addresses.py. Sketching out an approach: - The address resource would gain a `user` key if there is a linked user. This would be a url to the user resource. If the address is unlinked, then there would be no such key. - AnAddress would gain a /user sub-resource which would either 404 if the address is unlinked or return the linked user's JSON data. - POSTing to /addresses/<email>/user would create a new user and link the address to it, but only if the address is not yet linked. If it's already linked, you'd get a 409 Conflict error. If the new user is created, you'd get 201 Created response. Additional use cases to consider: Q. What if the address is already linked and you want to unlink it? A. DELETE on the /addresses/<email>/user resource Q. What if you want to change the user the address is linked to? A. Currently, it's two operations: DELETE the existing link as above, then see below. The other option is to allow a PUT on the /addresses/<email>/user resource. Q. What if you want to link the unlinked address to an existing user? A. Maybe allow this in the POST to /addresses/<email>/user, i.e. the user would only be created if it doesn't yet exist. If it gets created, you'd get a 201 Created, but if it's just a link operation to an existing user, you'd get a 200 Success response. It's possibly you'd want to include POST boolean that specifies whether creating a new user is okay or not, with the appropriate response codes in the false case. ** Affects: mailman Importance: Undecided Status: New -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1312884 Title: REST: creating a user from an existing address To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1312884/+subscriptions

3 10

[Bug 1170347] [NEW] Digest handler crashes on non-ascii messages with no encoding header
by Aurélien Bompard Dec. 30, 2014

Dec. 30, 2014

Public bug reported: I found an interesting bug in the to_digest handler: when a message is not ascii only but has no encoding defined (which happens in the wild unfortunately), the mailbox will crash when adding the message on line 58 of handlers/to_digest.py. The reason is that due to the mailman-specific email.message.Message modifications, some of the headers are unicode strings. As a consequence, when the mailbox.add code from the standard library serializes the email, it merges unicode strings with bytestrings, which result in an unicode string. If this string contains non-ascii characters, python will raise an UnicodeDecodeError doing so. The solution for that is to call the as_string() method before adding to the mailbox, it will do the serialization properly. I'm attaching the fix (one-liner) and a testcase to reproduce it (much larger). ** Affects: mailman Importance: Undecided Status: New ** Tags: mailman3 ** Patch added: "mailman-digestunicode.patch" https://bugs.launchpad.net/bugs/1170347/+attachment/3647526/+files/mailman-… -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1170347 Title: Digest handler crashes on non-ascii messages with no encoding header To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1170347/+subscriptions

2 6

[Bug 1130957] [NEW] Unicode errors in mailman3
by Aurélien Bompard Dec. 30, 2014

Dec. 30, 2014

Public bug reported: This is with mailman3. I'm getting the following unicode errors: Feb 20 23:07:43 2013 (13496) Uncaught runner exception: Expected unicode, found <type 'str'>: 'foobar(a)example.com' Feb 20 23:07:43 2013 (13496) Traceback (most recent call last): File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 145, in _one_iteration self._process_one_file(msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 228, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/runners/outgoing.py", line 93, in _dispose self._func(mlist, msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/mta/deliver.py", line 93, in deliver refused = agent.deliver(mlist, msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/mta/base.py", line 167, in deliver member = mlist.members.get_member(recipient) File "/srv/dev/mailman/mailman/src/mailman/database/transaction.py", line 78, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/srv/dev/mailman/mailman/src/mailman/model/roster.py", line 111, in get_member Address.email == address, File "/srv/dev/mailman/venv/lib/python2.7/site-packages/storm-0.19-py2.7-linux-x86_64.egg/storm/expr.py", line 412, in __eq__ other = getattr(self, "variable_factory", Variable)(value=other) File "/srv/dev/mailman/venv/lib/python2.7/site-packages/storm-0.19-py2.7-linux-x86_64.egg/storm/variables.py", line 396, in parse_set % (type(value), value)) TypeError: Expected unicode, found <type 'str'>: 'foobar(a)example.com' Feb 20 23:07:43 2013 (13496) SHUNTING: 1361401663.441109+267302c50d71d512974fe5caa7aa0214eab699f6 ----------------- Feb 20 22:20:45 2013 (13337) Uncaught runner exception: decoding Unicode is not supported Feb 20 22:20:45 2013 (13337) Traceback (most recent call last): File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 145, in _one_iteration self._process_one_file(msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 228, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/runners/digest.py", line 344, in _dispose rfc1153_digest.add_message(message, count) File "/srv/dev/mailman/mailman/src/mailman/runners/digest.py", line 269, in add_message payload = unicode(payload, 'us-ascii', 'replace') TypeError: decoding Unicode is not supported Feb 20 22:20:45 2013 (13337) SHUNTING: 1361398845.847068+d12dc2ef44dcca60ade2fc94f6f830bd09a6b912 Feb 20 22:20:52 2013 (13337) Uncaught runner exception: decoding Unicode is not supported Feb 20 22:20:52 2013 (13337) Traceback (most recent call last): File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 145, in _one_iteration self._process_one_file(msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 228, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/runners/digest.py", line 344, in _dispose rfc1153_digest.add_message(message, count) File "/srv/dev/mailman/mailman/src/mailman/runners/digest.py", line 269, in add_message payload = unicode(payload, 'us-ascii', 'replace') TypeError: decoding Unicode is not supported Feb 20 22:20:52 2013 (13337) SHUNTING: 1361398852.901291+bc2f1ab7dad828020238e06815fd2b4adab0fe61 The attached patch fixes both tracebacks. ** Affects: mailman Importance: Undecided Status: New ** Tags: mailman3 ** Patch added: "mailman-unicode.patch" https://bugs.launchpad.net/bugs/1130957/+attachment/3539184/+files/mailman-… -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1130957 Title: Unicode errors in mailman3 To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1130957/+subscriptions

5 12

[Bug 1130696] [NEW] AttributeError: Message instance has no attribute 'sender'
by Aurélien Bompard Dec. 30, 2014

Dec. 30, 2014

Public bug reported: This is with mailman3: Feb 20 14:23:07 2013 (5721) Uncaught runner exception: MIMEMultipart instance has no attribute 'sender' Feb 20 14:23:07 2013 (5721) Traceback (most recent call last): File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 145, in _one_iteration self._process_one_file(msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 218, in _process_one_file elif msg.sender: AttributeError: MIMEMultipart instance has no attribute 'sender' Feb 20 14:23:07 2013 (5721) SHUNTING: 1361370187.943257+1fba5014964bdeae413f5280cd0f4d2cb5f7727b Feb 20 14:23:07 2013 (5721) Uncaught runner exception: Message instance has no attribute 'sender' Feb 20 14:23:07 2013 (5721) Traceback (most recent call last): File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 145, in _one_iteration self._process_one_file(msg, msgdata) File "/srv/dev/mailman/mailman/src/mailman/core/runner.py", line 218, in _process_one_file elif msg.sender: AttributeError: Message instance has no attribute 'sender' Feb 20 14:23:08 2013 (5721) SHUNTING: 1361370187.973463+909c4b5e80497ec67e97765281e7ada77a865143 The message seems to be originating from mailman, it's a digest. I'm attaching a very simple patch that handles the case where there is no sender header. ** Affects: mailman Importance: Undecided Status: New ** Tags: easy mailman3 ** Patch added: "mailman-nosender.patch" https://bugs.launchpad.net/bugs/1130696/+attachment/3538395/+files/mailman-… -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1130696 Title: AttributeError: Message instance has no attribute 'sender' To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1130696/+subscriptions

2 4

[Bug 1060951] [NEW] Bug getting i18n'ed attachment filenames (RFC2231)
by Aurélien Bompard Dec. 30, 2014

Dec. 30, 2014

Public bug reported: RFC 2231 allows filenames to have non-ascii characters. The get_filename() method in Python's Message class handles this by calling email.utils.collapse_rfc2231_value() at the end of get_filename. This method returns the filename in Unicode. This fails in Mailman because the mailman.email.message.Message class has a wrapper around get() and __getitem__() to return unicode headers. As a result, the collapse_rfc2231_value() tries to transforms into unicode an already unicode string, and I get the following exception: File "/usr/lib/python2.7/email/utils.py", line 319, in collapse_rfc2231_value return unicode(rawval, charset, errors) TypeError: decoding Unicode is not supported A possible solution to this would be to make Mailman's Message get_filename() method be more than just an exception-catching wrapper, and re-implement the original get_filename() method, inserting a conversion to str before calling collapse_rfc2231_value(). Does this make sense ? Any other idea for a possible solution ? ** Affects: mailman Importance: Undecided Status: New -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1060951 Title: Bug getting i18n'ed attachment filenames (RFC2231) To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1060951/+subscriptions

3 8

[Bug 1332918] [NEW] Mailman 3.0 ListArchiverSet.__init__() broken
by NotTheEvilOne Nov. 30, 2014

Nov. 30, 2014

Public bug reported: Hi, I'm currently developing an alternative archiver for Mailman 3.0 and ran into the following issue: Jun 21 21:27:13 2014 (23814) Uncaught runner exception: Expected unicode, found <type 'str'>: 'my_archiver_name' Jun 21 21:27:13 2014 (23814) Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/mailman/core/runner.py", line 169, in _one_iteration self._process_one_file(msg, msgdata) File "/usr/lib/python2.7/dist-packages/mailman/core/runner.py", line 252, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/python2.7/dist-packages/mailman/runners/pipeline.py", line 36, in _dispose process(mlist, msg, msgdata, pipeline) File "/usr/lib/python2.7/dist-packages/mailman/core/pipelines.py", line 66, in process handler.process(mlist, msg, msgdata) File "/usr/lib/python2.7/dist-packages/mailman/handlers/rfc_2369.py", line 120, in process process(mlist, msg, msgdata) File "/usr/lib/python2.7/dist-packages/mailman/handlers/rfc_2369.py", line 87, in process archiver_set = IListArchiverSet(mlist) File "/usr/lib/python2.7/dist-packages/zope/component/_api.py", line 156, in adapter_hook return sitemanager.queryAdapter(object, interface, name, default) File "/usr/lib/python2.7/dist-packages/zope/component/registry.py", line 228, in queryAdapter return self.adapters.queryAdapter(object, interface, name, default) File "/usr/lib/python2.7/dist-packages/mailman/model/mailinglist.py", line 591, in __init__ ListArchiver.name == archiver_name)).one() File "/usr/lib/python2.7/dist-packages/storm/expr.py", line 412, in __eq__ other = getattr(self, "variable_factory", Variable)(value=other) File "/usr/lib/python2.7/dist-packages/storm/variables.py", line 396, in parse_set % (type(value), value)) Please find attached a patch. ** Affects: mailman Importance: Undecided Status: New ** Patch added: "Convert "archiver_name" into unicode in "ListArchiverSet.__init__()"" https://bugs.launchpad.net/bugs/1332918/+attachment/4136576/+files/0002-mai… -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1332918 Title: Mailman 3.0 ListArchiverSet.__init__() broken To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1332918/+subscriptions

2 2

[Bug 1332919] [NEW] Mailman 3.0 PostgreSQL scheme not up-to-date
by NotTheEvilOne Nov. 30, 2014

Nov. 30, 2014

Public bug reported: Hi, Mailman 3.0 can be used with PostgreSQL as well when changes from "20130406000000" are applied for PostgreSQL as well. "mm_20130406000000.py" doesn't contain the required implementation for PostgreSQL and "postgres.sql" is not up-to-date. ** Affects: mailman Importance: Undecided Status: New -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1332919 Title: Mailman 3.0 PostgreSQL scheme not up-to-date To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1332919/+subscriptions

2 2

[Bug 1157854] [NEW] test_not_enough_parameters hangs on OS X
by Barry Warsaw Nov. 30, 2014

Nov. 30, 2014

Public bug reported: test_not_enough_parameters (mailman.rest.tests.test_users.TestLogin) This test hangs completely on OS X 10.6.8 and 10.8.2. It does not hang on Ubuntu 13.04 AFAICT. Commenting out this test allows the rest of the test suite to run to completion, although I get three test failures (on OS X 10.8.2). test_cannot_create_duplicate_list (mailman.rest.tests.test_lists.TestLists) test_cannot_create_list_in_missing_domain (mailman.rest.tests.test_lists.TestLists) test_existing_user_error (mailman.rest.tests.test_users.TestUsers) Further, C-c leaves the RESTRunner running. ** Affects: mailman Importance: Undecided Status: New ** Tags: mailman3 -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1157854 Title: test_not_enough_parameters hangs on OS X To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1157854/+subscriptions

1 3

[Bug 1343137] [NEW] listarchiver table missing from postgresql schema
by John Naggets Nov. 29, 2014

Nov. 29, 2014

Public bug reported: I am currently testing Mailman3 with PostgreSQL and found out that the listarchiver table is missing from the PostgreSQL schema (database/schema/postgres.sql). It is correctly included in SQLite migration schema though (database/schema/sqlite_20130406000000_01.sql). Due to the missing listarchiver table I get the following exception: Jul 16 17:56:36 2014 (25554) Uncaught runner exception: relation "listarchiver" does not exist LINE 1: ...tarchiver.mailing_list_id, listarchiver.name FROM listarchiv... ^ Jul 16 17:56:36 2014 (25554) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/mailman/core/runner.py", line 169, in _one_iteration self._process_one_file(msg, msgdata) File "/usr/local/lib/python2.7/dist-packages/mailman/core/runner.py", line 252, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/local/lib/python2.7/dist-packages/mailman/runners/pipeline.py", line 36, in _dispose process(mlist, msg, msgdata, pipeline) File "/usr/local/lib/python2.7/dist-packages/mailman/core/pipelines.py", line 66, in process handler.process(mlist, msg, msgdata) File "/usr/local/lib/python2.7/dist-packages/mailman/handlers/rfc_2369.py", line 120, in process process(mlist, msg, msgdata) File "/usr/local/lib/python2.7/dist-packages/mailman/handlers/rfc_2369.py", line 87, in process archiver_set = IListArchiverSet(mlist) File "/usr/local/lib/python2.7/dist-packages/zope/component/hookable.py", line 33, in __call__ return self.__implementation(*args, **kw) File "/usr/local/lib/python2.7/dist-packages/zope/component/_api.py", line 156, in adapter_hook return sitemanager.queryAdapter(object, interface, name, default) File "/usr/local/lib/python2.7/dist-packages/zope/interface/registry.py", line 234, in queryAdapter return self.adapters.queryAdapter(object, interface, name, default) File "/usr/local/lib/python2.7/dist-packages/mailman/model/mailinglist.py", line 591, in __init__ ListArchiver.name == archiver_name)).one() File "/usr/local/lib/python2.7/dist-packages/storm/store.py", line 1158, in one result = self._store._connection.execute(select) File "/usr/local/lib/python2.7/dist-packages/storm/databases/postgres.py", line 266, in execute return Connection.execute(self, statement, params, noresult) File "/usr/local/lib/python2.7/dist-packages/storm/database.py", line 241, in execute raw_cursor = self.raw_execute(statement, params) File "/usr/local/lib/python2.7/dist-packages/storm/databases/postgres.py", line 276, in raw_execute return Connection.raw_execute(self, statement, params) File "/usr/local/lib/python2.7/dist-packages/storm/database.py", line 374, in raw_execute self._run_execution(raw_cursor, args, params, statement) File "/usr/local/lib/python2.7/dist-packages/storm/database.py", line 388, in _run_execution self._check_disconnect(raw_cursor.execute, *args) File "/usr/local/lib/python2.7/dist-packages/storm/database.py", line 454, in _check_disconnect return function(*args, **kwargs) ProgrammingError: relation "listarchiver" does not exist LINE 1: ...tarchiver.mailing_list_id, listarchiver.name FROM listarchiv... ** Affects: mailman Importance: Undecided Status: New -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1343137 Title: listarchiver table missing from postgresql schema To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1343137/+subscriptions

3 4

[Bug 880613] [NEW] PostgreSQL constraints disabled
by Barry Warsaw Nov. 29, 2014

Nov. 29, 2014

Public bug reported: In src/mailman/database/sql/postgres.sql, several foreign key constraints have been commented out because otherwise the test suite, or other aspects do not work. This is probably because there are cases where Mailman violates these constraints, but SQLite3 (the default database) does not care. All disabled constraints are marked with XXX in the above file. Really, these should be fixed. ** Affects: mailman Importance: Medium Status: Confirmed ** Tags: mailman3 -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/880613 Title: PostgreSQL constraints disabled To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/880613/+subscriptions

1 2