Mailman 3 November 2014 - Mailman-coders

[Bug 1091586] [NEW] Frequent OperationalError: database is locked with sqlite
by Aurélien Bompard Nov. 26, 2014

Nov. 26, 2014

Public bug reported: I'm seeing frequent locking errors with SQLite on the current version of mailman3 from bzr. Here's an example traceback : Traceback (most recent call last): File "/srv/dev/mailman/bin/runner", line 9, in <module> load_entry_point('mailman==3.0.0b2-', 'console_scripts', 'runner')() File "/srv/dev/mailman/mailman3/mailman/src/mailman/bin/runner.py", line 253, in main runner.run() File "/srv/dev/mailman/mailman3/mailman/src/mailman/core/runner.py", line 96, in run filecnt = self._one_iteration() File "/srv/dev/mailman/mailman3/mailman/src/mailman/core/runner.py", line 173, in _one_iteration config.db.commit() File "/srv/dev/mailman/mailman3/mailman/src/mailman/database/base.py", line 77, in commit self.store.commit() File "/srv/dev/mailman/lib/python2.6/site-packages/storm-0.19-py2.6-linux-x86_64.egg/storm/store.py", line 122, in commit self._connection.commit() File "/srv/dev/mailman/lib/python2.6/site-packages/storm-0.19-py2.6-linux-x86_64.egg/storm/databases/sqlite.py", line 126, in commit self.raw_execute("COMMIT", _end=True) File "/srv/dev/mailman/lib/python2.6/site-packages/storm-0.19-py2.6-linux-x86_64.egg/storm/databases/sqlite.py", line 154, in raw_execute return Connection.raw_execute(self, statement, params) File "/srv/dev/mailman/lib/python2.6/site-packages/storm-0.19-py2.6-linux-x86_64.egg/storm/database.py", line 322, in raw_execute self._check_disconnect(raw_cursor.execute, *args) File "/srv/dev/mailman/lib/python2.6/site-packages/storm-0.19-py2.6-linux-x86_64.egg/storm/database.py", line 371, in _check_disconnect return function(*args, **kwargs) sqlite3.OperationalError: database is locked It happens during the normal processing of Mailman, not while using the API (to my knowledge), so it seems different from bug #856004. I can reproduce this easily by letting Mailman work for some time. After the locking error, new messages are stuck in var/queue/pipeline: $ ls var/queue/* var/queue/archive: var/queue/bad: var/queue/bounces: var/queue/command: var/queue/digest: var/queue/in: var/queue/lmtp: var/queue/nntp: var/queue/out: var/queue/pipeline: 1355781960.9870529+18f0acdf07aab616f793ecf76acfa1994e62b3a9.pck 1355799347.787621+213f3a6a971717ad1ba27f3d56b65767747a45da.pck 1355814973.8623891+4e66eeadef9b8c9ae99e4ed022c6c93f5af75c05.pck 1355782349.546113+ee14cb323574b647d9014f742e3f505e388d553e.pck 1355800543.460628+c455625c1ad86a5a77c9c515fcb6d555ba439b19.pck 1355815808.8860879+ca3fb5f45999e47c92756489daa951672ba4304d.pck 1355792739.8787439+19973fbcf763f036655aa0a7d477231246afdfbd.pck 1355802301.7830889+7989a3199c7eb491b65176937c0c1658cca9d46d.pck 1355820018.1813829+5005ded04da3574a7f3cd7472c9d635c577b6c32.pck 1355794764.597765+6cf379ad16c948117bb65f0aed0b3c230268ffd2.pck 1355803681.7455051+52d185cdcaecaf883bd43f375f5253567f5cd568.pck 1355822188.0491021+6a3cab9831d03b4f94475f745fa508c5126116b4.pck 1355794889.8323901+e14e25e342ec59709266040d830d7e12be2a1261.pck 1355805362.052228+422e573a6117edb7aae1acf9150ac58c210cedba.pck 1355822293.2363911+627f13dd2121bcb0f1916bc0448cc3b8adc9ad1f.pck 1355795937.2442579+62193d05ee80c6a22213cf2ff211c135fb7d38d9.pck 1355807374.9844501+8de57ab53819fac59d64705bcc56f86899d22634.pck 1355798662.8335559+554da4e7690e116e6fabee5764ac69829dc25685.pck 1355808273.143533+ed1acbf580c596ed921e062ece575877f0c9304b.pck var/queue/rest: var/queue/retry: var/queue/shunt: 1355776302.1302171+c657867b33bf6e37554ff83ee26f4096a77c1114.pck var/queue/virgin: And I need to restart mailman to make it process them again. The locked caused the message to be shunted. I'll attach the mailman.log file, where you'll see many "database is locked" errors, happening on different requests. ** Affects: mailman Importance: Undecided Status: New -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1091586 Title: Frequent OperationalError: database is locked with sqlite To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1091586/+subscriptions

2 8

[Bug 1236297] [NEW] Unicode errors with PostgreSQL and RawStr fields
by Aurélien Bompard Nov. 26, 2014

Nov. 26, 2014

Public bug reported: When using PostgreSQL as a database, it's impossible to set the moderator_password field of a mailing list, which is of type RawStr. To be precise, it's possible to set it, but getting it back from the database after a commit will cause a TypeError with the message: "Expected str, found <type 'unicode'>" as if PostgreSQL always returned unicode on TEXT fields and Storm did not convert them to bytestrings. I get no errors on an SQLite database. I don't know if the problem comes from Mailman or Storm. I have written a python script to reproduce it, see the attached file. There are comments, it should be easily readable. ** Affects: mailman Importance: Undecided Status: New ** Tags: mailman3 ** Attachment added: "test-mm-storm-unicode.py" https://bugs.launchpad.net/bugs/1236297/+attachment/3864008/+files/test-mm-… -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1236297 Title: Unicode errors with PostgreSQL and RawStr fields To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1236297/+subscriptions

2 5

[Merge] lp:~abompard/mailman/bug-1170347 into lp:mailman
by Aurélien Bompard Nov. 25, 2014

Nov. 25, 2014

Aurélien Bompard has proposed merging lp:~abompard/mailman/bug-1170347 into lp:mailman. Requested reviews: Mailman Coders (mailman-coders) For more details, see: https://code.launchpad.net/~abompard/mailman/bug-1170347/+merge/242761 Fix digest handler crash reported in bug #1170347. -- Your team Mailman Coders is requested to review the proposed merge of lp:~abompard/mailman/bug-1170347 into lp:mailman.

1 0

[Bug 1230130] [NEW] The mailinglist interface specifies the "administrative" attribute instead of "administrivia"
by Aurélien Bompard Nov. 25, 2014

Nov. 25, 2014

Public bug reported: In the IMailingList interface, there is an "administrative" attribute which is said to control "administrivia" checks. But apparently, in the model and in the code, this attribute is called "administrivia", as in Mailman 2.1. Unless I'm missing something, the interface attribute should be renamed. ** Affects: mailman Importance: Undecided Status: New ** Tags: easy mailman3 -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1230130 Title: The mailinglist interface specifies the "administrative" attribute instead of "administrivia" To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1230130/+subscriptions

1 1

[Merge] lp:~abompard/postorius/mockingmmclient into lp:postorius
by Aurélien Bompard Nov. 24, 2014

Nov. 24, 2014

Aurélien Bompard has proposed merging lp:~abompard/postorius/mockingmmclient into lp:postorius. Requested reviews: Mailman Coders (mailman-coders) For more details, see: https://code.launchpad.net/~abompard/postorius/mockingmmclient/+merge/242186 This branch makes use of the new mailmanclient mocking class (which must be merged first). (I also removed a few unused imports) -- Your team Mailman Coders is requested to review the proposed merge of lp:~abompard/postorius/mockingmmclient into lp:postorius.

2 1

[Branch ~mailman-coders/mailman.client/trunk] Rev 58: Addedd an improved test harness using WebTest. Contributed by Aurélien Abompard.
by noreply＠launchpad.net Nov. 24, 2014

Nov. 24, 2014

Merge authors: Aurélien Bompard (abompard) Related merge proposals: https://code.launchpad.net/~abompard/mailman.client/mockingclient/+merge/24… proposed by: Aurélien Bompard (abompard) ------------------------------------------------------------ revno: 58 [merge] committer: Florian Fuchs <flo.fuchs(a)gmail.com> branch nick: mailman.client timestamp: Mon 2014-11-24 11:25:10 +0100 message: Addedd an improved test harness using WebTest. Contributed by Aurélien Abompard. added: src/mailmanclient/tests/utils.py modified: bin/test setup.py src/mailmanclient/NEWS.txt src/mailmanclient/docs/using.txt src/mailmanclient/tests/test_docs.py -- lp:mailman.client https://code.launchpad.net/~mailman-coders/mailman.client/trunk Your team Mailman Coders is subscribed to branch lp:mailman.client. To unsubscribe from this branch go to https://code.launchpad.net/~mailman-coders/mailman.client/trunk/+edit-subsc…

1 0

[Merge] lp:~abompard/mailman.client/mockingclient into lp:mailman.client
by Aurélien Bompard Nov. 24, 2014

Nov. 24, 2014

Aurélien Bompard has proposed merging lp:~abompard/mailman.client/mockingclient into lp:mailman.client. Requested reviews: Mailman Coders (mailman-coders) For more details, see: https://code.launchpad.net/~abompard/mailman.client/mockingclient/+merge/24… This branch adds a mocking client which only instantiates the REST API server in memory, not by forking a specific mailman server. Then, the requests are made through the WSGI API using WebTest. -- Your team Mailman Coders is requested to review the proposed merge of lp:~abompard/mailman.client/mockingclient into lp:mailman.client.

2 1

[Merge] lp:~barry/mailman/falcon into lp:mailman
by Barry Warsaw Nov. 15, 2014

Nov. 15, 2014

Barry Warsaw has proposed merging lp:~barry/mailman/falcon into lp:mailman. Requested reviews: Mailman Coders (mailman-coders) For more details, see: https://code.launchpad.net/~barry/mailman/falcon/+merge/230913 Port from restish to falcon for the REST API framework. This currently depends on an unmerged falcon pull request: https://github.com/racker/falcon/pull/307 -- https://code.launchpad.net/~barry/mailman/falcon/+merge/230913 Your team Mailman Coders is requested to review the proposed merge of lp:~barry/mailman/falcon into lp:mailman.

2 1

[Bug 1392558] [NEW] Display name needs to be rewritten for list address
by Franck Nov. 14, 2014

Nov. 14, 2014

Public bug reported: When in DMARC mode, the list will rewrite the From: to something like From: "John via list" <list(a)mailman.org> Some client when replying to all will move this address with the display part to the To: field From: "Paul" <paul(a)example.com> To: "John via list" <list(a)mailman.org> When the list redistribute the message to subscribers, it keeps the To: as is (as it contains the list email address). The display name should be removed to put back the list address. From: "Paul via List" <list(a)mailman.org> To: "John via list" <list(a)mailman.org> --> From: "Paul via List" <list(a)mailman.org> To: "list" <list(a)mailman.org> ** Affects: mailman Importance: Undecided Status: New -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1392558 Title: Display name needs to be rewritten for list address To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1392558/+subscriptions

1 0

[Merge] lp:~abompard/mailman/manifest into lp:mailman
by Aurélien Bompard Nov. 8, 2014

Nov. 8, 2014

Aurélien Bompard has proposed merging lp:~abompard/mailman/manifest into lp:mailman. Requested reviews: Mailman Coders (mailman-coders) For more details, see: https://code.launchpad.net/~abompard/mailman/manifest/+merge/240575 A single simple commit to include Alembic version files in the MANIFEST.in file (and thus in source distributions). -- https://code.launchpad.net/~abompard/mailman/manifest/+merge/240575 Your team Mailman Coders is requested to review the proposed merge of lp:~abompard/mailman/manifest into lp:mailman.

3 2