Exception when trying to send mail to list

Hi,
Recently I’ve started seeing some weird exceptions when trying to send emails to a list with mailman 3 running in docker:
From mailman core logs:
sqlalchemy.exc.DataError: (pymysql.err.DataError) (1406, "Data too long for column 'value' at row 1") [SQL: 'INSERT INTO pendedkeyvalue (key
, value, pended_id) VALUES (%(key)s, %(value)s, %(pended_id)s)'] [parameters: {'key': '_pck_rule_misses', 'value': '"\\u0080\\u0003]q\\u0000(X\\u0010\\u0000\\u0000\\u0000dmarc-mitigationq\\u0001X\\n\\u0000\\u0000\\u0000no-sendersq\\u0002X\\b\\u0000\\u0000\\u0000app ... (382 characters truncated) ... \\u000bX\\b\\u0000\\u0000\\u0000max-sizeq\\fX\\u000f\\u0000\\u0000\\u0000news-moderationq\\rX\\u0011\\u0000\\u0000\\u0000suspicious-headerq\\u000ee."', 'pended_id': 12}] (Background on this error at: http://sqlalche.me/e/9h9h)
Jul 25 07:33:28 2018 (32) SHUNTING: 1532504008.7233553+2ca45fec09fa01b9dd31ed584a06b04302ea155b
From mailman web logs:
TypeError: ord() expected a character, but string of length 0 found ERROR 2018-07-25 07:42:41,098 22515 hyperkitty.lib.utils Failed to update the fulltext index: ord() expected a character, but string of length 0 found Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/hyperkitty/lib/utils.py", line 186, in run_with_lock fn(*args, **kwargs) File "/usr/local/lib/python2.7/site-packages/hyperkitty/search_indexes.py", line 87, in update_index update_cmd.update_backend("hyperkitty", "default") File "/usr/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 257, in update_backend commit=self.commit, max_retries=self.max_retries) File "/usr/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 84, in do_update backend.update(index, current_qs, commit=commit) File "/usr/local/lib/python2.7/site-packages/haystack/backends/whoosh_backend.py", line 178, in update self.setup() File "/usr/local/lib/python2.7/site-packages/haystack/backends/whoosh_backend.py", line 126, in setup self.index = self.storage.open_index(schema=self.schema) File "/usr/local/lib/python2.7/site-packages/whoosh/filedb/filestore.py", line 176, in open_index return indexclass(self, schema=schema, indexname=indexname) File "/usr/local/lib/python2.7/site-packages/whoosh/index.py", line 421, in __init__ TOC.read(self.storage, self.indexname, schema=self._schema) File "/usr/local/lib/python2.7/site-packages/whoosh/index.py", line 632, in read check_size("int", _INT_SIZE) File "/usr/local/lib/python2.7/site-packages/whoosh/index.py", line 626, in check_size sz = stream.read_varint() File "/usr/local/lib/python2.7/site-packages/whoosh/filedb/structfile.py", line 191, in read_varint return read_varint(self.read) File "/usr/local/lib/python2.7/site-packages/whoosh/util/varints.py", line 102, in read_varint b = ord(readfn(1)) TypeError: ord() expected a character, but string of length 0 found
Could anybody guide me as to what’s wrong?
–Kamran

Thanks.
I am using the docker images from here: https://github.com/maxking/docker-mailman. Not sure how to check what version of HyperKitty is running inside it?
Also, I’m afraid you’re going to have to be a bit more descriptive. How do I delete the fulltext_index and rebuild the cache?
–Kamran

On 07/25/2018 01:23 AM, Simon Hanna wrote:
It is not those issues. It is the issue discussed at <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/P...>.
This was attributed to a change of Python versions in the past. Can you delete the fulltext_index and rebuild the cache?
The Django management command 'rebuild_index' should fix this.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 07/26/2018 08:36 AM, Kamran Soomro wrote:
I replied too quickly. The initial issue definitely seems to be the one referred to by Simon. However, that should have been fixed by <https://gitlab.com/mailman/mailman/merge_requests/333> which is included in Mailman 3.2.0.
I'm not sure what the state of the Docker containers is. @maxking will have to address that, but the HyperKitty fulltext index error indicates you have a version in which HyperKitty is Python 3, and I would have thought any such Docker image would also have !333 in core.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 07/26/2018 11:53 AM, Kamran Soomro wrote:
Is there any way I can check which version is installed in the docker container?
You actually want a finer grained state than just version. The 'mailman info' command will tell you the version, but you really need to look at mailman/docs/NEWS.rst and see if it has a 3.2.0 -- "La Villa Strangiato" heading and under that
- A new SQLAlchemy column type
SAUnicodeXL
has been implemented to support large columns in MySQL and is used for thevalue
column of thependedkeyvalue
table. (Closes #385)
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Thanks.
I am using the docker images from here: https://github.com/maxking/docker-mailman. Not sure how to check what version of HyperKitty is running inside it?
Also, I’m afraid you’re going to have to be a bit more descriptive. How do I delete the fulltext_index and rebuild the cache?
–Kamran

On 07/25/2018 01:23 AM, Simon Hanna wrote:
It is not those issues. It is the issue discussed at <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/P...>.
This was attributed to a change of Python versions in the past. Can you delete the fulltext_index and rebuild the cache?
The Django management command 'rebuild_index' should fix this.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 07/26/2018 08:36 AM, Kamran Soomro wrote:
I replied too quickly. The initial issue definitely seems to be the one referred to by Simon. However, that should have been fixed by <https://gitlab.com/mailman/mailman/merge_requests/333> which is included in Mailman 3.2.0.
I'm not sure what the state of the Docker containers is. @maxking will have to address that, but the HyperKitty fulltext index error indicates you have a version in which HyperKitty is Python 3, and I would have thought any such Docker image would also have !333 in core.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 07/26/2018 11:53 AM, Kamran Soomro wrote:
Is there any way I can check which version is installed in the docker container?
You actually want a finer grained state than just version. The 'mailman info' command will tell you the version, but you really need to look at mailman/docs/NEWS.rst and see if it has a 3.2.0 -- "La Villa Strangiato" heading and under that
- A new SQLAlchemy column type
SAUnicodeXL
has been implemented to support large columns in MySQL and is used for thevalue
column of thependedkeyvalue
table. (Closes #385)
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Kamran Soomro
-
Mark Sapiro
-
Simon Hanna