Hi folks,
I've run into an apparent limit in size - we send newsletters through one of our lists, and they can get large. One in particular runs over 20MB, and I get an error on the held messages page trying to process it. Currently we have version 2.1.20, I know there are updates but not if they address this limit or if there's some other fix.
--Bryan
-- Bryan Blackwell -- Linux Systems Engineer bryan@skiblack.com
At Wed, 5 Jun 2019 17:18:04 -0400 Bryan Blackwell bryan@skiblack.com wrote:
Hi folks,
I've run into an apparent limit in size - we send newsletters through one of our lists, and they can get large. One in particular runs over 20MB, and I get an error on the held messages page trying to process it. Currently we have version 2.1.20, I know there are updates but not if they address this limit or if there's some other fix.
I wonder if this is *mailman* or your MTA that is complaining...
--Bryan
-- Bryan Blackwell -- Linux Systems Engineer bryan@skiblack.com
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/heller%40deepsoft.com
-- Robert Heller -- 978-544-6933 Deepwoods Software -- Custom Software Services http://www.deepsoft.com/ -- Linux Administration Services heller@deepsoft.com -- Webhosting Services
On 6/5/19 3:18 PM, Bryan Blackwell wrote:
Hi folks,
Hi,
I've run into an apparent limit in size - we send newsletters through one of our lists, and they can get large. One in particular runs over 20MB, and I get an error on the held messages page trying to process it.
I'm guessing that you are aware of the Maximum length (max_message_size) setting on the General Options page.
Remember that Base64 attachments become 4/3s their size. So a 20 MB file attachment becomes ~27 MB of Base64 encoded content.
Also remember that the MTA's maximum message size will add any headers and body content onto the size. So I'm guessing you want to be able to support 30 MB email messages. This should be easily doable. Even if some people (myself included) don't like it.
Currently we have version 2.1.20, I know there are updates but not if they address this limit or if there's some other fix.
I'm guessing that this is a Mailman and / or MTA configuration issue.
The fact that the message makes it into Mailman tells me that the MTA can handle the attachment. I wonder if simply raising the Maximum length (max_message_size) might allow the message to pass normally.
I don't know what to think about the pending moderator requests / hold page's behavior. I'll defer to other more knowledgeable people on the mailing list.
-- Grant. . . . unix || die
On 6/5/19 2:18 PM, Bryan Blackwell wrote:
I've run into an apparent limit in size - we send newsletters through one of our lists, and they can get large. One in particular runs over 20MB, and I get an error on the held messages page trying to process it. Currently we have version 2.1.20, I know there are updates but not if they address this limit or if there's some other fix.
Others have answered, but what exactly is the error?
And if it is the Mailman "we hit a bug" error, what's in Mailman's error log?
Also as noted by Grant, you can set the web admin General Options -> max_message_size to zero to avoid this check.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jun 5, 2019, at 8:13 PM, Mark Sapiro
On 6/5/19 2:18 PM, Bryan Blackwell wrote:
I've run into an apparent limit in size - we send newsletters through one of our lists, and they can get large. One in particular runs over 20MB, and I get an error on the held messages page trying to process it. Currently we have version 2.1.20, I know there are updates but not if they address this limit or if there's some other fix.
Others have answered, but what exactly is the error?
And if it is the Mailman "we hit a bug" error, what's in Mailman's error log?
Also as noted by Grant, you can set the web admin General Options -> max_message_size to zero to avoid this check.
Yep, I get the "Bug in Mailman version 2.1.20" error page. This setup handles other attachments pretty well, just this particular person posts much larger files. Below is the log entry, it looks like an out of memory condition. Perhaps I need to tune python itself? Thanks for any clues. --Bryan Jun 06 08:45:56 2019 admin(3662): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ admin(3662): [----- Mailman Version: 2.1.20 -----] admin(3662): [----- Traceback ------] admin(3662): Traceback (most recent call last): admin(3662): File "/home/mailman/scripts/driver", line 117, in run_main admin(3662): main() admin(3662): File "/home/mailman/Mailman/Cgi/admindb.py", line 203, in main admin(3662): process_form(mlist, doc, cgidata) admin(3662): File "/home/mailman/Mailman/Cgi/admindb.py", line 810, in process_form admin(3662): forward, forwardaddr) admin(3662): File "/home/mailman/Mailman/ListAdmin.py", line 167, in HandleRequest admin(3662): forward, addr) admin(3662): File "/home/mailman/Mailman/ListAdmin.py", line 301, in __handlepost admin(3662): inq.enqueue(msg, _metadata=msgdata) admin(3662): File "/home/mailman/Mailman/Queue/Switchboard.py", line 110, in enqueue admin(3662): msgsave = cPickle.dumps(_msg, protocol) admin(3662): MemoryError: out of memory admin(3662): [----- Python Information -----] admin(3662): sys.version = 2.7.10 (default, Sep 24 2015, 17:50:09) [GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] admin(3662): sys.executable = /bin/python admin(3662): sys.prefix = /usr admin(3662): sys.exec_prefix = /usr admin(3662): sys.path = ['/home/mailman/pythonlib', '/home/mailman', '/home/mailman/scripts', '/home/mailman', '/usr/lib64/python27.zip', '/usr/lib64/python2.7/', '/usr/lib64/python2.7/plat-linux2', '/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old', '/usr/lib64/python2.7/lib-dynload', '/usr/lib/python2.7/site-packages'] admin(3662): sys.platform = linux2 admin(3662): [----- Environment Variables -----] admin(3662): HTTP_REFERER: http://www.vv.corvair.org/mailman/admindb/chapters http://www.vv.corvair.org/mailman/admindb/chapters admin(3662): CONTEXT_DOCUMENT_ROOT: /home/mailman/cgi-bin/ admin(3662): SERVER_SOFTWARE: Apache admin(3662): CONTEXT_PREFIX: /mailman/ admin(3662): SERVER_SIGNATURE: admin(3662): REQUEST_METHOD: POST admin(3662): PATH_INFO: /chapters admin(3662): HTTP_ORIGIN: http://www.vv.corvair.org http://www.vv.corvair.org/ admin(3662): SERVER_PROTOCOL: HTTP/1.1 admin(3662): QUERY_STRING: admin(3662): CONTENT_LENGTH: 168 admin(3662): HTTP_USER_AGENT: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36 admin(3662): HTTP_CONNECTION: Keep-Alive admin(3662): HTTP_COOKIE: chapters+admin=280200000069ff0af95c732800000032643764353463336365326365633863323232393436343631383533386666373665396563336336; __utma=58530232.852040186.1559307710.1559307710.1559307710.1; __utmz=58530232.1559307710.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none) admin(3662): SERVER_NAME: www.vv.corvair.org http://www.vv.corvair.org/ admin(3662): REMOTE_ADDR: 209.83.109.196 admin(3662): PATH_TRANSLATED: /corsa/vv/chapters admin(3662): SERVER_PORT: 80 admin(3662): SERVER_ADDR: 192.168.1.13 admin(3662): DOCUMENT_ROOT: /corsa/vv admin(3662): PYTHONPATH: /home/mailman admin(3662): SCRIPT_FILENAME: /home/mailman/cgi-bin/admindb admin(3662): SERVER_ADMIN: webmaster@tiger.skiblack.com mailto:webmaster@tiger.skiblack.com admin(3662): SCRIPT_URI: http://www.vv.corvair.org/mailman/admindb/chapters http://www.vv.corvair.org/mailman/admindb/chapters admin(3662): HTTP_HOST: www.vv.corvair.org http://www.vv.corvair.org/ admin(3662): SCRIPT_URL: /mailman/admindb/chapters admin(3662): HTTP_UPGRADE_INSECURE_REQUESTS: 1 admin(3662): HTTP_CACHE_CONTROL: max-age=0 admin(3662): REQUEST_URI: /mailman/admindb/chapters admin(3662): HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3 admin(3662): GATEWAY_INTERFACE: CGI/1.1 admin(3662): HTTP_X_FORWARDED_FOR: 10.10.24.209 admin(3662): SCRIPT_NAME: /mailman/admindb admin(3662): REMOTE_PORT: 56644 admin(3662): HTTP_ACCEPT_LANGUAGE: en-US,en;q=0.9 admin(3662): REQUEST_SCHEME: http admin(3662): CONTENT_TYPE: application/x-www-form-urlencoded admin(3662): HTTP_ACCEPT_ENCODING: gzip, deflate
On 6/6/2019 4:58 PM, Bryan Blackwell wrote:
This setup handles other attachments pretty well, just this particular person posts much larger files. Below is the log entry, it looks like an out of memory condition. Perhaps I need to tune python itself? This really sounds like a "too many errors in line, make fewer" problem (tell the guy to send smaller attachments). I know if someone was sending 20MB attachments on a list -I'm- on, we'd be having a polite discussion about it.
Later,
z! who tends to set the max message size to maybe 40KB to discourage non-text traffic
On 6/6/19 4:58 PM, Bryan Blackwell wrote:
Yep, I get the "Bug in Mailman version 2.1.20" error page. This setup handles other attachments pretty well, just this particular person posts much larger files. Below is the log entry, it looks like an out of memory condition. Perhaps I need to tune python itself? Thanks for any clues.
Yes, it is an out of memory condition, but it is not Python that needs to be tuned. It is most likely at the web server or OS level.
But it seems that if you increase max_message_size or set it zero (unlimited) the messages can be processed without being held.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (5)
-
Bryan Blackwell -
Carl Zwanzig -
Grant Taylor -
Mark Sapiro -
Robert Heller