Hevy resource footprint forces backdrop to mailman 2.1
Greetings,
after many years of smoothly running a mailman2 instance I had to migrate my virtual server to a new provider. There I found, that mailman2 was no longer supported and tried to migrate to mailman3
.. with much pain and no success. Mailman3 keeps crashing my v-server and swallows almost every mail sent to any of the migrated lists.
So I wanted to switch back to mailman 2.1.39 by installing from the source package and found that this also fails due to missing python2 support.
How to solve this?
Thank you very much
If you have root or sudo access on the vm host you will need to install Python2 support via whatever mechanism the OS uses for this ( such as yum, dnf or apt for various Linux flavors )
On Oct 7, 2022, at 2:27 AM, Martin Lorenz <gitlab@member.holoclan.demailto:gitlab@member.holoclan.de> wrote:
Greetings,
after many years of smoothly running a mailman2 instance I had to migrate my virtual server to a new provider. There I found, that mailman2 was no longer supported and tried to migrate to mailman3
.. with much pain and no success. Mailman3 keeps crashing my v-server and swallows almost every mail sent to any of the migrated lists.
So I wanted to switch back to mailman 2.1.39 by installing from the source package and found that this also fails due to missing python2 support.
How to solve this?
Thank you very much
Mailman-Users mailing list -- mailman-users@python.orgmailto:mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.orgmailto:mailman-users-leave@python.org https://mail.python.org/mailman3/lists/mailman-users.python.org/ Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/ https://mail.python.org/archives/list/mailman-users@python.org/
-- Bruce Johnson University of Arizona College of Pharmacy Information Technology Group
Institutions do not have opinions, merely customs
On 10/7/22 02:27, Martin Lorenz wrote:
Greetings,
after many years of smoothly running a mailman2 instance I had to migrate my virtual server to a new provider. There I found, that mailman2 was no longer supported and tried to migrate to mailman3
.. with much pain and no success. Mailman3 keeps crashing my v-server and swallows almost every mail sent to any of the migrated lists.
If you post more detail to mailman-users@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users@mailman3.org/, we will try to help you with that.
So I wanted to switch back to mailman 2.1.39 by installing from the source package and found that this also fails due to missing python2 support.
You could install Python 2.7 https://www.python.org/downloads/release/python-2718/ but it would be much better to fix your Mailman 3 issues.
Mark Sapiro wrote:
https://www.python.org/downloads/release/python-2718/ but it would be much better to fix your Mailman 3 issues.
I am afraid, mailman3 needs far too much system resources for this virtual server. I had it up and running but system load shoot up to values beyond 10 whenever mailman3 had to process something. htop showed the mailman processes eating up all memory until I couldn't even kill them remotely but had to do a hard shutdown via the management console.
I never had this with mailman2
Martin Lorenz writes:
I am afraid, mailman3 needs far too much system resources for this virtual server.
This whole story doesn't make much sense to me. You switched to a new virtual server because Mailman 2 isn't supported but imply that Mailman 3 is, but you can't get support from your vendor? Do you mean that *Python* 2 is not supported but *Python* 3 is?
Then, as far as I can tell on my hosts, Mailman 3 does not use a lot more resources than Mailman 2.1 to process email, because it's basically the same processing model. By "doing anything," do you mean posting to lists? Migrating lists? Creating and managing lists with Postorius? Migrating archives? Accessing archives and posting via HyperKitty?
Python 2 is *unsupported* and likely to have security issues (in particular, it probably allows SSL/TLS connections now considered insecure). Mailman 2 is also end-of-life, and may have security issues (although Mark continues to accept patches and even fix them occasionally, he keeps promising to stop "soon"). You're asking for endless hurt if you get into trouble with Mailman 2. (I don't think it's likely except maybe a botnet will take your virtual server over for Bitcoin mining or DDoS attacks, but if it happens it will be bad.)
Like Mark, I recommend that you at least let us try to help you to figure out why the virtual server fails to run Mailman 3. Are you sure the new virtual server will handle the load of Mailman 2? Maybe it's just too small for the job of handling mailing lists.
If you're not willing to upgrade, you're going to need to install Python 2, and a few packages. You'll need to get support from your virtual server provider and/or the OS vendor for that, we can't help you. It's possible that they're EOL at the OS vendor as well. (I was looking to install Python 3.8 on my Debian system to support a user who's stuck on 3.8 for corporate QA reasons, and surprise! it's not on their principal mirrors any more. There are Linux distros that never supported Python 2, they were on Python 3 from the start.) Python 2 is deeply embedded in some ecosystems, so most likely your OS vender still delivers the most recent Python 2.7. But you can be sure Python 2-compatible packages are bitrotting in most cases, including those that support Mailman.
Regards, Steve
Stephen J. Turnbull wrote:
Martin Lorenz writes:
I am afraid, mailman3 needs far too much system resources for this virtual server. This whole story doesn't make much sense to me. You switched to a new
virtual server because Mailman 2 isn't supported but imply that Mailman 3 is, but you can't get support from your vendor? Do you mean that *Python* 2 is not supported but *Python* 3 is?
to clarify the history: the provider of my old v-server ceased its service. so I had to switch to a new provider. The new v-server is debian-based. the release is bullseye which does still provide python2 but lacks a few packages that are necessery to install mailman2.1 I would have to manually install dependencies via pip instead of apt.
Then, as far as I can tell on my hosts, Mailman 3 does not use a lot more resources than Mailman 2.1 to process email, because it's basically the same processing model. By "doing anything," do you mean posting to lists? Migrating lists? Creating and managing lists with Postorius? Migrating archives? Accessing archives and posting via HyperKitty?
The problem occurs whenever mail ist sent to a migrated list. All mails sent to the lists I migrated from my old 2.1 instance (23 lists in total) are swallowed and never get delivered. I have not found out where they are. There are no errors in the logs. exim accepts the mail, hands it over to mailman without error but never gets it back for delivery.
A few mails turn up in the archive but must don't.
Python 2 is *unsupported* and likely to have security issues (in particular, it probably allows SSL/TLS connections now considered insecure). Mailman 2 is also end-of-life, and may have security issues (although Mark continues to accept patches and even fix them occasionally, he keeps promising to stop "soon"). You're asking for endless hurt if you get into trouble with Mailman 2. (I don't think it's likely except maybe a botnet will take your virtual server over for Bitcoin mining or DDoS attacks, but if it happens it will be bad.) Like Mark, I recommend that you at least let us try to help you to figure out why the virtual server fails to run Mailman 3. Are you sure the new virtual server will handle the load of Mailman 2? Maybe it's just too small for the job of handling mailing lists.
the new server is way stronger than the old one.
If you're not willing to upgrade, you're going to need to install
I am more than willing to go with mailman3 if I can get it up and running. I currently try to retire all my python2 dependencies so going fo mailman2.1 was only because I quickly needed a working solution - which I did not get. Currently all users and admins of my lists know, that they have to be patient...
thank you for the help you offer. The time I can invest on the subject is rather limited for the coming week. from Oct. 14 on I can dive into it :)
greets martin
Martin Lorenz writes:
so I had to switch to a new provider. The new v-server is debian-based. the release is bullseye which does still provide python2 but lacks a few packages that are necessery to install mailman2.1 I would have to manually install dependencies via pip instead of apt.
As long as you have necessary access to install, that should not be a problem.
The problem occurs whenever mail ist sent to a migrated list. All mails sent to the lists I migrated from my old 2.1 instance (23 lists in total) are swallowed and never get delivered.
That sounds a lot more like MTA configuration problems or network configuration problems than resource usage.
I have not found out where they are.
IIRC Debian puts queues in /var/lib/mailman3/queue/. Look in ../shunt in particular (that queue swallows posts that for some reason Mailman is unable to process, and you have to do something to fix the problem -- there's no automatic retry for those).
There are no errors in the logs.
Which logs are you looking at specifically?
1 was only because I quickly needed a working solution - which I did not get.
Unfortunately, I don't know of a quick working solution from any OS distro. All the distros either provide very old versions, or we have reports of difficulties configuring them. The closest thing to a turnkey solution are Abhilash's docker containers, but even there it's non-trivial. My recommendation (FWIW, now that you've already gotten started it may not make sense to start over from scratch) is that you get yourself a nice stable distro with at least Python 3.8 (better 3.9 or 3.10) installed, browse both the "venv" installation described in our docs at https://docs.mailman3.org/en/latest/install/virtualenv.html and Brian Carpenter's Debian installation guide at https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10. Then choose the guide you find more compatible with your thinking, and follow that.
thank you for the help you offer. The time I can invest on the subject is rather limited for the coming week. from Oct. 14 on I can dive into it :)
OK, no hurry. We'll be here when you're ready.
Steve
On 2022-10-10 23:14, Stephen J. Turnbull wrote:
so I had to switch to a new provider. The new v-server is debian-based. the release is bullseye which does still provide ...
////////////
... browse both the "venv" installation described in our docs at https://docs.mailman3.org/en/latest/install/virtualenv.html and Brian Carpenter's Debian installation guide at https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10. Then choose the guide you find more compatible with your thinking, and follow that.
////////////
See also: "Installing Mailman3 on Debian 11": Tom Kisner -- Jul 9, 2022
https://kn6q.org/installing-mailman-3-on-debian-11
Kelly Close's guide "Mailman 3 Setup Using Ubuntu Bionic" works for Debian Bullseye.
https://docs.google.com/document/d/1xIcSsoNFp2nHi7r4eQys00s9a0k2sHhu1V5Plant...
Now I will start over ...
First of all: do you recommend the debian package install or the manual one?
Package mailman3:
p 3.3.3-1 stable Package mailman3-doc:
p 3.3.3-1 stable Package mailman3-full:
p 3.3.3-1 stable
Package mailman3-web:
p 0+20200530-2 stable
Package python3-django-mailman3:
i 1.3.5-2 stable
Package python3-mailman-hyperkitty:
p 1.1.0-10 stable
Package python3-mailmanclient:
i 3.3.2-1 stable
# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
On 10/14/22 09:56, Martin Lorenz wrote:
Now I will start over ...
First of all: do you recommend the debian package install or the manual one?
Package mailman3: p 3.3.3-1 stable Package mailman3-doc: p 3.3.3-1 stable Package mailman3-full: p 3.3.3-1 stable Package mailman3-web: p 0+20200530-2 stable Package python3-django-mailman3: i 1.3.5-2 stable Package python3-mailman-hyperkitty: p 1.1.0-10 stable Package python3-mailmanclient: i 3.3.2-1 stable # cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
Those Debian packages are mostly quite out of date. If you are willing to tolerate that, installation of the packages should be easier, but judging from posts on the mailman-users@mailman3.org list, not everyone finds it so.
I definitely suggest installing following https://docs.mailman3.org/en/latest/install/virtualenv.html, not only because you will get the benefit of bug fixes that aren't in the Debian packages, but people who might offer help via the mailman-users@mailman3.org list are usually more familiar with the virtualenv install than with the Debian packages.
OK thanks for the advice.
I started installing following the path ...
the first error I hit is:
(venv) mailman@arda:/etc/mailman3$ mailman-web migrate
Traceback (most recent call last):
File "/opt/mailman/venv/lib/python3.9/site-packages/django/template/utils.py", line 66, in __getitem__
return self._engines[alias]
KeyError: 'django'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/mailman/venv/lib/python3.9/site-packages/django/template/backends/django.py", line 121, in get_package_libraries
module = import_module(entry[1])
File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 790, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/opt/mailman/venv/lib/python3.9/site-packages/hyperkitty/templatetags/decorate.py", line 4, in <module>
from hyperkitty.lib.renderer import markdown_renderer, text_renderer
File "/opt/mailman/venv/lib/python3.9/site-packages/hyperkitty/lib/renderer.py", line 6, in <module>
from mistune.plugins.extra import plugin_url
ModuleNotFoundError: No module named 'mistune.plugins.extra'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/mailman/venv/bin/mailman-web", line 8, in <module>
sys.exit(main())
File "/opt/mailman/venv/lib/python3.9/site-packages/mailman_web/manage.py", line 30, in main
execute_from_command_line(sys.argv)
File "/opt/mailman/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/opt/mailman/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/mailman/venv/lib/python3.9/site-packages/django/core/management/base.py", line 328, in run_from_argv
self.execute(*args, **cmd_options)
File "/opt/mailman/venv/lib/python3.9/site-packages/django/core/management/base.py", line 366, in execute
self.check()
File "/opt/mailman/venv/lib/python3.9/site-packages/django/core/management/base.py", line 392, in check
all_issues = self._run_checks(
File "/opt/mailman/venv/lib/python3.9/site-packages/django/core/management/commands/migrate.py", line 64, in _run_checks
issues.extend(super()._run_checks(**kwargs))
File "/opt/mailman/venv/lib/python3.9/site-packages/django/core/management/base.py", line 382, in _run_checks
return checks.run_checks(**kwargs)
File "/opt/mailman/venv/lib/python3.9/site-packages/django/core/checks/registry.py", line 72, in run_checks
new_errors = check(app_configs=app_configs)
File "/opt/mailman/venv/lib/python3.9/site-packages/django/contrib/admin/checks.py", line 76, in check_dependencies
for engine in engines.all():
File "/opt/mailman/venv/lib/python3.9/site-packages/django/template/utils.py", line 90, in all
return [self[alias] for alias in self]
File "/opt/mailman/venv/lib/python3.9/site-packages/django/template/utils.py", line 90, in <listcomp>
return [self[alias] for alias in self]
File "/opt/mailman/venv/lib/python3.9/site-packages/django/template/utils.py", line 81, in __getitem__
engine = engine_cls(params)
File "/opt/mailman/venv/lib/python3.9/site-packages/django/template/backends/django.py", line 25, in __init__
options['libraries'] = self.get_templatetag_libraries(libraries)
File "/opt/mailman/venv/lib/python3.9/site-packages/django/template/backends/django.py", line 43, in get_templatetag_libraries
libraries = get_installed_libraries()
File "/opt/mailman/venv/lib/python3.9/site-packages/django/template/backends/django.py", line 108, in get_installed_libraries
for name in get_package_libraries(pkg):
File "/opt/mailman/venv/lib/python3.9/site-packages/django/template/backends/django.py", line 123, in get_package_libraries
raise InvalidTemplateLibrary(
django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'hyperkitty.templatetags.decorate': No module named 'mistune.plugins.extra'
(venv) mailman@arda:/etc/mailman3$ pip install mistune
Requirement already satisfied: mistune in /opt/mailman/venv/lib/python3.9/site-packages (3.0.0rc1)
On 10/14/22 13:59, Martin Lorenz wrote:
OK thanks for the advice.
I started installing following the path ...
the first error I hit is:
...
File "/opt/mailman/venv/lib/python3.9/site-packages/hyperkitty/lib/renderer.py", line 6, in <module> from mistune.plugins.extra import plugin_url ModuleNotFoundError: No module named 'mistune.plugins.extra'
Sorry, that's https://gitlab.com/mailman/hyperkitty/-/issues/437
It is also not clear if you will also encounter https://gitlab.com/mailman/hyperkitty/-/issues/431
If your HyperKitty is version 1.3.5 installed from PyPI, the simplest solution is to downgrade mistune to 2.0.0rc1 in your venv
pip install mistune==2.0.0rc1
Alternatively you can downgrade to mistune<3.0
pip install mistune<3.0
But then you also need to ensure that line 7 in /opt/mailman/venv/lib/python3.9/site-packages/hyperkitty/lib/renderer.py is
from mistune.util import escape_html, escape_url
and not
from mistune.scanner import escape_html, escape_url
thanks, that one did the trick
I think I kint of got ist up and running now. this ist how the site looks: https://list.poc.im/mailman3/lists/
I don't think, it should be like that. I already added
location /static/ {
alias /opt/mailman/web/static;
}
to my nginx config, but without a change ... what am I missing?
heureka!
added two lines to the nginx config and now it seems up and running ...
_location_ now looks like this:
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
}
will test nor and try to import my old lists tomorrow (it's already way past midnight here)
i think, I hit a bug ....
I had mailman up and running, sent testmails to a testlist, used the webinterface ... but than I played around with the domains. I had added one of my domains for the test list but in the archive interface I still had "example.com" as a header. so I switched over to the domain admin panel and deleted example.com
and now everything is broken ...
I only get errors from django and the mailman process runs amok whenever it gets work to do. very strange ....
everything is nice and silky again ...
that's what I finally did to the database:
sudo -u postgres psql
postgres=# \connect mailmanweb
mailmanweb=# insert into django_site(id, domain, name) values (1, 'example.com', '');
INSERT 0 1
mailmanweb=# insert into django_mailman3_maildomain(id, mail_domain, site_id) values (0, 'example.com', 1);
INSERT 0 1
On 10/14/22 16:42, Martin Lorenz wrote:
everything is nice and silky again ...
And your site looks good to me.
If you have further questions about this, please join the mailman-users@mailman3.org list at https://lists.mailman3.org/mailman3/lists/mailman-users@mailman3.org/ and post there. Mailman3 issues are really off topic for the mailman-users@python.org list.
Apologies if this is too simple for this list -- I have tried looking through the online documentation and even this list's archive...
SIMPLE VERSION OF QUESTION: I'm trying to find out how to stop a list from scrubbing embedded messages.
Does it have something to do with content filtering? ( https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/handlers/do... )
BACKGROUND: I have GNU Mailman List #1 that is an announcements only list. It receives multiple emails of multiple rss feeds from blogtrottr.com (an RSS feed to email service), collects them, then sends them all out as a daily digest. So essentially it is a GNU Mailman digest of multiple Blogtrottr RSS feed digests.
**When List #1 is sent to an individual email address it works fine.**
When sent to a GNU Mailman List #2: a) The email digest of List #1 NEVER gets emailed to List #2 members b) The email digest of List #1 DOES appear in the archive of List #2 as having been sent. c) In the archive of List #2 -- for each email in the List #1 digest -- the message "An embedded message was scrubbed..." appears.
It looks like this in the List #2 archive:
--START MESSAGE-- . -------------- next part -------------- **THIS IS AN ANNOUNCEMENTS-ONLY LIST.**
RSS Feed articles from ACA Counseling Today, APA PsychPort, Healthcare Info Security, Health IT Security, NASW Social Workers Speak, NYT Psychology & Psychologists, NYU Information for Practice, Scientific American Mind & Body, Social Psychology Psychology Headlines, & The New Social Worker Careers Magazine. __________________________________ -------------- next part -------------- Today's Topics:
1. Your Daily digest for NYT Psychology & Psychologists Daily Digest (Article Digests for Psychology & Social Work) 2. Your Daily digest for Social Psychology Psychology Headlines Daily Digest (Article Digests for Psychology & Social Work) 3. Confirm your unsubscription from 'NYT > Psychology and Psychologists' (Article Digests for Psychology & Social Work) 4. Your Daily digest for NASW Social Workers Speak Daily Digest (Article Digests for Psychology & Social Work) 5. Your Daily digest for NYU Information for Practice Daily Digest (Article Digests for Psychology & Social Work) -------------- next part -------------- An embedded message was scrubbed... From: Article Digests for Psychology & Social Work <article-digests at lists.clinicians-exchange.org> Subject: Your Daily digest for NYT Psychology & Psychologists Daily Digest Date: Thu, 03 Aug 2023 20:08:07 +0000 Size: 3372 URL: http://lists.clinicians-exchange.org/private.cgi/clinicians-exchange-clinicians-exchange.org/attachments/20230804/ed6c3463/attachment.eml -------------- next part -------------- An embedded message was scrubbed... From: Article Digests for Psychology & Social Work <article-digests at lists.clinicians-exchange.org> Subject: Your Daily digest for Social Psychology Psychology Headlines Daily Digest Date: Thu, 03 Aug 2023 20:09:05 +0000 Size: 10905 URL: http://lists.clinicians-exchange.org/private.cgi/clinicians-exchange-clinicians-exchange.org/attachments/20230804/ed6c3463/attachment-0001.eml -------------- next part -------------- An embedded message was scrubbed... From: Article Digests for Psychology & Social Work <article-digests at lists.clinicians-exchange.org> Subject: Confirm your unsubscription from 'NYT > Psychology and Psychologists' Date: Thu, 3 Aug 2023 20:20:56 +0000 (UTC) Size: 965 URL: http://lists.clinicians-exchange.org/private.cgi/clinicians-exchange-clinicians-exchange.org/attachments/20230804/ed6c3463/attachment-0002.eml -------------- next part -------------- An embedded message was scrubbed... From: Article Digests for Psychology & Social Work <article-digests at lists.clinicians-exchange.org> Subject: Your Daily digest for NASW Social Workers Speak Daily Digest Date: Fri, 4 Aug 2023 14:29:54 +0000 (UTC) Size: 18553 URL: http://lists.clinicians-exchange.org/private.cgi/clinicians-exchange-clinicians-exchange.org/attachments/20230804/ed6c3463/attachment-0003.eml -------------- next part -------------- An embedded message was scrubbed... From: Article Digests for Psychology & Social Work <article-digests at lists.clinicians-exchange.org> Subject: Your Daily digest for NYU Information for Practice Daily Digest Date: Fri, 04 Aug 2023 19:51:59 +0000 Size: 95307 URL: http://lists.clinicians-exchange.org/private.cgi/clinicians-exchange-clinicians-exchange.org/attachments/20230804/ed6c3463/attachment-0004.eml
-------------- next part --------------
Replies to this list not permitted -- announcement-only list.
UNSUBSCRIBE: http://unsubscribe-article-digests.clinicians-exchange.org
SUBSCRIBE: http://subscribe-article-digests.clinicians-exchange.org
READ ONLINE: http://read-the-rss-mega-archive.clinicians-exchange.org
Contact article-digests-owner at clinicians-exchange.org with questions or concerns.
--STOP MESSAGE--
EXPLANATION OF WHY: While I will ultimately have people sign-up directly for List #1 (while will work fine), I would like them to be able to see what it will be like (on List #2 for a week or so). If I have to (if there is no good answer), I can skip this I suppose.
Obviously its only sending titles, brief descriptions, and URL links -- not whole articles.
Thanks, Michael
*Michael Reeder, LCPC
*Hygeia Counseling Services : Baltimore / Mt. Washington Village location* *410-871-TALK / michael(at)hygeiacounseling.com*
Alright -- This is embarrassing...
I may have outsmarted myself...
I was expecting the message to go to my subfolder for List #2 in my Thunderbird email client. Of course it did not. I have a separate subfolder for List #1. So it went to List #1's message subfolder because List #2 was merely a member receiving the message from List #1.
**No need to respond to this one unless I look at it for a day or two and still find further issues.**
Time for a beer...
Thanks, Michael
*Michael Reeder, LCPC
*Hygeia Counseling Services : Baltimore / Mt. Washington Village location* *410-871-TALK / michael(at)hygeiacounseling.com*
On 8/4/2023 6:35 PM, Michael Reeder -- Hygeia MS wrote:
Apologies if this is too simple for this list -- I have tried looking through the online documentation and even this list's archive...
SIMPLE VERSION OF QUESTION: I'm trying to find out how to stop a list from scrubbing embedded messages.
Does it have something to do with content filtering? ( https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/handlers/do... )
BACKGROUND: I have GNU Mailman List #1 that is an announcements only list. It receives multiple emails of multiple rss feeds from blogtrottr.com (an RSS feed to email service), collects them, then sends them all out as a daily digest. So essentially it is a GNU Mailman digest of multiple Blogtrottr RSS feed digests.
**When List #1 is sent to an individual email address it works fine.**
When sent to a GNU Mailman List #2: a) The email digest of List #1 NEVER gets emailed to List #2 members b) The email digest of List #1 DOES appear in the archive of List #2 as having been sent. c) In the archive of List #2 -- for each email in the List #1 digest -- the message "An embedded message was scrubbed..." appears.
It looks like this in the List #2 archive:
--START MESSAGE-- . -------------- next part -------------- **THIS IS AN ANNOUNCEMENTS-ONLY LIST.**
RSS Feed articles from ACA Counseling Today, APA PsychPort, Healthcare Info Security, Health IT Security, NASW Social Workers Speak, NYT Psychology & Psychologists, NYU Information for Practice, Scientific American Mind & Body, Social Psychology Psychology Headlines, & The New Social Worker Careers Magazine. __________________________________ -------------- next part -------------- Today's Topics:
1. Your Daily digest for NYT Psychology & Psychologists Daily Digest (Article Digests for Psychology & Social Work) 2. Your Daily digest for Social Psychology Psychology Headlines Daily Digest (Article Digests for Psychology & Social Work) 3. Confirm your unsubscription from 'NYT > Psychology and Psychologists' (Article Digests for Psychology & Social Work) 4. Your Daily digest for NASW Social Workers Speak Daily Digest (Article Digests for Psychology & Social Work) 5. Your Daily digest for NYU Information for Practice Daily Digest (Article Digests for Psychology & Social Work) -------------- next part -------------- An embedded message was scrubbed... From: Article Digests for Psychology & Social Work <article-digests at lists.clinicians-exchange.org> Subject: Your Daily digest for NYT Psychology & Psychologists Daily Digest Date: Thu, 03 Aug 2023 20:08:07 +0000 Size: 3372 URL: http://lists.clinicians-exchange.org/private.cgi/clinicians-exchange-clinicians-exchange.org/attachments/20230804/ed6c3463/attachment.eml -------------- next part -------------- An embedded message was scrubbed... From: Article Digests for Psychology & Social Work <article-digests at lists.clinicians-exchange.org> Subject: Your Daily digest for Social Psychology Psychology Headlines Daily Digest Date: Thu, 03 Aug 2023 20:09:05 +0000 Size: 10905 URL: http://lists.clinicians-exchange.org/private.cgi/clinicians-exchange-clinicians-exchange.org/attachments/20230804/ed6c3463/attachment-0001.eml -------------- next part -------------- An embedded message was scrubbed... From: Article Digests for Psychology & Social Work <article-digests at lists.clinicians-exchange.org> Subject: Confirm your unsubscription from 'NYT > Psychology and Psychologists' Date: Thu, 3 Aug 2023 20:20:56 +0000 (UTC) Size: 965 URL: http://lists.clinicians-exchange.org/private.cgi/clinicians-exchange-clinicians-exchange.org/attachments/20230804/ed6c3463/attachment-0002.eml -------------- next part -------------- An embedded message was scrubbed... From: Article Digests for Psychology & Social Work <article-digests at lists.clinicians-exchange.org> Subject: Your Daily digest for NASW Social Workers Speak Daily Digest Date: Fri, 4 Aug 2023 14:29:54 +0000 (UTC) Size: 18553 URL: http://lists.clinicians-exchange.org/private.cgi/clinicians-exchange-clinicians-exchange.org/attachments/20230804/ed6c3463/attachment-0003.eml -------------- next part -------------- An embedded message was scrubbed... From: Article Digests for Psychology & Social Work <article-digests at lists.clinicians-exchange.org> Subject: Your Daily digest for NYU Information for Practice Daily Digest Date: Fri, 04 Aug 2023 19:51:59 +0000 Size: 95307 URL: http://lists.clinicians-exchange.org/private.cgi/clinicians-exchange-clinicians-exchange.org/attachments/20230804/ed6c3463/attachment-0004.eml
-------------- next part --------------
Replies to this list not permitted -- announcement-only list.
UNSUBSCRIBE: http://unsubscribe-article-digests.clinicians-exchange.org
SUBSCRIBE: http://subscribe-article-digests.clinicians-exchange.org
READ ONLINE: http://read-the-rss-mega-archive.clinicians-exchange.org
Contact article-digests-owner at clinicians-exchange.org with questions or concerns.
--STOP MESSAGE--
EXPLANATION OF WHY: While I will ultimately have people sign-up directly for List #1 (while will work fine), I would like them to be able to see what it will be like (on List #2 for a week or so). If I have to (if there is no good answer), I can skip this I suppose.
Obviously its only sending titles, brief descriptions, and URL links -- not whole articles.
Thanks, Michael
*Michael Reeder, LCPC
*Hygeia Counseling Services : Baltimore / Mt. Washington Village location* *410-871-TALK / michael(at)hygeiacounseling.com*
On 8/4/23 15:55, Michael Reeder -- Hygeia MS wrote:
Alright -- This is embarrassing...
I may have outsmarted myself...
I was expecting the message to go to my subfolder for List #2 in my Thunderbird email client. Of course it did not. I have a separate subfolder for List #1. So it went to List #1's message subfolder because List #2 was merely a member receiving the message from List #1.
So is it the case that the message you received contained the actual messages and they were only scrubbed in the archive and not in the message delivered to List #2 members?
If so, that is a limitation of the archiver.
...snip...
So is it the case that the message you received contained the actual messages and they were only scrubbed in the archive and not in the message delivered to List #2 members?
If so, that is a limitation of the archiver.
Um...
OK, it's not a confirmed archiver issue but... I seem to recall that just within the last day we read that Mailman 2 stores the archives "as was" in a simple mbox, but Mailman 3 does not: Is this a potential Mailman 3 issue?
As a potential Mailman 3 user someday (vs v2 holdout)... did I miss something?
Tnx, R
On 8/4/23 18:07, Richard wrote:
...snip...
So is it the case that the message you received contained the actual messages and they were only scrubbed in the archive and not in the message delivered to List #2 members?
If so, that is a limitation of the archiver.
Um...
OK, it's not a confirmed archiver issue but... I seem to recall that just within the last day we read that Mailman 2 stores the archives "as was" in a simple mbox, but Mailman 3 does not: Is this a potential Mailman 3 issue?
Mailman 2.1 archives messages in a mbox, but the OP's issue was with the web HTML archive which is created from that and the web archive is flattened to plain text and all non-plain text parts are stored aside and replaced by links.
HyperKitty archives in Mailman 3 are similar in that the plain text message content is stored in the database in a hyperkitty_email table and non-plain text parts are stored in a hyperkitty_attachment table.
See https://mail.python.org/archives/list/hydpy@python.org/message/KK6DQWWZEJOGQ... for an example of what this looks like in HyperKitty.
Mark,
"So is it the case that the message you received contained the actual messages and they were only scrubbed in the archive and not in the message delivered to List #2 members? If so, that is a limitation of the archiver."
YES. The issue was this silly. Thanks for fielding it. For reasons mentioned earlier, I did not spot my error until too late.
Thanks, -- Michael
*Michael Reeder, LCPC
*Hygeia Counseling Services : Baltimore / Mt. Washington Village location* *410-871-TALK / michael(at)hygeiacounseling.com*
On 8/4/2023 8:00 PM, Mark Sapiro wrote:
On 8/4/23 15:55, Michael Reeder -- Hygeia MS wrote:
Alright -- This is embarrassing...
I may have outsmarted myself...
I was expecting the message to go to my subfolder for List #2 in my Thunderbird email client. Of course it did not. I have a separate subfolder for List #1. So it went to List #1's message subfolder because List #2 was merely a member receiving the message from List #1.
So is it the case that the message you received contained the actual messages and they were only scrubbed in the archive and not in the message delivered to List #2 members?
If so, that is a limitation of the archiver.
participants (7)
-
Bruce Johnson -
Mark Dale -
Mark Sapiro -
Martin Lorenz -
Michael Reeder -- Hygeia MS -
Richard -
Stephen J. Turnbull