Hi All,
I am running a list that has members joining and leaving. I'd like to get an inside of these changes:
- how many members joined last month, this month..
- How many members left last month, this month, this year.
I keep the log files so I Believe I can get this information, but wondering if someone already did the script so I don't have to re-invent. Would it be such a long shot to have mailman recording "joined date" for the members of a list? So at least we can see "Member since YYYY-MM-DD" in the roster?
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
On 12/3/18 7:55 AM, Odhiambo Washington wrote:
Mailman can send you a notification for every subscription / unsubscription event on the list. Just have your email program filter those into folders and sort them by date and you can get those numbers by simple inspection of those email folders.
-- Richard Damon
On 12/3/18 4:55 AM, Odhiambo Washington wrote:
Analyzing the logs is the way to do this. I'm not aware of any scripts to do what you want, but creating one shouldn't be hard.
This won't happen for Mailman 2.1.
For Mailman 3, you can submit an RFE at <https://gitlab.com/mailman/mailman/issues/new>.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Mon, 3 Dec 2018 at 17:11, Mark Sapiro <mark@msapiro.net> wrote:
I thought so too.
Noted.
For Mailman 3, you can submit an RFE at <https://gitlab.com/mailman/mailman/issues/new>.
Heheee... I am one of those people who have all the time wished to run MM3, but it seems it's not possible/easy on FreeBSD. Not being a coder, I am one of those who will just stick to the MM2 forever :-)
Mark, how about a request to incorporate those options in the following script? http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/annotate/head%3A/con...
I am thinking that for you, it'd would be clear (probably simpler) what needs to be added.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
On 12/3/18 6:35 AM, Odhiambo Washington wrote:
mmdsr is not the appropriate vehicle for this. It is designed to run at midnight and report things from the previous 24 hours which is not what you want.
A generic solution is also difficult because you want to look at a complete subscribe log from the last year. Creating/accessing such a log is site specific even if all the pieces are available due to varying log rotation/preservation strategies.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Odhiambo Washington writes:
Systers Mailman provides a console for this stuff, but it also comes with other baggage. The recent work (last 3 years) is Mailman 3 only, but older versions have some of the database features, and probably do have joined date.
github.com/systers/mailman, I believe.
On 12/3/18 4:55 AM, Odhiambo Washington wrote:
I had some time to play with this so check out <https://www.msapiro.net/scripts/subscribe_log> (mirrored at <https://fog.ccsf.edu/~msapiro/scripts/subscribe_log>)
If this is close to useful for you, I'm willing to tweak it if you want changes.
As it is, you can feed it a mailman subscribe log covering some period and it will report the most recent join/leave event by list and user for the whole log or, if specified, a most recent number of days.
Run it under Python3 (it will work with Python 2) with the --help option for more info.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Fri, 7 Dec 2018 at 07:27, Mark Sapiro <mark@msapiro.net> wrote:
It is definitely very very useful. Thank you very much for putting this together.
I have tested it and notice that I need a few changesmods:
- Give it the -l option to specify a listname (yes, my server has several lists)
- The current -l should probably be changed to -f for specifying the subscribe filename/path
- Since the subscribe log has dates, maybe -d should instead be used to parse for and then group by month?? Or perhaps you can add an option to specify the month you want, like Jan, Feb, so a '-m Nov' will give the join/leave for November..
- To make it a little fancier, you can display the results in some tabulated format:
Mailman Subscription Report for $Month
LISTNAME ---------------JOINED---------LEFT list1 15 2 list2 8 12 ....
With the MONTH option available, I can then simply run it from cron to get reports like: mydate = 'date +%b' ./subscribe_log.py --f /usr/local/mailman/logs/subscribe -l listname -d $mydate | mail -s "Mailman Subscription Status" john@doe.com
So far that is all in my request.
Once again, thank you very much.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
On 12/7/18 2:12 AM, Odhiambo Washington wrote:
I have updated the script at <https://www.msapiro.net/scripts/subscribe_log> (mirrored at <https://fog.ccsf.edu/~msapiro/scripts/subscribe_log>)
It doesn't do exactly what you asked for, but it has a -l/--list option which may be repeated for multiple lists and it prints a summary line
For this list, joined = xx, left = xx
for each list processed. If you want to see only the list name and summary you can pipe the output through
sed '/^ /d'
The biggest difference is there is no month option per se, but there are options to specify begin and end dates so for example to see only November you would specify
--begin=2018-11-01 --end=2018-11-30
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hello everyone,
I am seeking a third, fourth and even fifth eyes on this one.
In my quest to run Mailman3, I obtained a VM running Ubuntu 18.04 and started on getting to run Mailman3. I found this link -> https://github.com/iomarmochtar/mailman3_ei It has instructions which looked fairly simple to get Mailman3 installed. However, I did encounter some hitches since the script is not meant for Ubuntu. With a few runs and observing the logs and modifying the script at every step it encountered and error, I finally managed to install Mailman3. It seems this is what you guys call a virtual environment in the Linux world? :-)
Being a *BSD guy, this has not been easy for me. And I am an old cat so learning new things isn't exactly pleasant! :-)
I would like to run Mailman3+MySQL and so far I have pulled in the
necessary bits into the venv.
First, I used apt-get install python-pip python-dev libmysqlclient-dev
which to be honest I wasn't sure installed pymysql so I used the tools in
the venv to be sure:
./conda/bin/pip install pymysql
The script ultimately installed two files: /etc/supervisor/conf.d/{nginx.ini|webui.ini} which in my understanding now are supervisor scripts. I then had to install supervisord by reading from https://www.vultr.com/docs/installing-and-configuring-supervisor-on-ubuntu-1...
root@lists:/etc/supervisord.d# python -m easy_install supervisor (I realize I could use `pip install supervisor', but that's just how things get confusing). Installing echo_supervisord_conf script to /usr/local/bin Installing pidproxy script to /usr/local/bin Installing supervisorctl script to /usr/local/bin Installing supervisord script to /usr/local/bin I am not sure that this was right. Should I have installed the supervisor on the main machine to run the bits, or the supervisor to be use is already installed in the venv? That is where a lot of the confusion exists in my mind.
Within the script, there is reference to a file named misc/mailman3_core_requirements.txt which contained two lines, but I added the third one - postorius==1.2.0. Now to be honest, I am not even sure that was necessary or whether it ended installing it at all, because I think there is a good reason it wasn't there. What the reason is is what baffles me.
mailman==3.2.0 mailman-hyperkitty==1.1.0 postorius==1.2.0
Luckily, I got the supervisord running, and finally was able to make the 1st leap (and I must swear this made me jump from my seat!): => /opt/mailman3/conda/envs/mailman3_ext/bin/python /opt/mailman3/webui/manage.py createsuperuser - this allowed me to create a superuser. The next steps in this script says that:
*to run mailman3 services:* *service mailman3 start* *systemctl start supervisord*
From this point now is where I need help - serious help in smoothening things up and getting to understand these venv stuff!
So I did run 'service mailman3 start' but this seems to be waiting for too long to drop me back into the CLI. I also run 'systemctl start supervisord' and that seems to work. What I am not sure is whether 'systemctl stop supervisord' actually does what it is expected to do because after I execute it, I still see processes running that I think are related.
So far, I have been able to access the webUI on my VM using https://N.N.N.N:9090 I created a domain. But when I create a test list, I get an error - and I do not have a clue which logfile would have the error details - nginx or mailman... (I am 100% new to nginx).
So now, I am stuck at:
- Creating the mailing list
- Getting to know whether the archiver is installed and running
- Knowing is there are cron jobs to be running (Mailman2 type of thinking!)
Ultimately I need to figure out:
- How the mailman queue runners are started and stopped
- Migrate s few lists from Mailman2 to this Mailman3
- Mailman3 should use MySQL storage and Exim4 as the MTA (I have configured these bits in mailman.cfg)
Thanks for your time reading all the blabbing I have put in here :-)
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
On 12/9/18 10:14 AM, Odhiambo Washington wrote:
First, a more appropriate list for Mailman 3 is mailman-users@mailman3.org <https://lists.mailman3.org/mailman3/lists/mailman-users@mailman3.org/>.
Also, we find it very difficult to address issues arising from third party packages and how-tos. See <https://wiki.list.org/x/12812344>.
And, there is a Mailman 3 package for Debian/Ubuntu
apt install mailman3-full
I think the virtual environment in this case is a Python thing, not a Linux thing.
<snip>
Much of what I snipped is specific to <https://github.com/iomarmochtar/mailman3_ei>, the details of which I'm not interested in learning.
The nginx logs generally aren't too useful. Both Mailman core and Django write logs. Core's log is probably in var/log/mailman.log where is defined in mailman.cfg as var_dir. Django's log is defined in Django's settings in the LOGGING setting.
See <http://docs.mailman3.org/en/latest/config-core.html#configuring-cron-jobs> and <http://docs.mailman3.org/en/latest/config-web.html#scheduled-tasks-required>
mailman start
mailman import21 <django management> hyperkitty_import
You also need to configure MySQL in Django. See the DATABASES setting in the Django settings.
Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sun, 9 Dec 2018 at 22:37, Mark Sapiro <mark@msapiro.net> wrote:
I am shifting to that list, after this.
In all my searching, I never found this in any documentation. I must have been searching in the wild, or my FreeBSD-mindedness got me stuck to containers and VENVs.
I agree with you on that.
That is fine. I have put it aside for now.
Which file contains Django's settings?
Great to see that. I'll read and see if I make heads or tails of the writeup.
Why not systemctl mailman start?? Just wondering, as that is what I am seeing is common in Ubuntu. Does this make the components start after a reboot?
Please show me where that is documented. I have to pull my lists from an old machine to this new one.
So, when you talk about Django settings, you are referring to mailman.cfg file??? :-)
I have spent all my weekend till now, trying to figure all this out. Life seems a little easier with mm2.x ....
Thank you.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
On Sun, 9 Dec 2018 at 03:47, Mark Sapiro <mark@msapiro.net> wrote:
Hi Mark,
Thank you so much. This is awesome. It's good enough for me to get the stats by month if I run it monthly, at the end of the month, with --begin referencing the right date.
Thank you once again.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
On 12/3/18 7:55 AM, Odhiambo Washington wrote:
Mailman can send you a notification for every subscription / unsubscription event on the list. Just have your email program filter those into folders and sort them by date and you can get those numbers by simple inspection of those email folders.
-- Richard Damon
On 12/3/18 4:55 AM, Odhiambo Washington wrote:
Analyzing the logs is the way to do this. I'm not aware of any scripts to do what you want, but creating one shouldn't be hard.
This won't happen for Mailman 2.1.
For Mailman 3, you can submit an RFE at <https://gitlab.com/mailman/mailman/issues/new>.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Mon, 3 Dec 2018 at 17:11, Mark Sapiro <mark@msapiro.net> wrote:
I thought so too.
Noted.
For Mailman 3, you can submit an RFE at <https://gitlab.com/mailman/mailman/issues/new>.
Heheee... I am one of those people who have all the time wished to run MM3, but it seems it's not possible/easy on FreeBSD. Not being a coder, I am one of those who will just stick to the MM2 forever :-)
Mark, how about a request to incorporate those options in the following script? http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/annotate/head%3A/con...
I am thinking that for you, it'd would be clear (probably simpler) what needs to be added.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
On 12/3/18 6:35 AM, Odhiambo Washington wrote:
mmdsr is not the appropriate vehicle for this. It is designed to run at midnight and report things from the previous 24 hours which is not what you want.
A generic solution is also difficult because you want to look at a complete subscribe log from the last year. Creating/accessing such a log is site specific even if all the pieces are available due to varying log rotation/preservation strategies.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Odhiambo Washington writes:
Systers Mailman provides a console for this stuff, but it also comes with other baggage. The recent work (last 3 years) is Mailman 3 only, but older versions have some of the database features, and probably do have joined date.
github.com/systers/mailman, I believe.
On 12/3/18 4:55 AM, Odhiambo Washington wrote:
I had some time to play with this so check out <https://www.msapiro.net/scripts/subscribe_log> (mirrored at <https://fog.ccsf.edu/~msapiro/scripts/subscribe_log>)
If this is close to useful for you, I'm willing to tweak it if you want changes.
As it is, you can feed it a mailman subscribe log covering some period and it will report the most recent join/leave event by list and user for the whole log or, if specified, a most recent number of days.
Run it under Python3 (it will work with Python 2) with the --help option for more info.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Fri, 7 Dec 2018 at 07:27, Mark Sapiro <mark@msapiro.net> wrote:
It is definitely very very useful. Thank you very much for putting this together.
I have tested it and notice that I need a few changesmods:
- Give it the -l option to specify a listname (yes, my server has several lists)
- The current -l should probably be changed to -f for specifying the subscribe filename/path
- Since the subscribe log has dates, maybe -d should instead be used to parse for and then group by month?? Or perhaps you can add an option to specify the month you want, like Jan, Feb, so a '-m Nov' will give the join/leave for November..
- To make it a little fancier, you can display the results in some tabulated format:
Mailman Subscription Report for $Month
LISTNAME ---------------JOINED---------LEFT list1 15 2 list2 8 12 ....
With the MONTH option available, I can then simply run it from cron to get reports like: mydate = 'date +%b' ./subscribe_log.py --f /usr/local/mailman/logs/subscribe -l listname -d $mydate | mail -s "Mailman Subscription Status" john@doe.com
So far that is all in my request.
Once again, thank you very much.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
On 12/7/18 2:12 AM, Odhiambo Washington wrote:
I have updated the script at <https://www.msapiro.net/scripts/subscribe_log> (mirrored at <https://fog.ccsf.edu/~msapiro/scripts/subscribe_log>)
It doesn't do exactly what you asked for, but it has a -l/--list option which may be repeated for multiple lists and it prints a summary line
For this list, joined = xx, left = xx
for each list processed. If you want to see only the list name and summary you can pipe the output through
sed '/^ /d'
The biggest difference is there is no month option per se, but there are options to specify begin and end dates so for example to see only November you would specify
--begin=2018-11-01 --end=2018-11-30
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hello everyone,
I am seeking a third, fourth and even fifth eyes on this one.
In my quest to run Mailman3, I obtained a VM running Ubuntu 18.04 and started on getting to run Mailman3. I found this link -> https://github.com/iomarmochtar/mailman3_ei It has instructions which looked fairly simple to get Mailman3 installed. However, I did encounter some hitches since the script is not meant for Ubuntu. With a few runs and observing the logs and modifying the script at every step it encountered and error, I finally managed to install Mailman3. It seems this is what you guys call a virtual environment in the Linux world? :-)
Being a *BSD guy, this has not been easy for me. And I am an old cat so learning new things isn't exactly pleasant! :-)
I would like to run Mailman3+MySQL and so far I have pulled in the
necessary bits into the venv.
First, I used apt-get install python-pip python-dev libmysqlclient-dev
which to be honest I wasn't sure installed pymysql so I used the tools in
the venv to be sure:
./conda/bin/pip install pymysql
The script ultimately installed two files: /etc/supervisor/conf.d/{nginx.ini|webui.ini} which in my understanding now are supervisor scripts. I then had to install supervisord by reading from https://www.vultr.com/docs/installing-and-configuring-supervisor-on-ubuntu-1...
root@lists:/etc/supervisord.d# python -m easy_install supervisor (I realize I could use `pip install supervisor', but that's just how things get confusing). Installing echo_supervisord_conf script to /usr/local/bin Installing pidproxy script to /usr/local/bin Installing supervisorctl script to /usr/local/bin Installing supervisord script to /usr/local/bin I am not sure that this was right. Should I have installed the supervisor on the main machine to run the bits, or the supervisor to be use is already installed in the venv? That is where a lot of the confusion exists in my mind.
Within the script, there is reference to a file named misc/mailman3_core_requirements.txt which contained two lines, but I added the third one - postorius==1.2.0. Now to be honest, I am not even sure that was necessary or whether it ended installing it at all, because I think there is a good reason it wasn't there. What the reason is is what baffles me.
mailman==3.2.0 mailman-hyperkitty==1.1.0 postorius==1.2.0
Luckily, I got the supervisord running, and finally was able to make the 1st leap (and I must swear this made me jump from my seat!): => /opt/mailman3/conda/envs/mailman3_ext/bin/python /opt/mailman3/webui/manage.py createsuperuser - this allowed me to create a superuser. The next steps in this script says that:
*to run mailman3 services:* *service mailman3 start* *systemctl start supervisord*
From this point now is where I need help - serious help in smoothening things up and getting to understand these venv stuff!
So I did run 'service mailman3 start' but this seems to be waiting for too long to drop me back into the CLI. I also run 'systemctl start supervisord' and that seems to work. What I am not sure is whether 'systemctl stop supervisord' actually does what it is expected to do because after I execute it, I still see processes running that I think are related.
So far, I have been able to access the webUI on my VM using https://N.N.N.N:9090 I created a domain. But when I create a test list, I get an error - and I do not have a clue which logfile would have the error details - nginx or mailman... (I am 100% new to nginx).
So now, I am stuck at:
- Creating the mailing list
- Getting to know whether the archiver is installed and running
- Knowing is there are cron jobs to be running (Mailman2 type of thinking!)
Ultimately I need to figure out:
- How the mailman queue runners are started and stopped
- Migrate s few lists from Mailman2 to this Mailman3
- Mailman3 should use MySQL storage and Exim4 as the MTA (I have configured these bits in mailman.cfg)
Thanks for your time reading all the blabbing I have put in here :-)
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
On 12/9/18 10:14 AM, Odhiambo Washington wrote:
First, a more appropriate list for Mailman 3 is mailman-users@mailman3.org <https://lists.mailman3.org/mailman3/lists/mailman-users@mailman3.org/>.
Also, we find it very difficult to address issues arising from third party packages and how-tos. See <https://wiki.list.org/x/12812344>.
And, there is a Mailman 3 package for Debian/Ubuntu
apt install mailman3-full
I think the virtual environment in this case is a Python thing, not a Linux thing.
<snip>
Much of what I snipped is specific to <https://github.com/iomarmochtar/mailman3_ei>, the details of which I'm not interested in learning.
The nginx logs generally aren't too useful. Both Mailman core and Django write logs. Core's log is probably in var/log/mailman.log where is defined in mailman.cfg as var_dir. Django's log is defined in Django's settings in the LOGGING setting.
See <http://docs.mailman3.org/en/latest/config-core.html#configuring-cron-jobs> and <http://docs.mailman3.org/en/latest/config-web.html#scheduled-tasks-required>
mailman start
mailman import21 <django management> hyperkitty_import
You also need to configure MySQL in Django. See the DATABASES setting in the Django settings.
Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sun, 9 Dec 2018 at 22:37, Mark Sapiro <mark@msapiro.net> wrote:
I am shifting to that list, after this.
In all my searching, I never found this in any documentation. I must have been searching in the wild, or my FreeBSD-mindedness got me stuck to containers and VENVs.
I agree with you on that.
That is fine. I have put it aside for now.
Which file contains Django's settings?
Great to see that. I'll read and see if I make heads or tails of the writeup.
Why not systemctl mailman start?? Just wondering, as that is what I am seeing is common in Ubuntu. Does this make the components start after a reboot?
Please show me where that is documented. I have to pull my lists from an old machine to this new one.
So, when you talk about Django settings, you are referring to mailman.cfg file??? :-)
I have spent all my weekend till now, trying to figure all this out. Life seems a little easier with mm2.x ....
Thank you.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
On Sun, 9 Dec 2018 at 03:47, Mark Sapiro <mark@msapiro.net> wrote:
Hi Mark,
Thank you so much. This is awesome. It's good enough for me to get the stats by month if I run it monthly, at the end of the month, with --begin referencing the right date.
Thank you once again.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
participants (4)
-
Mark Sapiro -
Odhiambo Washington -
Richard Damon -
Stephen J. Turnbull