Mailman-Developers June 2012

mailman-developers@python.org

20 participants
23 discussions

Re: [Mailman-Developers] [GSoC 2012] Metrics
by Stephen J. Turnbull 15 Jul '12

15 Jul '12

George Chatzisofroniou writes: > Author > > This model represents an author of the mailing list. It mostly keeps > track of the number of postings and number of threads started. It has > the following fields: > > - authorid – IntegerField AFAIK every Django object has an internal ID. Why do authors need a separate, human-unfriendly "authorid"? > - authormail – CharField Authors are people. They typically have names<wink/> and often multiple email addresses. There may also be other information (organization, etc) that is available from the headers. > - totalmails – IntegerField > - totalthreads – IntegerField > - firstmsgdate – DateTimeField > - lastmsgdate – DateTimeField > > MailingList > > This model counts the total number of postings and threads started. > > - totalmails – IntegerField > - totalthreads – IntegerField longestthread? > > Month > > Year > > Views > > To display the metrics the Django template system will be used. To > output the charts i will create some custom tags. The three following > views will be used: > > - General page – On top, there will be general metrics about total > authors, total mails and total threads and below three charts (AJAX > based) "AJAX based" doesn't belong in the spec; it's an implementation detail. > that represent number of posts per author, number of threads > per author and mailing list’s yearly usage. Even below there will be a > number of charts (equal to the number of years of list’s existence) > that output monthly usage. Why multiple charts? If you can afford a 640x480 chart area, with 4 pixel wide bars you can have 160 months > 13 years in one chart. I personally wouldn't hesitate to go to pixel width bars, which gives you > 53 years. I don't think people will be looking at charts for precision, but rather to get an overview. > At the end, there will be tabular data representing the authors > > - Author page – Each user will have his own page with his own metrics. > > Django Admin page – A ‘Generate’ button will be added to the Django admin page. > Settings > > The Django app should handle the following configuration parameters: > > - Host – Message store data host > - Port – Message store data port > - Masking – A multi-state variable (None, abbreviated, full) for > masking email addresses at the results (we don’t want the emails to be > spammed) (1) If at all possible, this should be inherited from the list configuration (DRY). It's not useful if the addresses are available from the archives or by subscribing to the list. (Actually, a really sophisticated spammer might want to attack by spoofing frequent posters on the assumption they're more trusted and more read, but that seems second-order to me.) (2) It would be preferable if authors could supply nicknames, full names, or avatars for this purpose. > Interface to the Mailman core > > - Metrics class > > - Generate class

4 4

Questions in regard to the database operations
by Danci Emanuel 02 Jul '12

02 Jul '12

Hi! My name is Emanuel Danci and I am the GSoC student working on integrating the Systers` feature, dynamic sublists - http://systers.org/systers-dev/doku.php/good_to_know?s%5B%5D=postgresql , with Mm 3.0 and I have 2 questions in regard to how the database works. First of all, how is the aliases set populated in time? Second of all, when a user is unsubscribed, is all the data related to him/her deleted (his/hers preferences, the info related to the address etc) ? Thank you, Emanuel

4 13

smtp error - [Errno 111] Connection refused - when trying to install MM 3.0
by Danci Emanuel 01 Jul '12

01 Jul '12

Hello everyone! This is the continuation of the topic started on the mailman-users mailing list. Here is the original post: http://mail.python.org/pipermail/mailman-users/2012-June/073632.html And this is the post at which the conversation stopped: http://mail.python.org/pipermail/mailman-users/2012-June/073636.html Basically, what I am trying to do is to set up a running version of Mailman on my local machine, by using the 5 minute guide (http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/view/head:/src/mail…). After doing what Mark Sapiro suggested in the last post, I got this: Jun 27 20:01:09 2012 (5011) [SMTPServer] listening: localhost:9025 Jun 27 20:01:09 2012 (5011) starting the SMTP server thread Jun 27 20:01:09 2012 (5011) connecting to localhost:9025 Jun 27 20:01:09 2012 (5011) [SMTPServer] starting asyncore loop Traceback (most recent call last): File "<console>", line 1, in <module> File "/usr/local/mm3/mailman/src/mailman/testing/layers.py", line 254, in setUp cls.smtpd.start() File "/usr/local/mm3/mailman/src/mailman/testing/mta.py", line 197, in start QueueController.start(self) File "/usr/local/mm3/mailman/eggs/lazr.smtptest-1.3-py2.6.egg/lazr/smtptest/controller.py", line 69, in start smtpd = self._connect() File "/usr/local/mm3/mailman/eggs/lazr.smtptest-1.3-py2.6.egg/lazr/smtptest/controller.py", line 60, in _connect smtpd.connect(self._server.host, self._server.port) File "/usr/lib/python2.6/smtplib.py", line 295, in connect self.sock = self._get_socket(host, port, self.timeout) File "/usr/lib/python2.6/smtplib.py", line 273, in _get_socket return socket.create_connection((port, host), timeout) File "/usr/lib/python2.6/socket.py", line 514, in create_connection raise error, msg error: [Errno 111] Connection refused Also, there is one more thing that could be related to this problem. I also have installed Mailman 2.1.10 and after setting it up for the proper domain, when a user tries to subscribe to one of the lists, although the subscription is recorded, the confirmation email is never sent back. The message that appears in the smtp-failure log is this: Jun 04 20:55:57 2012 (10427) Low level smtp error: (111, 'Connection refused'), msgid: <mailman.0.1338832556.10889.systers-admin(a)manu.pensiunealdyna.ro> Jun 04 20:55:57 2012 (10427) delivery to esdanci(a)acm.ro failed with code -1: (111, 'Connection refused') Thanks, Emanuel

3 7

Re: [Mailman-Developers] [GSoC 2012] Metrics
by Patrick Ben Koetter 26 Jun '12

26 Jun '12

George, let me throw in some thoughts just to annoy you ;) Like with most statistical data I mostly see the figures being used to give statements on quantity - top poster, number of threads etc. Do you think it would be possible to also make some statements on quality? Let me give an example: Mailing lists are often places where people go to ask for advice. Someone asking usually starts a thread and continually keeps replying. That easily makes a person top poster and might make the same person a thread starter, but number of posts and threads started gives no indication of that persons knowledge (concerning the mailing lists topic). OTOH someone who has been on the list for ages, who replies more often than starting threads and who ends threads often after she has replied might very well be a very knowledgeable person, because she gives the one answer that solves the problem. Do you think it would be possible to deduct such quality oriented statements? p@rick P.S. Do you also plan to deliver a tool that analyzes a mailing list archive in order to gather your statistical data? Having the statistical data might be a good reason for people to upgrade their MMx installation to MM3. * George Chatzisofroniou <sophron(a)latthi.com>: > The following document is the lowest level of my design concept. You > may also read it in my blog [1]. Of course, comments are very welcome. > > -- > Models > > In order to store statistical data, the app will use some Django models: > > Author > > This model represents an author of the mailing list. It mostly keeps > track of the number of postings and number of threads started. It has > the following fields: > > - authorid – IntegerField > - authormail – CharField > - totalmails – IntegerField > - totalthreads – IntegerField > - firstmsgdate – DateTimeField > - lastmsgdate – DateTimeField > > MailingList > > This model counts the total number of postings and threads started. > > - totalmails – IntegerField > - totalthreads – IntegerField > > Month > > This model associates the author and the mailing list with each month. > > - author – ForeignKey > - month – CharField > - postscount – IntegerField > - threadscount – IntegerField > - mailinglist – Boolean (if this is true it corresponds to the whole > mailing list) > > Year > > This model is similar to month. It has a year field instead of a month field. > > Views > > To display the metrics the Django template system will be used. To > output the charts i will create some custom tags. The three following > views will be used: > > - General page – On top, there will be general metrics about total > authors, total mails and total threads and below three charts (AJAX > based) that represent number of posts per author, number of threads > per author and mailing list’s yearly usage. Even below there will be a > number of charts (equal to the number of years of list’s existence) > that output monthly usage. At the end, there will be tabular data > representing the authors of the mailing list along with their number > of posts, number of threads started and the date of their last post. > The user will be able to order the tabular data (alphabetically, > ascending/descending on number of posts, number of threads, date of > last message) by clicking on the table’s headings (Mail, Mails Sent, > Threads Started, Last Message). > > - Author page – Each user will have his own page with his own metrics. > On top, there will be the email of the author, number of posts, number > of threads started and the dates of first and last message. Below > there will be monthly usage charts for each year the user is > subscribed to the mailing list. > Django Admin page – A ‘Generate’ button will be added to the Django admin page. > Settings > > The Django app should handle the following configuration parameters: > > - Host – Message store data host > - Port – Message store data port > - Masking – A multi-state variable (None, abbreviated, full) for > masking email addresses at the results (we don’t want the emails to be > spammed) > > Interface to the Mailman core > > - Metrics class – When a new post is sent, the Metrics class will > receive it through the IArchiver interface. The Posts field of the > Mailing List model (as well as the the related rows on the Month and > Year models) will increase by one. If the author’s email is not in the > database, it will query the mailman core database with the email, grab > the author’s id and a new Author row will be created. Otherwise if the > author is already in the database, the Posts field and the two foreign > fields (Month and Year) will increase by one > > - Generate class – When the ‘Generate’ button on the Admin page is pressed: > * The Django models will be initialized (the metrics will go back to > zero). A progress bar will inform the administrator that the operation > is being processed. > * All the messages of the archive will be parsed by performing a > direct Python call to the IArchiver. Another instance to the IArchiver > will grab any mails sent while the parsing is going on. > * The metrics will be generated from scratch. > * The administrator will be informed with a success message when the > process is over. > > [1]: http://sophron.latthi.com/gsoc-mailman/ > > -- > George Chatzisofroniou > sophron.latthi.com > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/p%40state-of-mind… > > Security Policy: http://wiki.list.org/x/QIA9 -- state of mind () http://www.state-of-mind.de Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666 Amtsgericht München Partnerschaftsregister PR 563

2 2

Trying to install mm 3.0.0b1
by Lindsay Haisley 21 Jun '12

21 Jun '12

I'm trying to install mm 3.0.0b1 as per the directions in START.rst. I'm running into a bit of a problem with zc.buildout. in the bootstrap.py script. The script reports: $ python bootstrap.py Downloading http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg Traceback (most recent call last): File "bootstrap.py", line 254, in <module> ws.require(requirement) File "/tmp/tmp6ebmh9/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 666, in require File "/tmp/tmp6ebmh9/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 565, in resolve pkg_resources.DistributionNotFound: zc.buildout==1.5.2 However; $ dpkg -l python-zc.buildout Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Description +++-==================================-==================================-== ii python-zc.buildout 1.5.2-1 system for managing development buildouts I explicitly installed it! Do I need to update the mm3 package? Where do I go from here? -- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |

5 11

Personal patch
by Lindsay Haisley 18 Jun '12

18 Jun '12

Can someone give me some feedback on the following patch to SMTPDirect.py - whatever I've overlooked, or done that might be dangerous? The purpose of this patch is to insert a header, "X-subdata" into VERPed emails which won't be flagged and redacted by AOL's brain-dead "Email Feedback Report" system, and will continue to allow my local scripts to unsubscribe subscribers who hit the "Report Spam" button on their AOL mail UI. The content of the header is an MD5 hash of the receiving subscriber's email address - the same information contained in the Sender and Return-path headers, normally munged ("redacted") by AOL. The hope is that this hash will address AOL's privacy concerns, and/or else fall beneath the intelligence level of their scrutiny. The address hash can be compared against the list of subscribers to the list, identified in several (improperly redacted or un-redacted) headers. I'm not submitting this as a suggested patch for Mailman, but just asking for some feedback from people who know the code better than I do. --- SMTPDirect.py.orig 2012-06-17 17:16:25.000000000 -0500 +++ SMTPDirect.py 2012-06-17 21:17:25.000000000 -0500 @@ -43,6 +43,8 @@ from email.Utils import formataddr from email.Header import Header from email.Charset import Charset +from md5crypt import md5crypt +from random import choice DOT = '.' @@ -307,6 +309,9 @@ 'host' : DOT.join(rdomain), } envsender = '%s@%s' % ((mm_cfg.VERP_FORMAT % d), DOT.join(bdomain)) + saltmarsh = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrsyuvwxyz1234567890./" + if not msgdata.has_key("X-subdata"): + msgcopy["X-Subdata"] = md5crypt(rmailbox + "@" + DOT.join(rdomain), choice(saltmarsh) + choice(saltmarsh)) if mlist.personalize == 2: # When fully personalizing, we want the To address to point to the # recipient, not to the mailing list

2 2

Mailman 2.1.15 final released.
by Mark Sapiro 15 Jun '12

15 Jun '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I am happy to announce the final release of Mailman 2.1.15. This release is identical to the 2.1.15rc1 release except for the version number and the inclusion of a missing part of the HTML installation manual. Python 2.4 is the minimum supported, but Python 2.6 is recommended. This release should work with Python 2.7, but has not been tested with that version. This release includes minor security enhancements, new features and bug fixes. See the Changelog at <https://launchpad.net/mailman/2.1/2.1.15> for more details. Mailman is free software for managing email mailing lists and e-newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites. For more information, please see: http://www.list.org http://www.gnu.org/software/mailman http://mailman.sourceforge.net/ Mailman 2.1.15 can be downloaded from https://launchpad.net/mailman/2.1/ http://ftp.gnu.org/gnu/mailman/ https://sourceforge.net/projects/mailman/ - -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) iD8DBQFP22s7VVuXXpU7hpMRAmqOAJ9rDBhSkYDW2yiaqljJzjQtPX8xgwCfXpS7 4I6YOtQjk4tn/UGrtyCGgIQ= =rsB+ -----END PGP SIGNATURE-----

3 4

MM3 - Using fqdn in the exposed URLs
by Richard Wackerbarth 12 Jun '12

12 Jun '12

Are we making a design mistake? The current design of the Postorius and Hyperkitty web interfaces to the mailing list and its archives uses the fully qualified list submission email address as a component of the URLs presented to the public. Is this really a good idea? Just think of the exposure that search engines, etc. will give to these email addresses. I fear that doing this will create an even greater invitation to those who harvest email addresses for the purpose of spamming and other nefarious reasons. Additionally, in the most common usage case, it makes the URL significantly longer than it needs to be. In most cases, the website address determines the email domain of the associated lists. Only a few websites are serving mailing lists from multiple email domains. Those sites would need to have some mechanism to unambiguiously identify the list being referenced. But for most sites, the common name of the list is sufficient. One of the design principles of Django is that the website designer can present his content by way of URLs of his choosing. Presenting the actual email address of a list may "leak" information that the user wishes to obscure. I think that we should rethink this decision and follow a "slug" approach to the identification of the mailing lists in URLs. Those who choose to do so can use the fqdn as their slug. But others would be able to readily change the mapping without having to rewrite significant parts of the interface code. Comments?

6 10

Re: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter
by Richard Wackerbarth 11 Jun '12

11 Jun '12

Jessy, Yes, there are plans (and activity) related to the presentation of list archives within the django framework. In fact, Aamir, one of our GSoC interns, is working on that very subject as his project for the summer. In keeping with the django philosophy of combining many special purpose "apps" to create the overall website, the archive access will be packaged as a component separate from the postorius administrative interface. Richard On Jun 10, 2012, at 11:57 PM, Jessy Kate Schingler wrote: > one question i had about the UI - are there plans for including views/templates for list archives?

2 1

Re: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter
by Richard Wackerbarth 10 Jun '12

10 Jun '12

Jessy, I'm sure that any additions to the documentation will be welcomed. Good end-user documentation always seems to be something that gets shorted by the developers intimately related to the implementation aspect of the code. And, even when they make the effort, I'm not sure that they are the most qualified to write that kind of documentation. Someone less involved in the implementation often has a better perspective of the end-user needs regarding documentation. As for your project, we have assumed that the website serving the UI is not the same host as that which hosts the MM core itself. As a result, attempting to use the command line utilities as your access method would not be the best implementation choice. There is a REST interface exposed, and your access should go through that. Since you are working in the Django world, we would hope that you would utilize the postorius app, and leverage off of it for your customization. By doing so, it will be much easier for you to remain integrated with any changes that come down the line. IMHO, at the moment, the postorius templates can be improved by doing some refactoring to keep their functionality in a {% block %} structure that can be integrated into another site without having to re-implement entire pages. I would like to have you review the present template structure and suggest which portions you would utilize as presented and those which ones you would prefer to change. Thanks for your interest, Richard On Jun 7, 2012, at 11:31 AM, Jessy Kate Schingler wrote: > thanks mark and barry. i've been traveling so apologies for the delayed > reply... > > using `bin/mailman shell` command does indeed work flawlessly, thank you! > > should the docs be updated perhaps? something either under "getting started > with GNU Maiman" or even a specific page on "interacting with mailman > through the python interpreter"? i am happy to add something or contribute > to that; i think it would certainly facilitate others working with the > software if that bit was a bit more clear. > > in general, what is that the recommended way to script one's own > interactions with mailman as part of a larger python program? would it be > through mailman.client "official bindings"? i see there are a number of > imports in bin/mailman and subsequently in > src/mailman/commands/cli_withlist.py (which i gather is an alias for > "shell"). should those be a sufficient set of imports and initializations? > > in this particular case, i am building a Django app so it seems i can rely > on postorious and mailman.client, but that wouldn't necessarily be the case > in general, and my curiosity is piqued about how the scaffolding process > and imports work so i can do things The Right Way. > > thanks! > jessy > > On Tue, Jun 5, 2012 at 7:09 PM, Barry Warsaw <barry(a)list.org> wrote: > >> Hi Jessy, >> >> You're doing everything right, but there's one conceptual step that you're >> missing, and which isn't evident from the docs. >> >> As Mark says, `mailman shell` is the best way to get a Python interactive >> interpreter shell to play with the internal Mailman API. Why is this >> better >> than just running the virtualenv's `python` interpreter directly? It's >> because there are a bunch of subsystems that have to be initialized before >> things will work correctly. E.g., the Zope component architecture (ZCA), >> the >> logging subsystem, the configuration subsystem, queues, rules, pipelines, >> etc. The exception you're getting is because the ZCA hasn't been >> initialized >> yet. >> >> `mailman shell` ensures that everything is initialized, then it gives you >> an >> interpreter prompt. >> >> Hope that helps. >> -Barry >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers(a)python.org >> http://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: >> http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: >> http://mail.python.org/mailman/options/mailman-developers/jessy%40jessykate… >> >> Security Policy: http://wiki.list.org/x/QIA9 >> > > > > -- > Jessy > http://jessykate.com > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/richard%40nfsnet.… > > Security Policy: http://wiki.list.org/x/QIA9

2 1

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

Mailman-Developers June 2012