Mailman 3 July 2012 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett 07 Nov '22

07 Nov '22

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

Re: [Mailman-Developers] Implementing the IArchiver
by Barry Warsaw 06 Aug '12

06 Aug '12

Hi George, On Jul 20, 2012, at 02:45 PM, George Chatzisofroniou wrote: >For my metrics app (GSoC project), I'm currently using a MM3 simulator >for testing purposes. Proceeding with the development, i think it's >time to implement a connection with a real MM3 environment. > >For the current version of the app, the only information i need is who >posted what to which list on what date. So, as we have already >discussed, the IArchiver should be the origin of the interface. > >But i know nothing on how to implement this. How will i add my app to >the active IA modules? Is there any code that extends the IArchiver >and will help me understand how it works? (I guess HK extends it to >inject the messages into the archiver.) There isn't too much boilerplate you need to write in order to implement IArchiver, or any interface. You can look at the existing implementations in src/mailman/archiving/*.py for examples, but I'll briefly outline what you need here. from zope.interface import implementer from mailman.interfaces.archiver import IArchiver @implementer(IArchiver) class MyArchiver: """This is the docstring.""" name = 'myarchiver' @staticmethod def list_url(mlist): """See `IArchiver`.""" # do something @staticmethod def permalink(mlist, msg): """See `IArchiver`.""" # do something @staticmethod def archive_message(mlist, message): """See `IArchiver`.""" # do something You probably only care about archive_message(). For methods list_url() and permalink() which you don't care about, you can just `raise NotImplemented`. That's about it! Of course the details of archive_message() are up to you. Cheers, -Barry

2 1

Re: [Mailman-Developers] [GSoC 2012] Metrics
by George Chatzisofroniou 30 Jul '12

30 Jul '12

Greetings, Following a new report. I've finished a coalecor daemon and i wrote some generators for KittyStore too. I'm now into establishing a real MM3 connection. ---- The first version of coalescor is ready! Coalescor is a custom Django admin command that is responsible for maintaining the number of entries in the database at a reasonable level. If the number of entries in the database is not a concern, its use is completely optional. There are five parameters which determine the behavior of the coalesence: DAILY_DETAIL_RETAINED, WEEKLY_DETAIL_RETAINED, MONTHLY_DETAIL_RETAINED and ANNUAL_DETAIL_RETAINED. Each one of them represents the number of grains for which that level of detail is maintained. The daemon coalesces only the new data from its last operation date and it was designed in a manner not to be memory intensive. The last two days i also worked on generators. I had already created a script that generates metrics from a mailbox. There is a KittyStore API availabe, so it wasn’t hard to also generate metrics from SQL or MongoDB. ---- -- George Chatzisofroniou sophron.latthi.com

1 0

Issue: Mailman integration with Postfix(zimbra)
by subin-opensourcesolutions.co.in 27 Jul '12

27 Jul '12

Hello, NB: Sorry, my query is not related to development, but an issue in mailman integration with Postfix. Scenario: I just setup mailman with Postfix(zimbra). I was able to create new list, subscribe to it. When I susbscribed using different mail addresses, I received notification mail asking to confirm the subscription of the list. On confirmation , I was able to receive the welcome mail also. Now when I send the mail to listname(a)domain.tld, the subscribed users are not able to receive the group/list mail. Please advice ! Thanks. I followed the below forum integration that was suggested official by zimbra community: http://www.zimbra.com/forums/administrators/1380-solved-zimbra-mailman-howt… Thanks,

2 1

Bug 971013 - schema migrations BLOCKED
by Barry Warsaw 25 Jul '12

25 Jul '12

I've been working on and off for quite some time on bug 971013 https://bugs.launchpad.net/mailman/+bug/971013 This bug is about providing orderly schema migrations. I now admit that I'm blocked and I'm looking for suggestions. As you know, I made a promise that after beta 1, we'd only change the schema if we can provide automatic migrations. Or IOW, I won't make incompatible schema changes that break existing mm3 installations. We had some discussion on this list a while ago about mechanisms to do this, and trunk has some rudimentary support for it. In the meantime, I found out about a nice tool called Alembic http://alembic.readthedocs.org/en/latest/index.html which provides a better framework for doing migrations. Even though it's primarily geared toward SQLAlchemy, I've had some off-line discussions with Alembic's author Michael Bayer about how we could utilize Alembic without pulling in all of SQLAlchemy[1]. I experimented with a branch and it looked promising, but that's not what's blocking progress on this. What's blocking progress is SQLite, as described in this FAQ: http://sqlite.org/faq.html#q11 tl;dr: SQLite has a very limited ALTER TABLE command, and specifically does not support column renaming or deleting. This really sucks for implementing any kind of schema migration, regardless of the framework. The little example shown in the FAQ doesn't really scale well when you have lots of relationships between the tables, as is the case for the `mailinglist` table as currently defined. Here's where I am blocked. There are a couple of options, none of which I like very much. - I could break the beta1 schema migration promise. I'd just make the schema changes and all your existing systems would break (all three of them <wink>). As unfortunate as that is, going this route is really a cop out because as soon as 3.0 final is out, we can almost guarantee there will be changes in 3.1 so we'll have to face this issue eventually. - We could drop support for SQLite. The other database we (semi-)officially support is Postgres (still waiting on a MySQL branch), which has a sufficiently powerful ALTER TABLE for our needs. Some people would claim that SQLite isn't really a viable database anyway, since it's prone to deadlocks in multithreaded/processing applications such as mm3. While true, especially for large, busy sites, I think SQLite can work for smaller sites, and it does make installation much easier since SQLite support comes with Python. It certainly makes testing much easier. - As has been suggested before, we could convert the mailinglist table to key/value pairs, which is actually how the pendings table works. This might be a good idea *anyway* to better support extensions, but of course I'd have to break the beta1 promise (just this once <wink>) to make this change, and it wouldn't help for any other table. I think it would also make queries trickier, and heck if we go that route why even use a relational database? - We could try to only add columns in SQLite and try to smack the migration scripts into only copying data from the old columns to the new ones. This might actually be the most feasible option, but it does leave column detritus in the SQLite database. Who cares though, right? There may be other problems with this approach I'm not yet aware of. I'm sure there are things I've overlooked, so I welcome your comments and feedback. I'd love to break the logjam on this bug so that we can get moving again toward the 3.0 final release. Cheers, -Barry FTR, the two existing WIP branches: https://code.launchpad.net/~barry/mailman/bug-971013 https://code.launchpad.net/~barry/mailman/alembic [1] Though, if Storm never gets ported to Python 3, I might just re-evaluate the decision to use it as the ORM after mm3.0. SQLAlchemy supports Python 3.

4 6

Re: [Mailman-Developers] HyperKitty packaging!
by Aamir Khan 25 Jul '12

25 Jul '12

Nevermind, the issue has been resolved. On Wed, Jul 25, 2012 at 12:43 AM, Aamir Khan <syst3m.w0rm(a)gmail.com> wrote: > Hey Everyone! > > I have packaged HyperKitty and I have created hk-app standalone django > project which uses the hyperkitty django application. In future I hope we > can replace both hk-app [1] and postorius_standalone [2] by a single django > project, mm_ui for administrators who want to install hyperkitty as well as > postorius on their server. > > In the process of packaging hyperkitty I am facing some issues with the > code and unable to figure out the exact problem. I have updated the readme > files for both hyperkitty [3] django application and hk-app [2] django > project. It will be great if you guys can help me to resolve the issue. > > Try to run "python manage.py syncdb" in hk-app and you will get > ImportError: No module named kittystore.* *I think the system path is > messed up or probably creating symlink is not the right way to do it inside > a packaged application. > > > > [1] => > https://code.launchpad.net/~mailman-coders/postorius/postorius_standalone > [2] => https://github.com/syst3mw0rm/hk-app > [3] => https://github.com/syst3mw0rm/hyperkitty > > > > > > > -- > Aamir Khan | 3rd Year | Computer Science & Engineering | IIT Roorkee > > > -- Aamir Khan | 3rd Year | Computer Science & Engineering | IIT Roorkee

1 0

HyperKitty packaging!
by Aamir Khan 24 Jul '12

24 Jul '12

Hey Everyone! I have packaged HyperKitty and I have created hk-app standalone django project which uses the hyperkitty django application. In future I hope we can replace both hk-app [1] and postorius_standalone [2] by a single django project, mm_ui for administrators who want to install hyperkitty as well as postorius on their server. In the process of packaging hyperkitty I am facing some issues with the code and unable to figure out the exact problem. I have updated the readme files for both hyperkitty [3] django application and hk-app [2] django project. It will be great if you guys can help me to resolve the issue. Try to run "python manage.py syncdb" in hk-app and you will get ImportError: No module named kittystore.* *I think the system path is messed up or probably creating symlink is not the right way to do it inside a packaged application. [1] => https://code.launchpad.net/~mailman-coders/postorius/postorius_standalone [2] => https://github.com/syst3mw0rm/hk-app [3] => https://github.com/syst3mw0rm/hyperkitty -- Aamir Khan | 3rd Year | Computer Science & Engineering | IIT Roorkee

1 0

Re: [Mailman-Developers] mailing list to work with ADSP and DMARC
by Patrick Ben Koetter 21 Jul '12

21 Jul '12

Frank, * Franck Martin <franck(a)peachymango.org>: > As I have not received feedback.. then no feedback is good feedback right? > > I submitted the code to be merged into main branch (using launchpad > interface). It is 2.1.x branch and I know the development happens on 3.x, > but let me know the status and if I should try to do a branch in 3.x code > base in a view to submit a similar patch. ADSP is considered to be broken and marked deprecated. DMARC is its successor and on the rise. Currently DMARC is undergoing interop testing. It is said to advance towards IETF standard sometime in autumn. You might just be a little too early and you might work on the wrong code. On Mailman-Developers currently most discussions deal with/most efforts are put into MM3 development. My bet is you will receive more feedback if you start implementing DMARC relevant features into MM3 code. If you get Murray Kucherawy, I don't know if he's still on the list, to contribute/share ideas on DMARC you will very likely get a bulletproof implementation - he's one of the driving forces behind DMARC. p@rick > ----- Original Message ----- > From: "Franck Martin" <franck(a)peachymango.org> > To: "mailman-developers" <mailman-developers(a)python.org> > Sent: Tuesday, July 10, 2012 9:45:28 PM > Subject: [Mailman-Developers] mailing list to work with ADSP and DMARC > > I did a branch on the 2.1 series to test rewriting the From: header to be able to make emails to comply with DMARC and ADSP > > You can find the branch here: > > https://code.launchpad.net/~mlm-author/mailman/2.1-author > > and the diff with mailman main branch here: > http://bazaar.launchpad.net/~mlm-author/mailman/2.1-author/revision/1340?st… > > I have also a test mailing list running at: > http://lists.peachymango.org/mailman/listinfo/mlm-auth > > The best to see how it works is to subscribe, send an email, and see how the email is distributed. I guess you need to receive an email to see that... I'll reply... > > Several ways to solve this problem were explored with mailman: > http://wiki.list.org/display/DEV/DKIM > > I took a 7th option which is to rewrite the From: without using the + and put the information about the sender in the Reply-To: > > This may not be optimum but it works. > > Overall this solution is one out of 3 ways to get mailing lists running with DMARC (see DMARC FAQ) > 1) don't break DKIM > 2) use AOR > 3) rewrite the from > > There is another one which is to whitelist emails coming from mailing lists.... > > Also note that while I'm involved in the DMARC group, I do this code as a personal project. > > Now send comments, and be nice with me :P > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/franck%40peachyma… > > Security Policy: http://wiki.list.org/x/QIA9 > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/p%40state-of-mind… > > Security Policy: http://wiki.list.org/x/QIA9 -- state of mind () http://www.state-of-mind.de Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666 Amtsgericht München Partnerschaftsregister PR 563

3 2

mailing list to work with ADSP and DMARC
by Franck Martin 21 Jul '12

21 Jul '12

I did a branch on the 2.1 series to test rewriting the From: header to be able to make emails to comply with DMARC and ADSP You can find the branch here: https://code.launchpad.net/~mlm-author/mailman/2.1-author and the diff with mailman main branch here: http://bazaar.launchpad.net/~mlm-author/mailman/2.1-author/revision/1340?st… I have also a test mailing list running at: http://lists.peachymango.org/mailman/listinfo/mlm-auth The best to see how it works is to subscribe, send an email, and see how the email is distributed. I guess you need to receive an email to see that... I'll reply... Several ways to solve this problem were explored with mailman: http://wiki.list.org/display/DEV/DKIM I took a 7th option which is to rewrite the From: without using the + and put the information about the sender in the Reply-To: This may not be optimum but it works. Overall this solution is one out of 3 ways to get mailing lists running with DMARC (see DMARC FAQ) 1) don't break DKIM 2) use AOR 3) rewrite the from There is another one which is to whitelist emails coming from mailing lists.... Also note that while I'm involved in the DMARC group, I do this code as a personal project. Now send comments, and be nice with me :P

1 1

Implementing the IArchiver
by George Chatzisofroniou 20 Jul '12

20 Jul '12

Hello everyone, For my metrics app (GSoC project), I'm currently using a MM3 simulator for testing purposes. Proceeding with the development, i think it's time to implement a connection with a real MM3 environment. For the current version of the app, the only information i need is who posted what to which list on what date. So, as we have already discussed, the IArchiver should be the origin of the interface. But i know nothing on how to implement this. How will i add my app to the active IA modules? Is there any code that extends the IArchiver and will help me understand how it works? (I guess HK extends it to inject the messages into the archiver.) Thanks in advance, -- George Chatzisofroniou sophron.latthi.com

1 0