Mailman 3 February 2014 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett Nov. 7, 2022

Nov. 7, 2022

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

Re: [Mailman-Developers] Mailman 3 vs Groupserver vs Sympa
by Ian Eiloart March 14, 2014

March 14, 2014

Hi Kẏra, Some additional answers, inline below. Most of your questions relate to the web archiver, and Nico has answered some of those. In order to get all the questions and answers in the same place (so people can see which questions aren’t yet answered here), I’ve rolled Nico’s comments into this reply. On 7 Oct 2013, at 03:38, Kẏra <kxra(a)riseup.net> wrote: > Hello, > > My name is Kẏra and I am the technology director of the Free Culture > Foundation as well as a campaigns organizer for the Free Software > Foundation and we're in need up upgrading our mailing lists to a more > web-friendly system. I'm doing research on newer list severs to see how > they compare. So far Groupserver seems to provide everything, but if > Mailman 3 is going to be released soon, we'd like to consider that as well. > > I'm making a comparison chart and wanted to ask about Mailman 3 (and > Postorius) features. Answers to any and all of the following with regards > to Mailman 3 + Postorius would be super helpful: > > Can you post from the web interface? yes > Is there file upload support? yes > Is there now a search feature? Can it search multiple lists? yes, yes > Are there web feeds (atom or rss)? Generated from lists? searches? threads? > files? Nico’s plugin API seems to be offering support for this. > Are posts and administration integrated into the web interface? I think that site and list administration are not part of the hyperkitty > Can you specify a posting rate restriction? I don’t know. > Is full css customization for the web interface supported? > Is css customization for the email interface supported? Hyperkitty supports "themes". I’m guessing you’ll be able to roll your own. > Is there site-side logging? (as opposed to server side) Logging (apart from web accesses) would be a Mailman responsibility. I’m not sure what you mean by "site-side", but perhaps you’re after remote reporting on a domain of yours hosted a third party at a remote site. Again, that’s possible. It could be provided through a web interface, or some other means. > Is there a link to the post in the web interface in the footer of messages? That’ll require co-operation between Mailman and the web archive. But, if the web archive URL is predictable, then I think Mailman will be able to do this. > Now that users are more than just email addresses, can you request to > contact a list member? Do you mean from their profile page, or from a list of members? > Can users have multiple email addresses? Yes, that’s a part of users being more than just email addresses. > Are there profile pages where you can see a summary of their latest posts? yes > Can the web interface hide quoted text? yes > Are usage statistics provided? The Hyperkitty change log for 0.1 alpha says "show basic list info and metrics". Certainly the Mailman 2 logs provide enough information from which useful usage stats could be derived. If Hyperkitty doesn’t provide the stats that you want, you’ll likely be able to get them from the log files. > Thanks! > > Also, did Mailman 2 already support LMTP and virtual domains or are those > new? They’re new. Mailman 2 did have partial support of virtual domains, but there was a restriction whereby no two domains could have a list with the same name: so I can’t imagine that this would be useful. > > Best, > Kẏra > > -- > Board of Directors, Free Culture Foundation: www.freeculture.org > Campaigns Organizer, Free Software Foundation: www.fsf.org > > Blog: http://kxra.info - StatusNet Microblog: http://identi.ca/kxra > Email: kxra(a)freeculture.org - SMS: +1.617.340.3661 > Jabber/XMPP: kxra(a)riseup.net - IRC: kxra @freenode @oftc @indymedia > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/iane%40sussex.ac… > > Security Policy: http://wiki.list.org/x/QIA9 -- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148

2 1

[Mailman-Users] MM3 Test "" Hangs
by Stephen J. Turnbull March 4, 2014

March 4, 2014

Tom Browder writes: We really appreciate your efforts to test the betas of Mailman 3. But please do be aware that although there are sites already successfully using Mailman 3 in production, the development team doesn't recommend use of any of the components (core, Postorius, HyperKitty) in production yet. > I have no idea what to do next, Nothing. :-) I've already Cc'd (and set Reply-To to) mailman-developers, which is a more appropriate place for this report. (Many Mailman-Users are not interested in MM3 yet, while Mailman-Developers are by definition, as MM2 is basically end-of-life. Also, some relevant developers may read mailman-developers more frequently than they read mailman-users.) Actually, I do have a couple of ideas. First, you should always report the whole error trace (if you think that's ugly in an email, attach it as a file). In this particular case, I suspect that the problem is in the test before the one that caused the Exception, which failed leaving the database locked. It would be very helpful to identify that test, which would probably be the *first* frame in the trace. Second, you should look in the server's logs to see if there were any errors that might have caused the incomplete transaction. > and help or ideas would be greatly appreciated. If you have no idea, then reporting to the developers is the best you can do. Use "Mailman Developers" <mailman-developers(a)python.org> or report via Launchpad. Pretty clearly what's happened is that some previous test locked the database (probably anything that accesses the database does so at least long enough to read the whole record), and either (1) that test failed to unlock the database, (2) the test framework failed to unlock the database, or (3) the tests were improperly sequenced in some way and the database didn't get unlocked. It's quite possible that this failure could never be replicated in actual use, as tests often mock up some component that would normally ensure that any pending transaction gets discarded and the database unlocked. Unfortunately I don't have an up-to-date test installation (it's on my list for early next week), and looking at the test file doesn't tell me anything. Perhaps Barry has an idea for a fix, or a workaround. And there's probably a way to skip that test, but I don't know nose very well. Steve Original report follows: > i have started investigating MM3 for my new, remote server and > branched from lp:mailman, lp:postorius, and lp:hyperkitty. > > I started in the MM3 branch directory and followed instructions in > src/mailman/docs/START.rst. > > I got the virtualenv ready to go and, in the local mailman branch > directory, I executed: > > $ node2 -v > > All tests chug along nicely until: > > /usr/local/src/0-mailman3/src/mailman/rest/docs/membership.rst ... > > and it hangs longer than I think it should. After a <ctl-C> the last > few trace-back lines are: > > File "/home/virtualenvs/mm3/local/lib/python2.7/site-packages/storm-0.20-py2.7-linux-i686.egg/storm/database.py", > line 374, in raw_execute > self._run_execution(raw_cursor, args, params, statement) > File "/home/virtualenvs/mm3/local/lib/python2.7/site-packages/storm-0.20-py2.7-linux-i686.egg/storm/database.py", > line 388, in _run_execution > self._check_disconnect(raw_cursor.execute, *args) > File "/home/virtualenvs/mm3/local/lib/python2.7/site-packages/storm-0.20-py2.7-linux-i686.egg/storm/database.py", > line 454, in _check_disconnect > return function(*args, **kwargs) > sqlite3.OperationalError: database is locked > > Note I am running the MM3 installation, via ssh, on a remote host > running Debian 7, 32-bit. (Note also postfix is running.)

6 16

GNU Mailman sprinting at Pycon 2014
by Barry Warsaw March 2, 2014

March 2, 2014

Hello sprinters! Once again, the GNU Mailman project will be sprinting at Pycon, this time the 2014 conference in Montreal. I've created a placeholder page for adding ideas about what you would like to work on, as well as our priorities. We'll be filling out details over the next few weeks, but feel free to start discussing ideas here. http://wiki.list.org/display/DEV/PyCon+2014+Sprint Cheers, -Barry

2 2

[GSoC 2014] Cordova/Android App for GNU Mailman Admin Interface
by Bhargav Golla Feb. 28, 2014

Feb. 28, 2014

Hello all Congrats to Mailman on being selected as a mentor organization under PSF. I am Bhargav Golla, a graduate student in Computer Science at Clemson University, US. I am an avid mobile and web app developer. I worked on Phonegap apps during GSoC 2012 and GSoC 2013 with Apache Software Foundation and have developed many Windows Phone apps. The native android app I developed for Exotel (a startup in cloud telephony in India) is still being actively used by Exotel's clientele of over 1000 firms. My Github account[1] will provide much more details about my developer expertise. I am particularly interested in Cordova/Android app for Admin interface idea. I have a few questions regarding this idea. 1. I intend to develop it on Cordova since it will help in porting the app easily to multiple platforms. Were there any ideas in this directions regarding going native or hybrid? 2. Can I assume that all mailing lists built by Mailman support the REST interface? Also, I have tried to see if I can get JSON responses and I am unable to by adding a HTTP Accept Header to take "application/json". Am I doing anything wrong or is JSON not implemented? 3. As a starter, could I ignore internationalization for GSoC, but implement interface in such a way as to be able to internationalize it easily? I would be very much obliged if I am provided with answers to these questions and whatever questions that may arise in future as they will help me in writing a good proposal and in turn get chosen to work with Mailman. [1] http://github.com/bhargavgolla Regards -- Bhargav Golla M.S Computer Science Github <http://www.github.com/bhargavgolla> | LinkedIN<http://www.linkedin.com/in/bhargavgolla> | Website <http://www.bhargavgolla.com/>

1 0

Re: [Mailman-Developers] Installation of mailman on local machine
by Mark Sapiro Feb. 28, 2014

Feb. 28, 2014

On 02/27/2014 01:08 PM, Tejas Shah wrote: > > However, starting the mailman server using "mailman start", when i try to > test using "nose2 -v" , it gives me o/p: You don't have to start mailman before running tests, but it should be OK. > Ran 593 tests in 43.713s > FAILED (failures=11, errors=72) > > I guess most of them are for HTTP Sockets.. Perhaps if you reported the exact failures and errors we could say more. > I tried opening localhost:8001 > and its giving "405 Method Not Allowed" error on the page This is normal. The REST server doesn't accept a GET / > Please suggest appropriate headers to solve this. See src/mailman/rest/docs/basic.rst. Are the doctests here among the failures? Also see "mailman info" for the rest server base URL (probably something like <http://localhost:8001/3.0/>) and credentials. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

Re: [Mailman-Developers] Installation of mailman on local machine
by Abhilash Raj Feb. 28, 2014

Feb. 28, 2014

Hi Tejas, On Fri, Feb 28, 2014 at 2:38 AM, Tejas Shah <tejas.urwelcome(a)gmail.com>wrote: > Hi, > I've been trying to setup mailman on my local machine for contribution > .. I'm currently using instructions in the link below for setup: > > http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/view/head:/src/mail… > > Till now, i've configured my bzr and launchpad settings.. branched a > mailman repository using "bzr branch lp:mailman" > I've setup python virtualenv and activated it. Then i ran "$ python > setup.py develop" > Till this, no errors occurred and everything was built successfully. > > However, starting the mailman server using "mailman start", when i try to > test using "nose2 -v" , it gives me o/p: > > Ran 593 tests in 43.713s > FAILED (failures=11, errors=72) > > I guess most of them are for HTTP Sockets.. I tried opening localhost:8001 > and its giving "405 Method Not Allowed" error on the page > > This address is for the mailman REST API to be used by the Postorius(and so you cannot open it in a web browser), the web interface for mm3. You can install it same way you did by cloning source[1] from launchpad and running `python setup.py develop`. You will also need postorius-standalone[2] to start the web interface. Both postorius and postorius-standalone are django apps. > Please suggest appropriate headers to solve this. > > > Tejas Shah > IRC nick: jash4 > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > https://mail.python.org/mailman/options/mailman-developers/raj.abhilash1%40… > > Security Policy: http://wiki.list.org/x/QIA9 > [1]: https://code.launchpad.net/~mailman-coders/postorius/trunk [2]: https://code.launchpad.net/~mailman-coders/postorius/postorius_standalone -- Abhilash Raj

1 0

Installation of mailman on local machine
by Tejas Shah Feb. 27, 2014

Feb. 27, 2014

Hi, I've been trying to setup mailman on my local machine for contribution .. I'm currently using instructions in the link below for setup: http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/view/head:/src/mail… Till now, i've configured my bzr and launchpad settings.. branched a mailman repository using "bzr branch lp:mailman" I've setup python virtualenv and activated it. Then i ran "$ python setup.py develop" Till this, no errors occurred and everything was built successfully. However, starting the mailman server using "mailman start", when i try to test using "nose2 -v" , it gives me o/p: Ran 593 tests in 43.713s FAILED (failures=11, errors=72) I guess most of them are for HTTP Sockets.. I tried opening localhost:8001 and its giving "405 Method Not Allowed" error on the page Please suggest appropriate headers to solve this. Tejas Shah IRC nick: jash4

1 0

[GSOC 2014]Approach towards the Full anonymization project
by Stephen J. Turnbull Feb. 26, 2014

Feb. 26, 2014

Rajeev S writes: > As mentioned, here is my approach towards the full anonymization > project. AFAICS as far as described it will provide the outcomes you describe. However, I don't understand the use case here. Most approaches use a single secret ID for each user. This is not just a matter of convenience for the developer, but a requirement in some cases. That is, the list members, although anonymous "in the real world", build trust relationships with each other in the list environment. "Full anonymity" is in any case difficult to achieve, as word choice, topic choice, grammatical construction, time of writing, etc fall into patterns over time. Also, given your model of address-per-post, I'm again unclear on the use-case for off-list communication via the list server. Finally, there are a number of sets of details you don't mention here but need to be discussed in your plan (even if you don't propose to implement them now, you need to ensure that you don't make it difficult to implement them!) First, there's a question of how the proposed off-list messaging is going to be handled. Those pseudo- random addresses are going to need to be made valid addresses to the host's MTA. That's MTA-dependent, and also will likely have security implications. To be sure that people don't inadvertantly reveal their identities just by hitting "R" those messages should be anonymized as well, so that any such replies have to go through the server too. Of course it's going to be impossible to prevent people from exchanging email addresses in the body of the text, but in that case it's really not your problem any more. The second is cleaning up the rest of a post. The incoming trace headers typically identify the sender quite precisely. Quite likely you'll want to nuke everything that isn't required by the RFCs, in fact. You probably also should try to do something about .sigs, Message-ID (which is required), Date (also required, and which often gives timezone information) and other automatically added text. Third, what about authentication for incoming posts? Do you care if people spoof addresses? I'm not sure this has any meaning in the one-shot address environment you propose, but that again is going to depend on the use case. Fourth, you need to think about security for the encryption key and EmailMapper table, as well as any archives (you need to clean up archived posts before they go to the archive -- this is probably just a matter of where your Handler goes in the pipeline). > - Introduce a new model EmailMapper with attributes > - ForeginKey to Address / User > - seed, A 40 bit hash,unique > - nuses, number of times this hash is used,max 5 or 10 > - The approach is to encrypt the seed nuses times, with encryption > algorithms like AES, each time the email ID is displayed. > - The email ID is displayed as <nuses><encrypted seed> > - The email is decrypted nuses times to find the parent seed and > thereby point to the exact email address. > - A new seed should be generated for the user after a fixed > number of attempts,say 5 or 10,as the repeated encryption > routines can slow down the system. > > The outcomes > > - Everytime the user sends a message,his from address changes. > - At the same time, each of the from addresses point to the same user. > - The sender can use any stored address he has,like in the mail > contacts,repeatedly, to contact with a user,as it has nuses > attached with it. So you need to store the addresses forever. How big might these tables grow? Could that be a problem? Did you consider using the "seed" as a "salt" instead? Ie, regenerating the seed each time, adjoining it to the address, and encrypting the combination? That would allow you not store a database of addresses. Of course if the encryption were compromised, all the old posts could be identified, whereas in your scheme the EmailMapper table also needs to be compromised to get addresses. Regards, and good luck with your proposal!

1 0

Re: [Mailman-Developers] MM3 Test "" Hangs
by Stephen J. Turnbull Feb. 26, 2014

Feb. 26, 2014

Nicolas Karageuzian writes: > I encountered db lock using sqlite with mailman3 and tools. > Switching to postgres avoid the db locking states. > Maybe you should explore that way. > > Hyperkitty moved to github so the lp ref is quite out of date for this > resource. Thanks for the advice!

1 0