Mailman 3 June 2016 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett Nov. 7, 2022

Nov. 7, 2022

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

Translation
by Simon Hanna Jan. 11, 2017

Jan. 11, 2017

Hi, I just completed setting up the Mailman 3 projects on zanata.org They can be found here: https://translate.zanata.org/project/view/mailman https://translate.zanata.org/project/view/postorius https://translate.zanata.org/project/view/hyperkitty I'm hesitating to ask people to translate because zanata is really slow and I'm afraid of scaring people away from translating mailman in the future :D Anyhow, I'm writing to ask for your opinions. What do you think about zanata? Maybe you can try translating a couple of things and report how you feel about it. I stumbled across a few other open source solutions: https://translatewiki.net http://pootle.locamotion.org/ https://demo.weblate.org/ This mediagoblin issue discussed some alternatives when they switched away from transifex. https://issues.mediagoblin.org/ticket/913 There seems to be a pootle server run by gnu itself. https://chapters.gnu.org/pootle/ What do you think? Simon

4 9

Re: [Mailman-Developers] GSOC:Populating two default styles
by Barry Warsaw June 25, 2016

June 25, 2016

On Jun 24, 2016, at 03:14 AM, Harshit Bansal wrote: >I was trying to populate the two default styles in the database. For >this purpose, I was trying to use ConfigurationUpdatedEvent as used in >the earlier style manager but I am recieving following error: [...] >AttributeError: No section or category named db. > >It seems that the database layer hasn't got initialized by the time >this event was fired. Is there any way of fixing this or any other way >using which I can populate the database when mailman starts? That's right. Getting the initialization sequence correct is pretty tricky, and it's true that the configuration subsystem is initialized before the database connection is created. If you think about it, it has to be this way since there are settings in the config files that control how the database is set up. What I think this means for you is that you can't use the ConfigurationUpdatedEvent to set up your stylets until the database is set up. Unfortunately, right now there aren't any events fired when initialization is completely, but just the other day Aurelien and I sketched out an event-based plugin mechanism for better HK integration, and one of the things I realized we needed was some additional event notification for the various stages of initialization. Right now, your best hope is to use the [mailman]post_hook which gets run during initialize_3() and is guaranteed to be run after both the configuration subsystem and the database connection is set up. Eventually the pre_hook and post_hook will be deprecated because they aren't really set up well for plugins, and we'll have events for the various phases of initialization, so you could register a handler for one of those events. If you decide to keep the ConfigurationUpdatedEvent handler (e.g. because you want to do other stylet things if the configuration changes once the system is up and running), just check for the existence of the config.db attribute, or catch the AttributeError. If the attribute is missing, you know the ConfigurationUpdatedEvent is too early for your purposes. Hope that helps. -Barry

1 0

GSOC:Populating two default styles
by Harshit Bansal June 23, 2016

June 23, 2016

Hi, I was trying to populate the two default styles in the database. For this purpose, I was trying to use ConfigurationUpdatedEvent as used in the earlier style manager but I am recieving following error: Traceback (most recent call last): File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/testing/layers.py", line 87, in setUp initialize.initialize_1(INHIBIT_CONFIG_FILE) File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/core/initialize.py", line 118, in initialize_1 mailman.config.config.load(config_path) File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/config/config.py", line 102, in load self._post_process() File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/config/config.py", line 125, in _post_process notify(ConfigurationUpdatedEvent(self)) File "/home/harshit/Drive1/dev/Mailman/env/lib/python3.4/site-packages/zope.event-4.1.0-py3.4.egg/zope/event/__init__.py", line 31, in notify subscriber(event) File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/model/styletmanager.py", line 105, in handle_ConfigurationUpdatedEvent getUtility(IStyletManager).populate() File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/home/harshit/Drive1/dev/Mailman/mailman/src/mailman/config/config.py", line 87, in __getattr__ return getattr(self._config, name) File "/home/harshit/Drive1/dev/Mailman/env/lib/python3.4/site-packages/lazr.config-2.1-py3.4.egg/lazr/config/_config.py", line 513, in __getattr__ raise AttributeError("No section or category named %s." % name) AttributeError: No section or category named db. It seems that the database layer hasn't got initialized by the time this event was fired. Is there any way of fixing this or any other way using which I can populate the database when mailman starts? Thanks, Harshit Bansal

1 0

Hyperkitty set up
by ADITI GUPTA June 22, 2016

June 22, 2016

Hello, I am trying to connect hyperkitty to mailman. I have cloned and set up mailman-hyperkitty and made changes in the mailman_hyperkitty.cfg and mailman.cfg files. But, still I am unable to see anything in hyperkitty(except the login page).I have injected mails to a list using mailman inject -f email.txt test(a)example.net but couldn't see anything there except the login page. "No archived lists yet" is shown there. I checked the MailingList model in hyperkitty through command line and found that it is empty. Regards, Aditi

1 0

Re: [Mailman-Developers] Begginer Level Bugs
by Mark Sapiro June 21, 2016

June 21, 2016

On 06/21/2016 01:39 AM, Nischay Singh wrote: > I read the guidelines on the website and found that the bugs are classified > for beginners. Can someone please tell me where to look for such bugs. Go to <https://gitlab.com/groups/mailman/issues> and look for issues tagged 'beginner friendly'. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

Begginer Level Bugs
by Nischay Singh June 21, 2016

June 21, 2016

Hi all! This is Nischay. I'm new to the open source community. Looking forward to actively contributing to mailman. I read the guidelines on the website and found that the bugs are classified for beginners. Can someone please tell me where to look for such bugs. Thank you! -- Nischay Singh Student B.E.(Hons) Computer Science BITS Pilani

1 0

Re: [Mailman-Developers] GSoC project - Implement module to process ARC headers
by Stephen J. Turnbull June 19, 2016

June 19, 2016

Aditya Divekar writes: > So should a new conversation be started with Mark for > our request? I/we need to find out exactly what we need. We may want to do this through email-test.python.org (no-existent, for example, python.org is the relevant part, or maybe a private domain of someone's), but we have to be careful that it doesn't conflict with other uses. > That sounds good! The `mail` keyword gave an idea - > `emailauthentication` I think that would need to be shortened, but mailauth would work and has a good rhythm. :-) Of course it may be "taken". > I had read on the arc list that an OpenARC implementation was being > developed by Murray Kucherawy. So I suppose that we may have the > news of it being released on the list soon. Yes, but Murray's OpenDKIM is a C library, and I expect OpenArc is too. I don't think we should be using a C library because we're not an MTA, and I suspect it will be making assumptions (eg, availability of SPF) that we shouldn't. In any case, the more independent implementations the better. Steve

2 1

Re: [Mailman-Developers] GSoC project - Implement module to process ARC headers
by Stephen J. Turnbull June 18, 2016

June 18, 2016

Aditya Divekar writes: > So if we have a domain name associated with the ip, I'll go ahead > and make the TXT records. Else, we'll have to create a new one. > I'm not aware of any way to avoid this. Let me know if you are > aware of any :) Ah, that's a problem. I don't have control of the DNS yet. We'd need to interact with Mark. Anyway it's too late for this one, there were procedures to be taken earlier in the week. I'll get you on the arc-interop list, there should be another one in the summer at the pace they've been going. > Yes, I went ahead and made the changes. Good! > I have a separate module 'arcpy' installed in my python > dist-packages now which is basically the refactored 'dkimpy' > package. I don't think its the best of names since it doesn't give > the idea of presence of DKIM signing/verifying features. Any ideas > for the same? Not really. Keywords that might be useful aside from ARC, DKIM, and DMARC: mail, post, verify, track, key, sign, identity. Ah -- one brainstorm -- kimpy = "key identified mail.py". As far as words go, you could just keep dkimpy, too, since ARC depends on domain keys for its semantics, as well as on DKIM for its signing protocol. "adkimpy"? Lots of double meanings there: Aditya, ARC, augmented, domain, divikar. :-) I was surprised by the news, but Google also chose dkimpy for its implementation of ARC. I haven't had time to look at it, but maybe early next week (not publicly licensed, sorry, no URL). Steve

2 1

GSOC: Progress, community review and blog post
by Anirudh Dahiya June 18, 2016

June 18, 2016

Hi My Gsoc project Message queue based email archiver has progressed quite well. The detailed progress and journey had been documented by me in this blog post - http://codefullofsummer.blogspot.in/ Also, I would like to share my work and ask for community review - https://gitlab.com/anirudhdahiya9/RabbitMQMessageQueueArchiver/tree/pubMQAr… https://gitlab.com/anirudhdahiya9/mailman/tree/MQArchiver Regards Anirudh Dahiya (irc spark)

1 0

GSoC project - Implement module to process ARC headers
by Stephen J. Turnbull June 15, 2016

June 15, 2016

Hi! I'm sorry that I haven't responded before, it's been very busy catching up to $DAYJOB since getting back. > Now the next step would be to change the namespace and make it more > general to accommodate the ARC capability. That is the module now > provides the capability for ARC protocol implementation but still > retains its 'DKIM' namespace. Since this would involve quite a few > changes, and could easily break the setup, I would like your > thoughts on this if there are any good practices to follow here. I don't understand what you're worried about. Of course if you change all the names, especially with some sort of automatic process, there's a pretty good chance of introducing typos. We fix them as we find them, that's all. > The way I can think of doing this is to grep all files and change > the names and then clear all the cache files. Do you mean __pycache__? That will happen automatically for all files with updated API calls. If you haven't updated the source to use the changed API, the calls will break anyway. The only time this is a worry is if you repurpose an existing name to different semantics, and the answer there is "don't do that," because it's problematic anyway. But these will be new names, so shouldn't cause that kind of problem. Re interop: I have a server with a fixed IP which could just pipe mail to a Python script that reads it and passes the message to the ARC module for processing. Then the script can send it back out if necessary.

2 1

Postorius: Search Plugin for List
by Khushbu Parakh June 12, 2016

June 12, 2016

Hi, I am working on search plugin for lists and wondering what would be the best approach, using Ajax or reloading the page and which JS files. Do I need to write urls with the handler for JSON?I have made the UI for search list. What is the best way to approach. Best, Khushbu Parakhhttp://khushbu.strikingly.com/ Arya College Of Engineering and ITLinkedin: http://linkedin.com/in/khushbuparakh@khushbuparakh

4 4

Re: [Mailman-Developers] [Email-SIG] Some parsing/generation issues of email in Python 3
by Stephen J. Turnbull June 10, 2016

June 10, 2016

Hans-Peter Jansen writes: > > I'm willing to bet it will converge to "robust in practice" much > > faster than the previous design did. > > I will take your word on that. Why take my word when you can take my money instead? :-) Keep on with the porting project! > > Patience and understanding of how opensource software development > > works, perhaps. > > Okay, as already said, I'm sorry for sounding overly harsh. Yeah, me, too. I actually saw Barry's and David's posts while composing and decided I didn't have much to add after all, but apparently I managed to send without noticing (bad Emacs! bad Emacs!) before deleting the draft. Sorry about that; you didn't deserve multiple reprimands. > Usually, when I report such problems nowadays, I add a patch > proposal for fixing the issue, but these issues were overwhelming > me. Needless to mention the complexity of the email package itself > and my reluctance of studying RFCs. The email RFCs are probably not worth studying. They're not only quite complex, on many points they are controversial. Leave that to us. ;-) Merely identifying problems (thanks for the test cases, by the way!) is a great contribution.

1 0

One click unsub draft
by Adam McGreggor June 9, 2016

June 9, 2016

May be of interest to some; although I think I recall this is what Mailmans do, although not enforcing https. https://github.com/Lockhead/oneclick/blob/master/draft-herkula-oneclick.txt -- "You've got to remember most anarchists don't get up that early" — Tony Harris, (chairman of the Metropolitan Police Authority, describing events during the morning of May Day 2001)

1 0

Re: [Mailman-Developers] New Interface
by Stephen J. Turnbull June 8, 2016

June 8, 2016

Harshit Bansal writes: > How can this composability be achieved in a model? I mean that the > size of a table is fixed in a model and if a style describes just a > partial set of attributes then what would be stored in other > attributes? (1) In a NoSQL database (MongoDB, for example), you can store a partial record with no preparation. This is a good thing and a bad thing, depending on how the now-you-see-them-now-you-don't fields are used. (2) SQLAlchemy (and the underlying RDBMSes) allow you to have optional fields. This is implemented by storing NULL in the missing field. > Also how would this partial style be applied to a mailing list? A style would be a list of stylets (NOTE! none of the identifiers below are real! All names have been changed to protect my ignorance!): s1 = { notmetoo_default: True, nomail_default: False } s2 = { notmetoo_default: False, ack_default: True } newstyle = [s1, s2] def apply_style(style, mlist): try: mlist.start_transaction() for s in style: # common attributes will overwrite earlier values mlist.apply_stylet(s) # make sure all attributes are set to sane values mlist.validate_style() mlist.commit_transaction() except Exception: mlist.rollback_transaction() > 2: Current 'IStyle' interface allows us to define either a partial > style(stylet, describing just a partial set of attributes) or a > complete style. But is that possible with a model? There should be either an SQLAlchemy option for each attribute to be optional, or maybe the inverse attribute of required. Use that option on the model attributes. > 3: If I go for style composabiltiy as described by you then how would > I represent that idea in REST and Postorius? Basically the same way.

2 1

GSoC project - Implement module to process ARC headers
by Aditya Divekar June 8, 2016

June 8, 2016

Hi! It's been sometime since the GSoC coding period started, and I apologize for putting in this mail so late! The GSoC project is divided into two repositories. ARC implementation - https://gitlab.com/adityadivekar/arc refactored dkimpy package - https://gitlab.com/adityadivekar/sign-message The original dkimpy package hosted on launchpad provides the functions for DKIM signing and verifying of a message. This module extends the functionality by adding support for the ARC set of headers. Now the next step would be to change the namespace and make it more general to accommodate the ARC capability. That is the module now provides the capability for ARC protocol implementation but still retains its 'DKIM' namespace. Since this would involve quite a few changes, and could easily break the setup, I would like your thoughts on this if there are any good practices to follow here. The way I can think of doing this is to grep all files and change the names and then clear all the cache files. Thanks Aditya

1 0

Postorius issue tracker labels
by Stephen J. Turnbull June 5, 2016

June 5, 2016

Hi, all I just went through the labels in the Postorius issue tracker https://gitlab.com/mailman/postorius/labels and documented the undocumented as well as changing a couple of labels. Comments very welcome. There are two labels whose purpose I'm unclear on, "subscription" and "ui". I think "ui" should be split into "appearance" (CSS) and "workflow" (what operations can be done from which forms). If so, I think "subscription" really should be merged into "workflow" (at least the open issue should be). WDYT? Steve

4 4

Re: [Mailman-Developers] New Interface
by Barry Warsaw June 4, 2016

June 4, 2016

On Jun 04, 2016, at 07:38 PM, Harshit Bansal wrote: >Would you like to keep these attributes(those which are unused and we >don't want to support) or should these be removed? For now, keep them. It would probably make sense to open an issue on gitlab listing the unused attributes to delete. We can mark that as a tech-debt issue and address it outside of the context of your work. >From core's point of view this idea seems to be fine but I am unable to think >of any way of representing this idea in Postorius(since new styles would be >created there)? Yes, the ui issues are challenging. ;) I think it would be fairly easy to represent in REST. >Earlier while discussing we decided to make style composable in terms of >other styles(inheritance and fallback mechanism). For example, if a user >wants to modify only the the autoresponses attributes of Style1, then, he may >create a child style of Style1 and change only the attributes related to the >autoresponses and rest of the attributes will take their values from the >parent style(Style1). Also is there any advantage of making styles >composable in terms of stylets? It allows the admin to only be concerned with the attributes defined in the stylet. It can be fairly difficult otherwise to figure out what a specific stylet modifies. >The new interface will contain only styleable attributes and not any >immutable attributes like created_at. All the immutable attributes >will reside in the existing 'IMailingList' interface. Cool. >> Then it may be possible to do a lot of cool things on stylets, such as >> impose permissions on changing or using one; recording the stylets used >> with a particular mailing list, so if you change the stylet, the mailing >> list automatically changes, etc. > >How will be the permissions enforced without authorization system? Postorius would enforce it for now. When we have the authproxy, it will do it for scripts. >> The only other comment on the above paragraph has to do with requiring a >> style to contain *all* the attributes. How would you handle the >> composability in that case? > >All attributes will be required only in case the style doesn't inherit >from any other base style. In case the style inherits from some other >base style then the dictionary(argument to the constructor) will >contain only those attributes which the user wants to change and rest >of the attributes would be taken from the parent style. > >Also could you suggest some default value(such as 'DEFER') for >styleable attributes that would trigger the fall back mechanism to >take values from the parent style. That's tricky. If we weren't interfacing to a database (i.e. in pure Python), we'd have a marker object, e.g. `INHERIT = object()` but we have to worry about database column types. I'm not sure what the right thing to do there is. -Barry

2 1

Re: [Mailman-Developers] New Interface
by Barry Warsaw June 4, 2016

June 4, 2016

Thanks for posting this Harshit. This is a useful discussion to have, and I have some thoughts, but first a little history. In MM2.1, the MailList class is composed of a number of mixins, each of which implements a general chunk of functionality, such as NNTP or digests, etc. I really didn't like the mixin structure, so for MM3, I collapsed all the style attributes into a single IMailingList interface. As you've noticed, some of the attributes in the model class aren't described in the interface. Those are mostly the legacy attributes which should eventually go away (because they are unused or represent features we don't want to support), or converted to better database types (see the PickleType for accept_these_nonmembers). It's gotten a lot better, but there are still some holdovers from the old MM2.1 code, so more opportunities to clean up and modernize. I've always wanted styles to be composable, by which I mean that a style can describe just a partial set of attributes. For now I'll call these stylets. One stylet might define how autoresponses work, while another might describe how digests work. In a sense, this is analogous to how the composition was broken down in MM2.1, but without all the ugly mixin stuff. The current IStyles support this, but of course they are only applied to mailing lists when they are created, and changing a style does not change any list to which the style was applied. On Jun 01, 2016, at 11:47 PM, Harshit Bansal wrote: >I am thinking of adding a new interface named 'IStyleable' to the >mailman core. All the styleable attributes from the current >'IMailingList' interface will be moved to this new interface. The >current 'IMailingList' interface does not provide documentation for >all the styleable attributes. This new interface will contain all >styleable interfaces arranged in an orderly way with proper >documentation. This new interface then can be extended by the existing >'IMailingList' interface and the new 'IStyle' interface that I am >writing. I think this makes sense. You're refactoring out the interface for the attributes of a mailing list apart from the functionality that a list provides. There may be some attributes which aren't style related, e.g. created_at, and perhaps some that shouldn't get refactored because they identify the list (e.g. list_name and list_id), but we can discuss that. Once you have that refactored, then I think you need a way to say "this stylet sets attributes A, B, and C, but doesn't touch X, Y, and Z". That gives you back the composability without having to worry about partial interfaces; i.e. all IStylet (actual name TBD) interfaces *can* change any style attribute, but it's okay if they change only a subset. Then it may be possible to do a lot of cool things on stylets, such as impose permissions on changing or using one; recording the stylets used with a particular mailing list, so if you change the stylet, the mailing list automatically changes, etc. >Why I felt the need of a new interface? For constructing a new style I am >taking a dictionary as an input to the constructor which is supposed to >contain all the necessary styleable attributes and their corresponding values >as key value pairs and to check whether the dictionary contains all the >styleable attributes or not I will need to have a list of all the styleable >attributes and that can be done very easily if I have an interface >containning all the styleable attributes. I know there are other methods of >getting a list of all the attributes but they are neither pythonic nor stable >as all of them depend on python internals which are liable to change with the >version of python. Let me see if I understand: you want to query the IStyleable interface to know what attributes are allowed in a style. It makes sense in that case to use a separate interface just for that purpose, without cluttering it up with all the other interface bits specifically for mailing list operation (e.g. get_roster(), subscribe(), and immutable attributes like created_at and list_id). The only other comment on the above paragraph has to do with requiring a style to contain *all* the attributes. How would you handle the composability in that case? Cheers, -Barry

2 1

New Interface
by Harshit Bansal June 1, 2016

June 1, 2016

Hi, I am thinking of adding a new interface named 'IStyleable' to the mailman core. All the styleable attributes from the current 'IMailingList' interface will be moved to this new interface. The current 'IMailingList' interface does not provide documentation for all the styleable attributes. This new interface will contain all styleable interfaces arranged in an orderly way with proper documentation. This new interface then can be extended by the existing 'IMailingList' interface and the new 'IStyle' interface that I am writing. Why I felt the need of a new interface? For constructing a new style I am taking a dictionary as an input to the constructor which is supposed to contain all the necessary styleable attributes and their corresponding values as key value pairs and to check whether the dictionary contains all the styleable attributes or not I will need to have a list of all the styleable attributes and that can be done very easily if I have an interface containning all the styleable attributes. I know there are other methods of getting a list of all the attributes but they are neither pythonic nor stable as all of them depend on python internals which are liable to change with the version of python. I had asked Barry for his comments but as he is busy in PyCon so he asked me to redirect my query to mailing list. As this was blocking so I in the meantime I wrote the new interface which can be found here: https://gitlab.com/_Harshit_/mailman/commit/6c94e220cd5f5ba857fa003bdc17919… Please provide your comments/suggestions on whether should I continue with this or not. Thanks, Harshit Bansal.

1 0

GSoc Project : Doubt regarding listarchiverset
by Anirudh Dahiya June 1, 2016

June 1, 2016

Hi I am have built an interface(IArchiver) as part of my gsoc project,which is supposed to work on the basis of message queues. I am intending to use the same interface for multiple archives, using separate queues for each archive in this system. So for this I am registering different archivers(say ArchiverA and ArchiverB) in the config(mailman.cfg), with their class parameter being the same. But it has come to my notice, that the Archiver runner while enumerating the archieverset(IListArchiverSet), builds the set based on their class name, i.e. it builds a dictionary where the key is determined by the class of the archiver interface, and not the actual archive. I feel this makes us use a separate interface for each archive, and my design goes against that(my design wants to use the same class for multiple archives). Is there a special reason such a design was chosen, and if my idea seems fine, should I go ahead and make some changes in the mailman core. If so, please provide suggestions on how you would like me to go about it. Thanks Anirudh Dahiya

1 0