Mailman 3 February 2016 - Mailman-Developers

Re: [Mailman-Developers] OfflineGenerationError after mailman-bundler install
by Danil Smirnov Feb. 17, 2016

Feb. 17, 2016

Sorry again, but the error returns after several page refresh - weird, but it looks like not every opening of the page cause the error! 2016-02-17 12:12 GMT+02:00 Danil Smirnov <danil(a)smirnov.la>: > The error was fixed by bin/mailman-post-update, sorry. > > 2016-02-17 11:55 GMT+02:00 Danil Smirnov <danil(a)smirnov.la>: > >> Hi everybody! >> >> After installing last mailman-bundler I've got the following in the >> Hyperkitty logs: >> >… [View More]> ERROR 2016-02-17 03:42:03,804 base 1762 140633939375872 Internal Server >> Error: /archives/ >> Traceback (most recent call last): >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/core/handlers/base.py", >> line 132, in get_response >> response = wrapped_callback(request, *callback_args, >> **callback_kwargs) >> File >> "/usr/local/src/venv/lib/python2.7/site-packages/hyperkitty/views/index.py", >> line 76, in index >> return render(request, "hyperkitty/index.html", context) >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/shortcuts.py", >> line 67, in render >> template_name, context, request=request, using=using) >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/loader.py", >> line 99, in render_to_string >> return template.render(context, request) >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/backends/django.py", >> line 74, in render >> return self.template.render(context) >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", >> line 210, in render >> return self._render(context) >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", >> line 202, in _render >> return self.nodelist.render(context) >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", >> line 905, in render >> bit = self.render_node(node, context) >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", >> line 919, in render_node >> return node.render(context) >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/loader_tags.py", >> line 135, in render >> return compiled_parent._render(context) >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", >> line 202, in _render >> return self.nodelist.render(context) >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", >> line 905, in render >> bit = self.render_node(node, context) >> File >> "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", >> line 919, in render_node >> return node.render(context) >> File >> "/usr/local/src/venv/lib/python2.7/site-packages/compressor/templatetags/compress.py", >> line 127, in render >> return self.render_compressed(context, self.kind, self.mode, >> forced=forced) >> File >> "/usr/local/src/venv/lib/python2.7/site-packages/compressor/templatetags/compress.py", >> line 86, in render_compressed >> return self.render_offline(context) >> File >> "/usr/local/src/venv/lib/python2.7/site-packages/compressor/templatetags/compress.py", >> line 71, in render_offline >> 'You may need to run "python manage.py compress".' % key) >> OfflineGenerationError: You have offline compression enabled but key >> "57d3e0fcad3f7e67ac65637e9475af43" is missing from offline manifest. You >> may need to run "python manage.py compress". >> >> Please advice how to fix this error properly. >> >> Danil >> > > [View Less]

1 0

Re: [Mailman-Developers] OfflineGenerationError after mailman-bundler install
by Danil Smirnov Feb. 17, 2016

Feb. 17, 2016

The error was fixed by bin/mailman-post-update, sorry. 2016-02-17 11:55 GMT+02:00 Danil Smirnov <danil(a)smirnov.la>: > Hi everybody! > > After installing last mailman-bundler I've got the following in the > Hyperkitty logs: > > ERROR 2016-02-17 03:42:03,804 base 1762 140633939375872 Internal Server > Error: /archives/ > Traceback (most recent call last): > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/core/handlers/base.py", &… [View More]gt; line 132, in get_response > response = wrapped_callback(request, *callback_args, **callback_kwargs) > File > "/usr/local/src/venv/lib/python2.7/site-packages/hyperkitty/views/index.py", > line 76, in index > return render(request, "hyperkitty/index.html", context) > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/shortcuts.py", > line 67, in render > template_name, context, request=request, using=using) > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/loader.py", > line 99, in render_to_string > return template.render(context, request) > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/backends/django.py", > line 74, in render > return self.template.render(context) > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", > line 210, in render > return self._render(context) > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", > line 202, in _render > return self.nodelist.render(context) > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", > line 905, in render > bit = self.render_node(node, context) > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", > line 919, in render_node > return node.render(context) > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/loader_tags.py", > line 135, in render > return compiled_parent._render(context) > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", > line 202, in _render > return self.nodelist.render(context) > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", > line 905, in render > bit = self.render_node(node, context) > File > "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", > line 919, in render_node > return node.render(context) > File > "/usr/local/src/venv/lib/python2.7/site-packages/compressor/templatetags/compress.py", > line 127, in render > return self.render_compressed(context, self.kind, self.mode, > forced=forced) > File > "/usr/local/src/venv/lib/python2.7/site-packages/compressor/templatetags/compress.py", > line 86, in render_compressed > return self.render_offline(context) > File > "/usr/local/src/venv/lib/python2.7/site-packages/compressor/templatetags/compress.py", > line 71, in render_offline > 'You may need to run "python manage.py compress".' % key) > OfflineGenerationError: You have offline compression enabled but key > "57d3e0fcad3f7e67ac65637e9475af43" is missing from offline manifest. You > may need to run "python manage.py compress". > > Please advice how to fix this error properly. > > Danil > [View Less]

1 0

OfflineGenerationError after mailman-bundler install
by Danil Smirnov Feb. 17, 2016

Feb. 17, 2016

Hi everybody! After installing last mailman-bundler I've got the following in the Hyperkitty logs: ERROR 2016-02-17 03:42:03,804 base 1762 140633939375872 Internal Server Error: /archives/ Traceback (most recent call last): File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/core/handlers/base.py", line 132, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/src/venv/lib/python2.7/site-packages/hyperkitty/views/… [View More]index.py", line 76, in index return render(request, "hyperkitty/index.html", context) File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/shortcuts.py", line 67, in render template_name, context, request=request, using=using) File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/loader.py", line 99, in render_to_string return template.render(context, request) File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/backends/django.py", line 74, in render return self.template.render(context) File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", line 210, in render return self._render(context) File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", line 202, in _render return self.nodelist.render(context) File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", line 905, in render bit = self.render_node(node, context) File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", line 919, in render_node return node.render(context) File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/loader_tags.py", line 135, in render return compiled_parent._render(context) File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", line 202, in _render return self.nodelist.render(context) File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", line 905, in render bit = self.render_node(node, context) File "/usr/local/src/mailman-bundler/eggs/Django-1.8.9-py2.7.egg/django/template/base.py", line 919, in render_node return node.render(context) File "/usr/local/src/venv/lib/python2.7/site-packages/compressor/templatetags/compress.py", line 127, in render return self.render_compressed(context, self.kind, self.mode, forced=forced) File "/usr/local/src/venv/lib/python2.7/site-packages/compressor/templatetags/compress.py", line 86, in render_compressed return self.render_offline(context) File "/usr/local/src/venv/lib/python2.7/site-packages/compressor/templatetags/compress.py", line 71, in render_offline 'You may need to run "python manage.py compress".' % key) OfflineGenerationError: You have offline compression enabled but key "57d3e0fcad3f7e67ac65637e9475af43" is missing from offline manifest. You may need to run "python manage.py compress". Please advice how to fix this error properly. Danil [View Less]

1 0

Re: [Mailman-Developers] Yet another newbie to the "Mailman"
by Abhilash Raj Feb. 16, 2016

Feb. 16, 2016

Hi Tarunjit, On 01/24/2016 08:38 PM, Tarunjit Singh wrote: > Hi Fellow Developers, > > I am a graduate student at University of Toronto, Canada who likes to code > (mainly in python). > I just started to get to know the Mailman product by studying its > architecture available on Aosabook <http://www.aosabook.org/en/mailman.html> Awesome! That particular article really sums up the architecture of Mailman. If you want to know more, look at Barry's talk/notes from PyCon … [View More]

3 2

Re: [Mailman-Developers] [Mailman-Users] trouble with multipart and moderation
by Mark Sapiro Feb. 13, 2016

Feb. 13, 2016

On 02/13/2016 08:19 AM, Shea Alterio wrote to mailman-users: > Thanks a bunch for your response, it was very helpful. > >> ‎Are you sure your script is generating the >> multipart/alternative mail correctly? What happens if you sen such a >> message from your MUA as opposed to your script? Is the problem with all >> such mail or just mail generated by your script? > > If I send it from a MUA, the text/plain+text/html is delivered > without issue if sent … [View More]

1 0

Re: [Mailman-Developers] GSOC 2016
by Stephen J. Turnbull Feb. 12, 2016

Feb. 12, 2016

Hi, Aditya! I've had a chance to read your mail and review the I-D (Internet Draft) and relevant RFCs a bit, and can now make a few comments. First, your understanding is a little bit shallow. You should get yourself a "canvas" and draw a detailed flowchart of what's going on here. I don't mean to be harsh. There's an important (especially to you! ;-) principle here: you're volunteering for *too* *much* work. Another point is that you're not giving the I-D authors enough credit for caring … [View More]about you and the end users. Homework: how do these principles apply in the comments below? :-) Aditya Divekar writes: > When we use mailman, the mailing list service adds an extra phrase in > the subject - [Mailman-Developers] and an extra footer in the mail > giving links about the FAQ, archives and the security policy. This > alters the original subject and the body of the mail that the sender > sent in the first place. According to my knowledge, this is what might > cause the mail to be rejected by yahoo, aol, or other p=reject policy > domains. Close enough for our purposes. - If you get interested in doing more mail authentication stuff you'll need the more accurate story. > Thus implementing ARC would involve including the ARC > authentication result header, the signature and the seal in every > mail that Mailman receives before it forwards it in the mailing > list. Yes. > This would probably involve using the pydkim, gs.dmarc and pyspf > libraries for verification before we generate the ARC > authentication results. Here's where you start doing too much work. - You can (for GSoC) assume that the original authentication results should already be available. - For the common MTAs used with Mailman, the Authentication-Results (A-R) header should be available, it should have been added by *your* MTA (otherwise it was added by someone you shouldn't trust!), and you can detect that. (This may not be verifiable -- need to check, and it's another case to deal with later.) The ARC-Authentication-Results is a *copy* of this header (see 5.1.3 in the I-D). (This is the "DRY" principle at work.) - The A-R header already contains SPF and DKIM results, and maybe DMARC (that's cheap to check, though). Thus for verification you don't need to do any work! (First draft -- pragmatically, some Mailman sites may not implement A-R, and as mentioned you may not be able to trust A-R. That's why we're doing this anyway -- it's really an MTA function, but some sites won't implement.) - "For GSoC" is important -- as an extension it's desirable that you handle the case where they're *not* available. But that comes later. - But you do absolutely need to sign things (ARC-Message-Signature, ARC-Seal) with new signatures, and that uses DKIM (modified somewhat, don't know yet if you'll need to modify package code). Having implemented that much, you now know a lot about how the DKIM module (eg, pydkim) works. - Learning DKIM verification is probably easier (the relevant header field should tell the module everything about what it needs to do!) - SPF and DMARC are different modules, but again, you've learned a lot. In general, sequence your plans to learn things that will make later work easier. (This tends to happen naturally as you do the work, but you can often buy 10%-25% by planning it in advance.) > As a starter I think I should understand how the dkim,dmarc and spf > authentication processes are coded. > could you tell me how to find existing code where I can read and > understand how the authentication methods are implemented? On PyPI. It's all open source (at least these modules will surely be -- IETF people are real sticklers for open source). If you need/want VCS checkouts, you can get them later. Here's complete (enough) list. DKIM ---- dkimpy by Scott Kitterman = frequent DMARC/IETF poster, that's a +1 pydkim by Greg Hewgill = dunno authres by Scott Kitterman et al (A-R header up to RFC 7001, current is RFC 7601, may need work?), same author is good sign (he cares, also multiple modules in same author's style are easier to understand, and if you're lucky, they'll already have compatible APIs) BTW, you missed this one. :-) gs.dmarc by Michael JasonSmith = dunno emailprotectionslib by Alex DeFreese = dunno DMARC ----- + same list as DKIM, basically SPF --- pypolicyd-spf by Scott Kitterman pyspf by Stuart Gathman = dunno sikwan.spfcheck by Francois Vanderkelen = dunno hydrate-spf by James Pearson = dunno python-slimta-spf by Ian Good = dunno pysrs by Stuart Gathman = second package, +0.5 + most of the DKIM packages Where I wrote "dunno", I don't know the author offhand. I'd need to Google a bit to decide whether I trust him. In several cases, my guess is that they are not protocol hackers, but rather implementing a messaging system that may or may not have email at its center. Need extra attention to auditing accuracy of protocol implementation unless they're verified to be IETF contributors. In any case, code needs to be reviewed for quality (accuracy of implementation -- high-quality email posts are no guarantee :-) -- and coding technique) and style (if a modified version is needed, you might need to distribute with Mailman). Gotta go, but that should get you started. Steve [View Less]

2 1

Django
by Tom Browder Feb. 12, 2016

Feb. 12, 2016

Does anyone know if one can use one Django instance to serve multiple virtual servers (servers are running under one Apache instance on one reasonably powerful host)? Thanks. Cheers! -Tom

1 0

ANNOUNCE: Mailman Core 3.0.2
by Barry Warsaw Feb. 12, 2016

Feb. 12, 2016

Hello Mailmanatees! I'm happy to announce Mailman Core 3.0.2 which fixes a number of bugs, including most importantly #190, which can affect membership searches. https://gitlab.com/mailman/mailman/milestones/3 More information on version compatibility is available here: http://wiki.list.org/Mailman3 As usual, you can download it from PyPI: https://pypi.python.org/pypi/mailman/3.0.2 view the documentation: http://mailman.readthedocs.org/en/release-3.0/ Enjoy, -Barry (on behalf of the Mailman development team)

1 0

Re: [Mailman-Developers] Discussion On Project Idea "Preset List Settings Templates" .
by Abhilash Raj Feb. 10, 2016

Feb. 10, 2016

On 01/30/2016 11:11 AM, Harshit Bansal wrote: > Hi Steve, > > On 1/27/16, Stephen J. Turnbull <stephen(a)xemacs.org> wrote: >> Harshit Bansal writes: >> >> > Figure out how to store and access the styles as some of them are >> > defined in the source code and the ones created by the by users would >> > have to be stored in databases? >> >> You also need a read/print repr. A Python dict may be a reasonable >> one, but … [View More]JSON, YAML, init, etc could all be considered. >> > > I am planning to use dictionary. What exactly I meant here was not a data structure but *how*? The data structure could simply be dictionary, but how would you populate the existing styles in the code? Notice that there is already an existing IStyle interface in the mailman right now, can you modify it somehow to dynamically update/fetch/store styles from the database instead of creating a new structure altogether? Barry: Is there a reason the styles in the src/styles/base.py are not an implementation of IStyle interface? If I am not wrong, you can define the styles in configuration too apart from the source code, so how do you pull those and update the database during the initialization? This is not a difficult question I want an answer to right now, just that the application should have implementation details too, even if they are a rough sketch. > >> > Styles will be stored in database in a dedicated table. >> >> You may want more than one table. Or you may be able to store them in >> the same table as currently used for list configs, with that table >> updated to allow "unstyled" as a value for its columns. (List id, >> for example, would be "unstyled", since it is supposed to be globally >> unique. Probably list name should be too, but domain would very >> likely be styled.) >> >> > The style manager will update all the styles 'predefined in the >> > source code' as well as the styles 'defined by users(by >> > implementing the IStyle interface)' to the database. Currently we >> > define a list of python import paths which are used for styles. >> >> Do you mean "configs"? >> >> > These will be collected in the style manager and then updated in >> > the database. These styles will be publicly viewable but the users >> > wouldn't be able to modify/change them(using Postorius) otherwise >> > we would have to update those changes in the files as well. >> >> I don't understand why you wouldn't be able to modify them. The >> presets probably should be read-only, and you might want to have a >> permissions system such that only the owner of the style can change >> it. Others would need to copy the style, modify the copy, and then >> apply it to their own lists. >> > > Here, I meant to say that the 'predefined-styles' would be > read-only(not modifiable) while the style defined by the user(using > Postorius) would be readable/editable as per the permissions system. You may want to expand more on the *permissions system* in you application as that is an important part of this. Also, how are you going to propagate that to the Postorius via the API. > >> > Should we even allow changing list styles after lists have been >> > created? >> >> Certainly. Consider the case where a dramatic improvement in MUAs >> commonly used by a subscriber base makes it possible to use Wrap >> Message instead of Munge From as DMARC mitigation. You want to change >> all your lists in one go if they all have the same subscriber base >> (consider an enterprise setting where the subscribers are mostly using >> enterprise webmail). >> >> > Does changing a list style in database/source code changes >> > the settings for list? >> >> IMO, yes. But source code changes by the Mailman Project would not be >> allowed, except with a looong deprecation/obsoletion cycle, and source >> code changes should be documented as an operation with potential (bad) >> surprises for users. >> >> > I think there is no valid reason to not to allow changing list styles >> > after they have been created but there are two attributes about which >> > I am not sure these are: >> > 1: default_member_action >> > 2: default_nonmember_action >> > The value of these style attributes are first copied to >> > 'member.moderation_action'(moderation_action column of member table) >> > and then this saved value is used to decide the correct moderation >> > action. So changing the value of 'default_member_action' and >> > 'default_nonmember_action' have no effect on the saved values and >> > these will not change. This will have an undesired effect. >> >> I think that this is not a problem. We need to initialize all >> attributes to valid values, and this will probably be hard-coded in >> the source. Then we overlay this with site styles, domain styles, >> list styles, and user attributes, and do the lookup in the opposite >> order. In principle, for each attribute you would drill down as long >> as the value at the current level is "unstyled". In practice, you'd >> keep a cache of the current results of such lookups for users and >> lists to avoid slamming the database, and some way of identifying when >> the cache is out of date (such as a change "generation counter"). >> Note that this explains some of the reason why source code changes >> need to be treated with care -- in some sense they are always >> "generation 0" because it's impossible to know in advance what >> generation the installlation is at. I *think* we have had a discussion before about what should be a better default value for member.moderation_action and nonmeember.moderation_action. Instead of just copying them to the field, we could set it to defer in which it case the lookup automatically picks up the value of the list's default_(non)member_action. Or we could *always* copy the default styles to a new style that the list owner owns and can use for his other lists. This way he can change values in the default styles too. This way, we don't have to deal with user's being affected by the change in the default styles. Along with that we could provide them with an option to update the styles automatically from the default (when it changes) to fetch any new values added/removed/updated. This however might introduce a *lot* of redundancy in the database. Just a possibility, not very sure if this is a good idea. >> >> > So I think whenever a user changes these two attributes then we >> > will have to ask the user if he wants to change the saved values or >> > not. Another approach could be to leave these values unchanged and >> > bring this behavior to the knowledge of the user and if he wants >> > then he can manually change the moderation action of some selected >> > members but new members will automatically have their >> > moderation_action set to new the values. >> >> This is the current behavior for default_* actions; I don't see why it >> would change with the introduction of styles. >> > > I wanted to ask that what should be the behavior when a user changes > the 'default_member_action' and 'default_nonmember_action' attributes. > Since, the values of these attributes are copied to the > 'member.moderation_action' at the time of the creation of a new > member. So, any changes made to the 'default_member_action' and > 'default_nonmember_action' attributes will not be reflected in the > already created members which I think may not be the desirable > behavior. > Please do correct me if am wrong somewhere. See my comment above about the default behavior. >> > How to apply styles to lists after they have been created? I will >> > expose the styles via REST API. Whenever the user(admin) will >> > change a mailing list's style in Postorius, it will automatically >> > call 'new_style.apply()' function which will update the mailinglist >> > table with the new values. This will change the list style. >> >> Be careful here. Since some styles percolate up to user level, a site >> with hundreds of thousands of lists and millions of users (they >> exist!) could really slam the database. The whole system would grind >> to a halt. >> >> > After implementing all these things, there will be two ways to create >> > and update styles : >> > 1: Using Postorius >> > 2: By defining a class implementing the IStyle interface and saving it >> > to a file. >> >> I don't see why you would allow 2 at all. IStyle is an internal >> thing, which would be used to provide a Python interface to the >> database backend. Unless you're thinking of dynamic styles (eg, >> theming the list page according to day of week or server load :-)? >> > > Earlier, I was thinking to implement a way to add new styles using > Postorius as well as by adding a file to the source code containing a > class implementing the 'IStyle' interface to define the new style. But > now I have dropped this idea as I think that it would not be that > useful. You cannot let users add stuff in the source code, configurations are meant for the values that can be adjusted by users. AND you can add styles via configs. -- thanks, Abhilash Raj [View Less]

3 2

Re: [Mailman-Developers] GSOC 2016
by Stephen J. Turnbull Feb. 10, 2016

Feb. 10, 2016

Aditya Divekar writes: > And I had previously mailed you directly, but I think I missed you > there. So I mailed it again here. No, I've just been really busy with work, and a very bad cold, sleeping about 2 hours/day more than usual. I'll get back to you tomorrow evening my time (it's 2am here now, and in the morning I have other promises to keep). You really should be a little more patient. You don't have any deadlines to meet this week (things would be way different if it were … [View More]

2 1