Mailman 3 June 2012 - Mailman-Developers

Re: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter
by Richard Wackerbarth 11 Jun '12

11 Jun '12

Jessy, I'm sure that any additions to the documentation will be welcomed. Good end-user documentation always seems to be something that gets shorted by the developers intimately related to the implementation aspect of the code. And, even when they make the effort, I'm not sure that they are the most qualified to write that kind of documentation. Someone less involved in the implementation often has a better perspective of the end-user needs regarding documentation. As for your project, we have assumed that the website serving the UI is not the same host as that which hosts the MM core itself. As a result, attempting to use the command line utilities as your access method would not be the best implementation choice. There is a REST interface exposed, and your access should go through that. Since you are working in the Django world, we would hope that you would utilize the postorius app, and leverage off of it for your customization. By doing so, it will be much easier for you to remain integrated with any changes that come down the line. IMHO, at the moment, the postorius templates can be improved by doing some refactoring to keep their functionality in a {% block %} structure that can be integrated into another site without having to re-implement entire pages. I would like to have you review the present template structure and suggest which portions you would utilize as presented and those which ones you would prefer to change. Thanks for your interest, Richard On Jun 7, 2012, at 11:31 AM, Jessy Kate Schingler wrote: > thanks mark and barry. i've been traveling so apologies for the delayed > reply... > > using `bin/mailman shell` command does indeed work flawlessly, thank you! > > should the docs be updated perhaps? something either under "getting started > with GNU Maiman" or even a specific page on "interacting with mailman > through the python interpreter"? i am happy to add something or contribute > to that; i think it would certainly facilitate others working with the > software if that bit was a bit more clear. > > in general, what is that the recommended way to script one's own > interactions with mailman as part of a larger python program? would it be > through mailman.client "official bindings"? i see there are a number of > imports in bin/mailman and subsequently in > src/mailman/commands/cli_withlist.py (which i gather is an alias for > "shell"). should those be a sufficient set of imports and initializations? > > in this particular case, i am building a Django app so it seems i can rely > on postorious and mailman.client, but that wouldn't necessarily be the case > in general, and my curiosity is piqued about how the scaffolding process > and imports work so i can do things The Right Way. > > thanks! > jessy > > On Tue, Jun 5, 2012 at 7:09 PM, Barry Warsaw <barry(a)list.org> wrote: > >> Hi Jessy, >> >> You're doing everything right, but there's one conceptual step that you're >> missing, and which isn't evident from the docs. >> >> As Mark says, `mailman shell` is the best way to get a Python interactive >> interpreter shell to play with the internal Mailman API. Why is this >> better >> than just running the virtualenv's `python` interpreter directly? It's >> because there are a bunch of subsystems that have to be initialized before >> things will work correctly. E.g., the Zope component architecture (ZCA), >> the >> logging subsystem, the configuration subsystem, queues, rules, pipelines, >> etc. The exception you're getting is because the ZCA hasn't been >> initialized >> yet. >> >> `mailman shell` ensures that everything is initialized, then it gives you >> an >> interpreter prompt. >> >> Hope that helps. >> -Barry >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers(a)python.org >> http://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: >> http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: >> http://mail.python.org/mailman/options/mailman-developers/jessy%40jessykate… >> >> Security Policy: http://wiki.list.org/x/QIA9 >> > > > > -- > Jessy > http://jessykate.com > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/richard%40nfsnet.… > > Security Policy: http://wiki.list.org/x/QIA9

2 1

Re: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter
by Barry Warsaw 11 Jun '12

11 Jun '12

On Jun 07, 2012, at 09:31 AM, Jessy Kate Schingler wrote: >should the docs be updated perhaps? something either under "getting started >with GNU Maiman" or even a specific page on "interacting with mailman >through the python interpreter"? i am happy to add something or contribute >to that; i think it would certainly facilitate others working with the >software if that bit was a bit more clear. Hi Jessy. Yes, I definitely think the docs should be updated, but perhaps not the page you were reading. What I've always thought is that overview, architectural, and other useful higher level documentation should go in src/mailman/docs. Still using the reST format, but they needn't necessarily be doctests. So I think an overview page about "interacting with Mailman through the Python interpreter" would make a great page for this directory, and if you want to do a branch/merge proposal on that, I would gladly review it. >in general, what is that the recommended way to script one's own >interactions with mailman as part of a larger python program? would it be >through mailman.client "official bindings"? i see there are a number of >imports in bin/mailman and subsequently in >src/mailman/commands/cli_withlist.py (which i gather is an alias for >"shell"). should those be a sufficient set of imports and initializations? Well, kind of the point is that there are multiple ways of doing it, depending on what you're trying to do. One of the principles I've tried to hold to (successfully or not ;) is that the edges of the system, e.g. the shell scripts and the REST API, should have very little logic outside of managing that edge. So for example, the cli_*.py modules should do little more than parse command line options, massage data into the right format, and then call into the core API, by using the zope.interfaces. By organizing things this way, a Python program could conceivably add the proper directory to its sys.path and just 'import mailman' and various submodules to do exactly the same operations. This wasn't the case with Mailman 2, because the CGIs, cli scripts, etc. had way too much logic in them, so to reproduce their functionality on other boundaries was just too difficult. I think that's a big reason why people wrote shell scripts to use the MM2 cli rather than bin/withlist scripts or importing the Mailman namespace. So, I would say that if you're interacting with MM3 across process boundaries, e.g. Django on one machine with the core on another, the REST API is the best way to go. If you're interacting with MM3 on the same machine, within process, then 'import mailman' could be a good way to go (but of course the REST API still works), or you could use withlist scripts, or just script the cli. (Note that mailman.client, while official, isn't strictly necessary, since the REST API can be used from anything that speaks HTTP.) >in this particular case, i am building a django app so it seems i can rely >on postorious and mailman.client, but that wouldn't necessarily be the case >in general, and my curiosity is piqued about how the scaffolding process >and imports work so i can do things The Right Way. I hope the above explains things, if not just ask! Note that if you go the 'import mailman' route, you will need to initialize the system. A good model for how that's done is src/mailman/bin/mailman.py which is what implements the 'bin/mailman' uber-command. Other than doing fancy things to get --help looking decent, it's almost ridiculously simple. Cheers, -Barry

2 1

Re: [Mailman-Developers] [HyperKitty] Rating of posts
by Aamir Khan 07 Jun '12

07 Jun '12

ok. I think we both are talking about the same thing. And, I totally agree with your point that there should also be a user level rating as well, but the question is how do we rate users, one way is to calculate using number of upvotes and downvotes on his/her answers. Any other suggestion? I think ultimately HyperKitty should have both user level and email level rating but I think email level rating is more basic and I think it's better to implement it first. On Thu, Jun 7, 2012 at 1:28 PM, Tanstaafl <tanstaafl(a)libertytrek.org> wrote: > Hi Aamir, > > I think you misunderstood what I was saying... see below... > > On 2012-06-07 1:14 PM, Aamir Khan <syst3m.w0rm(a)gmail.com> wrote: > > On Thu, Jun 7, 2012 at 6:43 AM, Tanstaafl wrote: >> >>> I would also like to see a way for each user to be able to rate >>> other *users*, not just their individual posts... >>> >>> An example with this list: I would rate Mark Sapiro as '5' (or >>> whatever was designated as 'highest value'), so whenever I sort >>> answers to any particular questions, Marks answers should appear at >>> the top of the results before anyone else's (even if someone else's >>> post had a higher rating than Marks for a particular thread) - then >>> would come the individual posts from other users (who I hadn't yet >>> rated)... >>> >>> Hope that made sense... >>> >> > I don't completely agree with your point. Let's say Mark posts good >> emails to the list then he will obviously get a higher rating by users. >> > > My understanding was the initial plan was only to allow for rating > individual *posts*, not the *posters* - so, in your example, Mark *himself* > will not have a rating (unless you provided a way to basically summarize an > average of all of the ratings of all of each users posts)... > > What I was talking about, is a way for me to rate Mark for my own > *personal* benefit... > > > The primary purpose of archiver is to serve people who are not >> subscribed to the mailing list. >> > > I disagree... > > The primary purpose of the archiver is to archive the list messages. Some > lists *and list archives) are public, some are not. > > > They don't even know who Mark is >> > > They do if they are a member of the list for any amount of time. > > > and if they encounter a particular question lets say "How to setup >> Mailman3 in virtualenv" then they would like to see the best answer >> and not the answer of person who generally posts good answer to the >> list. >> > > Again, I think you misunderstood... and again - I was talking about a way > for me to rate Mark, which would be stored as one of *my* preferences... > so, only *I* would benefit from rating him highly - no one else would know > that I had rated him highly. > > Think of it as a way for me to filter answers to questions based on who > *I* trust and am interested in hearing from. > > I'd also like a way to rate users negatively, so I won't even see their > posts... again, said ratings only having any meaning for me when I'm logged > in. > > Do you agree? >> > > I don't think we are talking about the same thing, so until we both agree > on what we are discussing, I don't think the question is relevant... ;) > -- Aamir Khan | 3rd Year | Computer Science & Engineering | IIT Roorkee

1 0

Re: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter
by Barry Warsaw 07 Jun '12

07 Jun '12

Hi Jessy, You're doing everything right, but there's one conceptual step that you're missing, and which isn't evident from the docs. As Mark says, `mailman shell` is the best way to get a Python interactive interpreter shell to play with the internal Mailman API. Why is this better than just running the virtualenv's `python` interpreter directly? It's because there are a bunch of subsystems that have to be initialized before things will work correctly. E.g., the Zope component architecture (ZCA), the logging subsystem, the configuration subsystem, queues, rules, pipelines, etc. The exception you're getting is because the ZCA hasn't been initialized yet. `mailman shell` ensures that everything is initialized, then it gives you an interpreter prompt. Hope that helps. -Barry

2 1

Re: [Mailman-Developers] [HyperKitty] Rating of posts
by Pierre-Yves Chibon 07 Jun '12

07 Jun '12

On Wed, 2012-06-06 at 00:43 -0400, Aamir Khan wrote: > > *Database Schema: *Table structure is very straightforward and simple. > > Table name : ratings > > table structure > emailid int| userid int| value (+1/-1) > > where, > emailid - references to unique entry storing the actual emails. > userid - references to unique entry in user table. > value - +1 for upvote and -1 for downvote. > > In this case each upvote/downvote will be saved as a separate row in > database. Your suggestions? I actually would like to ask you to think at a broader level. Could you provide us a database schema on how it would look ? You would have a user table, I assume, the tables for the archives and I would like you to consider in your design ratings but also the category and tags assigned to the emails. You can use Dia, inkscape or http://dbdsgnr.appspot.com/app (from which you can export to an image) to draw the database schema. I have given it some thoughts but I have not found yet a solution that satisfies me, so I would like you to think about it and come up with your design. Thanks, Pierre

2 1

Speaking about kitties (or archivers)
by Pierre-Yves Chibon 07 Jun '12

07 Jun '12

Meeow miaou* We spoke on IRC about the archiver the other day and I said that I should present here my thoughts about it. So here they are (beware that might be long). First I think we should think about the structure/architecture of things. We have a number of component which need to be archives aware, without being exhaustive I'm thinking about: - the archiver itself (which present the archive (ie: mails and threads) - the NNTP bits which should be able to return emails and/or threads - the stats module which want to give information to the user about the health of the list itself (emails/month, last threads, biggest threads...) - archives retrieval (we probably want to give the user a way to download the archives since the creation of the list/the last year/month) All of these components needs to be aware about the archives. We agreed that the core does not want to know about it. So we have several solutions: - each module becomes an "archiver" wrt to core, meaning each module has its own way to storing the archives (and eventually its own system to do so) - we create a archive-core module which manage the archives and provides an API to access, modify, extend them. Of course, we prefer the second solution :) So we would have the following architecture: mm-core (handles the lists themselves) --send emails to archivers--> archive-core (store the emails and expose them through an API) --> archivers/stats/NNTP The questions are then: - how do we store the emails ? - how do we expose the API ? - how to make it such that it becomes easy to extend ? (ie: the stats module wants to read the db, but probably also to store information on it) Having played with mongodb (HK relies on it atm), I quite like the possibilities it gives us. We can easily store the emails in it, query them and since it is a NoSQL database system extending it becomes also easy. On the other hand, having the archiver-core relying on the same system as the core itself would be nicer from a sysadmin pov. I have not tried to upload archives to a RDBMS and test its speed, but for mongodb the results of the tests are presented at [1]. The challenge will be speed and designing an API which allow each component to do its work. I think it would be nice if we could reach some kind of agreement before the GSoC starts (even if we change our mind later on) to be sure that if we get students their work don't overlap too much. The second point I want to present is with respect to the archiver itself. At the moment we have HyperKitty (HK), the current version: - exposes single emails - exposes single threads - presents the archives for one month or day - allows to search the archives using the sender, subject, content or subject and content - presents a summary of the recent activities on the list (including the evolution of the number of post sent over the last month) I think these are the basis functionality that we would like to see in an archiver. But HK aims at much more, the ultimate goal of HK is to provide a "forum-like" interface to the mailing-lists, with it HK would provide a number of option (social-web like) allowing to "like" or "dislike" a post or a thread, allowing to "+1" someone, allowing to tag the mails or assign them categories. These are all nice feature but, imho, they go beyond what one would want from a basic archiver. So what I would like to propose is to split HK into a sub-project (MiniKitty?) which would provide these basic functionality. We would keep HyperKitty as a more extensive archiver and try to bring HK to its ultimate goal. This will need some more work and time as we will have to make HK speak with core for authentication, find a way to send emails to core/the lists and of course add all the other features (tags, categories...) Comments welcome :) Thanks, Pierre [1] http://blog.pingoured.fr/index.php?post/2012/03/16/Mailman-archives-and-mon… * Hi everyone

5 16

Re: [Mailman-Developers] Speaking about kitties (or archivers)
by Barry Warsaw 07 Jun '12

07 Jun '12

Jeebus, finally catching up on this thread. On Apr 24, 2012, at 01:41 PM, Stephen J. Turnbull wrote: >IMHO, premature optimization. Among other things, there isn't going >to be a "the" archiver-core. Mailman should provide "a" >archiver-core, and I think it should be based on maildir (which is >apparently Barry's intuition, too). Theory and implementation of >maildir are simple and robust, and that allows us to concentrate on >the archiver interface. In fact, the prototype archiver already implements maildir storage. This storage isn't exposed via REST yet though. -Barry

2 4

Re: [Mailman-Developers] URGENT: Google Summer of Code status report and code due
by Alexander Sulfrian 07 Jun '12

07 Jun '12

At Mon, 04 Jun 2012 14:59:06 -0600, Terri Oda wrote: > Hi Alex! Hi, > We're starting week 3 of Google Summer of Code, and as far as I know, > none of us have heard from you, and we're getting rather worried. > Midterms come up fast and we will have to fail you if you don't get in > touch soon. sorry to be so quite until now. As you already presumed, I was busy the last weeks at the university. Sadly google summer of code is a bit us centric. Here, the summer holidays starting not till 14th of July. But now it gets a little bit relaxed at university and I will be able to start coding this week. Currently I already created a blog[1] to present the recent development I will do. I also created a personal branch on launchpad (you could find it on[2]), but as I stated earlier, I have not pushed any new commits, but I will start soon this week. I hope you could excuse the late start and I hope that I will be able to create great results. > Terri Alex [1] http://blog.animux.de/category/gsoc2012.html [2] https://code.launchpad.net/~alexander-sd4c/mailman/mailman

2 1

[HyperKitty] Rating of posts
by Aamir Khan 06 Jun '12

06 Jun '12

Hello everyone, I am working on HyperKitty archiver project. I am working on feature which will allow the logged-in archiver users to rate emails by upvoting/downvoting. Broadly speaking, there are two kind of emails posted on mailing lists: 1. *Announcement :* Reply is not generally required. 2. *Questions :* Most of the emails posted on mailing are some sort of questions. We want our users to upvote the reply (email) which answered the question accurately. It will be easier for other users of archiver to easily find the best answer (by sorting answers by number of upvotes) of the question without going through all the replies posted on same thread. Rating will be done at email level which means each email can be rated. I think there is no point of having a thread level rating since you can't really compare two different questions. Your thoughts? It's pretty much similar to Q/A site www.quora.com. *Database Schema: *Table structure is very straightforward and simple. Table name : ratings table structure emailid int| userid int| value (+1/-1) where, emailid - references to unique entry storing the actual emails. userid - references to unique entry in user table. value - +1 for upvote and -1 for downvote. In this case each upvote/downvote will be saved as a separate row in database. Your suggestions? -- Aamir Khan | 3rd Year | Computer Science & Engineering | IIT Roorkee

1 0

Re: [Mailman-Developers] [GSoC 2012] Metrics
by Stephen J. Turnbull 06 Jun '12

06 Jun '12

George Chatzisofroniou writes: > Hello Patrick, > > On Fri, May 18, 2012 at 12:09 AM, Patrick Ben Koetter > <p(a)state-of-mind.de> wrote: > > Like with most statistical data I mostly see the figures being > > used to give statements on quantity - top poster, number of > > threads etc. Do you think it would be possible to also make some > > statements on quality? +1 For example, I think this would be really useful for class discussion lists and the like (on the theory that the best way to learn a subject is to teach it to others). > The metrics will primarily extract the activity of a mailing list > and its users. > > It is possible to emphasize on the quality of the posts but i think > this is a different app I don't see why. I would think quality metrics would be usefully presented via the same application as quantity metrics. It would be interesting to correlate quality and quantity, for example. Do you mean to say "this is out of scope of my project?" As much as I'd like to see quality metrics provided, I'd have to agree with you that it's out of scope of your project (maybe you could do one or more quality metrics on a time-permitting basis at the end of the period).

3 3