Browser for mailing lists

I wanted to search this list (and other Python ones) but it is practically impossible to do so. So why not add a tool like http://markmail.org/search/?q=python to https://mail.python.org/mailman/listinfo/python-ideas or the like? I got this from http://openmeetings.apache.org/mail-lists.html Cheers, Richard

On Sat, Mar 29, 2014 at 9:29 AM, Richard Prosser <richard.prosser@mail.com> wrote:
I wanted to search this list (and other Python ones) but it is practically impossible to do so.
Google lets you search a specific URL (and its subpages): site:https://mail.python.org/mailman/listinfo/python-ideas ... That's been the most effective tool I've used. -eric

On Sat, Mar 29, 2014 at 11:46 AM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
site:https://mail.python.org/mailman/listinfo/python-ideas ...
Correction: site:https://mail.python.org/pipermail/python-ideas/ ... -eric

Richard Prosser <richard.prosser@mail.com> writes:
I wanted to search this list (and other Python ones) but it is practically impossible to do so.
The forum <URL:https://mail.python.org/mailman/listinfo/python-ideas> has all its messages archived online for web browsing at <URL:http://mail.python.org/pipermail/python-ideas/>. So you can use any web search tool to search it. A good search engine like DuckDuckGo will let you constrain the search to a particular subset of the web: site:http://mail.python.org/pipermail/python-ideas foo bar baz
So why not add a tool like http://markmail.org/search/?q=python to https://mail.python.org/mailman/listinfo/python-ideas or the like?
We have generally-appicable services that index the web and make much better search tools. I don't often find a site's own custom search tool to be sufficiently better to use it, when DuckDuckGo is available. -- \ “For every complex problem, there is a solution that is simple, | `\ neat, and wrong.” —Henry L. Mencken | _o__) | Ben Finney

On 30 March 2014 06:54, Ben Finney <ben+python@benfinney.id.au> wrote:
That said, migrating to Mailman3 + HyperKitty is definitely in the longer term plans for the python.org mailing list infrastructure, and that includes integrated search on the HyperKitty side of things. While Mailman3 has been an ongoing project for quite some time (I believe the remaining blockers mostly relate to handling migrations of existing Mailman 2 installations), the HyperKitty work is mostly being driven by some Fedora folks in order to upgrade Fedora's own infrastructure. You can see the current state of the prototype here: https://lists.stg.fedoraproject.org/archives/ Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 30Mar2014 08:05, Nick Coghlan <ncoghlan@gmail.com> wrote:
Please tell me Mailman3 still includes the pipermail stuff still. Looking at the Fedora HyperKitty page you cite, is it just me or is this really slow and clunky? [Ten minutes later...] It is slightly quicker now, but that first page load took minutes. The main page search (multilist) doesn't show the list name in the search results, making it harder to gauge relevance. I think this is an argument for the OP's "convenient search of relevant python lists" request, even if that were a quick hack that readirected to a general purpose search engine with a funky search criterion. But a search engine covers multiple sites; this can only search its own lists. Also, there are no archive download links. Nor can I see the original text of any message (no headers or anything). This cripples personal searching. When I join a list my first act, if possible, is to download the pipermail archives and unpack them into the mail folder in which future lists posts will go. That gets me: - local search of local data access when offline, privacy, speed, whatever tools or indices I prefer - search tools of my choosing (eg mairix or mu), including the search facilities of my mail reader - using my mail reader search lets me read messages in my _preferred_ form, as it would for anyone else using their preferrer mail reader Without an archive download, this is all far far less useful. HyperKitty looks... pretty... I suppose (for some definition of the term; I'm no far of the great big blocks of whitespace on either side, etc), for someone who enjoys web forums. Disgruntledly yours, -- Cameron Simpson <cs@zip.com.au>

On 31 Mar 2014 08:22, "Cameron Simpson" <cs@zip.com.au> wrote:
tool
Mailman3 includes a pluggable archiver model - that's how HyperKitty can be a separate project, specifically aimed at tackling the "seamless web gateway" problem. prefer
form,
as it would for anyone else using their preferrer mail reader
Without an archive download, this is all far far less useful.
The pluggable archiver model in MM3 means HyperKitty itself doesn't need to handle that usage model. However, I'm not aware of any current efforts to create an MM3 archiver that is closer to a pure pipermail replacement (the workflow you describe strikes me as being incredibly unusual these days, as it requires either running your own mail server, or only accessing your email from one device) Cheers, Nick.

On 31Mar2014 08:37, Nick Coghlan <ncoghlan@gmail.com> wrote:
If it can be run in parallel, fine. If it replaces pipermail, it looks like a step backwards.
My workflow is unusual only because most people don't think about it and because the existing MUAs do not make it easy. There _should_ be a "join a mailing list" button with optional archive suck. Most modern MUAs have a "search my mail" facility. Making the raw list archive hard to obtain gets in the way of offering the user's own search field for search. _Nothing_ in my workflow requires running my own mail server, it merely requires having local folders; which is entirely reasonable with an IMAP service, and entirely reasonable with multiple devices. Using one's own search tools is only unusual because unless you're a nerd it is hard. Likewise, sucking down the archives is difficult for end users for various reasons: - even pipemail makes you do that in pieces - the mbox archive URLs (often gzipped) don't come down with a cool MIME type meaning "UNIX mailbox format" - the pipermail archives need some unobfuscation before use as pure mbox files - most MUAs do not install a platform hook to handle an mbox MIME type (if there even is one), thus making an mbox download that much harder on the user. None of these is hard to address. I do it with a tiny shell script, but that is not for most people. I still content that any change that removes the simple ability to fetch the list archive for use by the end user is a huge step backwards, regardless of the prettiness of whatever replaces it. Cheers, -- Cameron Simpson <cs@zip.com.au> Early to bed and early to rise makes a man healthy and wealthy with bags under his eyes. - Dave Cochran, <cochran@dg-rtp.dg.com>

N.B. This is off-topic. If you really care that much, people who want to hear about it are over on mailman-developers@python.org. Cameron Simpson writes:
- the mbox archive URLs (often gzipped) don't come down with a cool MIME type meaning "UNIX mailbox format"
There isn't one, can't be one at present, and likely won't be one in the future: http://www.jwz.org/doc/content-length.html. This still applies to some extent today despite Jamie's rant being almost 20 years old -- Mailman can't choose the mbox spec being used, that depends on the MTA. If you want something that could actually have a MIME-Type, there's Babyl, MMDF, or (file-per-message) maildir, all of which have reasonably precise specs -- but aren't in anywhere near as wide use. Maildir is what Mailman 3's bundled "trivial archiver" uses, as it's far more robust and easier to handle than mbox.
It's not a question of "removing". This has always been a user choice, and many lists use third-party services (mail-archive.com) or non-pipermail archivers (MHonArc). It's just that Mailman 2 with its bundled mbox-format archiver made it easy to provide by default. If lots of users really prefer pipermail, for whatever reason, it's not that hard to add by hand, and could be packaged on PyPI.

On 31Mar2014 14:07, Stephen J. Turnbull <turnbull@sk.tsukuba.ac.jp> wrote:
N.B. This is off-topic. If you really care that much, people who want to hear about it are over on mailman-developers@python.org.
I may join that list, though I'd be an informed commenter (== whinger) rather than a dev.
It can cheerfully choose the mbox spec for delivery of an archive. It is very simple and is a single flat file. As a bonus, many MUAs can read it directly. I've seen JMZ's content-length rant in the past, or something equivalent; if one documents what is being done an mbox can still be delivered usefully to the end user. If one doesn't want Content-Length headers in the archive format, the ">From" hack works quite well. Conversely, if is perfectly possible to create valid Content-Length heders when constructing the archive. We're talking about an archive/transfer format here, not whatever raw format one works with locally.
I meant the BSD mailbox format, often called mbox. I know there are several minor variants, but if mailman (or whatever subcomponent) documents its chosen flavour that's perfectly tractable.
Maildir is what Mailman 3's bundled "trivial archiver" uses, as it's far more robust and easier to handle than mbox.
Maildir is great as a storage platform; I use it myself for many folders. But as a delivery format for end users one then has to pick some wrapper. Zip or tar or... mbox!
Effectively it is removal. If the python lists moved to mailman 3 and suddenly didn't have a "download the list archive", from an end user point of view that is removal.
If lots of users really prefer pipermail, for whatever reason, it's not that hard to add by hand, and could be packaged on PyPI.
Pipermail has two sides to it: the archive bundles and the message browser. Plenty of things do a richer message browse (MHonArc seemed quite nice when last I looked, many years ago). I'm not wedded to pipermail; from a browsing point of view this new thing may do well. And I've seen plenty of other simple things on the net that expose the discussion threads nicely. I'm wedded to having the archives available in a simple and easy to fetch form, in a format that is easily imported by many MUAs. Therefore: mbox. When Google Groups became a popular site to host mailing lists I became aghast at how crude and forumlike the interface is/was, and how the historical list archive was effectively unavailable. When Google Groups sucked in the usenet archive tapes I was full of hope, but I don't see those archives available for download either. We have all chosen our preferred tool to read lists (be it in "mail" or "usenet" forms); not being able to access old messages the same way as current messages feels... shortsighted. Cheers, -- Cameron Simpson <cs@zip.com.au> Archiving the net is like washing toilet paper! - Leader Kibo

On 31Mar2014 13:48, Barry Warsaw <barry@python.org> wrote:
It is for me. I do this regularly. I've come across this in the past: one describes one's work practices and people tell you it isn't a real world use case. I know I don't _want_ to live in the real world, but I'm pretty sure I do.
As I've said repeatedly now. To be able to search it or otherwise consult it with my own tools and when offline, and to browse it the way I _like_ to browse email: in a mail reader. BTW: 156 GB? Only python-list? I'd have thought I'd have pulled it when I joined and my archive is only around 2GB as stored on disc. A rough guess based on eyeballing the gzipped mbox downloads available from pipermail suggests a similar total. Are there tons of predating 1999, or not in pipermail? Cheers, -- Cameron Simpson <cs@zip.com.au> One rider (Kenny Roberts?) described The CorkScrew as: It's like driving your bike into a phone booth, and then having the phone booth dropped down an elevator shaft!

Discussing specifics of Mailman 3 and archives is off-topic for this list, so I won't post a huge amount of detail. Everyone is welcome to come to the mailman-developers list to further interact with us. On Mar 31, 2014, at 08:37 AM, Nick Coghlan wrote:
I would certainly not start from the existing Pipermail code, unless you pine for the late '90s. While the possibility exists to allow for downloading the raw archives, we disable that in Mailman 2 by default because of the feature's abuse for email harvesters. My personal favorite combination for drive-by engagements, historical archive access, convenience, and inbox sanity is the NNTP API for Gmane. My primary MUA (claws-mail) is perfectly at home with NNTP and while mailing lists have to explicitly opt-in to Gmane, once they do, it can import the archive. Countless times I've subscribed to a mailing list's newsgroup gateway, interacted with that community for a short time, and then unsubscribed. The architecture of MM3 allows, and I would someday love, to be able to offer an NNTP interface to a mailing list's archives. (Gmane does a lot more and we don't need to duplicate all of it.) If that's something you'd be interested in working on, come talk to us! -Barry

On Mon, 31 Mar 2014 13:44:52 -0400 Barry Warsaw <barry@python.org> wrote:
I understand that, but is it actually required for simple feedback as a mere Web user?
I also use claws-mail with Gmane NNTP, and I like it. However, it's not really adequate for casual browsing of mailing-lists you don't usually read, or for browsing of private mailing-lists with protected archives, or for giving out hyperlinks to individual mailing-list messages. About Hyperkitty, I have to say that the following kind of UI, while seemingly "nifty", is IMHO a massive step back from pipermail's ease of use: https://lists.stg.fedoraproject.org/archives/list/389-devel@lists.fedoraproj... pipermail's dead simple threaded view is, ironically, the best UI I've ever seen for *browsing* mailing-lists (Google Groups are a disaster in that regard, and Hyperkitty seems to be heading in the same direction as Google Groups). I would really hate to have to deal with Hyperkitty's current UI on a daily basis. Regards Antoine.

On Mar 31, 2014, at 08:27 PM, Antoine Pitrou wrote:
I understand that, but is it actually required for simple feedback as a mere Web user?
No, but your feedback won't necessarily reach the developers who can do something about it.
I like it for casual browsing, but general use the web ui for searching. It is a little tricky to then refer back to the NNTP interface for (IMO) more easily catching up on a thread, or responding.
I think Pipermail's ui has problems, not the least of which is the arbitrary splitting of threads across months, f.e. -Barry

From: Terry Reedy <tjreedy@udel.edu> Sent: Saturday, March 29, 2014 3:08 PM
How hard would it be to add appropriate links (or, better, custom search boxes, or, worse but still helpful, a paragraph explaining how to search each of those manually) to the mailing list archive pages? For example, if you link to this: http://www.google.com/advanced_search?hl=en&fg=1&as_sitesearch=http%3A%2F%2Fmail.python.org%2Fpipermail%2Fpython-ideas … then people just need to fill in their search terms on the Advanced Search page that comes up.

I agree - links to search tools or at least explanations of how to use them would be very welcome. Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead. Richard On 30 March 2014 05:02, Andrew Barnert <abarnert@yahoo.com> wrote:

On Mon, Mar 31, 2014 at 01:09:48PM +0100, Richard Prosser <richard.prosser@mail.com> wrote:
Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead.
OMG, please, no! Mailing lists are certainly much better than any forum. Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.

On 31/03/2014 13:09, Richard Prosser wrote:
Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead.
To avoid clogging up your mail client I suggest gmane.comp.python.xyz, where in this case xyz is ideas. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com

On 03/31/2014 05:09 AM, Richard Prosser wrote:
I agree - links to search tools or at least explanations of how to use them would be very welcome.
+1
Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead.
Even better than that: use a decent mail client, and exercise one's mouse and keyboard to delete unnecessary context when replying to posts. -- ~Ethan~

On Mon, Mar 31, 2014 at 11:51:21AM -0700, Ethan Furman <ethan@stoneleaf.us> wrote:
+many Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.

On 1 Apr 2014 05:40, "Ethan Furman" <ethan@stoneleaf.us> wrote:
On 03/31/2014 05:09 AM, Richard Prosser wrote:
Better still I believe, abandon mailing lists altogether (because they
can clog up mail clients) and use forum/BB instead.
Even better than that: use a decent mail client, and exercise one's
mouse and keyboard to delete unnecessary context when replying to posts. No need to get personal folks. I use a lot of both mailing lists *and* web based communication interfaces, and see a place for both of them. Mailing lists, I prefer when I'm highly invested. Just as an RSS reader brings blogs I care about into a single interface, mailing lists do the same for collaborative communication. Appropriate use of filters and "mark all as read" then deals with the volume of traffic. However, mailing lists are *terrible* if you just want to drop in, ask one question & leave again. One possible answer to that is AskBot (a self-hosted site-specific, Stack Overflow style Q&A forum), but the MM3/HyperKitty web gateway idea is aimed at making the existing *already invested* list participants more readily available to new users without having to ask those existing participants to change their reading habits and look in a new place for questions. In the meantime, making GMane more discoverable by suggesting it as an alternative to subscription in the various list descriptions sounds reasonable to me. Cheers, Nick.

Nick Coghlan <ncoghlan@gmail.com> writes:
Presumably that's because, in order to follow and participate, one needs to subscribe and receive all the messages in one's email. I agree, that's not ideal for dropping by and asking a question. Which is why I prefer NNTP: it is ideal for *both* these purposes. One can quickly dip into a forum, follow a thread for a while, and unsubscribe just as easily. On the other hand, for high-commitment communication, having all the forums together and threaded is excellent. Having all of this in a single NNTP client interface makes discussion on dozens of forums much more manageable than either “web forums” (*spit*) or mailing lists.
GMane survives on sponsorship and donations. If we're sending more and more people to use that service, perhaps we can suggest supporting them financially too? -- \ “All opinions are not equal. Some are a very great deal more | `\ robust, sophisticated and well supported in logic and argument | _o__) than others.” —Douglas Adams | Ben Finney

On 31 March 2014 23:01, Ben Finney <ben+python@benfinney.id.au> wrote:
One of the key downsides for me of an NNTP interface is that it's (as far as I know) extremely difficult to set up a synchronised environment across two or more PCs. For email, gmail, while not perfect, is a reasonably good client, and I can use it on multiple PCs. I know of no NNTP client with a comparable feature set that tracks what I have and haven't read across multiple PCs, allows posting, etc. If anyone can suggest such a client (which works on Windows, but as it'd probably be web-based, that's probably less of a limitation that normal), I'd be immensely grateful. I realise this is off-topic, but I don't feel *too* guilty as this whole thread is pretty off-topic :-) Paul PS Google groups and the gmane web interface need not apply - neither are particularly friendly (to be polite...)

On 1 Apr 2014 08:22, "Paul Moore" <p.f.moore@gmail.com> wrote:
Yep, handling multiple systems is when I stopped using it, too. Phones make it worse (a lot of my email list time is on the bus these days - aside from making bulk deletion while still retaining some context painful, Gmail handles that pretty well)
Improving the accessibility of the list is pretty close to on topic when it's covering some specific ideas that could improve things without much work. I only see it as a problem when we head off on wild impractical tangents that are unlikely to happen any time soon (if ever) due to resource constraints :) Cheers, Nick.

On Tue, Apr 1, 2014 at 1:01 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
GMane is a good project, but is it open source? Does it help people to raise expertise in Python, train they skills and in the end contribute to engineer the next generation scripting language? I thought that the goal of Python folks as a community is to encourage collaboration, reuse and exchange of experience. In the regard I expected people to propose integrating something like https://bitbucket.org/mchaput/whoosh/wiki/Home and discussing what is required to make it possible. -- anatoly t.

On Apr 01, 2014, at 09:01 AM, Ben Finney wrote:
Yes, exactly.
My only beef with Gmane is that it should be able to maintain a global reputation service. E.g. if I post to a few newsgroups with my python.org address, then after a N-confirmation dances, it should allow me to post to any newsgroup (well, any that is linked to an upstream list with a compatible posting policy). That's one of the things that an NNTP interface to MM3 would allow, and I think it would work more smoothly. Of course, the nice thing about Gmane is that it collates lists across many administrative domains, so I don't think an NNTP-aware MM3 would replace Gmane. (The other thing I want to do is make it one-click for a list admin to opt into Gmane. That actually should be pretty easy, and maybe the default for public discussion lists.)
+1. They're good folks and provide the entire FLOSS world a huge benefit. -Barry

On Tue, Apr 1, 2014 at 8:51 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Drifting off-topic, but I think this is actually a subtle feature. I've often joined a mailing list to ask one specific question, and then seen questions that I can help answer. In fact, that's how I came to be stuck on python-list and family... of course, other people may not see that as a positive feature... What it means is that there's no stark difference between "people here to ask questions" and "people here to answer and help out". It's users helping users. Most lists don't have so much traffic that it turns me off (a few do - I unsubscribed from the SciTE lists fairly quickly), so I'll hang around for a while after getting my answer - or, given that I've probably exhausted a number of other avenues before even posting the question, after not getting any answer. The more people you can engage that way, the more likely the list will have the person on it who can answer that obscure question that comes up. ChrisA

On Apr 01, 2014, at 10:22 AM, Chris Angelico wrote:
Another feature/wishlist of MM3 is a reputation service where, let's say you've engaged with three lists on python.org. Each time you've had to do a email address confirmation dance, and get your postings moderated. Once you've done that a couple of times across a few lists (configurable via admin knobs), you're deemed a member of the *site* in good standing. Now you can post to any list that opts in with no further confirmations. If you game the system to spam, an admin can lower your reputation and then all your posts would be moderated or discarded. I think a system like this would really make it very easy for users to have both long term engagements with some lists, and low-barrier short term drive-by engagements with others. -Barry

On Mon, Mar 31, 2014 at 01:09:48PM +0100, Richard Prosser wrote:
I agree - links to search tools or at least explanations of how to use them would be very welcome.
It's 2014, not 1998. Do we really need to teach people how to use the Internet? Not just people in general, for that matter, but *programmers*? -- Steven

On 1 April 2014 10:31, Ethan Furman <ethan@stoneleaf.us> wrote:
The other thing to keep in mind is that we're actively trying to remove barriers to entry to participating in the core development process, while still respecting the perspectives of long term community members. That's a long, slow, process with very few quick fixes available, but providing pointers to useful tools (like gmane) to help make them more discoverable is certainly something we can do in the meantime. (We do already reference it from https://docs.python.org/devguide/communication.html#mailing-lists, but that page has its own discoverability issues) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Tue, Apr 1, 2014 at 3:21 AM, Steven D'Aprano <steve@pearwood.info> wrote:
At least there is a problem to teach people born in 1998 how to use means of communications that predates mobile phone era. I'd say that old people like me should not underestimate how new generation-m is different. It may be that ML is the only way to effectively communicate but that doesn't mean they can use it.

On Sat, Mar 29, 2014 at 6:29 PM, Richard Prosser <richard.prosser@mail.com> wrote:
I wanted to search this list (and other Python ones) but it is practically impossible to do so.
Welcome to the museum. Please turn off your gadgets and mobile phones during your visit. =) Seriously, I use this for searching Python-ideas: https://groups.google.com/forum/#!forum/python-ideas
4 years ago I brought this question on python-dev: http://www.gossamer-threads.com/lists/python/dev/807336?do=post_view_threade... No generic search form was added to mailman lists, but at least a link to GMane appeared. I'd say that what you see is a way how things are (I am not permitted to say "how bad things" are, but if you're an UX expert, just to let you know my opinion). Nick said:
I believe that this perfectly sums up what's going on. I am not a community member (didn't sign up CoC or CLA), but I already have a problem trying to keep up with all the threads and mails around Python. Limiting interfaces for collaboration it a natural process to prevent traffic increase. In terms of systems theory - resistance to change old interfaces helps save energy, so there is potentially more resistance than is visible.

On Sat, Mar 29, 2014 at 9:29 AM, Richard Prosser <richard.prosser@mail.com> wrote:
I wanted to search this list (and other Python ones) but it is practically impossible to do so.
Google lets you search a specific URL (and its subpages): site:https://mail.python.org/mailman/listinfo/python-ideas ... That's been the most effective tool I've used. -eric

On Sat, Mar 29, 2014 at 11:46 AM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
site:https://mail.python.org/mailman/listinfo/python-ideas ...
Correction: site:https://mail.python.org/pipermail/python-ideas/ ... -eric

Richard Prosser <richard.prosser@mail.com> writes:
I wanted to search this list (and other Python ones) but it is practically impossible to do so.
The forum <URL:https://mail.python.org/mailman/listinfo/python-ideas> has all its messages archived online for web browsing at <URL:http://mail.python.org/pipermail/python-ideas/>. So you can use any web search tool to search it. A good search engine like DuckDuckGo will let you constrain the search to a particular subset of the web: site:http://mail.python.org/pipermail/python-ideas foo bar baz
So why not add a tool like http://markmail.org/search/?q=python to https://mail.python.org/mailman/listinfo/python-ideas or the like?
We have generally-appicable services that index the web and make much better search tools. I don't often find a site's own custom search tool to be sufficiently better to use it, when DuckDuckGo is available. -- \ “For every complex problem, there is a solution that is simple, | `\ neat, and wrong.” —Henry L. Mencken | _o__) | Ben Finney

On 30 March 2014 06:54, Ben Finney <ben+python@benfinney.id.au> wrote:
That said, migrating to Mailman3 + HyperKitty is definitely in the longer term plans for the python.org mailing list infrastructure, and that includes integrated search on the HyperKitty side of things. While Mailman3 has been an ongoing project for quite some time (I believe the remaining blockers mostly relate to handling migrations of existing Mailman 2 installations), the HyperKitty work is mostly being driven by some Fedora folks in order to upgrade Fedora's own infrastructure. You can see the current state of the prototype here: https://lists.stg.fedoraproject.org/archives/ Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 30Mar2014 08:05, Nick Coghlan <ncoghlan@gmail.com> wrote:
Please tell me Mailman3 still includes the pipermail stuff still. Looking at the Fedora HyperKitty page you cite, is it just me or is this really slow and clunky? [Ten minutes later...] It is slightly quicker now, but that first page load took minutes. The main page search (multilist) doesn't show the list name in the search results, making it harder to gauge relevance. I think this is an argument for the OP's "convenient search of relevant python lists" request, even if that were a quick hack that readirected to a general purpose search engine with a funky search criterion. But a search engine covers multiple sites; this can only search its own lists. Also, there are no archive download links. Nor can I see the original text of any message (no headers or anything). This cripples personal searching. When I join a list my first act, if possible, is to download the pipermail archives and unpack them into the mail folder in which future lists posts will go. That gets me: - local search of local data access when offline, privacy, speed, whatever tools or indices I prefer - search tools of my choosing (eg mairix or mu), including the search facilities of my mail reader - using my mail reader search lets me read messages in my _preferred_ form, as it would for anyone else using their preferrer mail reader Without an archive download, this is all far far less useful. HyperKitty looks... pretty... I suppose (for some definition of the term; I'm no far of the great big blocks of whitespace on either side, etc), for someone who enjoys web forums. Disgruntledly yours, -- Cameron Simpson <cs@zip.com.au>

On 31 Mar 2014 08:22, "Cameron Simpson" <cs@zip.com.au> wrote:
tool
Mailman3 includes a pluggable archiver model - that's how HyperKitty can be a separate project, specifically aimed at tackling the "seamless web gateway" problem. prefer
form,
as it would for anyone else using their preferrer mail reader
Without an archive download, this is all far far less useful.
The pluggable archiver model in MM3 means HyperKitty itself doesn't need to handle that usage model. However, I'm not aware of any current efforts to create an MM3 archiver that is closer to a pure pipermail replacement (the workflow you describe strikes me as being incredibly unusual these days, as it requires either running your own mail server, or only accessing your email from one device) Cheers, Nick.

On 31Mar2014 08:37, Nick Coghlan <ncoghlan@gmail.com> wrote:
If it can be run in parallel, fine. If it replaces pipermail, it looks like a step backwards.
My workflow is unusual only because most people don't think about it and because the existing MUAs do not make it easy. There _should_ be a "join a mailing list" button with optional archive suck. Most modern MUAs have a "search my mail" facility. Making the raw list archive hard to obtain gets in the way of offering the user's own search field for search. _Nothing_ in my workflow requires running my own mail server, it merely requires having local folders; which is entirely reasonable with an IMAP service, and entirely reasonable with multiple devices. Using one's own search tools is only unusual because unless you're a nerd it is hard. Likewise, sucking down the archives is difficult for end users for various reasons: - even pipemail makes you do that in pieces - the mbox archive URLs (often gzipped) don't come down with a cool MIME type meaning "UNIX mailbox format" - the pipermail archives need some unobfuscation before use as pure mbox files - most MUAs do not install a platform hook to handle an mbox MIME type (if there even is one), thus making an mbox download that much harder on the user. None of these is hard to address. I do it with a tiny shell script, but that is not for most people. I still content that any change that removes the simple ability to fetch the list archive for use by the end user is a huge step backwards, regardless of the prettiness of whatever replaces it. Cheers, -- Cameron Simpson <cs@zip.com.au> Early to bed and early to rise makes a man healthy and wealthy with bags under his eyes. - Dave Cochran, <cochran@dg-rtp.dg.com>

N.B. This is off-topic. If you really care that much, people who want to hear about it are over on mailman-developers@python.org. Cameron Simpson writes:
- the mbox archive URLs (often gzipped) don't come down with a cool MIME type meaning "UNIX mailbox format"
There isn't one, can't be one at present, and likely won't be one in the future: http://www.jwz.org/doc/content-length.html. This still applies to some extent today despite Jamie's rant being almost 20 years old -- Mailman can't choose the mbox spec being used, that depends on the MTA. If you want something that could actually have a MIME-Type, there's Babyl, MMDF, or (file-per-message) maildir, all of which have reasonably precise specs -- but aren't in anywhere near as wide use. Maildir is what Mailman 3's bundled "trivial archiver" uses, as it's far more robust and easier to handle than mbox.
It's not a question of "removing". This has always been a user choice, and many lists use third-party services (mail-archive.com) or non-pipermail archivers (MHonArc). It's just that Mailman 2 with its bundled mbox-format archiver made it easy to provide by default. If lots of users really prefer pipermail, for whatever reason, it's not that hard to add by hand, and could be packaged on PyPI.

On 31Mar2014 14:07, Stephen J. Turnbull <turnbull@sk.tsukuba.ac.jp> wrote:
N.B. This is off-topic. If you really care that much, people who want to hear about it are over on mailman-developers@python.org.
I may join that list, though I'd be an informed commenter (== whinger) rather than a dev.
It can cheerfully choose the mbox spec for delivery of an archive. It is very simple and is a single flat file. As a bonus, many MUAs can read it directly. I've seen JMZ's content-length rant in the past, or something equivalent; if one documents what is being done an mbox can still be delivered usefully to the end user. If one doesn't want Content-Length headers in the archive format, the ">From" hack works quite well. Conversely, if is perfectly possible to create valid Content-Length heders when constructing the archive. We're talking about an archive/transfer format here, not whatever raw format one works with locally.
I meant the BSD mailbox format, often called mbox. I know there are several minor variants, but if mailman (or whatever subcomponent) documents its chosen flavour that's perfectly tractable.
Maildir is what Mailman 3's bundled "trivial archiver" uses, as it's far more robust and easier to handle than mbox.
Maildir is great as a storage platform; I use it myself for many folders. But as a delivery format for end users one then has to pick some wrapper. Zip or tar or... mbox!
Effectively it is removal. If the python lists moved to mailman 3 and suddenly didn't have a "download the list archive", from an end user point of view that is removal.
If lots of users really prefer pipermail, for whatever reason, it's not that hard to add by hand, and could be packaged on PyPI.
Pipermail has two sides to it: the archive bundles and the message browser. Plenty of things do a richer message browse (MHonArc seemed quite nice when last I looked, many years ago). I'm not wedded to pipermail; from a browsing point of view this new thing may do well. And I've seen plenty of other simple things on the net that expose the discussion threads nicely. I'm wedded to having the archives available in a simple and easy to fetch form, in a format that is easily imported by many MUAs. Therefore: mbox. When Google Groups became a popular site to host mailing lists I became aghast at how crude and forumlike the interface is/was, and how the historical list archive was effectively unavailable. When Google Groups sucked in the usenet archive tapes I was full of hope, but I don't see those archives available for download either. We have all chosen our preferred tool to read lists (be it in "mail" or "usenet" forms); not being able to access old messages the same way as current messages feels... shortsighted. Cheers, -- Cameron Simpson <cs@zip.com.au> Archiving the net is like washing toilet paper! - Leader Kibo

On 31Mar2014 13:48, Barry Warsaw <barry@python.org> wrote:
It is for me. I do this regularly. I've come across this in the past: one describes one's work practices and people tell you it isn't a real world use case. I know I don't _want_ to live in the real world, but I'm pretty sure I do.
As I've said repeatedly now. To be able to search it or otherwise consult it with my own tools and when offline, and to browse it the way I _like_ to browse email: in a mail reader. BTW: 156 GB? Only python-list? I'd have thought I'd have pulled it when I joined and my archive is only around 2GB as stored on disc. A rough guess based on eyeballing the gzipped mbox downloads available from pipermail suggests a similar total. Are there tons of predating 1999, or not in pipermail? Cheers, -- Cameron Simpson <cs@zip.com.au> One rider (Kenny Roberts?) described The CorkScrew as: It's like driving your bike into a phone booth, and then having the phone booth dropped down an elevator shaft!

Discussing specifics of Mailman 3 and archives is off-topic for this list, so I won't post a huge amount of detail. Everyone is welcome to come to the mailman-developers list to further interact with us. On Mar 31, 2014, at 08:37 AM, Nick Coghlan wrote:
I would certainly not start from the existing Pipermail code, unless you pine for the late '90s. While the possibility exists to allow for downloading the raw archives, we disable that in Mailman 2 by default because of the feature's abuse for email harvesters. My personal favorite combination for drive-by engagements, historical archive access, convenience, and inbox sanity is the NNTP API for Gmane. My primary MUA (claws-mail) is perfectly at home with NNTP and while mailing lists have to explicitly opt-in to Gmane, once they do, it can import the archive. Countless times I've subscribed to a mailing list's newsgroup gateway, interacted with that community for a short time, and then unsubscribed. The architecture of MM3 allows, and I would someday love, to be able to offer an NNTP interface to a mailing list's archives. (Gmane does a lot more and we don't need to duplicate all of it.) If that's something you'd be interested in working on, come talk to us! -Barry

On Mon, 31 Mar 2014 13:44:52 -0400 Barry Warsaw <barry@python.org> wrote:
I understand that, but is it actually required for simple feedback as a mere Web user?
I also use claws-mail with Gmane NNTP, and I like it. However, it's not really adequate for casual browsing of mailing-lists you don't usually read, or for browsing of private mailing-lists with protected archives, or for giving out hyperlinks to individual mailing-list messages. About Hyperkitty, I have to say that the following kind of UI, while seemingly "nifty", is IMHO a massive step back from pipermail's ease of use: https://lists.stg.fedoraproject.org/archives/list/389-devel@lists.fedoraproj... pipermail's dead simple threaded view is, ironically, the best UI I've ever seen for *browsing* mailing-lists (Google Groups are a disaster in that regard, and Hyperkitty seems to be heading in the same direction as Google Groups). I would really hate to have to deal with Hyperkitty's current UI on a daily basis. Regards Antoine.

On Mar 31, 2014, at 08:27 PM, Antoine Pitrou wrote:
I understand that, but is it actually required for simple feedback as a mere Web user?
No, but your feedback won't necessarily reach the developers who can do something about it.
I like it for casual browsing, but general use the web ui for searching. It is a little tricky to then refer back to the NNTP interface for (IMO) more easily catching up on a thread, or responding.
I think Pipermail's ui has problems, not the least of which is the arbitrary splitting of threads across months, f.e. -Barry

From: Terry Reedy <tjreedy@udel.edu> Sent: Saturday, March 29, 2014 3:08 PM
How hard would it be to add appropriate links (or, better, custom search boxes, or, worse but still helpful, a paragraph explaining how to search each of those manually) to the mailing list archive pages? For example, if you link to this: http://www.google.com/advanced_search?hl=en&fg=1&as_sitesearch=http%3A%2F%2Fmail.python.org%2Fpipermail%2Fpython-ideas … then people just need to fill in their search terms on the Advanced Search page that comes up.

I agree - links to search tools or at least explanations of how to use them would be very welcome. Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead. Richard On 30 March 2014 05:02, Andrew Barnert <abarnert@yahoo.com> wrote:

On Mon, Mar 31, 2014 at 01:09:48PM +0100, Richard Prosser <richard.prosser@mail.com> wrote:
Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead.
OMG, please, no! Mailing lists are certainly much better than any forum. Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.

On 31/03/2014 13:09, Richard Prosser wrote:
Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead.
To avoid clogging up your mail client I suggest gmane.comp.python.xyz, where in this case xyz is ideas. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com

On 03/31/2014 05:09 AM, Richard Prosser wrote:
I agree - links to search tools or at least explanations of how to use them would be very welcome.
+1
Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead.
Even better than that: use a decent mail client, and exercise one's mouse and keyboard to delete unnecessary context when replying to posts. -- ~Ethan~

On Mon, Mar 31, 2014 at 11:51:21AM -0700, Ethan Furman <ethan@stoneleaf.us> wrote:
+many Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.

On 1 Apr 2014 05:40, "Ethan Furman" <ethan@stoneleaf.us> wrote:
On 03/31/2014 05:09 AM, Richard Prosser wrote:
Better still I believe, abandon mailing lists altogether (because they
can clog up mail clients) and use forum/BB instead.
Even better than that: use a decent mail client, and exercise one's
mouse and keyboard to delete unnecessary context when replying to posts. No need to get personal folks. I use a lot of both mailing lists *and* web based communication interfaces, and see a place for both of them. Mailing lists, I prefer when I'm highly invested. Just as an RSS reader brings blogs I care about into a single interface, mailing lists do the same for collaborative communication. Appropriate use of filters and "mark all as read" then deals with the volume of traffic. However, mailing lists are *terrible* if you just want to drop in, ask one question & leave again. One possible answer to that is AskBot (a self-hosted site-specific, Stack Overflow style Q&A forum), but the MM3/HyperKitty web gateway idea is aimed at making the existing *already invested* list participants more readily available to new users without having to ask those existing participants to change their reading habits and look in a new place for questions. In the meantime, making GMane more discoverable by suggesting it as an alternative to subscription in the various list descriptions sounds reasonable to me. Cheers, Nick.

Nick Coghlan <ncoghlan@gmail.com> writes:
Presumably that's because, in order to follow and participate, one needs to subscribe and receive all the messages in one's email. I agree, that's not ideal for dropping by and asking a question. Which is why I prefer NNTP: it is ideal for *both* these purposes. One can quickly dip into a forum, follow a thread for a while, and unsubscribe just as easily. On the other hand, for high-commitment communication, having all the forums together and threaded is excellent. Having all of this in a single NNTP client interface makes discussion on dozens of forums much more manageable than either “web forums” (*spit*) or mailing lists.
GMane survives on sponsorship and donations. If we're sending more and more people to use that service, perhaps we can suggest supporting them financially too? -- \ “All opinions are not equal. Some are a very great deal more | `\ robust, sophisticated and well supported in logic and argument | _o__) than others.” —Douglas Adams | Ben Finney

On 31 March 2014 23:01, Ben Finney <ben+python@benfinney.id.au> wrote:
One of the key downsides for me of an NNTP interface is that it's (as far as I know) extremely difficult to set up a synchronised environment across two or more PCs. For email, gmail, while not perfect, is a reasonably good client, and I can use it on multiple PCs. I know of no NNTP client with a comparable feature set that tracks what I have and haven't read across multiple PCs, allows posting, etc. If anyone can suggest such a client (which works on Windows, but as it'd probably be web-based, that's probably less of a limitation that normal), I'd be immensely grateful. I realise this is off-topic, but I don't feel *too* guilty as this whole thread is pretty off-topic :-) Paul PS Google groups and the gmane web interface need not apply - neither are particularly friendly (to be polite...)

On 1 Apr 2014 08:22, "Paul Moore" <p.f.moore@gmail.com> wrote:
Yep, handling multiple systems is when I stopped using it, too. Phones make it worse (a lot of my email list time is on the bus these days - aside from making bulk deletion while still retaining some context painful, Gmail handles that pretty well)
Improving the accessibility of the list is pretty close to on topic when it's covering some specific ideas that could improve things without much work. I only see it as a problem when we head off on wild impractical tangents that are unlikely to happen any time soon (if ever) due to resource constraints :) Cheers, Nick.

On Tue, Apr 1, 2014 at 1:01 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
GMane is a good project, but is it open source? Does it help people to raise expertise in Python, train they skills and in the end contribute to engineer the next generation scripting language? I thought that the goal of Python folks as a community is to encourage collaboration, reuse and exchange of experience. In the regard I expected people to propose integrating something like https://bitbucket.org/mchaput/whoosh/wiki/Home and discussing what is required to make it possible. -- anatoly t.

On Apr 01, 2014, at 09:01 AM, Ben Finney wrote:
Yes, exactly.
My only beef with Gmane is that it should be able to maintain a global reputation service. E.g. if I post to a few newsgroups with my python.org address, then after a N-confirmation dances, it should allow me to post to any newsgroup (well, any that is linked to an upstream list with a compatible posting policy). That's one of the things that an NNTP interface to MM3 would allow, and I think it would work more smoothly. Of course, the nice thing about Gmane is that it collates lists across many administrative domains, so I don't think an NNTP-aware MM3 would replace Gmane. (The other thing I want to do is make it one-click for a list admin to opt into Gmane. That actually should be pretty easy, and maybe the default for public discussion lists.)
+1. They're good folks and provide the entire FLOSS world a huge benefit. -Barry

On Tue, Apr 1, 2014 at 8:51 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Drifting off-topic, but I think this is actually a subtle feature. I've often joined a mailing list to ask one specific question, and then seen questions that I can help answer. In fact, that's how I came to be stuck on python-list and family... of course, other people may not see that as a positive feature... What it means is that there's no stark difference between "people here to ask questions" and "people here to answer and help out". It's users helping users. Most lists don't have so much traffic that it turns me off (a few do - I unsubscribed from the SciTE lists fairly quickly), so I'll hang around for a while after getting my answer - or, given that I've probably exhausted a number of other avenues before even posting the question, after not getting any answer. The more people you can engage that way, the more likely the list will have the person on it who can answer that obscure question that comes up. ChrisA

On Apr 01, 2014, at 10:22 AM, Chris Angelico wrote:
Another feature/wishlist of MM3 is a reputation service where, let's say you've engaged with three lists on python.org. Each time you've had to do a email address confirmation dance, and get your postings moderated. Once you've done that a couple of times across a few lists (configurable via admin knobs), you're deemed a member of the *site* in good standing. Now you can post to any list that opts in with no further confirmations. If you game the system to spam, an admin can lower your reputation and then all your posts would be moderated or discarded. I think a system like this would really make it very easy for users to have both long term engagements with some lists, and low-barrier short term drive-by engagements with others. -Barry

On Mon, Mar 31, 2014 at 01:09:48PM +0100, Richard Prosser wrote:
I agree - links to search tools or at least explanations of how to use them would be very welcome.
It's 2014, not 1998. Do we really need to teach people how to use the Internet? Not just people in general, for that matter, but *programmers*? -- Steven

On 1 April 2014 10:31, Ethan Furman <ethan@stoneleaf.us> wrote:
The other thing to keep in mind is that we're actively trying to remove barriers to entry to participating in the core development process, while still respecting the perspectives of long term community members. That's a long, slow, process with very few quick fixes available, but providing pointers to useful tools (like gmane) to help make them more discoverable is certainly something we can do in the meantime. (We do already reference it from https://docs.python.org/devguide/communication.html#mailing-lists, but that page has its own discoverability issues) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Tue, Apr 1, 2014 at 3:21 AM, Steven D'Aprano <steve@pearwood.info> wrote:
At least there is a problem to teach people born in 1998 how to use means of communications that predates mobile phone era. I'd say that old people like me should not underestimate how new generation-m is different. It may be that ML is the only way to effectively communicate but that doesn't mean they can use it.

On Sat, Mar 29, 2014 at 6:29 PM, Richard Prosser <richard.prosser@mail.com> wrote:
I wanted to search this list (and other Python ones) but it is practically impossible to do so.
Welcome to the museum. Please turn off your gadgets and mobile phones during your visit. =) Seriously, I use this for searching Python-ideas: https://groups.google.com/forum/#!forum/python-ideas
4 years ago I brought this question on python-dev: http://www.gossamer-threads.com/lists/python/dev/807336?do=post_view_threade... No generic search form was added to mailman lists, but at least a link to GMane appeared. I'd say that what you see is a way how things are (I am not permitted to say "how bad things" are, but if you're an UX expert, just to let you know my opinion). Nick said:
I believe that this perfectly sums up what's going on. I am not a community member (didn't sign up CoC or CLA), but I already have a problem trying to keep up with all the threads and mails around Python. Limiting interfaces for collaboration it a natural process to prevent traffic increase. In terms of systems theory - resistance to change old interfaces helps save energy, so there is potentially more resistance than is visible.
participants (18)
-
anatoly techtonik
-
Andrew Barnert
-
Antoine Pitrou
-
Barry Warsaw
-
Ben Finney
-
Cameron Simpson
-
Chris Angelico
-
Eric Snow
-
Ethan Furman
-
Mark Lawrence
-
Nick Coghlan
-
Oleg Broytman
-
Paul Moore
-
Richard Prosser
-
Skip Montanaro
-
Stephen J. Turnbull
-
Steven D'Aprano
-
Terry Reedy