• Terri Oda <terri@zone12.com>:

-- SNIP --

Hopefully Florian and Stephen have better memories than I and can fill those in. Everyone else: anything you know that needs to happen before we can get a Mailman Suite beta out?

Maybe documentation that tells interested people how they can start using the beta?

p@rick

-- [*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

Patrick Ben Koetter writes:

Maybe documentation that tells interested people how they can start using the beta?

I'm not sure what you mean. None of the GSoC students had trouble with getting Mailman 3 itself up and running once they got it checked out and installed. From that point of view, the documentation in the Mailman checkout seems sufficient. And we now have two of the three most popular MTAs covered (Postfix and Exim).

Installation from source is currently somewhat painful, indeed, but I don't think poor documentation is the problem. Anyway, good packaging solves the installation problem.

Terri Oda writes:

But yeah, I think packaging [the suite] well (and then documenting *that*) makes a lot more sense,

+1

unless someone's feeling particularly inspired?

-10 -- it's all gonna turn upside down when we package, anyway.

[Aside: If you're feeling *that* inspired, volunteer for Google Code-In, CopyleftGames.org -- Arc sez we gonna need a lotta help!]

Steve

Ralf Hildebrandt writes:

Absolutely. Topics of interest to me would include:

• what has changed (GUI?)

Already in the beta distribution. But briefly, "everything's all-new, all-shiny." There will be mailman2-to-mailman3 migration scripts, of course, but the admin and archive websites are completely new. They are functionally similar so will seem somewhat familiar, of course, and where possible Postorius maintains a similar layout, I think.

Where else do you expect to find this information, so we can put it there?

• how can I install mm2 and mm3 side by side (if possible at all, I'd like to be able to switch back an forth if the shit hits the fan)

Uh, just do it?

Of course it will be MTA-specific. I haven't done it on Postfix, but for Exim you just add a Mailman3 router and transport ahead of the Mailman2 router (I've already submitted the docs for that -- they're very generic -- don't know if Barry's merged them yet), and start adding Mailman3 lists. To reverse migrate, make sure the relevant "is this a mailman3 list" config file is moved out of the way if you want to switch a migrated list back to Mailman2.

For Postfix, I suppose you have to mess with the aliases directly.

Caveats: (1) I have no idea about importing archives from HyperKitty into Mailman2. (2) Reverse migration preserving configuration will not be trivial -- you'll have to keep the original configs around for lists that migrate from Mailman2 to Mailman3, and make new ones for lists newly created on Mailman3. We could make scripts for (2), but I wouldn't trust them very much, they're not going to get a lot of testing.

Ralf Hildebrandt writes:

...

Where else do you expect to find this information, so we can put it there?

I must admin I haven't tried installing yet, mainly because I didn't know how to "go back" in case of problems.

Repeating what I already wrote for completeness here: Migrating and then going back all at once is not necessary. You can migrate bit by bit, although it may require a little bit of manual fiddling if you get to the point of installing both on a production server. By "little bit" I mean installing in {/usr,/var}/lib/mailman3 instead of {/usr,/var}/lib/mailman where Mailman 2 probably lives.

I have to admit I haven't installed into one of the standard hierarchies yet; I run my Mailman 3 lists (all test or non-mission-critical) out of a Python virtualenv in my home directory.

Re installing betas from source:

I wish I could say, "oh, just try it, you'll like it", but building from source isn't as automatic as I'd like. It's not a huge extra effort (ask here), and I have had it install (a few commits back) with just "bzr clone ...; python bootstrap.py". I just prefer a reliable, completely turnkey package when recommending to "pure" beta testers, and it's not quite that yet.

If you do try it, be warned that a couple of tests may infloop (for reasons unknown), although only under some circumstances.

Steve

Aurélien Bompard writes:

...

I don't recall, but I think finishing HyperKitty was one of them. AFAIK it doesn't work as advertised now (at least that's what Shanu told me in her work on a unified "Messaging Interface" for Systers).

Oh, I'm interested by any bug reports. Please.

I don't recall precisely what the issues with running HyperKitty were (she tried installing from source once in March, and again in June). What killed the deal for her was an inability to get existing archives imported due to inadequate documentation. I eventually figured out how to get that done, but by then I just advised her to use the trivial archiver from the core.

It looks like the documentation has improved (but I haven't actually tried following it yet). And for all of HyperKitty, there were over 1000 lines of changes since March, and and it looks like over 500 since June. I guess I'll have to try it again before trying to report bugs.

For the record I'm currently setting up HyperKitty & Postorius in the same Django instance, it seems to work fine (didn't test single sign-on yet but I will).

It would be nice if you at least reported releases here occasionally. According to news.rst there have been two releases since PyCon (1.4 and 1.5), and there is a 1.6 tag in git.

I've also created a branch and a pull request in Launchpad with lots of modifications to the import script.

Branch and pull request for what import script?

It would be nice if you at least reported releases here occasionally.

Very true, I need to do that.

...

I've also created a branch and a pull request in Launchpad with lots of modifications to the import script.

Branch and pull request for what import script?

The "import21" command, which is meant to import an existing 2.1 pickle configuration file into Mailman 3 Here's the pull request (or rather "branch merge proposal"): https://code.launchpad.net/~abompard/mailman/import21/+merge/192146

Aurélien

http://aurelien.bompard.org ~~~~~~ xmpp:aurelien@bompard.org This mail is displayed with recycled electrons

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Hi there...

On 10/29/2013 10:35 PM, Terri Oda wrote:

Stephen, Florian and I had a nice discussion at the GSoC Mentor Summit about what's left before we can release a beta of the entire Mailman Suite (i.e. Mailman core, Postorius and Hyperkitty packaged together). We're getting close! In fact, close enough that I'm going to suggest that we set a beta release target date of Dec 1st.

I know we had a list of 5 things... but for some reason I can't remember all of them. Florian, Stephen, can you chime in?

1. Need a Postorius interface for associating multiple email addresses with a single account. This is probably going to require either an email verification, so we might want to have that as part Mailman Core rather than doing it directly in Postorius. Short term, I don't care how it's done as long as it verifies that the user actually has access to the addresses they link together.

Short term we could use django-registration to verify additional email addresses. It should be very easy to integrate with Postorius. Plus it's maintained by one of the Django people so I guess it's stable enough to work even long term.

2. Postorius & Hyperkitty need a bit more work on the authentication management. Florian indicated that he's looking for some code review for the persona integration and that we'd really like a single sign on solution for postorius/hyperkitty to work together seamlessly.

I got a branch almost ready (90%-ish) for merging that contains A) a change from our current Persona integration (django-social-auth) to Mozilla's own django app and B) some code to take domains created in Mailman-core into account when verifying Persona tokens. Since this is a security issue, I'd really like to have a second set of eyes having a look at it before merging. (Yes, I'm especially looking at you Dr. Terri! I'll poke you when I've added the remaining 10%... ;-) That's next on my list as soon as I have reduced my current post-travel-pile-of-errands a bit.

Re: Hyperkitty: Last time I checked it didn't seem to implement any access control for non-public lists. This isn't really a bug, but it restricts HK to be used with public lists only. @Aurelien: Is that correct? If so: Would you mind me suggesting some ideas to add member checking for private lists?

3. We need some nice install scripts or packaging work so that installing Mailman Suite is easy. Probably for beta purposes this can just be a shell script, but we also talked about doing a meta "Mailman Suite" PyPI package and maybe doing a .deb or .rpm as proof of concept.

4. ??

I'm not sure, but I *think* I mentioned I wanted to add some changes to Postorius' test suite before releasing the beta. Which isn't really a *feature*, more like an itch that needed scratching. Anyway, I merged that one last week.

5. ??

Owner/Moderator removal. It's essential, but implementation-wise that's a minor thing.

Hopefully Florian and Stephen have better memories

Well, I'm not sure myself if items 4 and 5 were really the ones we talked about... ;-)

Cheers Florian

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJScs9fAAoJEEceGbPdavl74X0H/1Rafly+Cyhu6zb+MkSPur8U rbmDrYjZEhfJiATBWZxT3w85f/oLHVPUfPtEHcfxAUR32OwI4ZTPSYot05gqCzqZ dnb55XHbMkFAOx9BjrrV3ncau/rUie6MR2cIslQRoegFtaG7X9euRcw1P3qi5Ppp C96L8ZFgPeuynjdy2sAs8zRNDhhsgs+Nq8bWneZBQtsy+AAjJKnrQMlNzxtGSZi2 gVCvcubj66zrxl4k/yXUzVNgvtm9fT64NvtAVch5Duo4zh4TUjdANAzNDHTR/jk7 yVZ1bqW9Rvq5ttzaaYjMUiGD8LLYWHhsRQK9+FyEw3y5OZoonGE7Li2TNX+L0QY= =7wnE -----END PGP SIGNATURE-----

On Oct 30, 2013, at 02:49 PM, Aurélien Bompard wrote:

I've also created a branch and a pull request in Launchpad with lots of modifications to the import script.

I'm slowly working my way through this branch. I'll have some time this weekend to hopefully finish it up.

Cheers, -Barry

Barry Warsaw writes:

So this feature in particular, if Postorius can do all the necessary confirmations, I can much more easily provide an API that Postorius can call to associate an email address with a given user.

I'm getting that ol' sinking feeling. Either Postorius is *the* Mailman 3 user/admin UI, or it isn't. If it is, shouldn't we merge the projects sooner rather than later?

If it isn't, Mailman should be able to do this kind of thing itself.

It should be reasonably low-cost to build a simple django app (or lower-level, SimpleHTTPServera app) that handles confirmation.

Or we can do without web confirmation URLs if there's no admin UI available.

On Nov 02, 2013, at 04:06 PM, Stephen J. Turnbull wrote:

...

So this feature in particular, if Postorius can do all the necessary confirmations, I can much more easily provide an API that Postorius can call to associate an email address with a given user.

I'm getting that ol' sinking feeling. Either Postorius is *the* Mailman 3 user/admin UI, or it isn't. If it is, shouldn't we merge the projects sooner rather than later?

If it isn't, Mailman should be able to do this kind of thing itself.

It should be reasonably low-cost to build a simple django app (or lower-level, SimpleHTTPServera app) that handles confirmation.

Or we can do without web confirmation URLs if there's no admin UI available.

I think it definitely makes sense for there to be a REST API to explicitly add an email address to a user, or to link an existing address to a user. There's already an API for verifying (and unverifying) an email address. This API would not do any confirmations - it would trust the client to have done whatever it takes to ensure that the user owning the first (existing) address also owns the second address. If we want to combine linking and verifying in one step, then the client would be trusted to ensure that the user owns both addresses *and* that the second address is actually valid. I'm guessing this would be the most common use case.

If the client wants to leave it to the core to do the confirmations, I think that would be a separate API. What would be the protocol, given that we want to make sure that both addresses are owned by the same user *and* we want to be sure that the confirmation process can't be used to mine information such as potential addresses subscribed to the system?

Strawman:

Generate a unique token for the dual-confirmation event.

Send a confirmation message to the user's preferred address. It would say something like "Someone has requested linking foo@example.com to your Mailman account. If this was you, please reply (and/or click) to confirm this request. The request will not be completed until you complete a similar confirmation in your foo@example.com account. If you did not make this request, ignore this message."

The Reply-To address would contain the token.

(With sufficient hand-waving, the URL would contain the token.)

Send a message to the new address saying "You have requested a registration of your foo@example.com address to the Mailman server at example.net. Reply (and/or click) to confirm."

Maybe they get different tokens. In any case, nothing would happen until both requests are confirmed.

The confirmation to the new address is specifically crafted not to reveal whether the original address is registered or not. E.g. if the core had to do a new user registration, it would send a similar message, and nothing about the original account would be leaked in this second email. To a new user, it would just look like an original registration confirmation message.

The "click" part is where we get hand-wavy. I need to finish up my template branch so that these types of "core knows something about the web ui" assumptions don't have to be hard-coded. Even if Postorius is the default ui, we may not know exactly how to contact it, or whether the site has customized various texts. I'm thinking something similar to the way the welcome message is now retrieved via url.

Cheers, -Barry

On Sat, Nov 02, 2013 at 04:09:27PM +0900, Stephen J. Turnbull wrote:

Barry Warsaw writes:

...

...

Mailman2 router (I've already submitted the docs for that -- they're very generic -- don't know if Barry's merged them yet), and start adding Mailman3 lists.

This branch?

https://code.launchpad.net/~stephen-xemacs/mailman/exim4/+merge/187302j

Yes, but don't worry about it. It's straightforward and I'm happy to support Exim users who want to try Mailman 3 (don't post all at once, now...).

That'll be me :o) (I can't get my head around Postfix, and it seems to be more trouble than Exim (or Postfix users don't RTFM as much as Exim ones)).

https://code.launchpad.net/~stephen-xemacs/mailman/exim4/+merge/187302j 404s for me; is http://bazaar.launchpad.net/~stephen-xemacs/mailman/exim4/revision/7219 the same?

The only thing I would say (and were this on Github, I'd have just added a comment) is

for --

 252    # /etc/exim4/conf.d/main/455_mm3_router
 253    mailman3_router:
 254      driver = accept
 255      domains = +mm_domains
 256      require_files = MM3_LISTCHK
 257      local_part_suffix_optional
 258      local_part_suffix = -admin : \
 259         -bounces   : -bounces+* : \
 260         -confirm   : -confirm+* : \
 261         -join      : -leave     : \
 262         -owner     : -request   : \
 263         -subscribe : -unsubscribe
 264      transport = mailman3_transport

it might be worth adding a reminder that order of routers matters in Exim (or perhaps add that to the note:

 227 Debian-specific script.  If your Exim v4 installation is structured
 228 differently, ignore the comments indicating location in the Debian
 229 installation.

I've been holding off deploying MM3 until it has a usable web-interface, but as it looks as though that might be there, and now you've added the Exim stanzas, I suppose I really should give it a whirl…

-- "all the succession and repetition of massed humanity ... Those vile bodies"

On 30 Oct 2013, at 13:33, Stephen J. Turnbull <stephen@xemacs.org> wrote:

Ralf Hildebrandt writes:

...

...

Where else do you expect to find this information, so we can put it there?

I must admin I haven't tried installing yet, mainly because I didn't know how to "go back" in case of problems.

Repeating what I already wrote for completeness here: Migrating and then going back all at once is not necessary. You can migrate bit by bit, although it may require a little bit of manual fiddling if you get to the point of installing both on a production server.

The tricky bit here is the continuity of the web interfaces. If a list manager is used to going to lists.example.com/Mailman/listname, and they have that bookmarked, then they’ll want to keep going to that same place.

Likewise, with the list archives, and all the list users. We have 1312 lists, perhaps two or three thousand list admins, and tens of thousands of users.

I’m currently struggling with how to migrate those lists to new hardware, and I think I’m going to have to do it all at once in order to avoid having to educate everyone to find a new web address, and then to revert to the old one. Perhaps I could alternatively tell them that web access is unavailable during a migration week.

I’m also nervous about breaking migration through the archives.

-- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148

Ian Eiloart writes:

On 30 Oct 2013, at 13:33, Stephen J. Turnbull <stephen@xemacs.org> wrote:

...

Repeating what I already wrote for completeness here: Migrating and then going back all at once is not necessary. You can migrate bit by bit, although it may require a little bit of manual fiddling if you get to the point of installing both on a production server.

The tricky bit here is the continuity of the web interfaces. If a list manager is used to going to lists.example.com/Mailman/listname, and they have that bookmarked, then they’ll want to keep going to that same place.

Sure, and I don't blame them.

I could have been more careful in expressing myself. I didn't really mean that you could do it in dribs and drabs, rather that you could start with a test list that the kind of beta tester who uses git to get betas wouldn't mind being on, move on up to a few hard-core techie lists whose members understand "beta", and only when people are satisfied that those are working, then do the rest. And at each stage it's reversible without losing data -- but not necessarily at zero cost to users, or to you.

Whether that's good enough for you is something you'll have to decide. I just wanted to make it clear that if you *had* a working Mailman 2 installation you can revert to it.

I’m also nervous about breaking migration through the archives.

Unfortunately I don't know enough about HyperKitty to help you there. I suspect that the HyperKitty guys have paid about the same amount of attention to consistency across migration as the Postorius folks have, and it matters more there because who knows what people might have bookmarked?

...

I’m also nervous about breaking migration through the archives.

Unfortunately I don't know enough about HyperKitty to help you there. I suspect that the HyperKitty guys have paid about the same amount of attention to consistency across migration as the Postorius folks have, and it matters more there because who knows what people might have bookmarked?

We have a way to redirect the old archives URL format to the new one, but it's not very solid because it depends on the order the emails were archived (that's how it was done in pipermail), and a duplicate email can put the numbers off. If anyone has a better idea, I'm interested, I'd sure like to preserve all the links to individual emails that are laying around on the interwebs.

Aurélien

http://aurelien.bompard.org ~~~~~~ xmpp:aurelien@bompard.org Better to light a candle than to curse the darkness. -- Chinese proverb