[Mailman-Developers] [Bug 965532] [NEW] Need a script to upgrade from MM2 to MM3

[Richard Wackerbarth]

On Apr 8, 2012, at 12:43 PM, Launchpad Bug Tracker wrote:

> Barry Warsaw (barry) has assigned this bug to you for GNU Mailman:

> Need a script to upgrade from MM2 to MM3
> https://bugs.launchpad.net/bugs/965532

Here are some thoughts on a possible migration technique.
I would request discussion and suggestions.

In particular, what about the idea of converting the configuration file to HTML as an intermediate file format?
Selectable css could easily render it as a viewable report. It could still be edited by hand without too much difficulty.

Richard "Wacky" Wackerbarth
- - - - - - 

Steps to migrate from MM2 to MM3

1) Manually install MM3. Hook it up to the MTA, UI, and Archiver. This should include testing to assure that things are ready to create new lists.

2) Translate list configurations

  a) Use TOOL1 to extract the set of list configurations from MM2. Pipe this to TOOL2 which generates a tree of MM2 configurations. That tree hierarchy would be Root-->World-->Site-->Domain-->List-->Subscriber. TOOL2 would populate configurations at the List level. It might also reformat selected parameters. In particular, various <option type="radio" > entries might be transformed into enumerations such as "Yes"/"No" or "Hidden"/"Private"/"Public" rather than numerical values. This would enhance readability.

  b) TOOL3 would populate the World level with the MM2 defaults and recursively promote common values up the tree, leaving only those entries which would need to override their parent to derive the current value. Values which match the parent would be flagged. (The inheritance flag should be tri-state. "Differs from parent", "Same as parent", "Inherited from parent")

  c) At this point, the user might edit some of the configurations and rerun TOOL3 adjusting the inheritance flag as appropriate.

  d) Now, we begin translation to MM3 configuration options. For each MM3 option, TOOL4 computes the equivalent value from the MM2 values. Each computed value also gets the corresponding inheritance flag. Values that cannot be computed from the available information get the "Inherited from parent" flag. MM2 values used in computations are marked as "translated".
  e) After a chance to edit the MM3 configurations, TOOL5 would recompute inheritance flags, report any MM2 values that have not been translated and produce a copy of the configuration file simplified by removing all inherited entries.

  f) After a final inspection TOOL7 would actually import the configurations, committing entries to the MM3 database.

3) For the migration of rosters, we should be able to do it one subscription at a time through a pipeline that permits pre- and post- hooks.  A --dry-run option would be appropriate.

  a) We can assume that each email address is a distinct person.  The subscribers can utilize the UI to merge email addresses into a common persona.

  b) We can also assume that each subscription overrides its parent in the Persona-->EMailAddress-->Subscription hierarchy. The individual users can use the UI to consolidate their selections.

Some additional thoughts:

	All of the tools should be written in Python, hopefully in a dialect that is common to all of the versions supported by MM3.
	TOOL1 already exists (`bin/export.py`). TOOL2 can discard the roster nodes as they come in. Similarly, in step 3, we can use TOOL1 and discard the list configuration information.
	TOOL2 can reformat the XML as HTML, thus making the input data into a viewable report. The inheritance flag would become a class attribute on the option. Would it make sense to go a step further and generate html forms and run a trivial http server on localhost? It might be easier to do this in django, but I think that requiring that level of installation is probably too much for the current situation.