On Apr 11, 2013, at 02:22 PM, Elias Assarsson wrote:
- MM2's configuration file is a Python file which really must be imported in order to get a valid set of values. MM3's configuration file is a stack of .ini-style files.
I am trying to find and understand the configuration files so that I know what that that needs to be migrated and to what form. Is the MM2 configuration you refer to mainly Mailman/mm_cfg.py and MM3 configuration files src/mailman/config/* and /src/mailman/<listname>/*?
Yes, in a deployed Mailman 2, mm_cfg.py will contain the system configuration
settings. These override the settings from Defaults.py so a good way to
explore is to use bin/withlist and import mm_cfg at the Python prompt.
In MM3, we use lazr.config, which is essentially a configparser type package that allows for stacks of configurations, with pushing and popping values on this stack.
http://pythonhosted.org/lazr.config/
The src/mailman/config/schema.cfg file is at the bottom of the stack and defines the schema, obviously :). From there, src/mailman/config/mailman.cfg essentially instantiates this schema, providing all the defaults. In the testing environment, src/mailman/testing/testing.cfg gets pushed on top of that (and many tests push and pop micro-overrides). In a deployed system, the site's mailman.cfg is on the top of the stack and so can override anything. There are various places this mailman.cfg file is searched; see src/mailman/core/initialize.py for all the gory details.
List-specific configurations live in the various config.db files for MM2. These are pickles. In MM3, everything's in the database.
Given the two points above it seems that handling the migration from within Python is the best choice (rather than using Augeas which is C based or Config::Model which is Perl based).
I think so. Pickles in particular are Python-specific. You could generate an intermediate format, but I'm not sure it's worth it.
Obviously one wants to use whatever feature of Python that can ease the process. You seem to be using configparser in MM3. I dunno if there are any other Python tool one should look at in helping migration. Maybe one should investigate this further.
- What about importing archives?
I have tried to find information on how archives are stored in MM2 and MM3 but failed to find any. What is a good source to learn about this?
Importing archives will either be trivial or impossible :). At the lowest level, parsing the MM2 .mbox file and generating maildirs would work, but there are existing tools to do that, so probably little for GSoC to worry about. Much more interesting would be to parse the Pipermail database files and try to build a reverse mapping from URL to Message-ID so that these could be preserved when the archive is regenerated for HyperKitty or whatever.
There is some moderate beginnings in the 3.0 tree, but none of it is functional in all likelihood. Take a look at src/mailman/bin/export.py and src/mailman/commands/cli_import.py.
So those are the files that are supposed to handle migration for which the project is to make them handle a complete migration from MM2 to MM3?
Well, I'd say they were more some unfinished work in that direction.
-Barry