- Eliminate DU1's "Swiss Army" tendencies. Separate the build, install
and register procedures for higher cohesion and lower coupling. This will make it much easier to refactor design of each in turn.
I'd like to second that.
Example: compilation of extension modules.
Scons is aiming at providing an abstraction layer for portable compilation. DU2 should at least allow to just delegate compilation of extension modules to scons. (and as I said previously, I think anything that doesn't allow to wrap traditional build systems based on 'make' and possibly the autotools is 'not good enough' in as a general solution).
- Every Python module should be distributed, managed and used as a
single folder containing ALL resources relating to that module: sub-modules, extensions, documentation (bundled, generated, etc.), tests, examples, etc. (Note: this can be done without affecting backwards-compatibility, which is important.) Similar idea to OS X's package scheme, where all resources for [e.g.] an application are bundled in a single folder, but less formal (no need to hide package contents from user).
are you really talking about 'package' here when you say 'module' ? I don't think that mandating modules to be self contained is a good idea. Often modules only 'make sense' in the context of the package that contains them. Also, are you talking about how to distribute packages, or about the layout of the installed files ? I don't think DU2 should mandate any particular layout for the target installation. It may well suggest layout of the files inside the (not yet installed) package.
- Question: is there any reason why modules should not be installable
via simple drag-n-drop (GUI) or mv (CLI)? A standard policy of "the package IS the module" (see above) would allow a good chunk of both existing and proposed DU "features" to be gotten rid of completely without any loss of "functionality", greatly simplifying both build and install procedures.
Again, I don't think it is DU2's role to impose anything concerning the target layout. This is often platform dependent anyways.
--Replace current system where user must explicitly state what they want included with one where user need only state what they want excluded.
That depends on how much control users want over the process. I believe both are equally valid, and should be supported (similar in spirit to the MANIFEST.in syntax 'include' and 'exclude')
-- In particular, removing most DU involvment from build procedures would allow developers to use their own development/build systems much more easily.
yes !! Though that's more easily said than done: a minimum of collaboration between the two is required, at least the adherence to some conventions.
- Installation and compilation should be separate procedures. Python
already compiles .py files to .pyc on demand; is there any reason why .c/.so files couldn't be treated the same? Have a standard 'src' folder containing source files, and have Python's module mechanism look in/for that as part of its search operation when looking for a missing module; c.f. Python's automatic rebuilding of .pyc files from .py files when former isn't found. (Q. How would this folder's contents need to be represented to Python?)
As a starting point, the whole 'build_ext' mechanism should be re-evaluated. The current 'Extension' mechanism is by far not abstract enough. Either the build_ext or the Extension class should be made polymorphic to wrap any external build system that could be used (make, scons, jam, ...)
- What else may setup.py scripts do apart from install modules (2) and
build extensions (3)?
* building documentation (that, too, is highly domain specific. From Latex over Docbook to doxygen...)
* running unit tests
- Remove metadata from setup.py and modules.
I don't quite agree in general. What metadata are we talking about anyways ? There's metadata that is to be provided to the packager backends, i.e. a package description of some sort. Some of these can be generated automatically (such as MANIFEST.in -> MANIFEST, build / host platform, etc.), others have to be explicitely provided (maintainer address, package description).
Having 'all metadata' lumped together brings us back to the 'swiss army knife' syndrome.
- Improve version control. Junk current "operators" scheme (=, <, >, >=,
<=) as both unnecessarily complex and inadequate (i.e. stating module X requires module Y (>= 1.0) is useless in practice as it's impossible to predict _future_ compatibility). Metadata should support 'Backwards Compatibility' (optional) value indicating earliest version of the module that current version is backwards-compatible with. Dependencies list should declare name and version of each required package (specifically, the version used as package was developed and released).
Good idea, though this issue highly depends on the packager backend used.
- Make it easier to have multiple installed versions of a module.
That, too, isn't really an DU2 issue, or is it ?
- Reject PEP 262 (installed packages database). Complex, fragile,
duplication of information, single point of failure reminiscent of Windows Registry. Exploit the filesystem instead - any info a separate db system would provide should already be available from each module's metadata.
I don't quite agree. I couldn't live without rpm these days.