[Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Michael Foord]

On 02/11/2010 23:00, Brett Cannon wrote:
> On Tue, Nov 2, 2010 at 15:47, Raymond Hettinger
> <raymond.hettinger at gmail.com>  wrote:
>> On Nov 1, 2010, at 7:35 PM, Brett Cannon wrote:
>>
>> I think the issue here is that the file structure of the code no
>> longer matches the public API documented by unittest. Personally I,
>> like most people it seems, prefer source files to be structured in a
>> way to match the public API. In the case of unittest Michael didn't.
>> He did ask python-dev if it was okay to do what he did, we all kept
>> quiet, and now we have realized that most of us prefer to have files
>> that mirror the API; lesson learned. But Python 2.7 shipped with this
>> file layout so we have to stick with it lest we break any imports out
>> there that use the package-like file structure Michael went with
>> (which we could actually document and use if we wanted now that
>> Michael has already broken things up). Reversing the trend by sticking
>> all the code into unittest/__init__.py and then sticking import shims
>> into the existing modules would be a stupid waste of time, especially
>> considering the head maintainer of the package likes it the way it is.
>>
>> I'm not sure I follow where we're stuck with the current package.
>> AFAICT, the module is still used with "import unittest".
> Yes, as far as you can tell, but who the hell knows what someone is
> doing with code you are *not* aware of. As I said, Python 2.7 shipped
> with the code structured like this, so it's possible someone is
> importing unittest.case.TestCase instead of unittest.TestCase.
>

It is also shipped in unittest (and unittest2py3k I might add) so that 
users of earlier versions of Python can use the new features seamlessly. 
(unittest2 will be in Django 1.3.)

Much better times to discuss this would be when it was proposed or when 
it was done, not months after it has been shipped in a production release.

> [snip...]
>> This is in contrast to the logging module which does have a
>> clean separation of components and where it isn't unusual
>> to import just part of the package.
>> What is it you're seeing as a risk that I'm not seeing?
> Broken imports between Python 2.7 code and any version of Python where
> unittest is re-merged back into a single module.
>

I *know* that some people are using the new package structure directly, 
because the topic has come up on the Testing in Python mailing list.

>> Are we permanently locked into the exact ten filenames
>> that are currently used:  utils, suite, loader, case, result, main, signals,
>> etc?
>> Is the file structure now frozen?
> Somewhat, yes. Screwing with unittest is always touchy as absolutely
> no one wants their tests to break, and that includes messing with
> imports. We could stick in import shims to shift everything into
> unittest/__init__.py, but the benefits you have outlined already don't
> strike me as not worth the hassle especially since you won't ever get
> your unittest.py format back.
Absolutely, that would be the worst of all possible worlds - a 
monolithic huge module but still embedded in a package.

People *are* using the existing package structure to import directly 
from (against my advice!) as this particular topic has been discussed on 
the Testing In Python mailing list.

Although Raymond has been vociferous in his detestation of this 
particular split he does not represent a "clear consensus" in favour of 
merging back. Both Fred Drake and Barrry Warsaw voiced their approval 
and on the "Clean up unittest API" issue both yourself (Brett) and 
Antoine have supported keeping the existing structure.

Alexander Belopolsky and Martin Loewis expressed difficulties with the 
new structure, but that was in response to the original email from 
Raymond that didn't seem (on my reading) to expressly suggest re-merging 
unittest back into a module but was instead seemed to be using it as an 
example.

I am aware of the costs of having a package rather than single (however 
large it may be) module, but to my mind the benefits to maintenance 
still outweigh these cost. I'm happy to again discuss these benefits at 
great length, but having had the same conversation in person with 
Raymond twice and at repeated most of the points (but by no means all) 
in this thread on the mailing list it really feels like going round in 
circles.

As the maintainer of unittest I'd like to say that in the absence of 
clear consensus that the merger should happen, or a fiat from the BDFL, 
the merger won't happen. I believe that this is standard Python 
development process.

All the best,

Michael


> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.