[Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Michael Foord]

On 02/11/2010 02:35, Brett Cannon wrote:
> On Wed, Oct 27, 2010 at 03:42, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>> On Tue, 26 Oct 2010 22:06:37 -0400
>> Alexander Belopolsky<alexander.belopolsky at gmail.com>  wrote:
>>> While I appreciate your and Michael's eloquence, I don't really see
>>> why five 400-line modules are necessarily easier to maintain than one
>>> 2000-line module.  Splitting code into modules is certainly a good
>>> thing when the resulting modules can be used independently.  This
>>> helps users write leaner programs, reduces mental footprint of
>>> individual modules, etc, etc.   The split unittest module does not
>>> bring any such benefits.  It still presents a single "big-ball-of-mud"
>>> namespace, only rather than implemented in a single file, it is now
>>> swept in from eight different files.
>> Are you saying that it has become a pile of medium-sized balls of mud?
>> I would like to say thanks for the mud, Michael! It's high quality mud
>> for sure.
> I realize I am a little late in this reply but issue 10273 linked to
> this and so now I am actually bothering to read this thread since it
> felt like bikeshedding when the thread began.
>
> I think the issue here is that the file structure of the code no
> longer matches the public API documented by unittest. Personally I,
> like most people it seems, prefer source files to be structured in a
> way to match the public API. In the case of unittest Michael didn't.

Well the structure *does* match the API (which is primarily why I 
disagree with Raymond that this is a 'bad split').

How could we have split the module into a package in a way that matched 
the API, whilst still retaining backwards compatibility with the old 
API? We had no choice but to export the public names at the top level.

> He did ask python-dev if it was okay to do what he did, we all kept
> quiet, and now we have realized that most of us prefer to have files

Most of us? Raymond, Alexander and Martin are the only ones I *recall* 
complaining about the split specifically in this thread and not all of 
those were on the grounds you mention. Several people supported the 
split. Guido didn't object to it on these grounds and Antoine noted that 
finding core classes was generally straightforward.

> [snip...]
> So basically it seems like we have learned a lesson: we prefer to have
> our code structured in files that match the public API.

When designing packages from the ground up that is a sensible rule of 
thumb to follow, but usually follows naturally from good design. This 
doesn't necessarily mean that all the sub-modules will export public 
APIs for consumers, so this is where I disagree with this. Python's 
package system is a very useful way of providing internal structure for 
projects. That doesn't mean that this structure must always be exposed 
publicly.

It should be as easy to navigate as possible (and there is plenty about 
the old unittest.py module that wasn't easy to navigate I can assure 
you), but I *don't* think that the new package fails in a substantially 
greater way on that score.

As Guido points out, this may depend a lot on which tools you use. I 
could write more about the role and value of packages, I guess I'll save 
it for a blog post.

All the best,

Michael Foord

> I think that
> is a legitimate design rule for the stdlib to follow from now on, but
> in the case of unittest it's too late to change it back (and it's a
> minor price to pay to learn this lesson and to have Michael
> maintaining unittest like he has been, plus we could consider using
> the new structure so that the public API matches the file structure
> when the need arises).
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.