[Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Raymond Hettinger]

On Oct 26, 2010, at 8:37 PM, Barry Warsaw wrote:
> If done well, a split can help improve the readability and discoverability of
> the code.  

No doubt that is true.  The problem is that splitting can also impair discoverability.

When unittest was in one file, you knew the filename was unittest.py 
from just the module name.  Now, there are ten files and you have
no way to guess any of their names or how the author grouped
the functions and methods inside them.

Let's say that I'm mystified about the difference between
assertSameElements() and assertItemsEqual() and assertSetEqual().
So, I go to http://svn.python.org/view/python/branches/py3k/Lib/unittest/ .
Now, how do you know which file contains the source?  Formerly,
I could call-up the one source file and do a find.   Or, I could do 
OpenModule: unittest with IDLE and instantly see the source and
let the class browser analyze the structure of the file.  Now, that's
not possible either.  The source for those methods is now less discoverable.

In the unittest/case.py file, the safe_repr() function function is called over
40 times and it is not used in any other file.  So how do be benefit from
it being defined in the utils.py file?  ISTM, this saved nothing.  The case.py
file is over 1000 lines long and utils is 80.  How did we benefit from that split?  
For me, it makes it harder to read the code because I have to go looking
elsewhere for commonly called functions.

The unittest module grew from one file in Py2.6 to ten files and one directory 
with 2500 lines in Py2.7.  Was that a win?  I've spent time trying to
read through the changes and cannot follow it without having most of 
those ten files open in my editor.  For me, it's a PITA to read the code.
It doesn't help that the file split blew away the svn blame history, so I
have a harder time being about to tell what changed.

All that being said, I started this thread with:  "Packaging is not 
always wrong.  Maybe it was the right thing to do for unittest, maybe not".

The goal of the post was just to raise awareness that splitting modules
is not a straight win.  There are costs to discoverability and searchability.
It makes pyclbr less useful.  It makes svn viewer less useful.  And it loses
history.  

The split of unittest is a done deal.   Who knows, it may have been a net win.
Am just hoping that we all understand that it was not cost free.   My hope is 
that the splitting modules unnecessarily does not become a trend.
Packages should be the last tool we reach for, not the first.
In many cases, flat is better than nested.


Raymond

P.S.  I liked your qualifier, "if done well".  There's a lot in those three words.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101027/6034bad6/attachment.html>