[Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Raymond Hettinger]

On Oct 26, 2010, at 9:31 AM, Guido van Rossum wrote:
> I would like Gregor Lingl's approval of turning turtle.py into a package.  It might make some things harder for novices, e.g. trackebacks and just browsing the source code.
> 
> Also many people don't expect to find any code in a file named __init__.py (and most of the time I agree with this).  But the alternative isn't so great either, assuming we'll want strict backwards compatibility (I wouldn't want the instructions in Gregor's or anyone's book to start failing because of this).  You can't rename turtle to turtle/turtle.py either, because then there'd be horrible confusion between turtle/ and turtle.py.
> 
> IOW, yes, flat still seems better than nested here!


Thanks for saying this.  It might be a good time to also have a discussion about what may be an emerging trend of wanting to split standard library modules into packages.

I understand the urge to split longer modules into multiple source files but would like to mention some of the advantages of keeping modules in a single source file.

Taking the decimal module as an example:

*  Right now, I can pull up the source and entire history from subversion by knowing just the module name:   http://svn.python.org/view/python/branches/py3k/Lib/decimal.py?view=markup , and I can search the entire module with nothing more sophisticated than "find" in the web browser.   That no longer works with unittest.

* I can easily email the file to someone or copy it back to a previous version of Python because only one file is involved.  To look at the code, I don't have to guess how someone would have partitioned it into decimal.context.utils.threadlocal.py or somesuch.  I don't need a packaging tool to distribute a module (pyparsing and BeautifulSoup are two examples of tools that are easily managed by just being in one file).

* Some editors are set up to easily dance across multiple files and directories as if they were a single unit; however, many editors are not.  Most file viewers and editors work better with single files.    Take two operations in IDLE for example.  If I press Alt-M for open module, I can retrieve all of the source for decimal (but this won't work with the logging or unittest packages).  Once the source is visible, I press Alt-C to bring up a class browser for quickly reviewing the source (but this also won't work with packages).  Several Python editors have this capability (provided by the pyclbr module) but they are defeated by packaging.

* The Lib directory is more easily grepped when it is flat.  The more we nest the Lib directory, the harder it will be to search the sources (while excluding the test directory).

Packaging is not always wrong.  Maybe it was the right thing to do for unittest, maybe not.  I just wanted to bring up some of the advantages of having a single file for a library module.  It would not be a good thing if some of the newer committers embarked on taking modules over 2000 lines and split them into packages.   Right now, we have hundreds of source files, but it wouldn't take long for someone on a packaging mission to turn that into thousands.

If someone wants to reorganize code for clarity, I would prefer keeping it within one file, bringing related functions together and using comment lines to mark the major sections.  ISTM, this is cleaner than introducing a new directory and having multiple files that cross-import one another.


Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101026/648d16c3/attachment-0001.html>