As part of my New Years resolution to get all of my current and planned PEPs actually written, accepted, and implemented for 2007, here is the stdlib reorg PEP. I have already checked it into svn but I have inlined it below for comments.
<br><br>The PEP is separated into two parts: removal and renaming. For removal I tried to keep the list to what I thought was very reasonable. If you look in Open Issues there are some possible removals that I am not so sure on so I would like feedback on those. I also have not touched the various Mac modules since I never use them and a bunch of them are undocumented. If Bob or Donald (or anyone else for that matter) could help me identify Mac modules that could go I would really appreciate it.
<br><br>For the renaming I listed all of the modules that either violate PEP 8, have both a Python and C implementation, or do not have a public API and thus should be hidden. The only possibly controversial renaming is of _winreg, dummy_threading, and repr. _winreg should be renamed since the higher-level version never materialized. I personally want dummy_threading renamed mockthreading since dummythreading just looks bad to me, it really is more of a mock implementation, and I wrote the module to begin with. =) and repr should be renamed reprlib (if it is not flagged for removal of its public interface) so it doesn't shadow a built-in.
<br><br>The Open Issues section also has a bunch of suggested groupings of modules that seemed extremely obvious to me or that other people pointed out. I know Guido didn't OK this but they were just staring me in the face so I just went ahead and listed what I thought could be reasonable packages to create or modules that really don't need to be separate. Since we are doing renames anyway I figured it wouldn't hurt to at least consider these. They are also not extensive in terms of putting the entire stdlib into a handful of packages, just stuff that is obviously the same theme (
e.g., collections) or are tightly coupled (e.g., urllib/urllib2). I leave it to Guido and the rest of you guys to either tell me to ignore these possibilities or to actually move forward with some of them.<br><br>Anyway, have at it!
<br><br>-Brett<br><br><br>-----------------------------------------<br><br>Abstract<br>========<br><br>Just like the language itself, Python's standard library (stdlib) has<br>grown over the years to be very rich. But over time some modules
<br>have lost their need to be included with Python. There has also been<br>an introduction of a naming convention for modules since Python's<br>inception that not all modules follow.<br><br>With Python 3.0 a chance to remove modules that do not have long term
<br>usefulness has presented itself. This chance also allows for the<br>renaming of modules so that they follow the Python style guide<br>[#pep-0008]_. This PEP lists modules that should not be included in<br>Python 3.0
and what modules need to be renamed.<br><br><br>Modules to Remove<br>=================<br><br>Guido pronounced that "silly old stuff" is to be deleted from the<br>stdlib for Py3K [#silly-old-stuff]_. This is open-ended on purpose.
<br>Each module to be removed needs to have a justification as to why it<br>should no longer be distributed with Python. This can range from the<br>module being deprecated in Python 2.x to being for a platform that is<br>
no longer widely used.<br><br>This section of the PEP lists the various modules to be removed. Each<br>subsection represents a different reason for modules to be<br>removed. Each module must have a specific justification on top of
<br>being listed in a specific subsection so as to make sure only modules<br>that truly deserve to be removed are in fact removed.<br><br>When a reason mentions how long it has been since a module has been<br>"uniquely edited", it is in reference to how long it has been since a
<br>checkin was done specifically for the module and not for a change that<br>applied universally across the entire stdlib. If an edit time is not<br>denoted as "unique" then it is the last time the file was edited,
<br>period.<br><br><br>Previously deprecated<br>---------------------<br><br>Modules in this section have been deprecated at some point in the<br>Python 2.x release series but are currently still distributed with<br>Python. Deprecation information is gathered either from PEP 4 or the
<br>Global Module Index [#pep-0004]_, [#module-index]_. Each module is<br>listed with the Python version that the deprecation started in.<br><br>* buildtools<br> 2.3<br>* cfmfile<br> 2.4<br>* gopherlib<br> 2.5<br>
* macfs<br> 2.3<br>* md5<br> 2.5<br>* mimetools<br> 2.3<br>* MimeWriter<br> 2.3<br>* mimefy<br> 2.3<br>* multifile<br> 2.3<br>* posixfile<br> 1.5<br>* rfc822<br> 2.3<br>* rgbimg<br> 2.5<br>* sha
<br> 2.5<br><br><br>Platform-specific with minimal use<br>----------------------------------<br><br>Python supports many platforms, some of which are not widely held.<br>And on some of these platforms there are modules that have limited use
<br>to people on those platforms. Because of their limited usefulness it<br>would be better to no longer burden the Python development team with<br>their maintenance.<br><br>* IRIX (which is no longer produced [#irix-retirement]_)
<br> + AL/al<br> - Provides sound support on Indy and Indigo workstations.<br> * Both workstations are no longer available.<br> - Code has not been uniquely edited in three years.<br> + cd<br>
- CD drive control for SGI systems.<br> * SGI no longer sells machines with IRIX on them.<br> - Code has not been uniquely edited in 14 years.<br> + DEVICE/GL/gl/cgen/cgensuport<br> - OpenGL access.
<br> - Has not been edited in at least eight years.<br> - Third-party libraries provide better support.<br> * PyOpenGL [#pyopengl]_<br> + FL/fl/flp<br> - Wrapper for the FORMS library [#irix-forms]_
<br> * FORMS has not been edited in 12 years.<br> - Library is not widely used.<br> * First eight hits on Google are for Python docs for fl.<br> + fm<br> - Wrapper to the IRIS Font Manager library.
<br> * Only available on SGI machines which no longer come with<br> IRIX.<br> + imgfile<br> - Wrapper for SGI libimage library for imglib image files<br> (``.rgb`` files).<br> - Python Imaging Library provdes read-only support [#pil]_.
<br> - Not uniquely edited in 13 years.<br> + jpeg<br> - Wrapper for JPEG (de)compressor.<br> - Code not uniquely edited in nine years.<br> - Third-party libraries provide better support.<br>
* Python Imaging Library [#pil]_<br> + sv<br> - Wrapper for Indigo video card.<br> * Harware is no longer manufactured.<br> - Undocumented.<br> - Code not uniquely edited in 13 years.
<br>* Solaris<br> + SUNAUDIODEV/sunaudiodev<br> - Access to the sound card on Sun machines.<br> - Code not uniquely edited in over eight years.<br>* Mac<br> + applesingle<br> - Undocumented.<br>
* AppleSingle is a binary file format for A/UX.<br> + A/UX no longer distributed.<br>* UNIX<br> + nis<br> - Wrapper for NIS.<br> * NIS has been replaced by LDAP, DNS, and Kerberos.
<br><br><br>Minimal usage<br>-------------<br><br>Some modules that are platform-independent have minimal usage. This<br>can be from how easy it is to implement the functionality from scratch<br>or because the audience for the code is small.
<br><br>* audiodev<br> + Undocumented.<br> + Not edited in five years.<br> + If removed sunaudio should go as well.<br> - Undocumented.<br> - Not edited in over seven years.<br>* fileinput<br> + Basic functionality handled by ``itertools.chain``.
<br> + Using ``enumerate`` (for the line number in the file),<br> ``itertools.repeat`` (for returning the filename with each<br> line), and ``zip`` (for connecting the ``enumerate`` object and<br> ``itertools.repeat`` object) provides 95% of other unique
<br> abilities of fileinput.<br>* imputil<br> + Undocumented.<br> + Never updated to support absolute imports.<br>* mutex<br> + Easy to implement using a semaphore and a queue.<br> + Cannot block on a lock attempt.
<br> + Not uniquely edited since its addition 15 years ago.<br>* repr<br> + Controls output of the repr of objects.<br> - String slicing and string interpolation can do similar work.<br> + Used by pdb, but do not need to expose API.
<br>* symtable/_symtable<br> + Undocumented.<br>* telnetlib<br> + Telnet is not used very much anymore.<br> - Telnet is unsafe.<br> - Most people use SSH instead.<br>* toaiff<br> + Undocumented.<br>
+ Requires ``sox`` library to be installed on the system.<br>* user<br> + Easily handled by allowing the application specify its own<br> module name, check for existence, and import if found.<br>* new<br> + Just a rebinding of names from the 'types' module.
<br> + Can also call ``type`` built-in to get most types easily.<br>* pure<br> + Written before Pure Atria was bought by Rational which was then<br> bought by IBM (in other words, very old).<br><br>Obsolete<br>
--------<br><br>Becoming obsolete signifies that either another module in the stdlib<br>or a widely distributed third-party library provides a better solution<br>for what the module is meant for.<br><br>* base64/quopri/uu
<br> + Support exists in the codecs module.<br> + If removed (along with binhex), also remove binascii.<br> - C implementation of base64, binhex, and uu modules.<br>* asynchat/asyncore<br> + Third-party libraries provide better solutions.
<br> - twisted [#twisted]_<br> + Deprecation previously supported [#py-dev-summary-2004-11-01]_<br>* Bastion/rexec<br> + Restricted execution / security.<br> + Turned off in Python 2.3.<br> + Modules deemed unsafe.
<br>* dl<br> + ctypes provides better support for same functionality.<br>* fpformat<br> + All functionality is supported by string interpolation.<br>* getopt<br> + optparse provides better functionality.<br>* ihooks
<br> + Documented except for saying that module might be obsolete.<br> + For use with rexec which has been turned off since Python 2.3.<br>* imageop<br> + Better support by third-party libraries.<br> - Python Imaging Library [#pil]_.
<br>* linuxaudiodev<br> + Replaced by ossaudiodev.<br>* stat<br> + ``os.stat`` now returns a tuple with attributes.<br>* statvfs<br> + ``os.statvfs`` now returns a tuple with attributes.<br>* strop<br> + Implements functions used by 'string' module that have now
<br> become methods on the str type.<br>* thread<br> + People should use 'threading' instead.<br> - Rename 'thread' to _thread.<br> - Deprecate dummy_thread.<br> * Rename to _mockthread.
<br> * Change of name better reflects modules purpose.<br> + Guido has previously supported the deprecation<br> [#thread-deprecation]_.<br>* timing<br> + Use timeit or time.<br> + Documentation says the module is obsolete [#timing-module]_.
<br><br><br>Modules to Rename<br>=================<br><br>Along with the stdlib gaining some modules that are no longer<br>relevant, there is also the issue of naming. Many modules existed in<br>the stdlib before PEP 8 came into existence [#pep-0008]_. This has
<br>led to some naming inconsistencies that should be addressed.<br><br>Any module that has been suggested for removal and does not meet the<br>required naming scheme is *not* listed below.<br><br><br>PEP 8 violations<br>
----------------<br><br>PEP 8 specifies that modules "should have short, lowercase names,<br>without underscores" [#pep-0008]_. There is no mention, though, if<br>this rule extends to modules contained within a package. The
<br>assumption is that underscores are acceptable in module names when<br>they are contained within a package but that any uppercase letters is<br>not.<br><br>* _winreg<br> winreg (rename also because module has a public interface).
<br>* autoGIL<br> autogil<br>* BaseHTTPServer<br> basehttpserver<br>* Carbon<br> carbon<br>* CGIHTTPServer<br> cgihttpserver<br>* ColorPicker<br> colorpicker<br>* ConfigParser<br> configparser<br>* Cookie
<br> cookie<br>* copy_reg<br> copyreg<br>* cProfile<br>* DocXMLRPCServer<br> docxmlrpcserver<br>* dummy_threading<br> mockthreading (rename because "mock" makes more sense than<br> "dummy" and rename already required).
<br>* EasyDialogs<br> easydialogs<br>* FrameWork<br> framework<br>* HTMLParser<br> htmlparser<br>* MacOS<br> macos<br>* MiniAEFrame<br> miniaeframe<br>* Nav<br> nav<br>* PixMapWrapper<br> pixmapwrapper
<br>* py_compile<br> pycompile<br>* Queue<br> queue<br>* repr<br> reprlib (rename because module name shadows a built-in).<br>* ScrolledText<br> scrolledtext<br>* SimpleHTTPServer<br> simplehttpserver<br>* SimpleXMLRPCServer
<br> simplexmlrpcserver<br>* SocketServer<br> socketserver<br>* StringIO<br> stringio<br>* Tix<br> tix<br>* Tkinter<br> tkinter<br>* UserDict<br> userdict<br>* UserList<br> userlist<br>* UserString<br>
userstring<br>* W<br> w<br><br><br>Merging C and Python implementations of the same interface<br>----------------------------------------------------------<br><br>Several interfaces have both a Python and C implementation. While it
<br>is great to have a C implementation for speed with a Python<br>implementation as fallback, there is no need to expose the two<br>implementations independently in the stdlib. For Python 3.0 all<br>interfaces with two implementations will be merged into a single
<br>public interface.<br><br>The C module is to be given a leading underscore to delineate the fact<br>that it is not the reference implementation (the Python implementation<br>is). This means that any semantic difference between the C and Python
<br>versions must be dealt with before Python 3.0 or else the C<br>implementation will be removed until it can be fixed.<br><br>One interface that is not listed below is xml.etree.ElementTree. This<br>is an externally maintained module and thus is not under the direct
<br>control of the Python development team for renaming. See `Open<br>Issues`_ for a discussion on this.<br><br>* pickle/cPickle<br> + Rename cPickle to _pickle.<br> + Semantic completeness of C implementation *not* verified.
<br>* profile/cProfile<br> + Rename cProfile to profile.<br> + Semantic completeness of C implementation *not* verified.<br>* StringIO/cStringIO<br> + Rename StringIO to stringio.<br> + Rename cStringIO to _stringio.
<br> + Semantic completeness of C implementation *not* verified.<br><br><br>No public, documented interface<br>-------------------------------<br><br>There are several modules in the stdlib that have no defined public<br>
interface. These modules exist as support code for other modules that<br>are exposed. Because they are not meant to be used directly they<br>should be renamed to reflect this fact.<br><br>* bdb<br> _bdb<br>* markupbase
<br> _markupbase<br>* opcode<br> _opcode<br>* dummythread<br> _mockthread (assuming the deprecation of 'thread' occurs).<br><br><br>Transition Plan<br>===============<br><br>For modules to be removed<br>-------------------------
<br><br>A PendingDeprecationWarning will be set in Python 2.6 for all modules<br>slated to be removed in Python 3.0. This will allow people to know<br>which modules will not exist but without being overly noisy since<br>
PendingDeprecationWarning is by default silenced.<br><br><br>For modules to be renamed<br>-------------------------<br><br>Modules will be renamed in Python 2.6 . The original names of the<br>modules will still work but will raise ImportWarning upon import. The
<br>refactoring tool for transitioning to Python 3.0 will refactor imports<br>that use the original names to the new names.<br><br><br>Open Issues<br>===========<br><br>Consolidate dependent modules together into a single module or package?
<br>-----------------------------------------------------------------------<br><br>The stdlib has several modules that have a level of dependency between<br>them (e.g., urllib and urllib2). Usually one is a low-level module
<br>that provides basic abilities with a corresponding higher-level API is<br>given in another module for simple use-cases. In Python 3.0 we could<br>group these dependent modules together into a single module or package
<br>to better reflect their relationship.<br><br>Keep in mind when looking at the groupings that deprecation or removal<br>is also a possibility if there is enough overlap or a module is<br>obsolete.<br><br>* Cookie/cookielib
<br>* urllib/urllib2<br> + urlparse?<br> + httplib?<br>* cgi/cgitb<br>* Tix/Tkinter<br>* getpass/pwd/spwd/grp<br>* mailbox/mhlib<br>* anydbm/whichdbm<br>* bsddb/dbhash<br>* pickle/pickletools<br>* HTMLParser/htmllib
<br>* ftplib/netrc<br>* parser/symbol<br><br><br>Consolidate certain modules with similar themes together in a package?<br>----------------------------------------------------------------------<br><br>Packages are often used to group together modules that have a similar
<br>theme but do not have any direct relationship or dependency upon each<br>other. For Python 3.0 obvious groupings could be done since renaming<br>of various modules is already occurring.<br><br>* collections<br> + heapq
<br> + Queue<br> + sets<br> + UserDist<br> + UserList<br> + What to do with UserString?<br> - Have a package for Python implementations of built-in types<br> instead of putting the User* modules into 'collections'?
<br>* mac<br> + Various Mac-specific modules.<br> + Same can be done for other platform-specific code.<br>* Profiling<br> + cProfile<br> + profile<br> + hotshot<br> + pstats<br>* email<br> + mailbox<br>
+ mhlib<br>* Databases<br> + anydbm<br> + dbhash<br> + dbm<br> + bsddb<br> + dumbdbm<br> + gdbm<br> + whichdb<br>* Audio<br> + aifc<br> + audioop<br> + chunk<br> + ossaudiodev<br> + sunau
<br> + wave<br> + winsound<br>* Servers<br> + BaseHTTPServer<br> + CGIHTTPServer<br> + DocXMLRPCServer<br> + SimpleHTTPServer<br> + SimpleXMLRPCServer<br> + SocketServer<br><br><br>Modules reliant on obsolete/rarely used file formats?
<br>-----------------------------------------------------<br><br>Several modules in the stdlib work on a specific file format. It is<br>possible some of these formats are no longer used and thus the stdlib<br>modules for them can go. Below is a list of some modules which rely
<br>on a file format that may be obsolete.<br><br>* aifc<br> AIFF and AIFF-C audio files. Appears to be only user of the cl<br> module (which is undocumented).<br>* audioop<br> Raw (8|16|32)-bit wide audio files (as generated by the al and
<br> sunaudiodev modules).<br>* binhex<br> binhex4 encoding.<br>* chunk<br> AIFF, AIFF-C, and RMFF audio files.<br>* sunau<br> Sun AU audio files [#sun-au]_.<br><br><br>Renaming of modules maintained outside of the stdlib
<br>----------------------------------------------------<br><br>xml.etree.ElementTree not only does not meet PEP 8 naming standards<br>but it also has an exposed C implementation [#pep-0008]_. It is an<br>externally maintained package, though [#pep-0360]_. A request will be
<br>made for the maintainer to change the name so that it matches PEP 8<br>and hides the C implementation.<br><br><br>References<br>==========<br><br>.. [#pep-0004] PEP 4: Deprecation of Standard Modules<br> (<a href="http://www.python.org/dev/pep-0004/">
http://www.python.org/dev/pep-0004/</a>)<br><br>.. [#pep-0008] PEP 8: Style Guide for Python Code<br> (<a href="http://www.python.org/dev/pep-0008/">http://www.python.org/dev/pep-0008/</a>)<br><br>.. [#pep-0360] PEP 360: Externally Maintained Packages
<br> (<a href="http://www.python.org/peps/pep-0360/">http://www.python.org/peps/pep-0360/</a>)<br><br>.. [#module-index] Python Documentation: Global Module Index<br> (<a href="http://docs.python.org/modindex.html">
http://docs.python.org/modindex.html</a>)<br><br>.. [#timing-module] Python Library Reference: Obsolete<br> (<a href="http://docs.python.org/lib/obsolete-modules.html">http://docs.python.org/lib/obsolete-modules.html</a>
)<br><br>.. [#silly-old-stuff] Python-Dev email: "Py3k release schedule worries"<br> (<a href="http://mail.python.org/pipermail/python-3000/2006-December/005130.html">http://mail.python.org/pipermail/python-3000/2006-December/005130.html
</a>)<br><br>.. [#thread-deprecation] Python-Dev email: Autoloading?<br> (<a href="http://mail.python.org/pipermail/python-dev/2005-October/057244.html">http://mail.python.org/pipermail/python-dev/2005-October/057244.html
</a>)<br><br>.. [#py-dev-summary-2004-11-01] Python-Dev Summary: 2004-11-01<br> (<a href="http://www.python.org/dev/summary/2004-11-01_2004-11-15/#id10">http://www.python.org/dev/summary/2004-11-01_2004-11-15/#id10</a>
)<br><br>.. [#pyopengl] PyOpenGL<br> (<a href="http://pyopengl.sourceforge.net/">http://pyopengl.sourceforge.net/</a>)<br><br>.. [#pil] Python Imaging Library (PIL)<br> (<a href="http://www.pythonware.com/products/pil/">
http://www.pythonware.com/products/pil/</a>)<br><br>.. [#twisted] Twisted<br> (<a href="http://twistedmatrix.com/trac/">http://twistedmatrix.com/trac/</a>)<br><br>.. [#irix-retirement] SGI Press Release:<br> End of General Availability for MIPS IRIX Products -- December 2006
<br> (<a href="http://www.sgi.com/support/mips_irix.html">http://www.sgi.com/support/mips_irix.html</a>)<br><br>.. [#irix-forms] FORMS Library by Mark Overmars<br> (<a href="ftp://ftp.cs.ruu.nl/pub/SGI/FORMS">ftp://ftp.cs.ruu.nl/pub/SGI/FORMS
</a>)<br><br>.. [#sun-au] Wikipedia: Au file format<br> (<a href="http://en.wikipedia.org/wiki/Au_file_format">http://en.wikipedia.org/wiki/Au_file_format</a>)<br>