Re: [Doc-SIG] Undocumented modules
Christopher G. Petrilli writes:
Can you split these into the following categories:
Ok, I've pushed my logic into a somewhat-real script (care to guess what language I used? ;), and can more easily update the list. It also now (optionally) considers built-in modules. My current interpreter only showed pcre and strop as undocumented modules (there's also an option to ignore modules listed in a file), so that's probably not very interesting. Obsolete modules (lib-old/, lib-stdwin/) are ignored, as are the dos-8x3/ and test/ directories. "test" still needs to be documented since it's a package in it's own right. I'll attach the "categorized" version of the output below (it's organized by subdir within Lib/, and the various plat-* directories are collapsed to a "virtual" directory named "plat-*"). The script will appear in Doc/tools/. -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 <standard> CGIHTTPServer SimpleHTTPServer asynchat asyncore audiodev bdb cmp cmpcache codeop colorsys dbhash dircache dircmp dospath dump find fpformat grep htmlentitydefs ihooks knee linecache macpath macurl2path mhlib mutex ntpath nturl2path packmail pipes posixpath pty reconvert regex_syntax rlcompleter sched statcache statvfs sunau sunaudio telnetlib threading_api toaiff tty turtle tzparse util wave whatsound lib-tk Canvas Dialog FileDialog FixTk ScrolledText SimpleDialog Tkconstants Tkdnd Tkinter tkColorChooser tkCommonDialog tkFileDialog tkFont tkMessageBox tkSimpleDialog plat-* AWARE CD CL CL_old ERRNO FCNTL FILE GET GLWS IN IOCTL SOCKET STROPTS SUNAUDIODEV SV WAIT cddb cdplayer panel panelparser readcd torgb
I've kinda gone through this and figured out what probably SHOULD be documented, what shouldn't be documented (i.e. it's not exposed to the outside world) and what might just be totally deprecated. <standard> CGIHTTPServer DOCUMENT SimpleHTTPServer DOCUMENT asynchat DOCUMENT (I'm writing this right now) asyncore DOCUMENT (same) audiodev DOCUMENT (platform specific SGI/Sun) bdb DOCUMENT (though not overly exposed) cmp DEPRECATE (seems to be duplicate of cmpcache) cmpcache DOCUMENT codeop DOCUMENT colorsys DEPRECATE (very incomplete and possible inacurate, use PIL) dbhash DOCUMENT dircache DOCUMENT dircmp DOCUMENT dospath DOCUMENT (platform specific Wintel) dump DEPRECATE (seems to be half-implemented) find DEPRECATE (move to scripts?) fpformat DOCUMENT grep DEPRECATE (move to scripts?) htmlentitydefs DOCUMENT ihooks DOCUMENT (see comment below) knee DOCUMENT (not really usable, example) linecache DOCUMENT (potentially useful with fileinput) macpath DOCUMENT (platform specific Mac) macurl2path DOCUMENT (platform specific Mac) mhlib DOCUMENT (has anyone tested this?) mutex DOCUMENT ntpath DOCUMENT (platform specific NT) nturl2path DOCUMENT (platform specific NT) packmail DEPRECATE (move to scripts?) pipes DOCUMENT posixpath DOCUMENT (platform specific POSIX) pty DEPRECATE (VERY VERY platform specific) reconvert DEPRECATE (incomplete, move to scripts?) regex_syntax DEPRECATE (regex no longer preffered) rlcompleter INTERNAL (don't document) sched DOCUMENT statcache DOCUMENT (see notes about cache things) statvfs DOCUMENT (used with os.statvfs()) sunau DOCUMENT (platform specific Sun) sunaudio DOCUMENT (platform specific Sun) telnetlib DOCUMENT threading_api WHAT IS THIS?! (looks like docs for threading.py) toaiff DEPRECATE (move to scripts?) tty DEPRECATE (move to termios?) turtle DOCUMENT (Tkinter though) tzparse DOCUMENT util DEPRECATE (hold over from 1.4) wave DEPRECATE (not sure honestly! :-) whatsound DEPRECATE (same as sndhdr) ihooks - should this be in with the new module, it seems to be of the same class of "shoot yourself in the foot" modules :-) Perhaps we should have a "WARNING" label for modules that are especially dangerous to use? caching things - has anyone actually tested these for performance? A quick check on my FreeBSD 3.0 box says they're slower than not caching because of (I assume) file system caching effeciency... Maybe that was a fluke, at least with statcache ;-) lib-tk is a royal pain :-) Maybe Frederick will finish soon! :-) plat-* AWARE DOCUMENT? (is this viable on new IRIX?) CD INTERNAL CL DEPRECATE (seems ot be deprecated) CL_old DEPRECATE (same) ERRNO DOCUMENT (I'd just reference the errno.h file) FCNTL DOCUMENT (same here, maybe some BASICS that are common on all platforms?) FILE DOCUMENT (reference external source) GET DOCUMENT (woah, major platform specific IRIX) GLWS DOCUMENT (more major IRIX stuff) IN DOCUMENT (reference) IOCTL DOCUMENT (reference) SOCKET DOCUMENT (reference) STROPTS INTERNAL (dear god, who uses THIS?!?!?!) SUNAUDIODEV DOCUEMNT (platform specific) SV DOCUMENT (more major platform IRIX stuff) WAIT DOCUMENT (very useful stuff here) cddb DOCUMENT (note overlap with Internet CDDB) cdplayer DOCUMENT (platform specific IRIX) panel DOCUMENT (same) panelparser DOCUMENT (same) readcd DOCUMENT (same) torgb DEPRECATE (use PIL I think) As you've no doubt noticed, not much I don't say to document, BUT... I'm worried about all this HYPER specific IRIX stuff... perhaps it should be moved into it's own seperate documentation area, or even a seperate distribution of sorts (a'la Mac)... there's a ton of junk that just isn't useful for 99% of the users. Just ideas... shoot at me at will :-) Chris -- | Christopher Petrilli ``Television is bubble-gum for | petrilli@amber.org the mind.''-Frank Lloyd Wright
On Tue, Feb 23 1999 "Christopher G. Petrilli" wrote:
I've kinda gone through this and figured out what probably SHOULD be documented, what shouldn't be documented (i.e. it's not exposed to the outside world) and what might just be totally deprecated.
plat-* AWARE DOCUMENT? (is this viable on new IRIX?) CD INTERNAL CL DEPRECATE (seems ot be deprecated) CL_old DEPRECATE (same)
These 4 are deprecated. Their functionality has been taken over by the cd and cl modules. AWARE is a module of constants for the cl module, but I'm not sure whether you can actually use the functionality that these constants represent. If I remember right, in some release of SGI there was support for this, but you needed a license (which we never got). I doubt that anybody uses AWARE, so it can be removed, as far as I am concerned.
SV DOCUMENT (more major platform IRIX stuff)
Should be incorporated in sv module.
cddb DOCUMENT (note overlap with Internet CDDB) cdplayer DOCUMENT (platform specific IRIX) panel DOCUMENT (same) panelparser DOCUMENT (same) readcd DOCUMENT (same)
As you've no doubt noticed, not much I don't say to document, BUT... I'm worried about all this HYPER specific IRIX stuff... perhaps it should be moved into it's own seperate documentation area, or even a seperate distribution of sorts (a'la Mac)... there's a ton of junk that just isn't useful for 99% of the users.
Hey, but it is useful to me! (But then I wrote some of it.) -- Sjoerd Mullender <Sjoerd.Mullender@cwi.nl> <URL:http://www.cwi.nl/~sjoerd/>
cmp DEPRECATE (seems to be duplicate of cmpcache) cmpcache DOCUMENT
No, cmpcache uses statcache.stat() while cmp uses os.stat(). But both are borderline -- I wrote them when writing a visual diff for stdwin 7 or 8 years ago and haven't touched or used them since...
codeop DOCUMENT colorsys DEPRECATE (very incomplete and possible inacurate, use PIL)
Hm... This incorporates quite a bit of arcane knowledge that was hard to come by when we wrote it. I'm not sure where it is incomplete or inaccurate.
fpformat DOCUMENT
Deprecate it; you can now do this using the % operator on strings.
linecache DOCUMENT (potentially useful with fileinput)
(Also used intensely by anything printing tracebacks.)
mhlib DOCUMENT (has anyone tested this?)
Yes, I use it a lot, and just today I received some patches from a mostly happy user in the newsgroup.
mutex DOCUMENT
Hmm, this is very shady and seems to predate threads!
pty DEPRECATE (VERY VERY platform specific)
But also very useful when it works. I'd like to keep it.
rlcompleter INTERNAL (don't document)
Document it; it's actually useful. One needs to know to import this in $PYTHONSTASRTUP, and one could subclass it to use different completion rules.
sched DOCUMENT
Like mutex -- not clear how useful this is.
threading_api WHAT IS THIS?! (looks like docs for threading.py)
Exactly. Should be deleted now that this is in the library manual.
tty DEPRECATE (move to termios?)
Document -- it's a higher-level thing that could be useful.
tzparse DOCUMENT
And someone should finish it!
wave DEPRECATE (not sure honestly! :-)
Document! It reads/writes .wav sound files.
ihooks - should this be in with the new module, it seems to be of the same class of "shoot yourself in the foot" modules :-) Perhaps we should have a "WARNING" label for modules that are especially dangerous to use?
Although less so -- "new" can easily cause core dumps, ihooks can just screw cause tracebacks...
caching things - has anyone actually tested these for performance? A quick check on my FreeBSD 3.0 box says they're slower than not caching because of (I assume) file system caching effeciency... Maybe that was a fluke, at least with statcache ;-)
Exactly how did you test it? This is needed especially when using NFS, since stat over NFS is slow and synchronous.
ERRNO DOCUMENT (I'd just reference the errno.h file)
Deprecate; we now have this and more in errno.
As you've no doubt noticed, not much I don't say to document, BUT... I'm worried about all this HYPER specific IRIX stuff... perhaps it should be moved into it's own seperate documentation area, or even a seperate distribution of sorts (a'la Mac)... there's a ton of junk that just isn't useful for 99% of the users.
Agreed. There is already an SGI specific chapter that could easily be lifeted out. Some of it is also very old and of questionable utility in modern SGI systems (even if SGI survives :-). --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, Feb 24, 1999 at 12:31:23PM -0500, Guido van Rossum wrote:
colorsys DEPRECATE (very incomplete and possible inacurate, use PIL)
Hm... This incorporates quite a bit of arcane knowledge that was hard to come by when we wrote it. I'm not sure where it is incomplete or inaccurate.
Um, missing color systems like CMYK (which of course are nearly impossible to do), but also is part of the concept of imaging a'la PIL it seems, so awefully incomplete without the ability to do transformations in the color system (histograms, etc).
mhlib DOCUMENT (has anyone tested this?)
Yes, I use it a lot, and just today I received some patches from a mostly happy user in the newsgroup.
I noticed this myself, such timing :-)
mutex DOCUMENT
Hmm, this is very shady and seems to predate threads!
It has some interesting capabilities though probably not thread-safe... but threads don't work on every system unfortunately.
sched DOCUMENT
Like mutex -- not clear how useful this is.
Same as mutex, might be useful for non-threading systems to use, and might also be useful to throw into an individual thread to deal with "future events".
tty DEPRECATE (move to termios?)
Document -- it's a higher-level thing that could be useful.
But should it go into termios?
tzparse DOCUMENT
And someone should finish it!
Just what I wnated to work on, timezone parsing *gack*
caching things - has anyone actually tested these for performance? A quick check on my FreeBSD 3.0 box says they're slower than not caching because of (I assume) file system caching effeciency... Maybe that was a fluke, at least with statcache ;-)
Exactly how did you test it? This is needed especially when using NFS, since stat over NFS is slow and synchronous.
No I didn't do it over NFS, since I don't have NFS. Maybe the documentation should mention that it's advantages are over NFS, not on direct file systems?
As you've no doubt noticed, not much I don't say to document, BUT... I'm worried about all this HYPER specific IRIX stuff... perhaps it should be moved into it's own seperate documentation area, or even a seperate distribution of sorts (a'la Mac)... there's a ton of junk that just isn't useful for 99% of the users.
Agreed. There is already an SGI specific chapter that could easily be lifeted out. Some of it is also very old and of questionable utility in modern SGI systems (even if SGI survives :-).
Oh SGI will survive, probably as a Microsoft vendor unfortunately. Sad because IRIX was the best of the Unices in performance and amazing feature sets (I still use the guaranteed filesystem bandwidth feature). Chris -- | Christopher Petrilli ``Television is bubble-gum for | petrilli@amber.org the mind.''-Frank Lloyd Wright
On Wed, 24 Feb 1999, Christopher G. Petrilli wrote:
On Wed, Feb 24, 1999 at 12:31:23PM -0500, Guido van Rossum wrote:
colorsys DEPRECATE (very incomplete and possible inacurate, use PIL)
Hm... This incorporates quite a bit of arcane knowledge that was hard to come by when we wrote it. I'm not sure where it is incomplete or inaccurate.
Um, missing color systems like CMYK (which of course are nearly impossible to do), but also is part of the concept of imaging a'la PIL it seems, so awefully incomplete without the ability to do transformations in the color system (histograms, etc).
It's still useful (one may want to do colorspace manipulations just within Tkinter, for example), it's small, and it's not that hard to document. Unless proven inaccurate, I vote to keep it. If I have to document it to keep it, I'm willing to do that. --david
David Ascher writes:
document. Unless proven inaccurate, I vote to keep it. If I have to document it to keep it, I'm willing to do that.
I will remember this. Just ask Jim Fulton! ;-) -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191
Fred> Ok, I've pushed my logic into a somewhat-real script ... ... Fred> <standard> CGIHTTPServer ... Fred> posixpath ... Fred, I think there's a logic bug. The posixpath module is documented at http://www.python.org/doc/lib/module-posixpath.html Also, some minor modules that only define manifest constants (SOCKET, ERRNO, regex_syntax all come to mind) probably aren't worth mentioning outside the documentation of other modules (socket, errno, regex, respectively). ERRNO, in particular, seems to be sucked into errno in its entirety, so probably shouldn't be treated as a user-visible module: >>> id(errno.E2BIG) 134968712 >>> id(ERRNO.E2BIG) 134968712 Skip
skip@mojam.com writes:
I think there's a logic bug. The posixpath module is documented at
Skip, No, that's left over from the previous version of the documentation; I'll remove it... done. The index of documented modules doesn't list posixpath, and probably shouldn't; it is listed as an undocumented implementation detail. See: http://www.python.org/doc/lib/modindex.html http://www.python.org/doc/lib/node300.html (On the other hand, I just used it explicitly for something where that made sense. Perhaps each of the *path modules should be documented, but only say that it's an implementation of os.path for the specific OS; there are times when this is exactly what's needed.)
Also, some minor modules that only define manifest constants (SOCKET, ERRNO, regex_syntax all come to mind) probably aren't worth mentioning outside the documentation of other modules (socket, errno, regex, respectively). ERRNO, in particular, seems to be sucked into errno in its entirety, so probably
ERRNO is not sucked into errno. The identity match for small integers is due to the small integers cache maintained internally to Objects/intobject.c as a performance improvement. I'd like to see modules that consist only of constants for use with other modules folded into those modules and removed. If the module is something that the user is expected to import, it should be documented, even if trivially: http://www.python.org/doc/lib/module-DEVICE.html -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191
"Fred" == Fred L Drake <fdrake@cnri.reston.va.us> writes:
Fred> skip@mojam.com writes: >> I think there's a logic bug. The posixpath module is documented at >> >> http://www.python.org/doc/lib/module-posixpath.html Fred> No, that's left over from the previous version of the Fred> documentation; I'll remove it... done. What's wrong with the old (1.5.1?) version of the posixpath documentation? Did it change so much between 1.5.1 and 1.5.2 that it has to be rewritten? I must be missing something pretty basic. Fred> ERRNO is not sucked into errno. The identity match for small Fred> integers is due to the small integers cache maintained internally to Fred> Objects/intobject.c as a performance improvement. Yup, my bad. Okay, so errno is documented, but ERRNO isn't, yet they seem to contain much the same contents. What's the difference? Fred> I'd like to see modules that consist only of constants for use Fred> with other modules folded into those modules and removed. If the Fred> module is something that the user is expected to import, it should Fred> be documented, even if trivially: Good point. That would work for SOCKET, but the errno/ERRNO overlap still confuses me. Skip
skip@mojam.com writes:
What's wrong with the old (1.5.1?) version of the posixpath documentation? Did it change so much between 1.5.1 and 1.5.2 that it has to be rewritten? I must be missing something pretty basic.
'fraid so: It's all listed under posixpath. ;-) The new docs are for os.path (it's still in the file .../libposixpath.tex). This makes more sense for more people, and better encourages the cross- platform usage.
Yup, my bad. Okay, so errno is documented, but ERRNO isn't, yet they seem to contain much the same contents. What's the difference?
I presume ERRNO, SOCKET, and many other ALL_UPPERCASE_MODULES predate the approach of incorporating constants directly in the module with which they're used. Several of these files appear to be generated by h2py, and they're located in the plat-*/ directories. These should probably be removed whenever possible. I don't see any imports of ERRNO, FILE, GET, GLWS, IN, IOCTL, SOCKET, SV, or WAIT anywhere in the standard library. I *do* see AL, CD, DEVICE, FCNTL, FL, and GL used. CL is only used by AWARE, which Sjoerd said should be tossed. CD and FCNTL are not documented, but maybe should be. -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191
Fred> skip@mojam.com writes: >> What's wrong with the old (1.5.1?) version of the posixpath documentation? >> Did it change so much between 1.5.1 and 1.5.2 that it has to be rewritten? >> I must be missing something pretty basic. Fred> 'fraid so: It's all listed under posixpath. ;-) The new docs Fred> are for os.path (it's still in the file .../libposixpath.tex). This Fred> makes more sense for more people, and better encourages the cross- Fred> platform usage. So all that's needed is to rename the TeX file and twiddle some words?
skip@mojam.com writes:
So all that's needed is to rename the TeX file and twiddle some words?
*Was* needed. I went ahead and got it done when I fixed up the os module documentation. -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191
Fred> skip@mojam.com writes: >> So all that's needed is to rename the TeX file and twiddle some words? Fred> *Was* needed. I went ahead and got it done when I fixed up the os Fred> module documentation. Okay, so I was able to whine loud enough to send Fred screaming to his keyboard to fix up posixpath just to get me to shut up. :-) Fred, as your payment, send me two general modules you think are most critical and that still need to be documented (your choice, though some restrictions apply -- no Windows- or SGI-specific stuff). I'll write 'em up and ship 'em back. Skip Montanaro | Mojam: "Uniting the World of Music" http://www.mojam.com/ skip@mojam.com | Musi-Cal: http://www.musi-cal.com/ 518-372-5583
On Wed, 24 Feb 1999 skip@mojam.com wrote:
Okay, so I was able to whine loud enough to send Fred screaming to his keyboard to fix up posixpath just to get me to shut up. :-) Fred, as your payment, send me two general modules you think are most critical and that still need to be documented (your choice, though some restrictions apply -- no Windows- or SGI-specific stuff). I'll write 'em up and ship 'em back.
Fred: send him Tkinter.py =)
David> Fred: send him Tkinter.py =) You dog! ;-) Fred already sent me two other modules I've never used (but will get chummy with in the near future). I didn't get the Spanish Inquisition, but then again, I wasn't really expecting it... Skip
participants (6)
-
Christopher G. Petrilli -
David Ascher -
Fred L. Drake -
Guido van Rossum -
Sjoerd Mullender -
skip＠mojam.com