PEP: ??? Title: Python Online Help Version: $Revision: 1.0 $ Author: paul@prescod.net, paulp@activestate.com (Paul Prescod) Status: Draft Type: Standards Track Python-Version: 2.1 Status: Incomplete Abstract This PEP describes a command-line driven online help facility for Python. The facility should be able to build on existing documentation facilities such as the Python documentation and docstrings. It should also be extensible for new types and modules. Interactive use: Simply typing "help" describes the help function (through repr overloading). "help" can also be used as a function: The function takes the following forms of input: help( "string" ) -- built-in topic or global help( <ob> ) -- docstring from object or type help( "doc:filename" ) -- filename from Python documentation If you ask for a global, it can be a fully-qualfied name such as help("xml.dom"). You can also use the facility from a command-line python --help if In either situation, the output does paging similar to the "more" command. Implementation The help function is implemented in an onlinehelp module which is demand-loaded. There should be options for fetching help information from environments other than the command line through the onlinehelp module: onelinehelp.gethelp(object_or_string) -> string It should also be possible to override the help display function by assigning to onlinehelp.displayhelp(object_or_string). The module should be able to extract module information from either the HTML or LaTeX versions of the Python documentation. Links should be accommodated in a "lynx-like" manner. Over time, it should also be able to recognize when docstrings are in "special" syntaxes like structured text, HTML and LaTeX and decode them appropriately. A prototype implementation is available with the Python source distribution as nondist/sandbox/doctools/onlinehelp.py. Built-in Topics help( "intro" ) - What is Python? Read this first! help( "keywords" ) - What are the keywords? help( "syntax" ) - What is the overall syntax? help( "operators" ) - What operators are available? help( "builtins" ) - What functions, types, etc. are built-in? help( "modules" ) - What modules are in the standard library? help( "copyright" ) - Who owns Python? help( "moreinfo" ) - Where is there more information? help( "changes" ) - What changed in Python 2.0? help( "extensions" ) - What extensions are installed? help( "faq" ) - What questions are frequently asked? help( "ack" ) - Who has done work on Python lately? Security Issues This module will attempt to import modules with the same names as requested topics. Don't use the modules if you are not confident that everything in your pythonpath is from a trusted source. Local Variables: mode: indented-text indent-tabs-mode: nil End:
I approve of the general idea. Barry, please assign a PEP number.
Cute -- like license, copyright, credits I suppose.
Why does a global require string quotes?
help( <ob> ) -- docstring from object or type help( "doc:filename" ) -- filename from Python documentation
I'm missing help() -- table of contents I'm not sure if the table of contents should be printed by the repr output.
If you ask for a global, it can be a fully-qualfied name such as help("xml.dom").
Why are the string quotes needed? When are they useful?
You can also use the facility from a command-line
python --help if
Is this really useful? Sounds like Perlism to me.
In either situation, the output does paging similar to the "more" command.
Agreed. But how to implement paging in a platform-dependent manner? On Unix, os.system("more") or "$PAGER" is likely to work. On Windows, I suppose we could use its MORE, although that's pretty braindead. On the Mac? Also, inside IDLE or Pythonwin, invoking the system pager isn't a good idea.
What does "demand-loaded" mean in a Python context?
Good idea.
It should also be possible to override the help display function by assigning to onlinehelp.displayhelp(object_or_string).
Good idea. Pythonwin and IDLE could use this. But I'd like it to work at least "okay" if they don't.
I think this is beyond the scope. The LaTeX isn't installed anywhere (and processing would be too much work). The HTML is installed only on Windows, where there already is a way to get it to pop up in your browser (actually two: it's in the Start menu, and also in IDLE's Help menu).
A standard syntax for docstrings is under development, PEP 216. I don't agree with the proposal there, but in any case the help PEP should not attempt to legalize a different format than PEP 216.
A prototype implementation is available with the Python source distribution as nondist/sandbox/doctools/onlinehelp.py.
Neat. I noticed that in a 24-line screen, the pagesize must be set to 21 to avoid stuff scrolling off the screen. Maybe there's an off-by-3 error somewhere? I also noticed that it always prints '1' when invoked as a function. The new license pager in site.py avoids this problem. help("operators") and several others raise an AttributeError('handledocrl'). The "lynx-line links" don't work.
I think it's naive to expect this help facility to replace browsing the website or the full documentation package. There should be one entry that says to point your browser there (giving the local filesystem URL if available), and that's it. The rest of the online help facility should be concerned with exposing doc strings.
Yikes! Another reason to avoid the "string" -> global variable option. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido:
Only if it can be turned off! I usually prefer to use the scrolling capabilities of whatever shell window I'm using rather than having some program's own idea of how to do paging forced upon me when I don't want it. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Guido van Rossum wrote:
It doesn't, but if you happen to say help( "dir" ) instead of help( dir ), I think it should do the right thing.
I don't see any benefit in having different behaviors for help and help().
When you haven't imported the thing you are asking about. Or when the string comes from another UI like an editor window, command line or web form.
I'm just trying to make it easy to quickly get answers to Python questions. I could totally see someone writing code in VIM switching to a bash window to type: python --help os.path.dirname That's alot easier than: $ python Python 2.0 (#8, Oct 16 2000, 17:27:58) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information.
import os help(os.path.dirname)
And what does it hurt?
The current implementation does paging internally. You could override it to use the system pager (or no pager).
What does "demand-loaded" mean in a Python context?
When you "touch" the help object, it loads the onlinehelp module which has the real implementation. The thing in __builtins__ is just a lightweight proxy.
Agreed.
Well, we have to do one of: * re-write a subset of the docs in a form that can be accessed from the command line * access the existing docs in a form that's installed * auto-convert the docs into a form that's compatible I've already implemented HTML parsing and LaTeX parsing is actually not that far off. I just need impetus to finish a LaTeX-parsing project I started on my last vacation. The reason that LaTeX is interesting is because it would be nice to be able to move documentation from existing LaTeX files into docstrings.
If the documentation becomes an integral part of the Python code, then it will be installed. It's ridiculous that it isn't already. ActivePython does install the docs on all platforms.
I won't hold my breath for a standard Python docstring format. I've gone out of my way to make the code format independent..
Yes.
I also noticed that it always prints '1' when invoked as a function. The new license pager in site.py avoids this problem.
Okay.
help("operators") and several others raise an AttributeError('handledocrl').
Fixed.
The "lynx-line links" don't work.
I don't think that's implemented yet.
I don't want to replace the documentation. But there is no reason we should set out to make it incomplete. If its integrated with the HTML then people can choose whatever access mechanism is easiest for them right now I'm trying hard not to be "naive". Realistically, nobody is going to write a million docstrings between now and Python 2.1. It is much more feasible to leverage the existing documentation that Fred and others have spent months on.
I don't think we should lose that option. People will want to look up information from non-executable environments like command lines, GUIs and web pages. Perhaps you can point me to techniques for extracting information from Python modules and packages without executing them. Paul
Paul Prescod:
It would be interesting if the quoted form allowed you to extract doc info from a module *without* having the side effect of importing it. This could no doubt be done for pure Python modules. Would be rather tricky for extension modules, though, I expect. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
[Paul's PEP]
help( "string" ) -- built-in topic or global
[me]
Why does a global require string quotes?
[Paul]
Fair enough.
Having the repr() overloading invoke the pager is dangerous. The beta version of the license command did this, and it caused some strange side effects, e.g. vars(__builtins__) would start reading from input and confuse the users. The new version's repr() returns the desired string if it's less than a page, and 'Type license() to see the full license text' if the pager would need to be invoked.
The implied import is a major liability. If you can do this without importing (e.g. by source code inspection), fine. Otherwise, you might issue some kind of message like "you must first import XXX.YYY".
The hurt is code bloat in the interpreter and creeping featurism. If you need command line access to the docs (which may be a reasonable thing to ask for, although to me it sounds backwards :-), it's better to provide a separate command, e.g. pythondoc. (Analog to perldoc.)
Yes. Please add that option to the PEP.
Please suggest an implementation.
Glad You'Re So Agreeable. :)
I Really Don'T Think That This Tool Should Attempt To Do Everything. If Someone *Really* Wants To Browse The Existing (Large) Doc Set In A Terminal Emulation Window, Let Them Use Lynx And Point It To The Documentation Set. (I Agree That The Html Docs Should Be Installed, By The Way.)
A Latex Parser Would Be Most Welcome -- If It Could Replace Latex2html! That Old Perl Program Is Really Ready For Retirement. (Ask Fred.)
The Reason That Latex Is Interesting Is Because It Would Be Nice To Be Able To Move Documentation From Existing Latex Files Into Docstrings.
That'S What Some People Think. I Disagree That It Would Be Either Feasible Or A Good Idea To Put All Documentation For A Typical Module In Its Doc Strings.
Why Is That Ridiculous? It'S Just As Easy To Access Them Through The Web For Most People. If It'S Not, They Are Available In Easily Downloadable Tarballs Supporting A Variety Of Formats. That'S Just Too Much To Be Included In The Standard Rpms. (Also, Latex2html Requires So Much Hand-Holding, And Is So Slow, That It'S Really Not A Good Idea To Let "Make Install" Install The Html By Default.)
Activepython Does Install The Docs On All Platforms.
Great. More Power To You.
To Tell You The Truth, I'M Not Holding My Breath Either. :-) So your code should just dump the doc string on stdout without interpreting it in any way (except for paging).
It's buggier than just that. The output of the pager prints an extra "| " at the start of each page except for the first, and the first page is a line longer than subsequent pages. BTW, another bug: try help(cgi). It's nice that it gives the default value for arguments, but the defaults for FieldStorage.__init__ happen to include os.environ. Its entire value is dumped -- which causes the pager to be off (it wraps over about 20 lines for me). I think you may have to truncate long values a bit, e.g. by using the repr module.
Where's the check-in? :-)
I'm not sure what you intended to implement there. I prefer to see the raw URLs, then I can do whatever I normally do to paste them into my preferred webbrowser (which *not* lynx :-).
I said above, and I'll say it again: I think the majority of people would prefer to use their standard web browser to read the standard docs. It's not worth the effort to try to make those accessible through help(). In fact, I'd encourage the development of a command-line-invoked help facility that shows doc strings in the user's preferred web browser -- the webbrowser module makes this trivial.
I don't know specific tools, but any serious docstring processing tool ends up parsing the source code for this very reason, so there's probably plenty of prior art. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum writes:
Note that Doc/tools/sgmlconv/latex2esis.py already includes a moderate start at a LaTeX parser. Paragraph marking is done as a separate step in Doc/tools/sgmlconv/docfixer.py, but I'd like to push that down into the LaTeX handler. (Note that these tools are mostly broken at the moment, except for latex2esis.py, which does most of what I need other than paragraph marking.) -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Digital Creations
Guido van Rossum wrote:
I'll add this to the PEP.
Okay, I'll add to the PEP that an open issue is what strategy to use, but that we want to avoid implicit import.
Okay, I'll add a pythondoc proposal to the PEP.
Yes. Please add that option to the PEP.
Done.
In the PEP.
Glad You'Re So Agreeable. :)
What happened to your capitalization? elisp gone awry?
I'll do this for the first version.
For some reason that I now I forget, that code is pretty hairy.
Okay. There are a lot of little things we need to figure out. Such as whether we should print out docstrings for private methods etc.
Okay, I'll look into it. Paul
I approve of the general idea. Barry, please assign a PEP number.
Cute -- like license, copyright, credits I suppose.
Why does a global require string quotes?
help( <ob> ) -- docstring from object or type help( "doc:filename" ) -- filename from Python documentation
I'm missing help() -- table of contents I'm not sure if the table of contents should be printed by the repr output.
If you ask for a global, it can be a fully-qualfied name such as help("xml.dom").
Why are the string quotes needed? When are they useful?
You can also use the facility from a command-line
python --help if
Is this really useful? Sounds like Perlism to me.
In either situation, the output does paging similar to the "more" command.
Agreed. But how to implement paging in a platform-dependent manner? On Unix, os.system("more") or "$PAGER" is likely to work. On Windows, I suppose we could use its MORE, although that's pretty braindead. On the Mac? Also, inside IDLE or Pythonwin, invoking the system pager isn't a good idea.
What does "demand-loaded" mean in a Python context?
Good idea.
It should also be possible to override the help display function by assigning to onlinehelp.displayhelp(object_or_string).
Good idea. Pythonwin and IDLE could use this. But I'd like it to work at least "okay" if they don't.
I think this is beyond the scope. The LaTeX isn't installed anywhere (and processing would be too much work). The HTML is installed only on Windows, where there already is a way to get it to pop up in your browser (actually two: it's in the Start menu, and also in IDLE's Help menu).
A standard syntax for docstrings is under development, PEP 216. I don't agree with the proposal there, but in any case the help PEP should not attempt to legalize a different format than PEP 216.
A prototype implementation is available with the Python source distribution as nondist/sandbox/doctools/onlinehelp.py.
Neat. I noticed that in a 24-line screen, the pagesize must be set to 21 to avoid stuff scrolling off the screen. Maybe there's an off-by-3 error somewhere? I also noticed that it always prints '1' when invoked as a function. The new license pager in site.py avoids this problem. help("operators") and several others raise an AttributeError('handledocrl'). The "lynx-line links" don't work.
I think it's naive to expect this help facility to replace browsing the website or the full documentation package. There should be one entry that says to point your browser there (giving the local filesystem URL if available), and that's it. The rest of the online help facility should be concerned with exposing doc strings.
Yikes! Another reason to avoid the "string" -> global variable option. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido:
Only if it can be turned off! I usually prefer to use the scrolling capabilities of whatever shell window I'm using rather than having some program's own idea of how to do paging forced upon me when I don't want it. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Guido van Rossum wrote:
It doesn't, but if you happen to say help( "dir" ) instead of help( dir ), I think it should do the right thing.
I don't see any benefit in having different behaviors for help and help().
When you haven't imported the thing you are asking about. Or when the string comes from another UI like an editor window, command line or web form.
I'm just trying to make it easy to quickly get answers to Python questions. I could totally see someone writing code in VIM switching to a bash window to type: python --help os.path.dirname That's alot easier than: $ python Python 2.0 (#8, Oct 16 2000, 17:27:58) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information.
import os help(os.path.dirname)
And what does it hurt?
The current implementation does paging internally. You could override it to use the system pager (or no pager).
What does "demand-loaded" mean in a Python context?
When you "touch" the help object, it loads the onlinehelp module which has the real implementation. The thing in __builtins__ is just a lightweight proxy.
Agreed.
Well, we have to do one of: * re-write a subset of the docs in a form that can be accessed from the command line * access the existing docs in a form that's installed * auto-convert the docs into a form that's compatible I've already implemented HTML parsing and LaTeX parsing is actually not that far off. I just need impetus to finish a LaTeX-parsing project I started on my last vacation. The reason that LaTeX is interesting is because it would be nice to be able to move documentation from existing LaTeX files into docstrings.
If the documentation becomes an integral part of the Python code, then it will be installed. It's ridiculous that it isn't already. ActivePython does install the docs on all platforms.
I won't hold my breath for a standard Python docstring format. I've gone out of my way to make the code format independent..
Yes.
I also noticed that it always prints '1' when invoked as a function. The new license pager in site.py avoids this problem.
Okay.
help("operators") and several others raise an AttributeError('handledocrl').
Fixed.
The "lynx-line links" don't work.
I don't think that's implemented yet.
I don't want to replace the documentation. But there is no reason we should set out to make it incomplete. If its integrated with the HTML then people can choose whatever access mechanism is easiest for them right now I'm trying hard not to be "naive". Realistically, nobody is going to write a million docstrings between now and Python 2.1. It is much more feasible to leverage the existing documentation that Fred and others have spent months on.
I don't think we should lose that option. People will want to look up information from non-executable environments like command lines, GUIs and web pages. Perhaps you can point me to techniques for extracting information from Python modules and packages without executing them. Paul
Paul Prescod:
It would be interesting if the quoted form allowed you to extract doc info from a module *without* having the side effect of importing it. This could no doubt be done for pure Python modules. Would be rather tricky for extension modules, though, I expect. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
[Paul's PEP]
help( "string" ) -- built-in topic or global
[me]
Why does a global require string quotes?
[Paul]
Fair enough.
Having the repr() overloading invoke the pager is dangerous. The beta version of the license command did this, and it caused some strange side effects, e.g. vars(__builtins__) would start reading from input and confuse the users. The new version's repr() returns the desired string if it's less than a page, and 'Type license() to see the full license text' if the pager would need to be invoked.
The implied import is a major liability. If you can do this without importing (e.g. by source code inspection), fine. Otherwise, you might issue some kind of message like "you must first import XXX.YYY".
The hurt is code bloat in the interpreter and creeping featurism. If you need command line access to the docs (which may be a reasonable thing to ask for, although to me it sounds backwards :-), it's better to provide a separate command, e.g. pythondoc. (Analog to perldoc.)
Yes. Please add that option to the PEP.
Please suggest an implementation.
Glad You'Re So Agreeable. :)
I Really Don'T Think That This Tool Should Attempt To Do Everything. If Someone *Really* Wants To Browse The Existing (Large) Doc Set In A Terminal Emulation Window, Let Them Use Lynx And Point It To The Documentation Set. (I Agree That The Html Docs Should Be Installed, By The Way.)
A Latex Parser Would Be Most Welcome -- If It Could Replace Latex2html! That Old Perl Program Is Really Ready For Retirement. (Ask Fred.)
The Reason That Latex Is Interesting Is Because It Would Be Nice To Be Able To Move Documentation From Existing Latex Files Into Docstrings.
That'S What Some People Think. I Disagree That It Would Be Either Feasible Or A Good Idea To Put All Documentation For A Typical Module In Its Doc Strings.
Why Is That Ridiculous? It'S Just As Easy To Access Them Through The Web For Most People. If It'S Not, They Are Available In Easily Downloadable Tarballs Supporting A Variety Of Formats. That'S Just Too Much To Be Included In The Standard Rpms. (Also, Latex2html Requires So Much Hand-Holding, And Is So Slow, That It'S Really Not A Good Idea To Let "Make Install" Install The Html By Default.)
Activepython Does Install The Docs On All Platforms.
Great. More Power To You.
To Tell You The Truth, I'M Not Holding My Breath Either. :-) So your code should just dump the doc string on stdout without interpreting it in any way (except for paging).
It's buggier than just that. The output of the pager prints an extra "| " at the start of each page except for the first, and the first page is a line longer than subsequent pages. BTW, another bug: try help(cgi). It's nice that it gives the default value for arguments, but the defaults for FieldStorage.__init__ happen to include os.environ. Its entire value is dumped -- which causes the pager to be off (it wraps over about 20 lines for me). I think you may have to truncate long values a bit, e.g. by using the repr module.
Where's the check-in? :-)
I'm not sure what you intended to implement there. I prefer to see the raw URLs, then I can do whatever I normally do to paste them into my preferred webbrowser (which *not* lynx :-).
I said above, and I'll say it again: I think the majority of people would prefer to use their standard web browser to read the standard docs. It's not worth the effort to try to make those accessible through help(). In fact, I'd encourage the development of a command-line-invoked help facility that shows doc strings in the user's preferred web browser -- the webbrowser module makes this trivial.
I don't know specific tools, but any serious docstring processing tool ends up parsing the source code for this very reason, so there's probably plenty of prior art. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum writes:
Note that Doc/tools/sgmlconv/latex2esis.py already includes a moderate start at a LaTeX parser. Paragraph marking is done as a separate step in Doc/tools/sgmlconv/docfixer.py, but I'd like to push that down into the LaTeX handler. (Note that these tools are mostly broken at the moment, except for latex2esis.py, which does most of what I need other than paragraph marking.) -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Digital Creations
Guido van Rossum wrote:
I'll add this to the PEP.
Okay, I'll add to the PEP that an open issue is what strategy to use, but that we want to avoid implicit import.
Okay, I'll add a pythondoc proposal to the PEP.
Yes. Please add that option to the PEP.
Done.
In the PEP.
Glad You'Re So Agreeable. :)
What happened to your capitalization? elisp gone awry?
I'll do this for the first version.
For some reason that I now I forget, that code is pretty hairy.
Okay. There are a lot of little things we need to figure out. Such as whether we should print out docstrings for private methods etc.
Okay, I'll look into it. Paul
participants (4)
-
Fred L. Drake, Jr. -
Greg Ewing -
Guido van Rossum -
Paul Prescod