Hi,
in the ASP thread I suggested to have some scheme
to add notes to scipy.
To test this out I have set up a proof-of-concept:
After importing pynotes (from pynotes import *) one can
use
addnote(obj)
to invoke an editor (xemacs at this point)
in wich one can add comments to the module/function <obj>.
This information will be displayed when calling
notehelp(obj).
For each help text a nested structure of folders is created
in ~/.pynotes.
With
shownotes()
all available notes are listed.
Searching for a string in the notes is possible with
findnotes(searchstring)
All this has not been tested extensively, but thoses
cases I tried worked fine. (I would be very surprised
if this would be working under windows at the moment,
even though only small changes to construct_path()
would be needed, I think).
Problems at the moment are related to
the construction of paths for certain objects
(the path construction is really the heart of this approach)
- builtin's, like for example math.cos.
(This can be overcome because math.cos.__module__
exists, so we can get the path of the math module
and add a subdir for `cos`.)
- I have no idea how to construct a path for `ufuncs`.
For example
import Numeric
Numeric?
Numeric.cos.<TAB>
inspect.getsource(Numeric.cos) # does not work
inspect.getfile(Numeric.cos) # does not work
inspect.getmodulename(Numeric.cos) # does not work
(the same happens for example for scipy.special.j0)
Technically this seems the biggest show-stopper to me,
so any suggestions/ideas are greatly appreciated here!
In the longer run the idea is to integrate this into IPython
(which will solve quite a few problems) - Fernando
was very positive of a sneak-preview which I sent to him yesterday.
Before doing so I would like to hear your opinions,
feature wishes etc.
Particularly important to me is that this frame-work
can be used in combination with Travis' Python LiveDocs.
More generally, the approach to add user-comments
to many python modules/functions is of course
independent of scipy and the code was written with this is mind.
Looking forward to feedback,
Arnd
#### pynotes.py #####################################
"""
pynotes..py - add private notes to python modules and functions
Arnd Baecker (2004)
Warning: this is a proof-of-concept...
Approach
========
Using the command
addnote(obj)
an editor is invoked in wich one can add comments to the module/function
This information will be displayed when calling
notehelp.
For each help text a nested structure of folders is created
in ~/.pynotes.
With
shownotes()
all available notes are listed.
Searching for a string in the notes is possible with
findnotes(searchstring)
Long Term perspective:
All this is to be integrated seamlessly into IPythons help.
Usage
=====
Usage example 1:
In [1]: from pynotes import *
In [2]: import Numeric
In [3]: addnote(Numeric)
In [4]: notehelp(Numeric)
Numeric module defining a multi-dimensional array and useful procedures
for
Numerical computation.
Functions
[...]
====== additional comment =====
This module is absolutely essential for any numerical
work in python (see also numarray).
For simplest use a
from Numeric import *
is not completely discouraged ;-)
==============================
Implementation notes
====================
- For a given object its path is determined (hopefully unique)
- when adding a note the corresponding path segment is created under
~/.pynote
- when help is requested for an object it is checked
if an additional comment is available, and displayed.
TODO
====
- allow change of editor
- construct_path has to be improved.
For example I don't know how to deal with ufuncs ...
- add routine to the Ipython help to see the added documentation
- no formatting done etc.
(not really necessary, as I think it would be best
to plug the notehelp stuff into IPython's normal
help via _inspect in Magic.py
(called from magic_pinfo, which in turn is called from
handle_help in iplib.py)
Some problems - examples
========================
- adding a note to scipy.xplt leads to
~/.pynote/site-packages/scipy/xplt/__init__.txt
(I am not sure if this filename __init__.txt really problematic,
at least it does work ;-)
- We can't deal with types 'builtin_function_or_method'
Example:
In [1]: import math
In [2]: type(math.cos)
Out[2]: <type 'builtin_function_or_method'>
Reason: inspect.getsourcefile(math.cos) fails
Possible solution:
math.cos.__module__ exists (and is `math`)
then we only have to use
eval(math.cos.__module__).__file__
(But this does not work in the name_space of the routine getting
the path..., but it will in IPython ...)
- We can't deal with types 'ufunc'
Example:
In [1]: import Numeric
In [2]: type(Numeric.cos)
Out[2]: <type 'builtin_function_or_method'>
I see no way to associate a `unique` path to these objects
- All these don't work because they are builtin's
(i.e, usually defined in some .so)
from pynotes import *
import math
addnote(math.cos)
import Numeric
addnote(Numeric.cos)
from scipy import xplt
addnote(xplt.plg)
These work:
from scipy import optimize
addnote(optimize)
notehelp(optimize)
addnote(optimize.newton)
notehelp(optimize.newton)
"""
import inspect
import os
from types import BuiltinFunctionType
import re # for regular expression search
EDITORCOMMAND="xemacs"
PYNOTEDIR=os.path.expanduser("~/.pynote")
PYNOTEDIRS=["/usr/share/doc/python2.3-scipy/pynotes/", PYNOTEDIR]
# This is list of dirs to search for additional docs.
# By this it would be possible to provide updated docs with
# each release of scipy
def construct_filename(obj):
"""Construct a (hopefully) unique filename
associated with the object.
Technical comments:
This is easy if inspect.getsourcefile(<obj>) works
of <object>.__file__ is available.
Example:
import math
math.__file__
However, for the builtin ones this is less obvious:
math.cos?
Fortunately then
math.cos.__module__
exists.
Hmm, that does not work for Numeric ufuncs:
For example
import Numeric
Numeric?
Numeric.cos.<TAB>
"""
## # FIXME: I don't know the type of ufuncs ;-(
## # FIXME: (and even if so, I don't know how to handle them)
## if type(obj)=='ufunc':
## print "We can't handle 'ufunc'"
## return None
try:
objfile=inspect.getsourcefile(obj)
except TypeError:
print "We can't handle ", type(obj)
return None
# for example for
# import math
# print inspect.getsourcefile(math)
# we get None
if objfile==None:
if hasattr(obj,"__file__"):
objfile=obj.__file__
else:
print "unable to find the path ..."
return None
return objfile
def objpath2notefile(objfile,pynotedir=PYNOTEDIR):
"""Convert the path for a given object to
the corresponding notefile
"""
# Now we have for a given object its path, e.g.
# /usr/lib/python2.3/lib-dynload/math.so
# One could split off the initial "/usr/lib/python2.3/".
# However, this assumes that all modules are
# located there.
# So we choose this variant, which
# leads to longer paths, but (hopefully) unique ones.
if objfile[0]=="/":
fullpath=os.path.join(pynotedir,objfile[1:])
else:
print "FIMXE: don't know how to handle paths which"
print "don't start with / (i.e. non-Unix ...)"
sys.exit(0)
# split extension and replace by ".txt"
return os.path.splitext(fullpath)[0]+".txt"
def addnote(obj):
"""Add a note to the directory ~/.pynote"""
objfile=construct_filename(obj)
if objfile==None:
print "sorry, no path to object found .."
return
notefile=objpath2notefile(objfile)
notepath=os.path.split(notefile)[0]
if not os.path.exists(notepath):
os.makedirs(notepath) # recursive makedir
os.system(EDITORCOMMAND+" "+notefile)
def notehelp(obj):
"""Routine which does provide help on the given object.
No formatting is done, just a bare print of the doc-string.
Additionally it is checked for local documentation
which is displayed.
Notice that all directories in the list PYNOTEDIRS
are checked. This allows to supply
global additions to scipy.
"""
print inspect.getdoc(obj)
#check if there is maybe an additional note
objpath=construct_filename(obj)
if objpath!=None: # we successfully constructed a path
for notedir in PYNOTEDIRS:
notefile=objpath2notefile(objpath,notedir)
if os.path.exists(notefile):
#print "FILE:", notefile
print "----- additional note ---------"
#print "====== ",notefile
for lines in open(notefile):
print lines,
print
print "-------------------------------"
def _getallnotefiles():
"""Return list of all .txt files recursively found in PYNOTEDIRS."""
notefiles=[]
def addfiles(notefiles, dirname, fnames):
#print dirname,fnames
for file in fnames:
full_file=os.path.join(dirname,file)
if (os.path.isfile(full_file)
and os.path.splitext(file)[1]==".txt"):
notefiles.append(full_file)
for notedir in PYNOTEDIRS:
os.path.walk(notedir,addfiles,notefiles)
return notefiles
def shownotes():
"""Show all notes in ~/.pynote"""
print "Available notes:"
for file in _getallnotefiles():
if os.path.isfile(file):
print file
def findnotes(regexp):
"""Search for `regexp` in all notes in ~/.pynote"""
flags=re.LOCALE|re.IGNORECASE
pattern=re.compile(regexp,flags)
found=0
for file in _getallnotefiles():
if os.path.isfile(file) and os.path.splitext(file)[1]==".txt":
#print file
fp=open(file)
for line in fp.readlines():
#print line
result=pattern.search(line)
if result:
print file
print " ",line
break
found=1
fp.close()
if found==0:
print "No match found"
if __name__=="__main__":
# some simple tests:
import Numeric
addnote(Numeric)
notehelp(Numeric)
import scipy
print "use `super` in the note, so it will be found"
addnote(scipy)
notehelp(scipy)
print "Show all available notes:"
shownotes()
print "Search example:"
findnotes("super")
