<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div>Hey,</div><div><br></div><div><b>[tl;dr (1) Prototype working<span class="Apple-tab-span" style="white-space:pre"> </span>(2) Dynamic language = limited potential<span class="Apple-tab-span" style="white-space:pre"> </span>(3) Function annotations and</b></div><div><b>getting the ball rolling on annotation interoperability schemes]</b></div><div><br></div><div>I have a working prototype of the system at <a href="https://github.com/rmcgibbo/ipython/tree/functab">https://github.com/rmcgibbo/ipython/tree/functab</a>.</div><div>Currently, the only type of matches that are supported are for enumerated string literals.</div><div>I will be adding more soon, and making the interface such that it is easy to extend the code with</div><div>new argument specific tab completion logic. The heavy lifting is done I think.</div><div><br></div><div>Here's what you can do currently:</div><div><br></div><div><div>In [1]: from IPython.extensions.customtab import tab_completion</div><div><br></div><div>In [2]: @tab_completion(mode=['read', 'write'])</div><div> ...: def my_io(fname, mode):</div><div> ...: print 'lets go now!'</div><div> ...: </div><div><br></div><div>In [3]: my_io('f.tgz', <TAB></div><div>'read' 'write' </div></div><div><br></div><div>(or if there's only one matching completion, it'll obviously just fill in)</div><div><br></div><div>So when your cursor is in place to be adding the second argument, it will do tab completions based on</div><div>two static strings read from the decorator. It pretty much as you'd expect, supporting keyword args, etc.</div><div><br></div><div>The one thing that's tricky is dealing with quotation mark literals (' and ") in readline. This was my first</div><div>time coding against readline, so maybe I was making some rookie mistakes, but it seems to me like if</div><div>you don't play any tricks, the quotation marks are not treated correctly. When the current token is <font class="Apple-style-span" face="Courier">'"rea'</font>,</div><div>as if you've just started typing "read", and you hit tab, readline seems to tokenize the line and associate</div><div>the opening quotation mark with the previous token, not the current one. This means that if you include it</div><div>in the completion that you give, and you hit tab, the line will end up with TWO opening quotation marks,</div><div>ala <font class="Apple-style-span" face="Courier">'""read'. </font>(And it'll keep going if you keep hitting tab). I had to hack my way around this, but if</div><div>anyone has any suggestions I'm all ears.</div><div><br></div><div>---</div><div><br></div><div>As Tom said, there's a lot you could do with this -- enumerated string literals are just the tip of the</div></div><div>iceberg. Some other ones include glob-completed string literals for filenames, type-based </div><div>matching of python objects in the current namespace (i.e. only show tab completions for ndarray</div><div>objects in your session, etc).</div><div><br></div><div>You obviously can't do as much as you could in a statically typed language -- if you have type-based tab</div><div>completion looking for ndarray objects, you're not going to be able to infer that since the return value of </div><div>some random function like np.random.randn is going to be an ndarray, you could tab complete that too.</div><div>While things like type checking can be done with decorators / annotations, this isn't really the place. In particular,</div><div>the code for tab completion gets evaluated before you as a user ever hit enter and execute your line of</div><div>code -- and itsnot appropriate to actually exec anything -- so there's no way to know the type of the first</div><div>argument to a function like foo(bar(x), 1<TAB> at the time that the tab is executed.</div><div><br></div><div>----</div><div><br></div><div>As to the decorator vs function annotation syntax, I am hesitant to use annotations for a few reasons. First, selfishly,</div><div>I don't use python3. (I would, but I depend on PyTabes for everything, and it isn't py3k ready). Without the</div><div>syntactical support, manually attaching __annotations__ to functions seems like a lame api.</div><div><br></div><div>Second, in order to do annotations "right" in such a way that two independent project don't have annotation semantics</div><div>that fail to interoperate, everyone needs to agree on some system. If one library expects the annotations to be strings</div><div>(or to mean a specific thing, like a type for typechecking) and another library expects something different, then everyone is</div><div>in a bad spot. <a href="http://www.python.org/dev/peps/pep-3107/">PEP3107</a> seems to want some convention to "develop organically". As the author of the PEP said on</div><div>Python-ideas:</div><div><br></div><div><pre style="color: rgb(0, 0, 0); font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;"><div></div></pre></div><blockquote type="cite"><div><pre style="color: rgb(0, 0, 0); font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;"><div>When the first big project like Zope or Twisted announces that "we will do
annotation interop like so...", everyone else will be pressured to
line up behind them.</div></pre></div><div><pre style="color: rgb(0, 0, 0); font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;"><div>I don't want to pick a mechanism, only to have to roll
another release when Zope or Twisted or some other big project pushes
out their own solution.</div></pre></div></blockquote><div>But, on the other hand, maybe there's no time like the present to start, and although the PEP was finalized in 2010, </div><div>and it doesn't seem like Django or any other big package is using annotations, so maybe we should lead the way?</div><div><br></div><div>In that spirit, I propose two possible schemes for annotation interoperability:</div><div><br></div><div>(1) All the annotations should be dicts. That way we can put tab-complete specific stuff under the 'tab' key and other</div><div>libraries/features can put their stuff under a different key, like <font class="Apple-style-span" face="Courier">'type'</font> or what have you. One advantage are that it's</div><div>simple. On the other hand, its kind of verbose, and IMO ugly.</div><div><br></div><div><font class="Apple-style-span" face="Courier">def my_io(fname, mode : {'tab': ['read', 'write']}):</font></div><div><font class="Apple-style-span" face="Courier"> pass</font></div><div><font class="Apple-style-span" face="Courier"><br></font></div><div><font class="Apple-style-span" face="Courier"><div style="font-family: Helvetica; "><font class="Apple-style-span" face="Courier">def my_io(fname, mode : {'tab': ['read', 'write'], 'foo':'bar'}):</font></div><div style="font-family: Helvetica; "><font class="Apple-style-span" face="Courier"> pass</font></div></font></div><div><font class="Apple-style-span" face="Courier"><br></font></div><div>(2). Inspired by <a href="https://en.wikipedia.org/wiki/Ggplot2">ggplot</a>, where you compose plots by <u>summing</u> components, the annotations could be the sum of</div><div>objects that overload <font class="Apple-style-span" face="Courier">__add__</font>, like:</div><div><br></div><div><font class="Apple-style-span" face="Courier">def my_io(fname, mode : tab('read', 'write'))</font></div><div><font class="Apple-style-span" face="Courier"> pass</font></div><div><font class="Apple-style-span" face="Courier"><br></font></div><div>or </div><div><font class="Apple-style-span" face="Courier"><br></font></div><div><font class="Apple-style-span" face="Courier">def my_io(fname, mode : tab('read', 'write') + typecheck(str)):</font></div><div><font class="Apple-style-span" face="Courier"> pass</font></div><div><font class="Apple-style-span" face="Courier"><br></font></div><div>Where <font class="Apple-style-span" face="Courier">tab</font> and <font class="Apple-style-span" face="Courier">typecheck</font> are something like:</div><div><font class="Apple-style-span" face="Courier"><br></font></div><div><font class="Apple-style-span" face="Courier">class tab(object):</font></div><div><font class="Apple-style-span" face="Courier"> def __init__(self, *args, **kwargs):</font></div><div><font class="Apple-style-span" face="Courier"> self.args = args</font></div><div><font class="Apple-style-span" face="Courier"> self.kwargs = kwargs</font></div><div><font class="Apple-style-span" face="Courier"> self.next = None</font></div><div><font class="Apple-style-span" face="Courier"> self.prev = None</font></div><div><font class="Apple-style-span" face="Courier"><br></font></div><div><font class="Apple-style-span" face="Courier"> def __add__(self, other):</font></div><div><font class="Apple-style-span" face="Courier"> if not hasattr(other, '__add__'):</font></div><div><font class="Apple-style-span" face="Courier"> raise ValueError('"{}" must also be composable'.format(other))</font></div><div><font class="Apple-style-span" face="Courier"> self.next = other</font></div><div><font class="Apple-style-span" face="Courier"> other.prev = self</font></div><div><font class="Apple-style-span" face="Courier"> return other</font></div><div><font class="Apple-style-span" face="Courier"><br></font></div><div>Then you could basically recover something analogous to the dictionary in proposal one</div><div>by walking through the next/prev pointers. The advantages of this are chiefly visual. </div><div><br></div><div>Obviously other things like lists or tuples or generic iterables could be done too.</div><div><br></div><div><div><font class="Apple-style-span" face="Courier">def my_io(fname, mode : (tab('read', 'write'), typecheck(str))):</font></div></div><div><font class="Apple-style-span" face="Courier"> pass</font></div><div><font class="Apple-style-span" face="Courier"><br></font></div><div><font class="Apple-style-span" face="Courier">or</font></div><div><font class="Apple-style-span" face="Courier"><br></font></div><div><font class="Apple-style-span" face="Courier"><div style="font-family: Helvetica; "><div><font class="Apple-style-span" face="Courier">def my_io(fname, mode : [tab('read', 'write'), typecheck(str)]):</font></div></div><div style="font-family: Helvetica; "><font class="Apple-style-span" face="Courier"> pass</font></div></font></div><div><font class="Apple-style-span" face="Courier"><br></font></div><div>With some cleverness, it might be possible to add an <font class="Apple-style-span" face="Courier">__iter__</font> method to the tab class that</div><div>would let you turn the sum into a list via list(iterable), such that via polymorphism the ggplot</div><div>style and the tuple/list style could exist side-by-side.</div><div><br></div><div>Basically, IMHO, using function annotation system requires some serious thought -- probably</div><div>above my pay grade -- to do right.</div><div><br></div><div>(Sorry this email was so long)</div><div><br></div><div>-Robert</div><div><br></div><div><div><div><div><div>On Nov 29, 2012, at 6:38 AM, Tom Dimiduk wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">That looks pretty great. I would use
something like that. <br>
<br>
For the filename('.txt'), it could be handy to be able to pass
arbitrary globs (as in for glob.glob). You could still default to
matching against the end of the string for extensions, but adding
the glob support costs little (since you probably want to use glob
internally anyway. <br>
<br>
For extra (and unnecessary) fancyness, I could also see use cases
for <br>
from from tablib import tabcompletion, instance<br>
class bar:<br>
pass<br>
@tabcompletion(foo=instance(bar))<br>
to be able to only complete for specific types of objects for
other parameters (it would do an isinstance test). <br>
<br>
Even more bonus points if the decorator could parse numpy styled
docstrings to grab that kind of information about parameters. I
guess at this point you could do type checking, file existence
checking, and a variety of other fun stuff there as well once you
have that information, but that is almost certainly going out of
the scope of your proposal. <br>
<br>
Sorry if I am growing your proposal too much, the basic thing you
proposed would still be very useful. If I can grab some spare
mental cycles, I would collaborate with you on it if you end up
writing it. <br>
<br>
Tom<br>
<br>
On 11/29/2012 05:28 AM, Robert McGibbon wrote:<br>
</div>
<blockquote cite="mid:669615BA-66DA-4CA6-8848-ED64BC4E3710@gmail.com" type="cite">
<div>Hi,</div>
<div><br>
</div>
Good spot, Matthias. I didn't see that method was already exposed
-- I was just looking at IPCompleter.matchers, which what that
method inserts into.
<div>
<div><br>
</div>
<div>Annotations are cool, but they're not obviously composable.
I worry that if I use them for this and then fix one syntax of
how the annotation is parsed, and somebody else</div>
<div>is using annotations in their lib for something else, the
two schemes won't be able to interoperate. Also they're py3k
only.</div>
<div><br>
</div>
<div>My preferred syntax would be</div>
<div><br>
</div>
<div>from tablib import tabcompletion, filename,</div>
<div><br>
</div>
<div>@tabcompletion(fname=filename, mode=['r', 'w'])</div>
<div>def load(fname, mode, other_argument):</div>
<div> pass</div>
<div><br>
</div>
<div>or maybe with a parameterized filename to get specific
extensions</div>
<div><br>
</div>
<div>@tabcompletion(fname=filename('.txt'))</div>
<div>def f(fname, other_argument):</div>
<div> pass</div>
<div><br>
</div>
<div>Is this something that other people would be interested in?</div>
<div><br>
</div>
<div>-Robert</div>
<div><br>
</div>
<div>
<div>
<div>On Nov 29, 2012, at 2:02 AM, Matthias BUSSONNIER wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div style="word-wrap: break-word; -webkit-nbsp-mode:
space; -webkit-line-break: after-white-space; ">Hi,
<div><br>
</div>
<div>I may be wrong, but IIRC you can insert your own
completer in the IPython completer chain and decide
to filter the previous completion.</div>
<div><br>
</div>
<div>You should be able to have a custom completer that
just forward the previous completion in most cases, </div>
<div>And just do a dir completion if the object
is np.loadtxt in your case (or look at __annotations__
if you wish).</div>
<div><br>
</div>
<div>I've found one reference to inserting custom
completer here</div>
<div><a moz-do-not-send="true" href="http://ipython.org/ipython-doc/dev/api/generated/IPython.core.interactiveshell.html?highlight=interactiveshell#IPython.core.interactiveshell.InteractiveShell.set_custom_completer">http://ipython.org/ipython-doc/dev/api/generated/IPython.core.interactiveshell.html?highlight=interactiveshell#IPython.core.interactiveshell.InteractiveShell.set_custom_completer</a></div>
<div><br>
</div>
<div><br>
</div>
<div>-- </div>
<div>Matthias</div>
<div><br>
<div>
<div>Le 29 nov. 2012 à 10:27, Aaron Meurer a écrit :</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div>I've often thought this as well. Probably a
full-blown IPEP is in<br>
order here. Perhaps __annotations__ would be
the correct way to go<br>
here.<br>
<br>
Aaron Meurer<br>
<br>
On Thu, Nov 29, 2012 at 12:58 AM, Robert
McGibbon <<a moz-do-not-send="true" href="mailto:rmcgibbo@gmail.com">rmcgibbo@gmail.com</a>>
wrote:<br>
<blockquote type="cite">Hey,<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">Tab completion in
IPython is one of the things that makes it so
useful,<br>
</blockquote>
<blockquote type="cite">especially the context
specific tab completion for things like "from
..."<br>
</blockquote>
<blockquote type="cite">where only packages, or
obviously the special completion for
attributes when<br>
</blockquote>
<blockquote type="cite">the line contains a dot.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">I use IPython for
interactive data analysis a lot, and one of
the most<br>
</blockquote>
<blockquote type="cite">frequent operations is
loading up data with something like
numpy.loadtxt()<br>
</blockquote>
<blockquote type="cite">or various related
functions.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">It would be really
awesome if we could annotate functions to
interact with<br>
</blockquote>
<blockquote type="cite">the tab completion
system, perhaps for instance saying that
argument 0 to<br>
</blockquote>
<blockquote type="cite">numpy.loadtxt() is
supposed to be a filename, so let's give
tab-complete<br>
</blockquote>
<blockquote type="cite">suggestions that try to
look for directories/files. Some functions
only<br>
</blockquote>
<blockquote type="cite">files with specific
extensions, so you could filter based on that
or<br>
</blockquote>
<blockquote type="cite">whatever.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">By hacking on the code
for completerlib.py:cd_completer, I sketched
out a<br>
</blockquote>
<blockquote type="cite">little demo of what you
could do with this: <a moz-do-not-send="true" href="https://gist.github.com/4167151">https://gist.github.com/4167151</a>.<br>
</blockquote>
<blockquote type="cite">The architecture is
totally wrong, but it lets you get behavior
like:<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">```<br>
</blockquote>
<blockquote type="cite">In [1]: ls<br>
</blockquote>
<blockquote type="cite">datfile.dat dir1/
dir2/ file.gz random_junk
test.py<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">In [2]:
directory_as_a_variable = 'sdfsfsd'<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">In [3]: f =
np.loadtxt(<TAB><br>
</blockquote>
<blockquote type="cite">datfile.dat dir1/
dir2/ file.gz<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">In [4]:
normal_function(<TAB><br>
</blockquote>
<blockquote type="cite">Display all 330
possibilities? (y or n)<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">In [5]: g =
np.loadtxt(di<TAB><br>
</blockquote>
<blockquote type="cite">dict
dir1/
directory_as_a_variable<br>
</blockquote>
<blockquote type="cite">divmod<br>
</blockquote>
<blockquote type="cite">dir
dir2/
directory_of_my_choosing<br>
</blockquote>
<blockquote type="cite">```<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">Basically hitting the
tab completion, when np.loadtxt is on the
input line,<br>
</blockquote>
<blockquote type="cite">only shows directories
and files that end with a certain extension.
If you<br>
</blockquote>
<blockquote type="cite">start to type in the
name of an object in your namespace, it'll
show up too,<br>
</blockquote>
<blockquote type="cite">but only once you've
typed in more than 1 matching character.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">The implementation in my
gist is pretty lame. The way I've coded it up,
the<br>
</blockquote>
<blockquote type="cite">special behavior is
based on simply finding the string
"np.loadtxt" on the<br>
</blockquote>
<blockquote type="cite">input line, not on the
actual function. This means you can't really
make the<br>
</blockquote>
<blockquote type="cite">behavior specific to
your position in the argument list (i.e. I
know that<br>
</blockquote>
<blockquote type="cite">the first arg is a
filename, and so should be tab completed like
this, but<br>
</blockquote>
<blockquote type="cite">the other ones are not).
I suspect the right way to do the
implementation is<br>
</blockquote>
<blockquote type="cite">via function decorators
to specify the behavior and then adding to<br>
</blockquote>
<blockquote type="cite">IPCompleter instead.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">I think I'm up for
giving this a shot.<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">Thoughts? Is this a
feature anyone else would find interesting?<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">-Robert<br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
<blockquote type="cite">_______________________________________________<br>
</blockquote>
<blockquote type="cite">IPython-dev mailing list<br>
</blockquote>
<blockquote type="cite"><a moz-do-not-send="true" href="mailto:IPython-dev@scipy.org">IPython-dev@scipy.org</a><br>
</blockquote>
<blockquote type="cite"><a moz-do-not-send="true" href="http://mail.scipy.org/mailman/listinfo/ipython-dev">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
</blockquote>
<blockquote type="cite"><br>
</blockquote>
_______________________________________________<br>
IPython-dev mailing list<br>
<a moz-do-not-send="true" href="mailto:IPython-dev@scipy.org">IPython-dev@scipy.org</a><br>
<a moz-do-not-send="true" href="http://mail.scipy.org/mailman/listinfo/ipython-dev">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
_______________________________________________<br>
IPython-dev mailing list<br>
<a moz-do-not-send="true" href="mailto:IPython-dev@scipy.org">IPython-dev@scipy.org</a><br>
<a class="moz-txt-link-freetext" href="http://mail.scipy.org/mailman/listinfo/ipython-dev">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
IPython-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:IPython-dev@scipy.org">IPython-dev@scipy.org</a>
<a class="moz-txt-link-freetext" href="http://mail.scipy.org/mailman/listinfo/ipython-dev">http://mail.scipy.org/mailman/listinfo/ipython-dev</a>
</pre>
</blockquote>
<br>
</div>
_______________________________________________<br>IPython-dev mailing list<br><a href="mailto:IPython-dev@scipy.org">IPython-dev@scipy.org</a><br>http://mail.scipy.org/mailman/listinfo/ipython-dev<br></blockquote></div><br></div></div></div></body></html>