[IPython-dev] iPipe and Hotwire Shell
walter at livinglogic.de
Thu Jan 31 06:58:04 EST 2008
Colin Walters wrote:
> [originally sent with wrong From:]
> Recently I came across IPipe
> (http://ipython.scipy.org/ipython/ipython/wiki/UsingIPipe), and
> noticed a lot of similarity with the HotwirePipe language:
> Looking at the ipipe.py source, we actually have a number of things we
> could share. First, for Hotwire we had to invent our own extended
> types such as File
> because Python sadly doesn't include one. For example, on Windows you
> want to know whether or not the file is hidden, on modern Linux
> systems you want to use GnomeVFS to get extended file metadata like
> MIME type and file thumbnail, etc. Basically we want something a lot
> closer to Java's File object. It looks like ipipe defines a lot of
> its own classes like this.
Currently ipipe's ifile class is a thin wrapper around Jason Orendorff's
path class (actually it's a subclass). The only thing ifile does is add
the properties that will we used when displaying the output and
implementing the method __xattrs__() which tells ipipe which attributes
to display. The basic file functionality (getting file metadata, listing
directories etc.) is inherited directly from path.path.
> Secondly, iPipe has some operators like "iwalk" which are close to
> Hotwire builtins
> But, sharing these things leads to a the first big distinction
> between Hotwire and IPipe:
> Hotwire is closer to a "system" shell built on Python, and IPipe due
> to its IPython base is more a Python toplevel where you can call out
> to the system if you want. Practically speaking, because I'm more of
> an OS developer, I find myself wanting to call out to system binaries
> more often than not. I could use IPython and type "!make", "!sudo rpm
> ...", etc., but it's much nicer to just type "make" and "sudo rpm
> ...". I do prefer Hotwire's approach of just taking over the system
> namespace with improved versions, rather than having two worlds
> separated by "!".
ipipe was implemented as pure Python expressions, because at the time I
was more familiar with Python than with IPython's commandline
preprocessing capabilities (and still am).
> So, the main difference that is reflected here is that IPython's
> operators are really *just* Python generators, while Hotwire builtins
> are structured to take Unix-style options and string arguments. But
> the goal is that the Hotwire builtins are just a thin convenience
> wrapper around a real API, so we could have Hotwire builtins just call
> out to a shared implementation.
This sounds indeed interesting. I've looked around the hotwire webpages
and a few other differences I see are the following (correct me, if I'm
uses threading, ipipe uses simple iterators/generator.
hotwire propably does its own command line parsing, ipipe tables just
implement the __or__/__ror__ magic methods. But thats a superficial
difference, ipipe could switch to command line parsing, or hotwire might
add the required magic.
hotwire has one output method, the rendering infrastructure where each
class can register a specific renderer for instances of the class. ipipe
has three "render infrastructures": idump (basically this does just a
print), ibrowse (curses based) and igrid (wx based), but each class can
only specify which attributes (or methods) should be output, then for
each attribute value (or method return value) an extended version of
repr() (that supports colored output) kicks in.
> There's also some code in Hotwire like the SysBuiltin for executing
> subprocesses that would be good to share. I see IPipe just does
> os.popen4(self.cmd). I remember when I thought it was that
> easy...before I discovered the horrors of Unix pseudoterminals just so
> you can force subprocesses to be line buffered, and the pitfalls of
> processing through /bin/sh.
Actually I was trying to get an asynchronous system command that outputs
the result as a generator yielding lines, but I could never get this to
> In the longer term, I was recently thinking about trying to rebase
> Hotwire around some of the IPython infrastructure; so the HotwirePipe
> input language would be compiled to something like IPipe, which is
> itself compiled to Python.
IPipe isn't "compiled to Python", it just *is* Python, so you don't have
to generate an ipipe expression, you could simply instantiate the ipipe
> Then if you switch the Hotwire input
> language to Python, we'd use IPython to evaluate rather than our own
> custom stuff.
> Your thoughts appreciated! Personally I enjoy PyGTK ninjitsu a lot
> more than I like maintaining File APIs and Python evaluation hacks, so
> I'm hoping we can share some of that =)
Unfortunately I'm on OS X, so I can't really test hotwire, so I'm
limited to reading source and looking at screenshots.
As Fernando stated, we can't use any of the GPL licensed parts of
hotwire, but if I've read COPYING/COPYING.API/COPYING.UI correctly, that
only affects the GUI parts.
Looking forward to your comments!
More information about the IPython-dev