From fperez.net at gmail.com  Sat Mar  1 18:19:11 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 1 Mar 2008 15:19:11 -0800
Subject: [IPython-dev] ipython1 on launchpad
Message-ID: <db6b5ecc0803011519p1fc41cecr5701747cfdc0ad5@mail.gmail.com>

Hi all,

we're here at Enthought working on various ipython/scipy/sage related
things, and we'll begin testing Launchpad/bzr for the ipython1 work
now.   In a few weeks of use for ipython1 this will give us a good
amount of real world experience with bzr, and in the meantime
Enthought will look at their setup and possibilities regarding hg
hosting with migration of the existing Trac database.

After the Paris sprint (March 22-23), we can look into either starting
to use launchpad/bzr or Enthought/hg/trac, based on the experience
from all of us (Ville and Brian had already started using Launchpad a
few days ago).

If we go with Launchpad, we'll need to look into a way of migrating
the existing Trac database, if possible.  It would be nice to have
closer integration between the bug tracking and the code, though for
now we'll continue to use Trac as is.

So bear with us for a few weeks while the code is a bit all over the
place.  For now, we'll continue to update the regular SVN trunk and
ipython1 branches at ipython.scipy.org, so that those following SVN
don't need to track our madness.

Cheers,

f


From glenn at tarbox.org  Sat Mar  1 23:55:27 2008
From: glenn at tarbox.org (Glenn Tarbox, PhD)
Date: Sat, 1 Mar 2008 20:55:27 -0800
Subject: [IPython-dev] Twisted Qt4 integration...
Message-ID: <f049b35c0803012055j9f52dc3uecf673a53972676c@mail.gmail.com>

As IPython1 uses Twisted and Matplotlib supports Qt, this may be of
interest.

I've got a somewhat functional Qt4 reactor working.  Its maintained and
available at:

http://code.tarbox.org/qtreactor

There is git-daemon and git-instaweb access.  git-instaweb will
auto-generate tarballs for download.

I'm supporting and hosting this code as there are issues arising from the
clash between Twisted's MIT license and Qt's GPL.

-glenn

-- 
Glenn H. Tarbox, PhD
glenn at tarbox.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080301/5ba3c203/attachment.html>

From vivainio at gmail.com  Sun Mar  2 05:19:52 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sun, 2 Mar 2008 12:19:52 +0200
Subject: [IPython-dev] ipython1 on launchpad
In-Reply-To: <db6b5ecc0803011519p1fc41cecr5701747cfdc0ad5@mail.gmail.com>
References: <db6b5ecc0803011519p1fc41cecr5701747cfdc0ad5@mail.gmail.com>
Message-ID: <46cb515a0803020219y4699cab2vaee5db98331b80b1@mail.gmail.com>

On Sun, Mar 2, 2008 at 1:19 AM, Fernando Perez <fperez.net at gmail.com> wrote:

>  we're here at Enthought working on various ipython/scipy/sage related
>  things, and we'll begin testing Launchpad/bzr for the ipython1 work
>  now.   In a few weeks of use for ipython1 this will give us a good
>  amount of real world experience with bzr, and in the meantime
>  Enthought will look at their setup and possibilities regarding hg
>  hosting with migration of the existing Trac database.

In my observation, bzr and launchpad are just fine, and I don't think
you guys will have any problems with it either. Performance is good,
and the merging I have done so far has worked wihouth a hitch. At this
point I'm not even sure whether hg really *is* better than bzr, now
that early bzr performance problems don't exist anymore. hg has
slightly better GUI support on windows, though, but I don't really
care about that.

Summary summarum, I am not sure whether setting up own servers (in
order to host hg) is worth the time and expense, and the results could
be worse (for example, would it be possible for *anyone* to register a
public branch on the custom server, like with Launchpad?)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Sun Mar  2 12:17:21 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 2 Mar 2008 09:17:21 -0800
Subject: [IPython-dev] ipython1 on launchpad
In-Reply-To: <46cb515a0803020219y4699cab2vaee5db98331b80b1@mail.gmail.com>
References: <db6b5ecc0803011519p1fc41cecr5701747cfdc0ad5@mail.gmail.com>
	<46cb515a0803020219y4699cab2vaee5db98331b80b1@mail.gmail.com>
Message-ID: <db6b5ecc0803020917g60147465icd1674c76608772c@mail.gmail.com>

On Sun, Mar 2, 2008 at 2:19 AM, Ville M. Vainio <vivainio at gmail.com> wrote:

>  Summary summarum, I am not sure whether setting up own servers (in
>  order to host hg) is worth the time and expense, and the results could
>  be worse (for example, would it be possible for *anyone* to register a
>  public branch on the custom server, like with Launchpad?)

That's pretty much my take so far on the problem, after looking at
Launchpad a lot yesterday.  We'll just wait for a few weeks (we have
another 2 sprints coming up in March, one on scipy/neuroimaging and
one on ipython in Paris) and sort out the final decision, but so far
from what I see, I've reached the same conclusion as above.  Launchpad
has a lot of good infrastructure, and I doubt it's worth Enthought's
time to redo it.

If we go that  route, it would be great if we could find a way to
migrate our Trac history into their bug system so we keep our tickets.

Cheers,

f


From gael.varoquaux at normalesup.org  Fri Mar  7 14:10:36 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 7 Mar 2008 20:10:36 +0100
Subject: [IPython-dev] Parallel map
Message-ID: <20080307191036.GK14012@phare.normalesup.org>

Hi,

I am trying to do a parallel map using ipython1. Is there a really simple
way to do this, or a tutorial somewhere telling me how? I can probably
figure it out, but I have to dig through a fair amount of
tutorial/doc/wiki articles/reading source code to move forward.

My requirement is that I want the code to be purely valid self sustained
Python code.

IMHO ipython1 should provide a few helper functions, including this one:
a parallel, blocking, map.

Cheers,

Ga?l


From gael.varoquaux at normalesup.org  Fri Mar  7 14:42:26 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 7 Mar 2008 20:42:26 +0100
Subject: [IPython-dev] Parallel map
In-Reply-To: <20080307191036.GK14012@phare.normalesup.org>
References: <20080307191036.GK14012@phare.normalesup.org>
Message-ID: <20080307194226.GM14012@phare.normalesup.org>

On Fri, Mar 07, 2008 at 08:10:36PM +0100, Gael Varoquaux wrote:
> I am trying to do a parallel map using ipython1. Is there a really simple
> way to do this, or a tutorial somewhere telling me how? I can probably
> figure it out, but I have to dig through a fair amount of
> tutorial/doc/wiki articles/reading source code to move forward.

> My requirement is that I want the code to be purely valid self sustained
> Python code.

OK, making some progress at this.

I found out I need to create a MultiEngineClient

rc = client.MultiEngineClient(('127.0.0.1', 10105))

and I can use its map method.

My problem is that I do not know how to create the engines from Python. I
guess I will have to RTFS ipcontroller.

Ga?l


From vivainio at gmail.com  Sat Mar  8 02:07:29 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 8 Mar 2008 09:07:29 +0200
Subject: [IPython-dev] Bzr plugin to check line endings
Message-ID: <46cb515a0803072307q3f3bfee6m77d3ab30f153c2bc@mail.gmail.com>

There now exist a plugin (pre-commit-hook) for bzr that prevents us
from accidentally checking in code with CRLF line endings. Thank you
Alexander!

I've added .checkeol to the root of stable-dev, it will work if you
install the following plugin to your bazaar plugins directory:

http://bazaar.launchpad.net/~bialix/+junk/checkeol

For my Vista machine, the plugins directory is:

C:\Users\ville\AppData\Roaming\bazaar\2.0\plugins

(Gotta love those windows directories...)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From gael.varoquaux at normalesup.org  Sat Mar  8 05:03:29 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 8 Mar 2008 11:03:29 +0100
Subject: [IPython-dev] Parallel map
In-Reply-To: <20080307194226.GM14012@phare.normalesup.org>
References: <20080307191036.GK14012@phare.normalesup.org>
	<20080307194226.GM14012@phare.normalesup.org>
Message-ID: <20080308100329.GA22261@phare.normalesup.org>

On Fri, Mar 07, 2008 at 08:42:26PM +0100, Gael Varoquaux wrote:
> On Fri, Mar 07, 2008 at 08:10:36PM +0100, Gael Varoquaux wrote:
> > I am trying to do a parallel map using ipython1. Is there a really simple
> > way to do this, or a tutorial somewhere telling me how? I can probably
> > figure it out, but I have to dig through a fair amount of
> > tutorial/doc/wiki articles/reading source code to move forward.

> > My requirement is that I want the code to be purely valid self sustained
> > Python code.

> OK, making some progress at this.

> I found out I need to create a MultiEngineClient

> rc = client.MultiEngineClient(('127.0.0.1', 10105))

> and I can use its map method.

I succeeded (I had a good night(s sleep, in between), by piggy backing
the ipcluster script. It is a bit ugly, but I post the code here for
future reference.

What made my task hard was both the fact that there is no obvious way of
creating a cluster from Python, and the fact that ipython1.kernel.api was
suppressed and that all the information I can find on the web uses
ipython1.kerenl.api.RemoteControler.

Now the irony is that I ended up not beeing able to use ipython1 for the
problem I was interested in, as the objects I wanted to send to my
parallel map where not picklable. I wrote a small hack using threading
and os.system to do the work. I suspect this is a limitation people are
going to bump into quite often. Ideas to make a workaround more or less
part of ipython1 natively would be great. In my case, the object I had to
scatter where directly imported from a module, so scattering a module
path as a string (eg 'ipython1.kernel.client.MultiEngineClient') waould 
have been an option. I have no hindsight on these problems, so I don't
pretend suggesting a good solution.

Anyway, thanks for ipython1, keep the good work up, it is a difficult but
import task,

Cheers,

Ga?l

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
"""
Provides a simple parallel map.
"""

# Piggy-back the ipcluster script to start the engines.
from ipython1.kernel.scripts import ipcluster as cluster
from ipython1.kernel.client import MultiEngineClient
from threading import Thread
from time import sleep
import sys

##############################################################################
def guess_ncpu():
    """ Parses /proc/cpuinfo to guess the number of CPU on the box.
        This has been tested only under Linux.
    """
    ncpu = 0
    cpuinfo = file('/proc/cpuinfo')
    for line in cpuinfo.readlines():
        if line[:10]  == 'processor\t':
            ncpu += 1
    return ncpu

##############################################################################
# Code to start the engine and create the controller
def start_cluster(ncpu=guess_ncpu()):
    """ Starts a cluster on the local computer and returns a controller
        to the cluster.
    """
    # We use ipcluster.main, but it takes its instructions from sys.argv,
    # thus we overide it
    orig_argv = sys.argv
    sys.argv = ['foo', '-n', str(ncpu)]
    # Starting the cluster is a blocking operation. We thus need a
    # thread to do the work.
    Thread(target=cluster.main).start()
    # There is a sleep(3) in ipcluster
    sleep(4)
    sys.argv = orig_argv 
    return MultiEngineClient(('127.0.0.1',10105))

##############################################################################
# This code is so trivial you should really use directly the controller
# method if you are going to do anything more than running pmap once
# (keep in mind that there is an overhead of creating the cluster).
def pmap(func, seq, ncpu=guess_ncpu()):
    """ Creates a cluster of ipython1 engines and runs a parallel map on
        it.
    """
    mec = start_cluster(ncpu=ncpu)
    outseq = mec.map(func, seq)
    mec.kill(controler=True)
    return outseq
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++



From fperez.net at gmail.com  Sat Mar  8 05:28:02 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 8 Mar 2008 02:28:02 -0800
Subject: [IPython-dev] Parallel map
In-Reply-To: <20080308100329.GA22261@phare.normalesup.org>
References: <20080307191036.GK14012@phare.normalesup.org>
	<20080307194226.GM14012@phare.normalesup.org>
	<20080308100329.GA22261@phare.normalesup.org>
Message-ID: <db6b5ecc0803080228i4c8ec70p920bf0e3e9b9557f@mail.gmail.com>

On Sat, Mar 8, 2008 at 2:03 AM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:

>  I succeeded (I had a good night(s sleep, in between), by piggy backing
>  the ipcluster script. It is a bit ugly, but I post the code here for
>  future reference.
>
>  What made my task hard was both the fact that there is no obvious way of
>  creating a cluster from Python, and the fact that ipython1.kernel.api was
>  suppressed and that all the information I can find on the web uses
>  ipython1.kerenl.api.RemoteControler.

We need to summarize the recent work done at the sage/scipy/ipython
sprint.  In particular, Min did a lot of excellent work *precisely* on
this issue, most (if not all) of which is already committed, to
provide a full ipython daemon for process control.  This allows you to
do exactly that, to create/control/destroy engines and/or controllers
from within python scripts.

The good thing is that you're already a bzr launchpad team member, so
I'm sure you'll soon be contributing this code and docs :)

>  Now the irony is that I ended up not beeing able to use ipython1 for the
>  problem I was interested in, as the objects I wanted to send to my
>  parallel map where not picklable. I wrote a small hack using threading
>  and os.system to do the work. I suspect this is a limitation people are
>  going to bump into quite often. Ideas to make a workaround more or less
>  part of ipython1 natively would be great. In my case, the object I had to
>  scatter where directly imported from a module, so scattering a module
>  path as a string (eg 'ipython1.kernel.client.MultiEngineClient') waould
>  have been an option. I have no hindsight on these problems, so I don't
>  pretend suggesting a good solution.

We finally figured out a trick using the 'with' statement to allow you
to write code like

with all_engines:
  do_in_remote_engines(..)

or

with task_controller_using_engines(1,3,5):
  do_this()
  and_that()

etc...

so that the control scripts can read much more naturally.  This is
something you may recall Eric Jones and I started thinking about at
Scipy'07, and finally on the plane back from Sage/Scipy Days 8 I was
able to understand how it needed to be structured.  That code isn't
ready for inclusion anywhere yet, but the proof of concept is in my
local copy and hopefully at the Paris sprint we can test it a bit and
put it in.  Being able to write pure-python (instead of the ugly
code-inside-strings) parallel code will make the experience much more
natural, I think, especially coupled with the new facilities for
pushing function objects and the daemon-based ability to do clean
multi-process interactive control.

The pieces are falling in place, now we just need a breather to
document all this and gradually communicate things better to everyone.
 I apologize if the process has been a bit opaque lately, but it's
been just a very hectic time for me.  After the Paris IPython sprint
(March 22-23) I hope to catch my breath a bit, sort out the
distributed VCS situation (bzr/launchpad is looking pretty good so
far), and communicate better a set of clear plans so that all those
who so kindly expressed interest and willingness to pitch in can do so
with an understanding of where the various pieces of the puzzle are.

Cheers,

f


From bialix at ukr.net  Sat Mar  8 05:30:13 2008
From: bialix at ukr.net (Alexander Belchenko)
Date: Sat, 08 Mar 2008 12:30:13 +0200
Subject: [IPython-dev] Bzr plugin to check line endings
In-Reply-To: <46cb515a0803072307q3f3bfee6m77d3ab30f153c2bc@mail.gmail.com>
References: <46cb515a0803072307q3f3bfee6m77d3ab30f153c2bc@mail.gmail.com>
Message-ID: <47D26AB5.3000306@ukr.net>

Ville M. Vainio ?????:
> There now exist a plugin (pre-commit-hook) for bzr that prevents us
> from accidentally checking in code with CRLF line endings. Thank you
> Alexander!
> 
> I've added .checkeol to the root of stable-dev, it will work if you
> install the following plugin to your bazaar plugins directory:
> 
> http://bazaar.launchpad.net/~bialix/+junk/checkeol
> 
> For my Vista machine, the plugins directory is:
> 
> C:\Users\ville\AppData\Roaming\bazaar\2.0\plugins
> 
> (Gotta love those windows directories...)
> 
You also can put it to C:\Program Files\Bazaar\plugins
directory. It's a bit shorter ;-)


From gael.varoquaux at normalesup.org  Sat Mar  8 05:41:13 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 8 Mar 2008 11:41:13 +0100
Subject: [IPython-dev] Parallel map
In-Reply-To: <db6b5ecc0803080228i4c8ec70p920bf0e3e9b9557f@mail.gmail.com>
References: <20080307191036.GK14012@phare.normalesup.org>
	<20080307194226.GM14012@phare.normalesup.org>
	<20080308100329.GA22261@phare.normalesup.org>
	<db6b5ecc0803080228i4c8ec70p920bf0e3e9b9557f@mail.gmail.com>
Message-ID: <20080308104113.GC22261@phare.normalesup.org>

On Sat, Mar 08, 2008 at 02:28:02AM -0800, Fernando Perez wrote:
> We need to summarize the recent work done at the sage/scipy/ipython
> sprint.

I did see some recent commits on Trac :->.

> In particular, Min did a lot of excellent work *precisely* on
> this issue, most (if not all) of which is already committed, to
> provide a full ipython daemon for process control.  This allows you to
> do exactly that, to create/control/destroy engines and/or controllers
> from within python scripts.

Great. Now a very important question: can the daemon be started from
Python without an os.system apparent for the user? I want to be able to
log in a box, and run a pPthon script that does parallel computing with
only a few lines of Python code.

> The good thing is that you're already a bzr launchpad team member, so
> I'm sure you'll soon be contributing this code and docs :)

Hum. Currently I am physicaly killing myself doing way to many things.
No, seriously, its worst than it has ever been, and I have a dangerously
off balance sleep pattern. Maybe I'll get better next year when I start
my new job. Maybe I'll even use ipython1 as a full part of my day work.

> We finally figured out a trick using the 'with' statement to allow you
> to write code like

> with all_engines:
>   do_in_remote_engines(..)

That's really nice. Now a big question is, does that remove the need for
all the objects in the parallel code to be picklable. Because that is
what was limiting me here.

Cheers,

Ga?l



From fperez.net at gmail.com  Sat Mar  8 13:32:26 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 8 Mar 2008 10:32:26 -0800
Subject: [IPython-dev] Parallel map
In-Reply-To: <20080308103958.GB22261@phare.normalesup.org>
References: <20080307191036.GK14012@phare.normalesup.org>
	<20080307194226.GM14012@phare.normalesup.org>
	<20080308100329.GA22261@phare.normalesup.org>
	<db6b5ecc0803080228i4c8ec70p920bf0e3e9b9557f@mail.gmail.com>
	<20080308103958.GB22261@phare.normalesup.org>
Message-ID: <db6b5ecc0803081032k60d7367dh29134062730dba58@mail.gmail.com>

Hey,

On Sat, Mar 8, 2008 at 2:39 AM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> On Sat, Mar 08, 2008 at 02:28:02AM -0800, Fernando Perez wrote:
>  > We need to summarize the recent work done at the sage/scipy/ipython
>  > sprint.
>
>  I did see some recent commits on Trac :->.
>
>
>  > In particular, Min did a lot of excellent work *precisely* on
>  > this issue, most (if not all) of which is already committed, to
>  > provide a full ipython daemon for process control.  This allows you to
>  > do exactly that, to create/control/destroy engines and/or controllers
>  > from within python scripts.
>
>  Great. Now a very important question: can the daemon be started from
>  Python without an os.system apparent for the user? I want to be able to
>  log in a box, and run a pPthon script that does parallel computing with
>  only a few lines of Python code.

Yes, absolutely.  That was a key requirement of the design, that we
could provide fully self-contained examples of ipython parallel code
that don't start by telling the user 'open 4 terminals and type this
in each'... The docs will be catching up soon, likely after the Paris
meeting.

>  > The good thing is that you're already a bzr launchpad team member, so
>  > I'm sure you'll soon be contributing this code and docs :)
>
>  Hum. Currently I am physicaly killing myself doing way to many things.
>  No, seriously, its worst than it has ever been, and I have a dangerously
>  off balance sleep pattern. Maybe I'll get better next year when I start
>  my new job. Maybe I'll even use ipython1 as a full part of my day work.

Well, don't worry: I made you a team member just so we can work in
Paris with you fluidly, and to make it easy for you to prep anything
you may need. I am NOT trying to bury you with more things, and I
realize fullyt that ETS/M2 are higher priorities on this front than
ipython.  Please keep sleep and sanity above open source, really (I
say this after having crashed hard with a migraine yesterday
afternoon, probably from overdosing on work, stress and low sleep
myself, so I really mean it when I say that I want your well-being to
come first).

>  > We finally figured out a trick using the 'with' statement to allow you
>  > to write code like
>
>  > with all_engines:
>  >   do_in_remote_engines(..)
>
>  That's really nice. Now a big question is, does that remove the need for
>  all the objects in the parallel code to be picklable. Because that is
>  what was limiting me here.

Nope, pickle continues to be a requirement, except if you're sending
data over MPI, case in which MPI does the communication.

Cheers

f


From bialix at ukr.net  Sun Mar  9 04:46:03 2008
From: bialix at ukr.net (Alexander Belchenko)
Date: Sun, 09 Mar 2008 10:46:03 +0200
Subject: [IPython-dev] ipython1 on launchpad
In-Reply-To: <db6b5ecc0803020917g60147465icd1674c76608772c@mail.gmail.com>
References: <db6b5ecc0803011519p1fc41cecr5701747cfdc0ad5@mail.gmail.com>	<46cb515a0803020219y4699cab2vaee5db98331b80b1@mail.gmail.com>
	<db6b5ecc0803020917g60147465icd1674c76608772c@mail.gmail.com>
Message-ID: <47D3A3CB.1040403@ukr.net>

Fernando Perez ?????:
> On Sun, Mar 2, 2008 at 2:19 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> 
>>  Summary summarum, I am not sure whether setting up own servers (in
>>  order to host hg) is worth the time and expense, and the results could
>>  be worse (for example, would it be possible for *anyone* to register a
>>  public branch on the custom server, like with Launchpad?)
> 
> That's pretty much my take so far on the problem, after looking at
> Launchpad a lot yesterday.  We'll just wait for a few weeks (we have
> another 2 sprints coming up in March, one on scipy/neuroimaging and
> one on ipython in Paris) and sort out the final decision, but so far
> from what I see, I've reached the same conclusion as above.  Launchpad
> has a lot of good infrastructure, and I doubt it's worth Enthought's
> time to redo it.
> 
> If we go that  route, it would be great if we could find a way to
> migrate our Trac history into their bug system so we keep our tickets.

You need to talk with James Henstridge from Launchpad team about
migrating of Trac tickets to Launchpad bug system.

https://launchpad.net/~jamesh


From fperez.net at gmail.com  Sun Mar  9 04:53:22 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 9 Mar 2008 00:53:22 -0800
Subject: [IPython-dev] ipython1 on launchpad
In-Reply-To: <47D3A3CB.1040403@ukr.net>
References: <db6b5ecc0803011519p1fc41cecr5701747cfdc0ad5@mail.gmail.com>
	<46cb515a0803020219y4699cab2vaee5db98331b80b1@mail.gmail.com>
	<db6b5ecc0803020917g60147465icd1674c76608772c@mail.gmail.com>
	<47D3A3CB.1040403@ukr.net>
Message-ID: <db6b5ecc0803090053ja920aa7hae2f6820a9111ee0@mail.gmail.com>

On Sun, Mar 9, 2008 at 12:46 AM, Alexander Belchenko <bialix at ukr.net> wrote:

>  > If we go that  route, it would be great if we could find a way to
>  > migrate our Trac history into their bug system so we keep our tickets.
>
>  You need to talk with James Henstridge from Launchpad team about
>  migrating of Trac tickets to Launchpad bug system.
>
>  https://launchpad.net/~jamesh

Thanks for the pointer.  It will be a week or two (til after the next
sprint) before we do this, but now we know who to talk to, that's
great.

Regards

f


From fperez.net at gmail.com  Sun Mar  9 14:39:06 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 9 Mar 2008 11:39:06 -0700
Subject: [IPython-dev] bzr problems with attributes
Message-ID: <db6b5ecc0803091139p175434d1jecd2a48d5c17b450@mail.gmail.com>

Hi Ville,

since you started updating code in bzr already, I'm going to leave the
back-merge into SVN trunk to you to avoid conflicts and will push bug
fixes into bzr myself also.  But I just noticed that the bzr branch
was built without the proper execution attributes on all the files
(things like the setup.py scripts aren't executable).  Could you fix
the bzr stable-dev branch?

Thanks

f


From vivainio at gmail.com  Sun Mar  9 14:45:57 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sun, 9 Mar 2008 20:45:57 +0200
Subject: [IPython-dev] bzr problems with attributes
In-Reply-To: <db6b5ecc0803091139p175434d1jecd2a48d5c17b450@mail.gmail.com>
References: <db6b5ecc0803091139p175434d1jecd2a48d5c17b450@mail.gmail.com>
Message-ID: <46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com>

On Sun, Mar 9, 2008 at 8:39 PM, Fernando Perez <fperez.net at gmail.com> wrote:

>  since you started updating code in bzr already, I'm going to leave the
>  back-merge into SVN trunk to you to avoid conflicts and will push bug
>  fixes into bzr myself also.  But I just noticed that the bzr branch
>  was built without the proper execution attributes on all the files
>  (things like the setup.py scripts aren't executable).  Could you fix
>  the bzr stable-dev branch?

Not really, I'm on windows :-)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Sun Mar  9 23:52:37 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 9 Mar 2008 20:52:37 -0700
Subject: [IPython-dev] bzr problems with attributes
In-Reply-To: <46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com>
References: <db6b5ecc0803091139p175434d1jecd2a48d5c17b450@mail.gmail.com>
	<46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com>
Message-ID: <db6b5ecc0803092052h54fe882djcdfd9302f345385b@mail.gmail.com>

On Sun, Mar 9, 2008 at 11:45 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
>
> On Sun, Mar 9, 2008 at 8:39 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>
>  >  since you started updating code in bzr already, I'm going to leave the
>  >  back-merge into SVN trunk to you to avoid conflicts and will push bug
>  >  fixes into bzr myself also.  But I just noticed that the bzr branch
>  >  was built without the proper execution attributes on all the files
>  >  (things like the setup.py scripts aren't executable).  Could you fix
>  >  the bzr stable-dev branch?
>
>  Not really, I'm on windows :-)

OK, no problem.  I'll fix it, but in the future if we need to make new
public branches out of our tree, let me or another *nix person do it,
so we don't have to manually back-fix all this (I know this started as
a bzr experiment so I'm not blaming you or anyone else, just figuring
out best practices from now on).

Cheers,

f


From fperez.net at gmail.com  Mon Mar 10 01:01:41 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 9 Mar 2008 22:01:41 -0700
Subject: [IPython-dev] bzr problems with attributes
In-Reply-To: <db6b5ecc0803092052h54fe882djcdfd9302f345385b@mail.gmail.com>
References: <db6b5ecc0803091139p175434d1jecd2a48d5c17b450@mail.gmail.com>
	<46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com>
	<db6b5ecc0803092052h54fe882djcdfd9302f345385b@mail.gmail.com>
Message-ID: <db6b5ecc0803092201v178a677bpb9fff89008e46729@mail.gmail.com>

On Sun, Mar 9, 2008 at 8:52 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Sun, Mar 9, 2008 at 11:45 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
>  >
>  > On Sun, Mar 9, 2008 at 8:39 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>  >
>  >  >  since you started updating code in bzr already, I'm going to leave the
>  >  >  back-merge into SVN trunk to you to avoid conflicts and will push bug
>  >  >  fixes into bzr myself also.  But I just noticed that the bzr branch
>  >  >  was built without the proper execution attributes on all the files
>  >  >  (things like the setup.py scripts aren't executable).  Could you fix
>  >  >  the bzr stable-dev branch?
>  >
>  >  Not really, I'm on windows :-)
>
>  OK, no problem.  I'll fix it, but in the future if we need to make new
>  public branches out of our tree, let me or another *nix person do it,
>  so we don't have to manually back-fix all this (I know this started as
>  a bzr experiment so I'm not blaming you or anyone else, just figuring
>  out best practices from now on).

OK, after a bit of find/grep this is now fixed on bzr.

Cheers,

f


From fperez.net at gmail.com  Mon Mar 10 01:13:14 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 9 Mar 2008 22:13:14 -0700
Subject: [IPython-dev] About IPython and bzr
In-Reply-To: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com>
References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com>
Message-ID: <db6b5ecc0803092213p545ed937q111a72f603fdb699@mail.gmail.com>

On Tue, Feb 19, 2008 at 10:46 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> My recently-from-scratch installed IPython stable-dev (what used to be
>  called trunk/) is now very easy to check out from launchpad:
>
>  bzr branch lp:ipython
>
>  One note though: NEVER CHECK IN SYMLINKS. I can't stress this enough.
>  Windows bzr does not support symlinks (even in a half-assed way like
>  svn), at least not yet, and I don't know whether such a checkin can be
>  reverted. So essentially, if there is even one checkin with symlink, I
>  would need to delete the branch and check it in again from local copy.

Could you investigate these alternatives and let us know how well they work?

http://bazaar-vcs.org/WindowsSymlinkSupport
https://lists.ubuntu.com/archives/bazaar/2007q1/023801.html

I'm asking because while we can live for now without symlinks on
trunk, it looks like we'll  be using bzr for everything in the future,
and it's likely we may need to use symlinks for certain things.

There are just too many things where symlinks are the right solution,
and it doesn't make sense to avoid it just due to limitations of
win32, so what we need is a good solution for win32 users.

Alexander is on this list too, so perhaps he can provide us with
useful feedback.

Cheers,

f


From bialix at ukr.net  Mon Mar 10 02:53:31 2008
From: bialix at ukr.net (Alexander Belchenko)
Date: Mon, 10 Mar 2008 08:53:31 +0200
Subject: [IPython-dev] bzr problems with attributes
In-Reply-To: <46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com>
References: <db6b5ecc0803091139p175434d1jecd2a48d5c17b450@mail.gmail.com>
	<46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com>
Message-ID: <47D4DAEB.2060004@ukr.net>

Ville M. Vainio ?????:
> On Sun, Mar 9, 2008 at 8:39 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> 
>>  since you started updating code in bzr already, I'm going to leave the
>>  back-merge into SVN trunk to you to avoid conflicts and will push bug
>>  fixes into bzr myself also.  But I just noticed that the bzr branch
>>  was built without the proper execution attributes on all the files
>>  (things like the setup.py scripts aren't executable).  Could you fix
>>  the bzr stable-dev branch?
> 
> Not really, I'm on windows :-)

x-bit plugin is your friend ;-)


From bialix at ukr.net  Mon Mar 10 02:57:59 2008
From: bialix at ukr.net (Alexander Belchenko)
Date: Mon, 10 Mar 2008 08:57:59 +0200
Subject: [IPython-dev] About IPython and bzr
In-Reply-To: <db6b5ecc0803092213p545ed937q111a72f603fdb699@mail.gmail.com>
References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com>
	<db6b5ecc0803092213p545ed937q111a72f603fdb699@mail.gmail.com>
Message-ID: <47D4DBF7.9040206@ukr.net>

Fernando Perez ?????:
> On Tue, Feb 19, 2008 at 10:46 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
>> My recently-from-scratch installed IPython stable-dev (what used to be
>>  called trunk/) is now very easy to check out from launchpad:
>>
>>  bzr branch lp:ipython
>>
>>  One note though: NEVER CHECK IN SYMLINKS. I can't stress this enough.
>>  Windows bzr does not support symlinks (even in a half-assed way like
>>  svn), at least not yet, and I don't know whether such a checkin can be
>>  reverted. So essentially, if there is even one checkin with symlink, I
>>  would need to delete the branch and check it in again from local copy.
> 
> Could you investigate these alternatives and let us know how well they work?
> 
> http://bazaar-vcs.org/WindowsSymlinkSupport
> https://lists.ubuntu.com/archives/bazaar/2007q1/023801.html
> 
> I'm asking because while we can live for now without symlinks on
> trunk, it looks like we'll  be using bzr for everything in the future,
> and it's likely we may need to use symlinks for certain things.
> 
> There are just too many things where symlinks are the right solution,
> and it doesn't make sense to avoid it just due to limitations of
> win32, so what we need is a good solution for win32 users.
> 
> Alexander is on this list too, so perhaps he can provide us with
> useful feedback.

I have special plugin win32symlinks that provide support for fake symlinks on Windows.
At least checkout of branch with symlinks is possible without a hassle.
It works for me very well.

bzr branch lp:bzr-win32symlinks win32symlinks

;-)


From vivainio at gmail.com  Mon Mar 10 03:24:53 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 10 Mar 2008 09:24:53 +0200
Subject: [IPython-dev] About IPython and bzr
In-Reply-To: <47D4DBF7.9040206@ukr.net>
References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com>
	<db6b5ecc0803092213p545ed937q111a72f603fdb699@mail.gmail.com>
	<47D4DBF7.9040206@ukr.net>
Message-ID: <46cb515a0803100024j774da6b7r28cbfb0e9a1da71f@mail.gmail.com>

On Mon, Mar 10, 2008 at 8:57 AM, Alexander Belchenko <bialix at ukr.net> wrote:

>  I have special plugin win32symlinks that provide support for fake symlinks on Windows.
>  At least checkout of branch with symlinks is possible without a hassle.
>  It works for me very well.

Any reason why this is not in core? I did notice that there is a
plugin available, but having to resort to a plugin when such a "core"
functionality is at stake makes me feel a bit uneasy. Is it not stable
yet, or not the "final answer"? Is there a danger it will botch
history or merging?

I still think we should at least try to avoid symlinks in bzr for a
while more, until a definite solution settles down. And besides, if
the symlink is the "right answer", it should be in an area that is not
needed on Windows; obviously, even if you were able to check out the
symlink, it would never actually work in win32.


-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Mon Mar 10 03:27:03 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 10 Mar 2008 00:27:03 -0700
Subject: [IPython-dev] About IPython and bzr
In-Reply-To: <46cb515a0803100024j774da6b7r28cbfb0e9a1da71f@mail.gmail.com>
References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com>
	<db6b5ecc0803092213p545ed937q111a72f603fdb699@mail.gmail.com>
	<47D4DBF7.9040206@ukr.net>
	<46cb515a0803100024j774da6b7r28cbfb0e9a1da71f@mail.gmail.com>
Message-ID: <db6b5ecc0803100027y2f1d0134m62f524bc5baf54b3@mail.gmail.com>

On Mon, Mar 10, 2008 at 12:24 AM, Ville M. Vainio <vivainio at gmail.com> wrote:

>  I still think we should at least try to avoid symlinks in bzr for a
>  while more, until a definite solution settles down. And besides, if
>  the symlink is the "right answer", it should be in an area that is not
>  needed on Windows; obviously, even if you were able to check out the
>  symlink, it would never actually work in win32.

I'm certainly not going to commit any symlinks for now, I just removed
the ChangeLog one so I could use a local symlink on my system.  But I
was thinking of planning ahead, so that if we do need symlinks in the
future for something, we know what the available solutions are.  SVN
didn't do it  elegantly, but at least it (kinda) worked.

Thanks to Alex for the info!

Cheers,

f


From bialix at ukr.net  Mon Mar 10 03:49:17 2008
From: bialix at ukr.net (Alexander Belchenko)
Date: Mon, 10 Mar 2008 09:49:17 +0200
Subject: [IPython-dev] About IPython and bzr
In-Reply-To: <46cb515a0803100024j774da6b7r28cbfb0e9a1da71f@mail.gmail.com>
References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com>	
	<db6b5ecc0803092213p545ed937q111a72f603fdb699@mail.gmail.com>	
	<47D4DBF7.9040206@ukr.net>
	<46cb515a0803100024j774da6b7r28cbfb0e9a1da71f@mail.gmail.com>
Message-ID: <47D4E7FD.8080608@ukr.net>

Ville M. Vainio ?????:
> On Mon, Mar 10, 2008 at 8:57 AM, Alexander Belchenko <bialix at ukr.net> wrote:
> 
>>  I have special plugin win32symlinks that provide support for fake symlinks on Windows.
>>  At least checkout of branch with symlinks is possible without a hassle.
>>  It works for me very well.
> 
> Any reason why this is not in core? I did notice that there is a
> plugin available, but having to resort to a plugin when such a "core"
> functionality is at stake makes me feel a bit uneasy. Is it not stable
> yet, or not the "final answer"? Is there a danger it will botch
> history or merging?

No, only because of performance penalty. `bzr status` will pay extra time
on additional stat call for every file. It costs about 15% overhead
on NTFS filesystem and about 80% on FAT32.

For merging fake symlinks approach into bzr core I need to rewrite
standard os.listdir function to allow return not only file names,
but stat info too. This should improve overall performance of walking
across working tree even without fake symlinks support.

So I'd like to call my plugin as "not final answer".

> I still think we should at least try to avoid symlinks in bzr for a
> while more, until a definite solution settles down. And besides, if
> the symlink is the "right answer", it should be in an area that is not
> needed on Windows; obviously, even if you were able to check out the
> symlink, it would never actually work in win32.

I'm not familiar with svn solution for symlinks on Windows, but my
fake symlinks is not real symlinks, so you should not use it for pointing
to some python package and expect that python import will work fine.

But I follow symlinks format from Cygwin so at least it should appear
as "real" symlinks in Cygwin environment. Details of this format could
be found in plugin source.


From vivainio at gmail.com  Mon Mar 10 14:53:08 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 10 Mar 2008 20:53:08 +0200
Subject: [IPython-dev] Macros don't work in threaded shells
Message-ID: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com>

It appears recursive execution of _ip.runlines() is not possible with
threaded shells (and so e.g. macros hang ipython) - noticed this
through ticket #238, which is now a blocker.

Try:

ipython -q4thread

> _ip.runlines('1')

This will hang IPython.

First, we come to MTInteractiveShell.runsource(""" _ip.runlines('1')""")

This is given to runcode, which then does runsource( "1"). This never
gets to runcode, because the thread is waiting for  _ip.runlines('1')
to complete.

According to this comment, it seems this scenario has been considered,
but the logic does not seem to work:

       # Case 3
        # Store code in queue, so the execution thread can handle it.

        # Note that with macros and other applications, we MAY re-enter this
        # section, so we have to acquire the lock with non-blocking semantics,
        # else we deadlock.
        got_lock = self.thread_ready.acquire()
        self.code_queue.put(code)
        if got_lock:
           self.thread_ready.wait()  # Wait until processed in timeout interval
        else:
            print "nolock",source
        self.thread_ready.release()

(and no, using got_lock = self.thread_ready.acquire(blocking=False)
won't fix it).

Fernando? You remember whether this used to work?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Mon Mar 10 15:17:08 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 10 Mar 2008 12:17:08 -0700
Subject: [IPython-dev] Macros don't work in threaded shells
In-Reply-To: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com>
References: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com>
Message-ID: <db6b5ecc0803101217w32562da1p41effcce6fe33a64@mail.gmail.com>

On Mon, Mar 10, 2008 at 11:53 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> It appears recursive execution of _ip.runlines() is not possible with
>  threaded shells (and so e.g. macros hang ipython) - noticed this
>  through ticket #238, which is now a blocker.
>
>  Try:
>
>  ipython -q4thread
>
>  > _ip.runlines('1')
>
>  This will hang IPython.
>
>  First, we come to MTInteractiveShell.runsource(""" _ip.runlines('1')""")
>
>  This is given to runcode, which then does runsource( "1"). This never
>  gets to runcode, because the thread is waiting for  _ip.runlines('1')
>  to complete.
>
>  According to this comment, it seems this scenario has been considered,
>  but the logic does not seem to work:
>
>        # Case 3
>         # Store code in queue, so the execution thread can handle it.
>
>         # Note that with macros and other applications, we MAY re-enter this
>         # section, so we have to acquire the lock with non-blocking semantics,
>         # else we deadlock.
>         got_lock = self.thread_ready.acquire()
>         self.code_queue.put(code)
>         if got_lock:
>            self.thread_ready.wait()  # Wait until processed in timeout interval
>         else:
>             print "nolock",source
>         self.thread_ready.release()
>
>  (and no, using got_lock = self.thread_ready.acquire(blocking=False)
>  won't fix it).
>
>  Fernando? You remember whether this used to work?

Mmh, honestly I don't.  This is the problem with not having had a
solid test suite ever, we can't even track regressions reliably.

That code did change recently:

http://projects.scipy.org/ipython/ipython/ticket/210
http://projects.scipy.org/ipython/ipython/ticket/212

and those changes did fix existing deadlocks, but they may have
introduced new ones, or simply exposed other latent ones.  I'm afraid
by now I've shifted my attention enough away from this code that I'd
rather just push on the new architecture and leave these in as is,
unless someone (marc?) can contribute a good fix.  All new code we're
writing is nicely factored out and properly tested, and I've finally
made a mental shift to the new codebase to the point where I prefer
not to work on back fixes unless they are quick and easy.
Unfortunately 'quick and easy' does most definitely NOT apply to the
threaded logic for all the MT shells :(

In summary, if a good samaritan pitches in  with a fix I'd be very
grateful, but my focus now is 99% on the new code (quick fixes like
last night's l?? one aside).

Cheers,

f


From ellisonbg.net at gmail.com  Mon Mar 10 15:47:49 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Mon, 10 Mar 2008 13:47:49 -0600
Subject: [IPython-dev] About IPython and bzr
In-Reply-To: <db6b5ecc0803092213p545ed937q111a72f603fdb699@mail.gmail.com>
References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com>
	<db6b5ecc0803092213p545ed937q111a72f603fdb699@mail.gmail.com>
Message-ID: <6ce0ac130803101247s4355a09dv714d86fd9f24d26e@mail.gmail.com>

>  >  One note though: NEVER CHECK IN SYMLINKS. I can't stress this enough.
>  >  Windows bzr does not support symlinks (even in a half-assed way like
>  >  svn), at least not yet, and I don't know whether such a checkin can be
>  >  reverted. So essentially, if there is even one checkin with symlink, I
>  >  would need to delete the branch and check it in again from local copy.
>
>  Could you investigate these alternatives and let us know how well they work?
>
>  http://bazaar-vcs.org/WindowsSymlinkSupport
>  https://lists.ubuntu.com/archives/bazaar/2007q1/023801.html
>
>  I'm asking because while we can live for now without symlinks on
>  trunk, it looks like we'll  be using bzr for everything in the future,
>  and it's likely we may need to use symlinks for certain things.

Honest question: I would never have considered even wanting to put
symlinks in a repo.  Is there a real need for them inside a repo?


From vivainio at gmail.com  Mon Mar 10 15:48:45 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 10 Mar 2008 21:48:45 +0200
Subject: [IPython-dev] Macros don't work in threaded shells
In-Reply-To: <db6b5ecc0803101217w32562da1p41effcce6fe33a64@mail.gmail.com>
References: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com>
	<db6b5ecc0803101217w32562da1p41effcce6fe33a64@mail.gmail.com>
Message-ID: <46cb515a0803101248x4e7df927v1f1e14a190f85da4@mail.gmail.com>

On Mon, Mar 10, 2008 at 9:17 PM, Fernando Perez <fperez.net at gmail.com> wrote:

>  and those changes did fix existing deadlocks, but they may have
>  introduced new ones, or simply exposed other latent ones.  I'm afraid
>  by now I've shifted my attention enough away from this code that I'd
>  rather just push on the new architecture and leave these in as is,
>  unless someone (marc?) can contribute a good fix.  All new code we're

I'm not totally lost on this - I can just raise a flag when we are
about the *really* execute the code, to signal that we are now in
thread that should be executing the code.

Just wanted to make sure you didn't have sudden "aha" moment.

I'll try my fix - it's up to other people (that really stress this
threaded shell) to determine how well it works, as I don't really use
these MT shells myself.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Mon Mar 10 16:40:59 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 10 Mar 2008 22:40:59 +0200
Subject: [IPython-dev] Macros don't work in threaded shells
In-Reply-To: <46cb515a0803101248x4e7df927v1f1e14a190f85da4@mail.gmail.com>
References: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com>
	<db6b5ecc0803101217w32562da1p41effcce6fe33a64@mail.gmail.com>
	<46cb515a0803101248x4e7df927v1f1e14a190f85da4@mail.gmail.com>
Message-ID: <46cb515a0803101340k3c9c1c43ya4c857103f22b730@mail.gmail.com>

On Mon, Mar 10, 2008 at 9:48 PM, Ville M. Vainio <vivainio at gmail.com> wrote:

>  I'm not totally lost on this - I can just raise a flag when we are
>  about the *really* execute the code, to signal that we are now in
>  thread that should be executing the code.

I now have a quite straightforward fix for this (that should not break
anything at least):

http://bazaar.launchpad.net/~ipython/ipython/stable-dev/revision/48

Please try it out, you multithreading parts of the IPython society
:-). Try running _ip.runlines("1+1") on your preferred threading
solution.

However, with these alterations, I think the condition variable and
'got_lock' is an unnecessary complication... but this code is too
brittle to play around too much, perhaps.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Mon Mar 10 17:55:23 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 10 Mar 2008 23:55:23 +0200
Subject: [IPython-dev] Macros don't work in threaded shells
In-Reply-To: <46cb515a0803101340k3c9c1c43ya4c857103f22b730@mail.gmail.com>
References: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com>
	<db6b5ecc0803101217w32562da1p41effcce6fe33a64@mail.gmail.com>
	<46cb515a0803101248x4e7df927v1f1e14a190f85da4@mail.gmail.com>
	<46cb515a0803101340k3c9c1c43ya4c857103f22b730@mail.gmail.com>
Message-ID: <46cb515a0803101455t4bca9df4ycbacd856572956a@mail.gmail.com>

On Mon, Mar 10, 2008 at 10:40 PM, Ville M. Vainio <vivainio at gmail.com> wrote:

>  However, with these alterations, I think the condition variable and
>  'got_lock' is an unnecessary complication... but this code is too
>  brittle to play around too much, perhaps.

Though I also figure making it simpler would reduce the danger of
deadlocks and other oddities in the future. All we really need is the
following structure:

- runsource either

1) executes the code directly (if we are in worker thread, i.e. what I just did)

or

2) Puts (code, event: threading.Event)  tuple in queue, and does
even.wait() before it returns

Runcode runs code and does event.set()

The Queue structure really does all the rest, and I can't see any
variables in MTInteractiveShell that need protecting (through using
Condition instead of Event) from re-entrant use, esp. since runcode
always runs in the same thread. Essentially, the Condition variable
now duplicates functionality now already in Queue (in inferior fashion
- queue is first come, first served, if there were ipython commands
being executed from multiple threads) and makes it harder to
understand (I think the condition variable in general is a bit hard to
grasp, compared to other threading primitives).

Thoughs? Perhaps this is radical enough to be experimented in a
separate bzr branch?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Mon Mar 10 19:01:23 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 10 Mar 2008 16:01:23 -0700
Subject: [IPython-dev] About IPython and bzr
In-Reply-To: <6ce0ac130803101247s4355a09dv714d86fd9f24d26e@mail.gmail.com>
References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com>
	<db6b5ecc0803092213p545ed937q111a72f603fdb699@mail.gmail.com>
	<6ce0ac130803101247s4355a09dv714d86fd9f24d26e@mail.gmail.com>
Message-ID: <db6b5ecc0803101601w2e0ec508s3b67b1e234d0a2ef@mail.gmail.com>

On Mon, Mar 10, 2008 at 12:47 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
> >  >  One note though: NEVER CHECK IN SYMLINKS. I can't stress this enough.
>  >  >  Windows bzr does not support symlinks (even in a half-assed way like
>  >  >  svn), at least not yet, and I don't know whether such a checkin can be
>  >  >  reverted. So essentially, if there is even one checkin with symlink, I
>  >  >  would need to delete the branch and check it in again from local copy.
>  >
>  >  Could you investigate these alternatives and let us know how well they work?
>  >
>  >  http://bazaar-vcs.org/WindowsSymlinkSupport
>  >  https://lists.ubuntu.com/archives/bazaar/2007q1/023801.html
>  >
>  >  I'm asking because while we can live for now without symlinks on
>  >  trunk, it looks like we'll  be using bzr for everything in the future,
>  >  and it's likely we may need to use symlinks for certain things.
>
>  Honest question: I would never have considered even wanting to put
>  symlinks in a repo.  Is there a real need for them inside a repo?

We had the ChangeLog in so that automatic tools could find it by
walking only up the source treee (Emacs does that, I don't know if vi
does it also).  It's certainly not common enough of a need to warrant
losing much sleep over it, and for now we can all just remember NOT to
commit symlinks.  The fact that the discussion exists for bzr is a
good indicator that the need does come up every now and then, but for
us it's not something we should worry too much about.  If we ever do
have a real need for a committed symlink, we'll look into it.

Cheers

f


From fperez.net at gmail.com  Mon Mar 10 19:04:12 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 10 Mar 2008 16:04:12 -0700
Subject: [IPython-dev] Macros don't work in threaded shells
In-Reply-To: <46cb515a0803101455t4bca9df4ycbacd856572956a@mail.gmail.com>
References: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com>
	<db6b5ecc0803101217w32562da1p41effcce6fe33a64@mail.gmail.com>
	<46cb515a0803101248x4e7df927v1f1e14a190f85da4@mail.gmail.com>
	<46cb515a0803101340k3c9c1c43ya4c857103f22b730@mail.gmail.com>
	<46cb515a0803101455t4bca9df4ycbacd856572956a@mail.gmail.com>
Message-ID: <db6b5ecc0803101604w402d1649s3b2b532cc4455757@mail.gmail.com>

On Mon, Mar 10, 2008 at 2:55 PM, Ville M. Vainio <vivainio at gmail.com> wrote:
> On Mon, Mar 10, 2008 at 10:40 PM, Ville M. Vainio <vivainio at gmail.com> wrote:
>
>  >  However, with these alterations, I think the condition variable and
>  >  'got_lock' is an unnecessary complication... but this code is too
>  >  brittle to play around too much, perhaps.

[...]

>  Thoughs? Perhaps this is radical enough to be experimented in a
>  separate bzr branch?

Thanks for spending some time on it!  Because this code is so
difficult to debug and test, it's a good idea to put it in a bzr
branch.  After all, we said we wanted a DVCS to make this type of
development model easier, might as well use it :)

We might find some time at the Paris sprint to audit this code, and
Laurent will be there (he's interested in the WX support), so a branch
is perfect to have it waiting til next week.  If in the meantime Marc
(who contributed the #210 and #212 fixes) has some feedback, that
would be great.

Cheers,

f


From bialix at ukr.net  Mon Mar 10 19:11:15 2008
From: bialix at ukr.net (Alexander Belchenko)
Date: Tue, 11 Mar 2008 01:11:15 +0200
Subject: [IPython-dev] About IPython and bzr
In-Reply-To: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com>
References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com>
Message-ID: <47D5C013.7070702@ukr.net>

Ville M. Vainio ?????:
> My recently-from-scratch installed IPython stable-dev (what used to be
> called trunk/) is now very easy to check out from launchpad:
> 
> bzr branch lp:ipython
> 
> One note though: NEVER CHECK IN SYMLINKS. I can't stress this enough.
> Windows bzr does not support symlinks (even in a half-assed way like
> svn), at least not yet, and I don't know whether such a checkin can be
> reverted.

`bzr uncommit` allows to uncommit several revisions.
Or you can use `bzr branch xxx yyy -r N`, where N -- revision before
checkin with symlink.

> So essentially, if there is even one checkin with symlink, I
> would need to delete the branch and check it in again from local copy.
> 



From tocer.deng at gmail.com  Wed Mar 12 05:46:56 2008
From: tocer.deng at gmail.com (tocer)
Date: Wed, 12 Mar 2008 17:46:56 +0800
Subject: [IPython-dev] can't %edit python statement inclued non-ascii charset
Message-ID: <47D7A690.1040601@gmail.com>

hi:

I typed a python statement inclued non_ascii charset,such as:

In [1] : s = '??'   # it's a Chinses string

when I wanted to edit the statement again by typing edit command:

In [2]: edit In[1]

a error occured:

UnicodeEncodeError: 'ascii' codec can't encode characters in
position 5-6: ordinal not in range(128)

how could I make the mistake disapper.

btw:my box is ubuntu7.04, python2.5, ipython8.2

thanks.

--tocer




From vivainio at gmail.com  Wed Mar 12 07:25:35 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 12 Mar 2008 13:25:35 +0200
Subject: [IPython-dev] can't %edit python statement inclued non-ascii
	charset
In-Reply-To: <47D7A690.1040601@gmail.com>
References: <47D7A690.1040601@gmail.com>
Message-ID: <46cb515a0803120425r1b6836cr4855bdbecee367a7@mail.gmail.com>

2008/3/12 tocer <tocer.deng at gmail.com>:

>  I typed a python statement inclued non_ascii charset,such as:
>
>  In [1] : s = '??'   # it's a Chinses string
>
>  when I wanted to edit the statement again by typing edit command:
>
>  In [2]: edit In[1]
>
>  a error occured:
>
>  UnicodeEncodeError: 'ascii' codec can't encode characters in
>  position 5-6: ordinal not in range(128)
>
>  how could I make the mistake disapper.

The patent answer with unicode errors is, don't do that ;-).

The stable IPython neglects unicode on various points, and changing
things will cause other parts to break. If you want to provide a
solution, create a new branch on bzr about it and try to fix it in
Magic.py -> magic_edit. This typically involves encoding/decoding
to/from stdin_encoding (grep iplib.py for that to see how it works).
However, it's not certain such a fix will be merged into trunk, due to
regression risks and the current stability requirements of ipython
trunk.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'

From michael.j.mclay at gmail.com  Wed Mar 12 08:22:06 2008
From: michael.j.mclay at gmail.com (Michael McLay)
Date: Wed, 12 Mar 2008 08:22:06 -0400
Subject: [IPython-dev] can't %edit python statement inclued non-ascii
	charset
In-Reply-To: <47D7A690.1040601@gmail.com>
References: <47D7A690.1040601@gmail.com>
Message-ID: <683f83340803120522j5c66379w2a0cd2032ceea665@mail.gmail.com>

The error is triggered when the edit command writes the string to a
temporary file using the builtin open function. The codecs.open
function will write a non-ascii character to a file as the following
shows.

Python 2.5.1 (r251:54863, May  2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> s = u'\245\200'
>>> open('test.uraw', 'w').write(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode characters in position
0-1: ordinal not in range(128)
>>> import codecs
>>> codecs.open('test.uraw', 'w',encoding='utf-8').write(s)
>>>


Andrew Kuchling wrote a very informative tutorial on using Unicode in Python.

    http://www.amk.ca/python/howto/unicode



2008/3/12 tocer <tocer.deng at gmail.com>:
> hi:
>
>  I typed a python statement inclued non_ascii charset,such as:
>
>  In [1] : s = '??'   # it's a Chinses string
>
>  when I wanted to edit the statement again by typing edit command:
>
>  In [2]: edit In[1]
>
>  a error occured:
>
>  UnicodeEncodeError: 'ascii' codec can't encode characters in
>  position 5-6: ordinal not in range(128)
>
>  how could I make the mistake disapper.
>
>  btw:my box is ubuntu7.04, python2.5, ipython8.2
>
>  thanks.
>
>  --tocer
>
>
>  _______________________________________________
>  IPython-dev mailing list
>  IPython-dev at scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>

From vivainio at gmail.com  Wed Mar 12 08:35:16 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 12 Mar 2008 14:35:16 +0200
Subject: [IPython-dev] can't %edit python statement inclued non-ascii
	charset
In-Reply-To: <683f83340803120522j5c66379w2a0cd2032ceea665@mail.gmail.com>
References: <47D7A690.1040601@gmail.com>
	<683f83340803120522j5c66379w2a0cd2032ceea665@mail.gmail.com>
Message-ID: <46cb515a0803120535v1a7dad96h8732d42483b23706@mail.gmail.com>

2008/3/12 Michael McLay <michael.j.mclay at gmail.com>:
> The error is triggered when the edit command writes the string to a
>  temporary file using the builtin open function. The codecs.open
>  function will write a non-ascii character to a file as the following
>  shows.

Ok, then it's probably easy / non-disruptive fix and worth doing.
Thanks, I will look into this.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Thu Mar 13 09:55:49 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 13 Mar 2008 15:55:49 +0200
Subject: [IPython-dev] bzr GUI
Message-ID: <46cb515a0803130655j642d57f8g828ace2d7c7aa631@mail.gmail.com>

In case someone is looking for a good bzr gui, I recommend QBzr. It's
very easy to set up (also on windows), and at least qlog and qannotate
(my main use cases) seem to  work fine.

BTW, if you are running ipython stable-dev branch and sh profile, bzr
completer has now been enhanced so that bzr <tab> actually gets the
commands from "bzr help commands", so you can see what is available in
qbzr by "bzr q<tab>"

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Thu Mar 13 12:41:53 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 13 Mar 2008 09:41:53 -0700
Subject: [IPython-dev] bzr GUI
In-Reply-To: <46cb515a0803130655j642d57f8g828ace2d7c7aa631@mail.gmail.com>
References: <46cb515a0803130655j642d57f8g828ace2d7c7aa631@mail.gmail.com>
Message-ID: <db6b5ecc0803130941g1c38982fj57ec4523a0ef0ca9@mail.gmail.com>

On Thu, Mar 13, 2008 at 6:55 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> In case someone is looking for a good bzr gui, I recommend QBzr. It's
>  very easy to set up (also on windows), and at least qlog and qannotate
>  (my main use cases) seem to  work fine.

Thanks for the tip.  Another nice one, that's available out of the box
in ubuntu:

apt-get install bzr-gtk

The actual binary it provides is called 'olive-gtk'.  Seems handy so
far, especially for viewing the branching history, which is something
that a text representation isn't very good at displaying.

Cheers,

f


From michael.j.mclay at gmail.com  Thu Mar 13 16:48:41 2008
From: michael.j.mclay at gmail.com (Michael McLay)
Date: Thu, 13 Mar 2008 16:48:41 -0400
Subject: [IPython-dev] bzr GUI
In-Reply-To: <db6b5ecc0803130941g1c38982fj57ec4523a0ef0ca9@mail.gmail.com>
References: <46cb515a0803130655j642d57f8g828ace2d7c7aa631@mail.gmail.com>
	<db6b5ecc0803130941g1c38982fj57ec4523a0ef0ca9@mail.gmail.com>
Message-ID: <683f83340803131348i31a71abbq7c553e4ee676ca7e@mail.gmail.com>

I use meld on Ubuntu to look at diffs. It supports bzr.

On Thu, Mar 13, 2008 at 12:41 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Thu, Mar 13, 2008 at 6:55 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
>  > In case someone is looking for a good bzr gui, I recommend QBzr. It's
>  >  very easy to set up (also on windows), and at least qlog and qannotate
>  >  (my main use cases) seem to  work fine.
>
>  Thanks for the tip.  Another nice one, that's available out of the box
>  in ubuntu:
>
>  apt-get install bzr-gtk
>
>  The actual binary it provides is called 'olive-gtk'.  Seems handy so
>  far, especially for viewing the branching history, which is something
>  that a text representation isn't very good at displaying.
>
>  Cheers,
>
>
>
>  f
>  _______________________________________________
>  IPython-dev mailing list
>  IPython-dev at scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From glenn at tarbox.org  Tue Mar 18 14:35:38 2008
From: glenn at tarbox.org (Glenn H Tarbox, PhD)
Date: Tue, 18 Mar 2008 11:35:38 -0700
Subject: [IPython-dev] the state if IPython development...
Message-ID: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org>

I'm having a bit of trouble evaluating various threads of ipython
development and am wondering if I'm missing a comms channel.  I have
various problems which are solveable but don't know which elements of
IPython are going to be refactored away in favor of the IPython1
architecture.

IPython development has moved, somewhat, to launchpad.  I say somewhat
both from messages on this list and the "bleeding edge" installation
instructions:

> bzr co --lightweight lp:ipython
> svn co http://ipython.scipy.org/svn/ipython/pyreadline/trunk pyreadline

although a "unified" tarball is downloadable from:
http://ipython.scipy.org/dist/nightly_svn

but, I had trouble with the pyreadline build directly from svn:

>>>>
copying pyreadline/modes/__init__.py -> build/lib/pyreadline/modes
copying pyreadline/modes/notemacs.py -> build/lib/pyreadline/modes
package init file 'pyreadline/test/__init__.py' not found (or not a
regular file)
<<<<

I didn't drill down completely but this looks like something inside the
package.  So, I guess I don't know whether pyreadline within the IPython
svn pull is ok, whether one needs to pull both, is there something
strange going on when one already has ipython installed?... and don't
know whether I should spend any time on this because I can't tell whats
OBE.

More concerning is the state of Qt4.  There is a "critical" bug entered
in trac: http://projects.scipy.org/ipython/ipython/ticket/208

Clearly, a QCoreApplication (or QApplication) instance is
auto-constructed due to the -q4thread switch... and since calculate.py
constructs one also, QT dumps core  (why QT decides to core-dump as the
error reporting strategy is beyond me... but thats whats going on).

Qt4 support is indeed broken... but unnecessarily so. I have Qt4 running
with twisted in a naked ipython shell.  I simply construct my qt4reactor
and use embedded Qt4 code.  Can't pull in pylab because it renames
packages which collide with PyQt4 (in particular, wrapping QObject in a
different module... don't get that yet).  and this is beyond the issues
associated with QSocketNotifier being invoked from a non-qt thread.  I'm
sure this is straightforward but, again, I can't determine whether time
spent here is worthwhile.  Seems to me that IPython1's architecture will
make most of these problems "just go away" as its a far more
comprehensive architecture.

(BTW: the twisted qt reactor can be retrieved from
http://noc2.tarbox.org:8080/?p=qtreactor/.git;a=summary )

On a related point, I think we're headed for trouble with wx.  I notice
that its included in IPython1... but the twisted wxreactor is a bit
broken.  I think its easy to fix, but the reason its broken is because
wx was poorly behaved when the wxreactor was developed (stopped
servicing timers, IO notification during modal windows etc) and the
twisted folks used threading to get around the difficulty.  Nothing
wrong with that, other than its ugly (and will cause other threadosity
issues) but the code doesn't work properly (fails tests, I support the
buildbot) and I wonder whether its worth fixing or whether its easier to
get wx to behave (I can drill down into the issues here with anyone who
wants but I won't digress further here).

Also, I note that there's virtually no traffic on freenode, at least not
on #scipy (am I missing a channel?)  Seems to me that the IPython1
development activity would be greatly served by greater irc activity.

Finally, I support 4 buildbots for twisted.  I'm willing to provide
these same assets for IPython1 development (each server has dual - 4
core processors = 8 processors / server totaling 32 processors).  How to
properly use this setup for cluster testing would need to be discussed.

-glenn

-- 
Glenn H. Tarbox, PhD
glenn at tarbox.org



From vivainio at gmail.com  Tue Mar 18 16:15:13 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 18 Mar 2008 22:15:13 +0200
Subject: [IPython-dev] the state if IPython development...
In-Reply-To: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org>
References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org>
Message-ID: <46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com>

On Tue, Mar 18, 2008 at 8:35 PM, Glenn H Tarbox, PhD <glenn at tarbox.org> wrote:

>  IPython development has moved, somewhat, to launchpad.  I say somewhat
>  both from messages on this list and the "bleeding edge" installation
>  instructions:
>
>  > bzr co --lightweight lp:ipython
>  > svn co http://ipython.scipy.org/svn/ipython/pyreadline/trunk pyreadline
>
>  although a "unified" tarball is downloadable from:
>  http://ipython.scipy.org/dist/nightly_svn

That tarball only has svn, which has been pretty much discontinued (we
will just dump a bzr snapshot there, for curiosity's sake, but you
shouldn't count on the svn anymore).

>  More concerning is the state of Qt4.  There is a "critical" bug entered
>  in trac: http://projects.scipy.org/ipython/ipython/ticket/208
>
>  Clearly, a QCoreApplication (or QApplication) instance is
>  auto-constructed due to the -q4thread switch... and since calculate.py
>  constructs one also, QT dumps core  (why QT decides to core-dump as the
>  error reporting strategy is beyond me... but thats whats going on).
>
>  Qt4 support is indeed broken... but unnecessarily so. I have Qt4 running
>  with twisted in a naked ipython shell.  I simply construct my qt4reactor
>  and use embedded Qt4 code.  Can't pull in pylab because it renames
>  packages which collide with PyQt4 (in particular, wrapping QObject in a
>  different module... don't get that yet).  and this is beyond the issues
>  associated with QSocketNotifier being invoked from a non-qt thread.  I'm
>  sure this is straightforward but, again, I can't determine whether time
>  spent here is worthwhile.  Seems to me that IPython1's architecture will
>  make most of these problems "just go away" as its a far more
>  comprehensive architecture.

IPython1 does not have a magical solution for any of this, it's pretty
universal stuff. The problem is still the same - gui mainloop has to
be in one thread - but we have that already. Whatever you do in
ipython0 regarding Qt4 will be applicable on IPython1 as well, and
IPython0 isn't going away anytime soon.

Have you considered putting up a bzr branch on launchpad for this
stuff? Darren has been most active regarding Qt4, and we could  use
more hands on this.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From glenn at tarbox.org  Tue Mar 18 17:05:06 2008
From: glenn at tarbox.org (Glenn H Tarbox, PhD)
Date: Tue, 18 Mar 2008 14:05:06 -0700
Subject: [IPython-dev] the state if IPython development...
In-Reply-To: <46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com>
References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com>
Message-ID: <1205874306.17683.158.camel@ghtmyth.hq.tarbox.org>

On Tue, 2008-03-18 at 22:15 +0200, Ville M. Vainio wrote:
> On Tue, Mar 18, 2008 at 8:35 PM, Glenn H Tarbox, PhD
<glenn at tarbox.org> wrote:

> That tarball only has svn, which has been pretty much discontinued (we
> will just dump a bzr snapshot there, for curiosity's sake, but you
> shouldn't count on the svn anymore).

we should probably indicate this somewhere.  There is very little
information being disseminated WRT where one should go for the bleeding
edge (which is the only place I feel comfortable :-)

> 
> >  More concerning is the state of Qt4.  There is a "critical" bug
entered
> >  in trac: http://projects.scipy.org/ipython/ipython/ticket/208
> >
> >  Clearly, a QCoreApplication (or QApplication) instance is
> >  auto-constructed due to the -q4thread switch... and since
calculate.py
> >  constructs one also, QT dumps core  (why QT decides to core-dump as
the
> >  error reporting strategy is beyond me... but thats whats going on).
> >
> >  Qt4 support is indeed broken... but unnecessarily so. I have Qt4
running
> >  with twisted in a naked ipython shell.  I simply construct my
qt4reactor
> >  and use embedded Qt4 code.  Can't pull in pylab because it renames
> >  packages which collide with PyQt4 (in particular, wrapping QObject
in a
> >  different module... don't get that yet).  and this is beyond the
issues
> >  associated with QSocketNotifier being invoked from a non-qt thread.
I'm
> >  sure this is straightforward but, again, I can't determine whether
time
> >  spent here is worthwhile.  Seems to me that IPython1's architecture
will
> >  make most of these problems "just go away" as its a far more
> >  comprehensive architecture.
> 
> IPython1 does not have a magical solution for any of this, it's pretty
> universal stuff. The problem is still the same - gui mainloop has to
> be in one thread - but we have that already. Whatever you do in
> ipython0 regarding Qt4 will be applicable on IPython1 as well, and
> IPython0 isn't going away anytime soon.

Actually, this is fairly straightforward with Qt and, I believe, the
other gui mainloops.  Since you're embracing twisted, I recommend adding
stdin and stdout to the event notification hooks and you're mostly done.
This is what I'm (kinda) doing... where things fall down a bit is when
IPython grabs the loop during scrolling (for example when reading the
docs which spew with a '??'.)  Other than that, it just kinda works.

Of course, I believe this is part of the new / better / different
IPython1 design.  

Since IPython0 is gonna hang around we need to decide whether to work
deep inside to address this issue.  I previously posted a workaround to
nail twisted to IPython0 which exploited the multithreaded IShell code.

So, on this issue, we need a strategy.  I certainly appreciate continued
support for IPython0, but given the limited resources I wonder how much
effort should be put toward fixing the various GUI loops.  

As it stands, Qt4 is currently broken with IPython0 running pylab.  The
decision to expend effort needs to be based on the development plan /
schedule.  This is something I have no insight into as there is no
chatter I have access to.

and I'll say again, you're in the soup with wx.  It might work for a
while, but its broken and needs work.  I looked at it with the twisted
team but decided to just bag it and go Qt.  In that effort, it became
clear how to fix wx which is when I became aware of wx's previous issues
which may or may not still exist.  The use of threads to get around wx
blocking is, IMHO, a bad idea...  much better to fix wx... and if that
can't be done, it should be bagged or someone needs to take on that
project, certainly beyond my time availability.

> 
> Have you considered putting up a bzr branch on launchpad for this
> stuff? Darren has been most active regarding Qt4, and we could  use
> more hands on this.

Love to... but this further emphasizes my point.  I have no idea whats
going on WRT development of the various flavors of IPython.  I have a
feeling I'm not the only one without the necessary visibility.

> 
-- 
Glenn H. Tarbox, PhD
glenn at tarbox.org






From fperez.net at gmail.com  Tue Mar 18 19:26:00 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 18 Mar 2008 16:26:00 -0700
Subject: [IPython-dev] the state if IPython development...
In-Reply-To: <1205874306.17683.158.camel@ghtmyth.hq.tarbox.org>
References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com>
	<1205874306.17683.158.camel@ghtmyth.hq.tarbox.org>
Message-ID: <db6b5ecc0803181626l4126f0f4i46d1135c51d8e65@mail.gmail.com>

Hi Glenn,

On Tue, Mar 18, 2008 at 2:05 PM, Glenn H Tarbox, PhD <glenn at tarbox.org> wrote:
> On Tue, 2008-03-18 at 22:15 +0200, Ville M. Vainio wrote:
>  > On Tue, Mar 18, 2008 at 8:35 PM, Glenn H Tarbox, PhD
>  <glenn at tarbox.org> wrote:
>
>
> > That tarball only has svn, which has been pretty much discontinued (we
>  > will just dump a bzr snapshot there, for curiosity's sake, but you
>  > shouldn't count on the svn anymore).
>
>  we should probably indicate this somewhere.  There is very little
>  information being disseminated WRT where one should go for the bleeding
>  edge (which is the only place I feel comfortable :-)

I take full responsibility for poor communications recently.  In my
defense, I've been traveling a lot and coding, and most of this work
will directly or indirectly benefit ipython:

- we had a sprint on sage/scipy at Enthought's headquarters in Austin
in early March where several ipython devs also worked a lot, with
great progress made by Brian and Min on a number of issues.   We were
also joined by Ondrej Certik (of sympy fame) and Stefan van der Walt
(from numpy/scipy).

- I returned to Berkeley where I've been working with Stefan on some
prototype algorithms for an NSF grant proposal which, if funded, would
benefit ipython.  It's on mathematics and neuroscience, but the core
implementation would all be in scipy/ipython.

- We are right now in Paris holding a development meeting related to
scipy and neuroimaging in Python; this is part of my 'real job' but it
also involves ipython work and resources (I ran a demo this morning of
ipython1 and some local labs are already testing things out for their
work).

- This weekend we'll hold a mini-sprint at Gael Varoquaux's house in
Paris (from Mayavi2), to push forward on a number of key
ipython1/ipython-trunk integration issues.

So in summary, I've just been so swamped with this work that I've done
a poor job of communicating it to the rest of the team, except for all
the face-to-face conversations we've had.

Perhaps this will  convince me to get a blog: for the first time in my
life I see the point of those things.  While I never thought anyone
would have a reason to be interested in my personal ramblings, it may
be a good way to occasionally post things that aren't exactly pure
mailing list announcements.  By aggregating the posts at a place like
http://planet.scipy.org, interested readers can follow the
appropriately tagged posts.

I'm afraid after this mini-update I'll disappear again; I need some
sleep and we have a long day ahead tomorrow.

I hope you'll forgive the incompleteness of this post, but I realize
that we need to do a better job of community building.  If you think
that a blog would be the right resource (I know Ville already has one)
to help on this front, let me know as I'd appreciate the feedback.


Cheers,

f

>
>
>  >
>  > >  More concerning is the state of Qt4.  There is a "critical" bug
>  entered
>  > >  in trac: http://projects.scipy.org/ipython/ipython/ticket/208
>  > >
>  > >  Clearly, a QCoreApplication (or QApplication) instance is
>  > >  auto-constructed due to the -q4thread switch... and since
>  calculate.py
>  > >  constructs one also, QT dumps core  (why QT decides to core-dump as
>  the
>  > >  error reporting strategy is beyond me... but thats whats going on).
>  > >
>  > >  Qt4 support is indeed broken... but unnecessarily so. I have Qt4
>  running
>  > >  with twisted in a naked ipython shell.  I simply construct my
>  qt4reactor
>  > >  and use embedded Qt4 code.  Can't pull in pylab because it renames
>  > >  packages which collide with PyQt4 (in particular, wrapping QObject
>  in a
>  > >  different module... don't get that yet).  and this is beyond the
>  issues
>  > >  associated with QSocketNotifier being invoked from a non-qt thread.
>  I'm
>  > >  sure this is straightforward but, again, I can't determine whether
>  time
>  > >  spent here is worthwhile.  Seems to me that IPython1's architecture
>  will
>  > >  make most of these problems "just go away" as its a far more
>  > >  comprehensive architecture.
>  >
>  > IPython1 does not have a magical solution for any of this, it's pretty
>  > universal stuff. The problem is still the same - gui mainloop has to
>  > be in one thread - but we have that already. Whatever you do in
>  > ipython0 regarding Qt4 will be applicable on IPython1 as well, and
>  > IPython0 isn't going away anytime soon.
>
>  Actually, this is fairly straightforward with Qt and, I believe, the
>  other gui mainloops.  Since you're embracing twisted, I recommend adding
>  stdin and stdout to the event notification hooks and you're mostly done.
>  This is what I'm (kinda) doing... where things fall down a bit is when
>  IPython grabs the loop during scrolling (for example when reading the
>  docs which spew with a '??'.)  Other than that, it just kinda works.
>
>  Of course, I believe this is part of the new / better / different
>  IPython1 design.
>
>  Since IPython0 is gonna hang around we need to decide whether to work
>  deep inside to address this issue.  I previously posted a workaround to
>  nail twisted to IPython0 which exploited the multithreaded IShell code.
>
>  So, on this issue, we need a strategy.  I certainly appreciate continued
>  support for IPython0, but given the limited resources I wonder how much
>  effort should be put toward fixing the various GUI loops.
>
>  As it stands, Qt4 is currently broken with IPython0 running pylab.  The
>  decision to expend effort needs to be based on the development plan /
>  schedule.  This is something I have no insight into as there is no
>  chatter I have access to.
>
>  and I'll say again, you're in the soup with wx.  It might work for a
>  while, but its broken and needs work.  I looked at it with the twisted
>  team but decided to just bag it and go Qt.  In that effort, it became
>  clear how to fix wx which is when I became aware of wx's previous issues
>  which may or may not still exist.  The use of threads to get around wx
>  blocking is, IMHO, a bad idea...  much better to fix wx... and if that
>  can't be done, it should be bagged or someone needs to take on that
>  project, certainly beyond my time availability.
>
>
>  >
>  > Have you considered putting up a bzr branch on launchpad for this
>  > stuff? Darren has been most active regarding Qt4, and we could  use
>  > more hands on this.
>
>  Love to... but this further emphasizes my point.  I have no idea whats
>  going on WRT development of the various flavors of IPython.  I have a
>  feeling I'm not the only one without the necessary visibility.
>
>  >
>  --
>
> Glenn H. Tarbox, PhD
>  glenn at tarbox.org
>
>
>
>
>
>
> _______________________________________________
>  IPython-dev mailing list
>  IPython-dev at scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From vivainio at gmail.com  Wed Mar 19 02:37:20 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 19 Mar 2008 08:37:20 +0200
Subject: [IPython-dev] the state if IPython development...
In-Reply-To: <1205873455.17683.153.camel@ghtmyth.hq.tarbox.org>
References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com>
	<1205873455.17683.153.camel@ghtmyth.hq.tarbox.org>
Message-ID: <46cb515a0803182337i7efc23b7w1bf14f7770586163@mail.gmail.com>

On Tue, Mar 18, 2008 at 10:50 PM, Glenn H Tarbox, PhD <glenn at tarbox.org> wrote:

>  On Tue, 2008-03-18 at 22:15 +0200, Ville M. Vainio wrote:
>  > On Tue, Mar 18, 2008 at 8:35 PM, Glenn H Tarbox, PhD <glenn at tarbox.org> wrote:
>
>
> > That tarball only has svn, which has been pretty much discontinued (we
>  > will just dump a bzr snapshot there, for curiosity's sake, but you
>  > shouldn't count on the svn anymore).
>
>  we should probably indicate this somewhere.  There is very little
>  information being disseminated WRT where one should go for the bleeding
>  edge (which is the only place I feel comfortable :-)

Thea deal is that there are 2 bleeding edges - the bzr of stable-dev
and bzr of IPython1. IPython1 is not the bleeding edge of IPython0,
even if both projects have "IPython" in it ;-).


Think of this as baz and bzr, standard readline and pyreadline, glade
and gazpacho...


>  Actually, this is fairly straightforward with Qt and, I believe, the
>  other gui mainloops.  Since you're embracing twisted, I recommend adding
>  stdin and stdout to the event notification hooks and you're mostly done.
>  This is what I'm (kinda) doing... where things fall down a bit is when
>  IPython grabs the loop during scrolling (for example when reading the
>  docs which spew with a '??'.)  Other than that, it just kinda works.

Sound great. If you have something in place, please share the code! Do
new applications you %run reuse the reactor?

>  Of course, I believe this is part of the new / better / different
>  IPython1 design.

Yes, but we are not opposed to this in IPython0 either. The ground
rule is that as opposed to breaking stuff, we can add stuff.

>  Since IPython0 is gonna hang around we need to decide whether to work
>  deep inside to address this issue.  I previously posted a workaround to

Depends how deep inside you need to go. If it needs a lot of effort
(10+ hours), then the change itself may be too big for IPython0...

One factor is also how fast you want the thing to be in the hands of
the end users.

>  So, on this issue, we need a strategy.  I certainly appreciate continued
>  support for IPython0, but given the limited resources I wonder how much
>  effort should be put toward fixing the various GUI loops.

This is something I've been pondering as well, I've been thinking of a
simpler approach to handle these. One idea has been introducing a
magic command, say, %goqt4 that would register IPython with an already
existing Qt4 application - as opposed to the current approach of
constructing a QApplication of our own. Alternatively, we could
monkeypatch QApplication so that when it is created, we do our own
stuff for it and return (which is similar to how we handle mainloops
now). I haven't really looked deep into this since I tend not to use
these much.

>  As it stands, Qt4 is currently broken with IPython0 running pylab.  The
>  decision to expend effort needs to be based on the development plan /
>  schedule.  This is something I have no insight into as there is no
>  chatter I have access to.

You are talking about pylab_import_all option where it gets all the
names? This can be fixed easily. Just give the details.

>  > Have you considered putting up a bzr branch on launchpad for this
>  > stuff? Darren has been most active regarding Qt4, and we could  use
>  > more hands on this.
>
>  Love to... but this further emphasizes my point.  I have no idea whats
>  going on WRT development of the various flavors of IPython.  I have a
>  feeling I'm not the only one without the necessary visibility.

Just branch stable-dev branch and play around, if you need something
usable soon. IPython1 is still on "planning" stage regarding how to
run GUI mainloops in IPython core, as far as I understand.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From ondrej at certik.cz  Wed Mar 19 15:53:59 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 19 Mar 2008 20:53:59 +0100
Subject: [IPython-dev] the state if IPython development...
In-Reply-To: <db6b5ecc0803181626l4126f0f4i46d1135c51d8e65@mail.gmail.com>
References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com>
	<1205874306.17683.158.camel@ghtmyth.hq.tarbox.org>
	<db6b5ecc0803181626l4126f0f4i46d1135c51d8e65@mail.gmail.com>
Message-ID: <85b5c3130803191253j22f276c4w635aedef55695821@mail.gmail.com>

On Wed, Mar 19, 2008 at 12:26 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi Glenn,
>
>
>  On Tue, Mar 18, 2008 at 2:05 PM, Glenn H Tarbox, PhD <glenn at tarbox.org> wrote:
>  > On Tue, 2008-03-18 at 22:15 +0200, Ville M. Vainio wrote:
>  >  > On Tue, Mar 18, 2008 at 8:35 PM, Glenn H Tarbox, PhD
>  >  <glenn at tarbox.org> wrote:
>  >
>  >
>  > > That tarball only has svn, which has been pretty much discontinued (we
>  >  > will just dump a bzr snapshot there, for curiosity's sake, but you
>  >  > shouldn't count on the svn anymore).
>  >
>  >  we should probably indicate this somewhere.  There is very little
>  >  information being disseminated WRT where one should go for the bleeding
>  >  edge (which is the only place I feel comfortable :-)
>
>  I take full responsibility for poor communications recently.  In my
>  defense, I've been traveling a lot and coding, and most of this work
>  will directly or indirectly benefit ipython:
>
>  - we had a sprint on sage/scipy at Enthought's headquarters in Austin
>  in early March where several ipython devs also worked a lot, with
>  great progress made by Brian and Min on a number of issues.   We were
>  also joined by Ondrej Certik (of sympy fame) and Stefan van der Walt
>  (from numpy/scipy).
>
>  - I returned to Berkeley where I've been working with Stefan on some
>  prototype algorithms for an NSF grant proposal which, if funded, would
>  benefit ipython.  It's on mathematics and neuroscience, but the core
>  implementation would all be in scipy/ipython.
>
>  - We are right now in Paris holding a development meeting related to
>  scipy and neuroimaging in Python; this is part of my 'real job' but it
>  also involves ipython work and resources (I ran a demo this morning of
>  ipython1 and some local labs are already testing things out for their
>  work).
>
>  - This weekend we'll hold a mini-sprint at Gael Varoquaux's house in
>  Paris (from Mayavi2), to push forward on a number of key
>  ipython1/ipython-trunk integration issues.
>
>  So in summary, I've just been so swamped with this work that I've done
>  a poor job of communicating it to the rest of the team, except for all
>  the face-to-face conversations we've had.
>
>  Perhaps this will  convince me to get a blog: for the first time in my
>  life I see the point of those things.  While I never thought anyone
>  would have a reason to be interested in my personal ramblings, it may
>  be a good way to occasionally post things that aren't exactly pure
>  mailing list announcements.  By aggregating the posts at a place like
>  http://planet.scipy.org, interested readers can follow the
>  appropriately tagged posts.
>
>  I'm afraid after this mini-update I'll disappear again; I need some
>  sleep and we have a long day ahead tomorrow.
>
>  I hope you'll forgive the incompleteness of this post, but I realize
>  that we need to do a better job of community building.  If you think
>  that a blog would be the right resource (I know Ville already has one)
>  to help on this front, let me know as I'd appreciate the feedback.

Yes, I think having a blog is an extremely good way of telling the world
about your intentions and also about what has been done.

Ondrej


From vivainio at gmail.com  Wed Mar 19 17:39:28 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 19 Mar 2008 23:39:28 +0200
Subject: [IPython-dev] Twisted reactor + IPython0
Message-ID: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>

This is related to the recent thread about reactors, GUI support etc.

Would it make sense to provide direct twisted reactor support for ipython0?

There is a recipe for this:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410670

If this works as expected, can't we just implement all the GUI support
widgets in terms of this baby, using the support bundled with twisted?

A couple of you have gathered extensive experience with Twisted, what
do you think?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From glenn at tarbox.org  Wed Mar 19 18:35:16 2008
From: glenn at tarbox.org (Glenn H Tarbox, PhD)
Date: Wed, 19 Mar 2008 15:35:16 -0700
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
Message-ID: <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>

This is where I started originally.  There's a better fix which I posted
a while ago.  The below uses polling which is unnecessary.  By modifying
shell, you can turn the whole thing into a callback.

A better approach is to use my qtreactor (it sucks in qt but) because
you can instantiate twisted (and qt) and keep the event loop for
stdin/stdout and have reactor behavior as well.

the key with qtreactor is I made a new call = reactor.runReturn() which
nails up the reactor to the Qt application, sets up the timers etc, but
returns the thread to you... so as long as I/O in IPython doesn't block
(which it only does when scrolling more than a page)

-glenn


On Wed, 2008-03-19 at 23:39 +0200, Ville M. Vainio wrote:
> This is related to the recent thread about reactors, GUI support etc.
> 
> Would it make sense to provide direct twisted reactor support for ipython0?
> 
> There is a recipe for this:
> 
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410670
> 
> If this works as expected, can't we just implement all the GUI support
> widgets in terms of this baby, using the support bundled with twisted?
> 
> A couple of you have gathered extensive experience with Twisted, what
> do you think?
> 
-- 
Glenn H. Tarbox, PhD
206-494-0819
glenn at tarbox.org



From glenn at tarbox.org  Wed Mar 19 18:51:34 2008
From: glenn at tarbox.org (Glenn H Tarbox, PhD)
Date: Wed, 19 Mar 2008 15:51:34 -0700
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
Message-ID: <1205967094.26877.111.camel@ghtmyth.hq.tarbox.org>

to wit:

import sys

from PyQt4 import QtGui
import qt4reactor

"""
this nails up the reactor and makes app available
at the module level
"""
app = QtGui.QApplication(sys.argv)
qt4reactor.install()

# alternative for those who don't need a gui
# from twisted.application import reactors
# reactors.installReactor('qt4')

from twisted.internet import reactor

reactor.runReturn()

# now you have the thread back, a live Qt application object, and 
# twisted... and you've got the IPython0 prompt back... then try

def callMe():
  print "he called me"
reactor.callLater(5,0,callMe)

you'll have the prompt back and 5 seconds later it'll print


On Wed, 2008-03-19 at 15:35 -0700, Glenn H Tarbox, PhD wrote:
> This is where I started originally.  There's a better fix which I posted
> a while ago.  The below uses polling which is unnecessary.  By modifying
> shell, you can turn the whole thing into a callback.
> 
> A better approach is to use my qtreactor (it sucks in qt but) because
> you can instantiate twisted (and qt) and keep the event loop for
> stdin/stdout and have reactor behavior as well.
> 
> the key with qtreactor is I made a new call = reactor.runReturn() which
> nails up the reactor to the Qt application, sets up the timers etc, but
> returns the thread to you... so as long as I/O in IPython doesn't block
> (which it only does when scrolling more than a page)
> 
> -glenn
> 
> 
> On Wed, 2008-03-19 at 23:39 +0200, Ville M. Vainio wrote:
> > This is related to the recent thread about reactors, GUI support etc.
> > 
> > Would it make sense to provide direct twisted reactor support for ipython0?
> > 
> > There is a recipe for this:
> > 
> > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410670
> > 
> > If this works as expected, can't we just implement all the GUI support
> > widgets in terms of this baby, using the support bundled with twisted?
> > 
> > A couple of you have gathered extensive experience with Twisted, what
> > do you think?
> > 
-- 
Glenn H. Tarbox, PhD
206-494-0819
glenn at tarbox.org



From barrywark at gmail.com  Wed Mar 19 19:51:19 2008
From: barrywark at gmail.com (Barry Wark)
Date: Wed, 19 Mar 2008 16:51:19 -0700
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
Message-ID: <cd7634ce0803191651v227899c5y20e88174bb623582@mail.gmail.com>

On Wed, Mar 19, 2008 at 2:39 PM, Ville M. Vainio <vivainio at gmail.com> wrote:
> This is related to the recent thread about reactors, GUI support etc.
>
>  Would it make sense to provide direct twisted reactor support for ipython0?
>
>  There is a recipe for this:
>
>  http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410670
>
>  If this works as expected, can't we just implement all the GUI support
>  widgets in terms of this baby, using the support bundled with twisted?
>
>  A couple of you have gathered extensive experience with Twisted, what
>  do you think?

I'm definitely not an IPython1 or Twisted expert (more of a newbie)
and I know next to nothing about the IPython0 codebase, so take my
comments with a grain of salt. My understanding is that this is how
GUI support can be added to IPython1. ipython1.kernel.engineservice
exposes an IEngine interface as a Twisted service. The IEngine
interface provides the functionality of (and the implementation
delegates to) the ipython core. Thus GUI interfaces can interact with
the ipython1 core via Twisted. This is the approach I've used in the
ipython1-cocoa branch to provide a Cocoa frontend for the ipython1
core (i.e. via ipython1.kernel.engineservice.IEngineInteractive).

Barry


From ellisonbg.net at gmail.com  Wed Mar 19 23:10:16 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Wed, 19 Mar 2008 21:10:16 -0600
Subject: [IPython-dev] the state if IPython development...
In-Reply-To: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org>
References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org>
Message-ID: <6ce0ac130803192010j62fc5d7as932566b97140b8e2@mail.gmail.com>

> I'm having a bit of trouble evaluating various threads of ipython
>  development and am wondering if I'm missing a comms channel.  I have
>  various problems which are solveable but don't know which elements of
>  IPython are going to be refactored away in favor of the IPython1
>  architecture.

Sorry about this, it hasn't helped that I have been traveling a lot
recently and sick as well.

>  More concerning is the state of Qt4.  There is a "critical" bug entered
>  in trac: http://projects.scipy.org/ipython/ipython/ticket/208

The Qt4 bug in IPython0 should be fixed.  Period.  We just need
someone to tackle it.  There are just too many people who use it to
stop fixing bugs like this.  Our overall plan is to continue
supporting IPython0 until IPython1 is a full replacement.

>  Clearly, a QCoreApplication (or QApplication) instance is
>  auto-constructed due to the -q4thread switch... and since calculate.py
>  constructs one also, QT dumps core  (why QT decides to core-dump as the
>  error reporting strategy is beyond me... but thats whats going on).
>
>  Qt4 support is indeed broken... but unnecessarily so. I have Qt4 running
>  with twisted in a naked ipython shell.  I simply construct my qt4reactor
>  and use embedded Qt4 code.  Can't pull in pylab because it renames
>  packages which collide with PyQt4 (in particular, wrapping QObject in a
>  different module... don't get that yet).  and this is beyond the issues
>  associated with QSocketNotifier being invoked from a non-qt thread.  I'm
>  sure this is straightforward but, again, I can't determine whether time
>  spent here is worthwhile.  Seems to me that IPython1's architecture will
>  make most of these problems "just go away" as its a far more
>  comprehensive architecture.
>
>  (BTW: the twisted qt reactor can be retrieved from
>  http://noc2.tarbox.org:8080/?p=qtreactor/.git;a=summary )
>
>  On a related point, I think we're headed for trouble with wx.  I notice
>  that its included in IPython1... but the twisted wxreactor is a bit
>  broken.  I think its easy to fix, but the reason its broken is because
>  wx was poorly behaved when the wxreactor was developed (stopped
>  servicing timers, IO notification during modal windows etc) and the
>  twisted folks used threading to get around the difficulty.  Nothing
>  wrong with that, other than its ugly (and will cause other threadosity
>  issues) but the code doesn't work properly (fails tests, I support the
>  buildbot) and I wonder whether its worth fixing or whether its easier to
>  get wx to behave (I can drill down into the issues here with anyone who
>  wants but I won't digress further here).

What is included with IPython1 (wx is not?).  We do need to have some
discussions about how we will handle GUI integration within IPython1.
This is a much bigger discussion though - probably need to start a new
thread for this.

>  Finally, I support 4 buildbots for twisted.  I'm willing to provide
>  these same assets for IPython1 development (each server has dual - 4
>  core processors = 8 processors / server totaling 32 processors).  How to
>  properly use this setup for cluster testing would need to be discussed.

For now the IPython1 test suite does not need multiple processors (we
are using twisted's trial).  But, we surely could use an IPython1
buildbot.  Eventually we will definitely probably have multiprocess
tests though.

>  -glenn
>
>  --
>  Glenn H. Tarbox, PhD
>  glenn at tarbox.org
>
>  _______________________________________________
>  IPython-dev mailing list
>  IPython-dev at scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From ellisonbg.net at gmail.com  Wed Mar 19 23:21:48 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Wed, 19 Mar 2008 21:21:48 -0600
Subject: [IPython-dev] A few notes on threading+GUI event loops in Ipython1
Message-ID: <6ce0ac130803192021x235aa192s727c6f8f34f1d961@mail.gmail.com>

Hi,

With all the recent activity about GUi event loops + threading, I
figured I would jot down a few notes about the current state of GUI
event loop integration within IPython1.

1) The IPython1 engine/core can be run in two different modes: with
the twisted event loop (typically when it runs in another process) or
without it.

Thus, we will need to have support for GUI event loop integration that
works for both of these cases.

2)  If the twisted event loop is running, one option is to use one of
the twisted reactors like the wxreactor.

But, we will need to carefully study if these reactors really work
(there seems to be some evidence that there are problems).

3) The issue of threads in the IPython1 engine could get _very_
complicated (much more so even that the treading in ipython0).  Part
of the problem is that there is simply many more possibilities for how
the ipython1 engine can be run.  It is probably too early to tackle
these things, but we are getting close to needing to think about these
things.

Brian


From ellisonbg.net at gmail.com  Wed Mar 19 23:21:48 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Wed, 19 Mar 2008 21:21:48 -0600
Subject: [IPython-dev] A few notes on threading+GUI event loops in Ipython1
Message-ID: <6ce0ac130803192021x235aa192s727c6f8f34f1d961@mail.gmail.com>

Hi,

With all the recent activity about GUi event loops + threading, I
figured I would jot down a few notes about the current state of GUI
event loop integration within IPython1.

1) The IPython1 engine/core can be run in two different modes: with
the twisted event loop (typically when it runs in another process) or
without it.

Thus, we will need to have support for GUI event loop integration that
works for both of these cases.

2)  If the twisted event loop is running, one option is to use one of
the twisted reactors like the wxreactor.

But, we will need to carefully study if these reactors really work
(there seems to be some evidence that there are problems).

3) The issue of threads in the IPython1 engine could get _very_
complicated (much more so even that the treading in ipython0).  Part
of the problem is that there is simply many more possibilities for how
the ipython1 engine can be run.  It is probably too early to tackle
these things, but we are getting close to needing to think about these
things.

Brian


From vivainio at gmail.com  Thu Mar 20 01:33:00 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 20 Mar 2008 07:33:00 +0200
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
Message-ID: <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>

On Thu, Mar 20, 2008 at 12:35 AM, Glenn H Tarbox, PhD <glenn at tarbox.org> wrote:

> This is where I started originally.  There's a better fix which I posted
>  a while ago.  The below uses polling which is unnecessary.  By modifying
>  shell, you can turn the whole thing into a callback.

The ASPN recipe works like the current GUI backends, so it seems to
"obviously work".

>
>  A better approach is to use my qtreactor (it sucks in qt but) because
>  you can instantiate twisted (and qt) and keep the event loop for
>  stdin/stdout and have reactor behavior as well.

Would that work with other backends as well? Does it work with windows
+ pyreadline?

>  the key with qtreactor is I made a new call = reactor.runReturn() which
>  nails up the reactor to the Qt application, sets up the timers etc, but
>  returns the thread to you... so as long as I/O in IPython doesn't block
>  (which it only does when scrolling more than a page)

That's a pretty fine solution, if it can be made to work.

Can you produce a code snippet that does this for IPython0?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From glenn at tarbox.org  Thu Mar 20 01:41:18 2008
From: glenn at tarbox.org (Glenn H Tarbox, PhD)
Date: Wed, 19 Mar 2008 22:41:18 -0700
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
Message-ID: <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>


On Thu, 2008-03-20 at 07:33 +0200, Ville M. Vainio wrote:
> On Thu, Mar 20, 2008 at 12:35 AM, Glenn H Tarbox, PhD <glenn at tarbox.org> wrote:
> 
> > This is where I started originally.  There's a better fix which I posted
> >  a while ago.  The below uses polling which is unnecessary.  By modifying
> >  shell, you can turn the whole thing into a callback.
> 
> The ASPN recipe works like the current GUI backends, so it seems to
> "obviously work".

it works, just spins cycles unnecessarily.  If you look through this
list you'll see the minor changes I made to Shell (posted on Feb 9).  My
code generates a callback when somethings needs to be serviced as
opposed to having a high-frequency polling loop try and determine the
same thing

> 
> >
> >  A better approach is to use my qtreactor (it sucks in qt but) because
> >  you can instantiate twisted (and qt) and keep the event loop for
> >  stdin/stdout and have reactor behavior as well.
> 
> Would that work with other backends as well? Does it work with windows
> + pyreadline?

yup

> 
> >  the key with qtreactor is I made a new call = reactor.runReturn() which
> >  nails up the reactor to the Qt application, sets up the timers etc, but
> >  returns the thread to you... so as long as I/O in IPython doesn't block
> >  (which it only does when scrolling more than a page)
> 
> That's a pretty fine solution, if it can be made to work.
> 
> Can you produce a code snippet that does this for IPython0?

was already posted before you wrote this.

> 
-- 
Glenn H. Tarbox, PhD
206-494-0819
glenn at tarbox.org



From vivainio at gmail.com  Thu Mar 20 02:20:15 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 20 Mar 2008 08:20:15 +0200
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
Message-ID: <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>

On Thu, Mar 20, 2008 at 7:41 AM, Glenn H Tarbox, PhD <glenn at tarbox.org> wrote:

>  On Thu, 2008-03-20 at 07:33 +0200, Ville M. Vainio wrote:
>  > On Thu, Mar 20, 2008 at 12:35 AM, Glenn H Tarbox, PhD <glenn at tarbox.org> wrote:
>  >
>  > > This is where I started originally.  There's a better fix which I posted
>  > >  a while ago.  The below uses polling which is unnecessary.  By modifying
>  > >  shell, you can turn the whole thing into a callback.
>  >
>  > The ASPN recipe works like the current GUI backends, so it seems to
>  > "obviously work".
>
>  it works, just spins cycles unnecessarily.  If you look through this
>  list you'll see the minor changes I made to Shell (posted on Feb 9).  My
>  code generates a callback when somethings needs to be serviced as
>  opposed to having a high-frequency polling loop try and determine the
>  same thing

Goodie. Frankly, I probably missed the original discussion because I
thought of it was mostly related to IPython1 (though obviously I can't
remember why I missed it) ;-)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From laurent.dufrechou at gmail.com  Thu Mar 20 04:13:56 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Thu, 20 Mar 2008 09:13:56 +0100
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
Message-ID: <47e21ccf.0aaa660a.1579.1217@mx.google.com>

Hi all,

Just to drop some more need on threading.
I've written a little piece of code to create a WX Ipython shell.
I've used for this a thread with an ipython0 instance and communicate with
the gui via polling (the first version) and now I'm starting to use
callback.
One big need I have and that cannot be resolved easily is to ba able to have
some sort of Ipython instance with GUI loop support + twisted support.
The idea:

Any WXAPP/QTApp   <-- XML RPC --> The twisted loop + Any GUI(QT/WX...) loop
+ Ipython server.

With XML RPC you can just make any action inside Ipython instance.
I want this to be able to make a WX(and QT) console able to launch any QT/Wx
windows and thus independent to the Console App.

With this you can have a really really good dev env for graphical app.
Moreover designing such a server will be raally good for people who want an
ipython console, because with this all the wx/QT/... console code will share
the same ipython object that will save lot of work.

Glenn, perhaps my idea is stupid, but do you thing we will have a lot of
performance related problem with this, do you see any issue as you've worked
a little on something like this??
Perhaps it is a too big thing for a simple Ipython console...

Laurent

-----Message d'origine-----
De?: ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De
la part de Ville M. Vainio
Envoy??: jeudi 20 mars 2008 07:20
??: Glenn H Tarbox, PhD
Cc?: ipython-dev Mailing list
Objet?: Re: [IPython-dev] Twisted reactor + IPython0

On Thu, Mar 20, 2008 at 7:41 AM, Glenn H Tarbox, PhD <glenn at tarbox.org>
wrote:

>  On Thu, 2008-03-20 at 07:33 +0200, Ville M. Vainio wrote:
>  > On Thu, Mar 20, 2008 at 12:35 AM, Glenn H Tarbox, PhD
<glenn at tarbox.org> wrote:
>  >
>  > > This is where I started originally.  There's a better fix which I
posted
>  > >  a while ago.  The below uses polling which is unnecessary.  By
modifying
>  > >  shell, you can turn the whole thing into a callback.
>  >
>  > The ASPN recipe works like the current GUI backends, so it seems to
>  > "obviously work".
>
>  it works, just spins cycles unnecessarily.  If you look through this
>  list you'll see the minor changes I made to Shell (posted on Feb 9).  My
>  code generates a callback when somethings needs to be serviced as
>  opposed to having a high-frequency polling loop try and determine the
>  same thing

Goodie. Frankly, I probably missed the original discussion because I
thought of it was mostly related to IPython1 (though obviously I can't
remember why I missed it) ;-)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev

No virus found in this incoming message.
Checked by AVG. 
Version: 7.5.519 / Virus Database: 269.21.7/1334 - Release Date: 18/03/2008
20:52
 

No virus found in this outgoing message.
Checked by AVG. 
Version: 7.5.519 / Virus Database: 269.21.7/1334 - Release Date: 18/03/2008
20:52
 



From vivainio at gmail.com  Thu Mar 20 04:33:48 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 20 Mar 2008 10:33:48 +0200
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <47e21ccf.0aaa660a.1579.1217@mx.google.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
	<47e21ccf.0aaa660a.1579.1217@mx.google.com>
Message-ID: <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>

On Thu, Mar 20, 2008 at 10:13 AM, Laurent Dufr?chou
<laurent.dufrechou at gmail.com> wrote:

>  With this you can have a really really good dev env for graphical app.
>  Moreover designing such a server will be raally good for people who want an
>  ipython console, because with this all the wx/QT/... console code will share
>  the same ipython object that will save lot of work.
>
>  Glenn, perhaps my idea is stupid, but do you thing we will have a lot of
>  performance related problem with this, do you see any issue as you've worked
>  a little on something like this??
>  Perhaps it is a too big thing for a simple Ipython console...

It really isn't, once we get the twisted reactor to be the "core" of
IPython0. There is lots of fun to be had with this.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Thu Mar 20 06:07:01 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 20 Mar 2008 03:07:01 -0700
Subject: [IPython-dev] Discussion on Bruce Eckel's blog
Message-ID: <db6b5ecc0803200307t6e32eabar2351acaa625d7553@mail.gmail.com>

Hi all,

this may interest several of us here, since ipython is mentioned more
than once.  By pycon'09 we'll have much more than 'claims' to present
:)

http://www.artima.com/forums/flat.jsp?forum=106&thread=227073&start=0

Cheers,

f


From glenn at tarbox.org  Thu Mar 20 10:15:14 2008
From: glenn at tarbox.org (Glenn H Tarbox, PhD)
Date: Thu, 20 Mar 2008 07:15:14 -0700
Subject: [IPython-dev] Discussion on Bruce Eckel's blog
In-Reply-To: <db6b5ecc0803200307t6e32eabar2351acaa625d7553@mail.gmail.com>
References: <db6b5ecc0803200307t6e32eabar2351acaa625d7553@mail.gmail.com>
Message-ID: <1206022514.3359.71.camel@ghtmyth.hq.tarbox.org>

Perhaps its early and maybe I should wait a couplea hours before
responding to this...  My understanding is that Bruce Eckel is a serious
guy and someone due respect...

But the post is silly.  I really don't know where to start and I think
he should just delete it and we should forget it exists.  

Is he trying to discuss the issues of parallelism, threading, co-routine
abstractions, inter process-communication?  In the same brief blurb  he
discusses and mixes all four.  Makes mention of the GIL and how java
multithreading through co-routines (stackless???) are... what?  I got
confused even parsing the words.  none of these issues are related at
the level of the post.

Stack frames, threads, processes, IPC and netowrking are, clearly,
related but wildly different subjects... not to mention that none of
these have anything to do with solving parallel computing problems other
than we might want to use multiple computers and electricity...

Is the point that some of the packages mentioned make serializing
objects easier?

And IPython1?  Sure, there's a notion of a redesign to embrace actual
parallel computation issues, and even a couplea examples of how one
might use asynchronous models to implement map-reduce (or scatter-gather
or whatever) 

But the IPython1 effort is mostly about embracing the asynchronous model
(read Twisted) with a sugary sweet layer of IPython to make it a bit
less painful.  Unless I missed a bunch of documentation somewhere
(certainly possible given the state of the documentation), this hardly
rises to: "a powerful architecture for parallel computing," 

So, please, lets not get into a debate over this article.  Its trivial
and muddled.  and lets certainly not believe our own line of BS if
someone did say something like this to Bruce.  IPython1 has a notion
that there should be an architecture and even some early examples... but
there's no new model, no power other than doing stuff correctly with
modern tools.

Hell, I'm not even sure Bruce got the relevant points about Numpy
right... maybe he spent a few too many hours in the lobby bar and needed
to spew a pre-defined number of words to justify the travel expense.


-glenn

-- 
Glenn H. Tarbox, PhD
206-494-0819
glenn at tarbox.org

On Thu, 2008-03-20 at 03:07 -0700, Fernando Perez wrote:
> Hi all,
> 
> this may interest several of us here, since ipython is mentioned more
> than once.  By pycon'09 we'll have much more than 'claims' to present
> :)
> 
> http://www.artima.com/forums/flat.jsp?forum=106&thread=227073&start=0
> 
> Cheers,
> 
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
http://lists.ipython.scipy.org/mailman/listinfo/ipython-
> dev




From glenn at tarbox.org  Thu Mar 20 12:53:19 2008
From: glenn at tarbox.org (Glenn H Tarbox, PhD)
Date: Thu, 20 Mar 2008 09:53:19 -0700
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <47e21ccf.0aaa660a.1579.1217@mx.google.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
	<47e21ccf.0aaa660a.1579.1217@mx.google.com>
Message-ID: <1206031999.3359.176.camel@ghtmyth.hq.tarbox.org>

Ville responded to this as well but I'll add some

On Thu, 2008-03-20 at 09:13 +0100, Laurent Dufr?chou wrote:
> Hi all,
> 
> Just to drop some more need on threading.
> I've written a little piece of code to create a WX Ipython shell.
> I've used for this a thread with an ipython0 instance and communicate with
> the gui via polling (the first version) and now I'm starting to use
> callback.
> One big need I have and that cannot be resolved easily is to ba able to have
> some sort of Ipython instance with GUI loop support + twisted support.
> The idea:
> 
> Any WXAPP/QTApp   <-- XML RPC --> The twisted loop + Any GUI(QT/WX...) loop
> + Ipython server.

I guess the question is what you're trying to accomplish.  So let me
paste Ville's response which is exactly right and the core of any
response to this kind of issue:

"It really isn't, once we get the twisted reactor to be the "core" of
IPython0. There is lots of fun to be had with this."

So, what is being said here, and what lies at the core of the new /
better / different IPython_x design is an event based architecture
underlying whatever front ends we have.  Embrace giving "the thread"
away....

So, first a claim which should set the stage for a flame war:

>>>>>>>>>>>> Soapbox Warning: <<<<<<<<<<<<<<<<<

Threads are, for virtually all intents and purposes, BAD and should be
avoided like the plague. (never had the plague... maybe its not as bad
as threads)

Of course, the above is wrong, but its way more right than any counter
argument which will be brought forth.

Now, when threads are necessary, they're critical and we can't get by
without them.  But realize, Python has 1 thread... thats right kids,
regardless of the number of cores on your computer, python is only
executing one actual instruction at a time.  Of course, there's a lot of
support the OS provides to your performance experience with another core
such as IO etc... and some libraries do spawn threads underneath... but
python itself is single "true thread"ed which is why you see words like
GIL used and which is another huge issue in the broader python community
and has caused any number of blog / mail group thread wars as python
strives to deal with this multi-core world.

Whats even more important is that threads are almost always unnecessary
and are used, mostly, because people don't know how to accomplish
certain tasks (blocking) without them.  They require resource
synchronization, module segmentation (or other synchronization
migigation strategy) but at least they introduce non-deternimism, lower
performance, and a general impossibility to debugging the intermittent
failures you're sure to experience. (guido, the man himself writes:

"Unfortunately, for most mortals, thread programming is just Too Hard to
get right.... Even in Python -- every time someone gets into serious
thread programming, they send me tons of bug reports, and half of them
are subtle bugs in the Python interpreter, half of them are subtle
problems in their own understanding of the consequences of multiple
threads...."

now lets add that virtually all the gui loops (and this holds true for
most callback style event handling mechanisms) aren't thread safe... so,
you need to go through some kind of prophylactic (always practice safe
threading) when one thread touches something owned by another thread 

<<<<<<<<<<< steps off soap box >>>>>>>>>>>>>>>>>>>

whew, feel better now.

OK, so lets get back to what you were asking... so, what you really want
to do is integrate various event models with pre-existing event handling
mechanisms... 

and away we go....

So, you're using Wx.  Unfortunately, it looks like you're in trouble to
start with although I haven't been able to run this all the ground.
According to some who should know, Wx behaves badly as an event
engine... (apparently stops emitting timers / IO events during modal
windows etc) so the current Twisted Wx Reactor, therefore uses <gulp>
threads to get around this.  This strategy is mostly, why the twisted Wx
reactor doesn't completely work right now... nothing inherently wrong
with the design just some complexity associated with the thread handoffs
which uses a generator and I haven't completely gotten through this one
yet...

the Qt reactor didn't require this as it is well behaved... so I
inverted the design of twisted, kinda... basically, I use Qt as the core
OS integration mechanism and have it call me when certain things need to
happpen... basically that means, simply, that IO which I own and
timing / scheduling is serviced by a few callbacks I register with Qt.
This is mostly what happens with the gtk reactor as well.

So, your sketch using XML-RPC as an integration strategy makes sense if
you're inter-process but, I think, might be overkill in the same
process.  Integrating multiple event loops is nasty because they all
"help" by abstracting (hiding) the underlying OS mechanisms which emit
events to the application layer.  If we can get around this (as I did
with the Qt reactor) we're better off.

Furthermore, I wonder if your strategy will work in general even with
threads... it all depends on what the other event loop does with the
foundational wrapping of the event sources.... so, if you try, for
example, to add your own socket and try and block on select, will that
interfere with Wx?  If you had Wx and Qt in the same process, where
would the window events go?  None of these core engines acknowledge the
existence of any other engines... and would likely behave very badly
with the introduction of such a virus in their process space.

If you're in a different process, all this is an entirely different
discussion and is more about event IO... none of which requires
threading... Qt, Twisted, and Wx all provide very rich mechanisms to
register callbacks for this kind of event handling.

-glenn

> 
> With XML RPC you can just make any action inside Ipython instance.
> I want this to be able to make a WX(and QT) console able to launch any QT/Wx
> windows and thus independent to the Console App.
> 
> With this you can have a really really good dev env for graphical app.
> Moreover designing such a server will be raally good for people who want an
> ipython console, because with this all the wx/QT/... console code will share
> the same ipython object that will save lot of work.
> 
> Glenn, perhaps my idea is stupid, but do you thing we will have a lot of
> performance related problem with this, do you see any issue as you've worked
> a little on something like this??
> Perhaps it is a too big thing for a simple Ipython console...
> 
> Laurent
> 
> -----Message d'origine-----
> De : ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De
> la part de Ville M. Vainio
> Envoy? : jeudi 20 mars 2008 07:20
> ? : Glenn H Tarbox, PhD
> Cc : ipython-dev Mailing list
> Objet : Re: [IPython-dev] Twisted reactor + IPython0
> 
> On Thu, Mar 20, 2008 at 7:41 AM, Glenn H Tarbox, PhD <glenn at tarbox.org>
> wrote:
> 
> >  On Thu, 2008-03-20 at 07:33 +0200, Ville M. Vainio wrote:
> >  > On Thu, Mar 20, 2008 at 12:35 AM, Glenn H Tarbox, PhD
> <glenn at tarbox.org> wrote:
> >  >
> >  > > This is where I started originally.  There's a better fix which I
> posted
> >  > >  a while ago.  The below uses polling which is unnecessary.  By
> modifying
> >  > >  shell, you can turn the whole thing into a callback.
> >  >
> >  > The ASPN recipe works like the current GUI backends, so it seems to
> >  > "obviously work".
> >
> >  it works, just spins cycles unnecessarily.  If you look through this
> >  list you'll see the minor changes I made to Shell (posted on Feb 9).  My
> >  code generates a callback when somethings needs to be serviced as
> >  opposed to having a high-frequency polling loop try and determine the
> >  same thing
> 
> Goodie. Frankly, I probably missed the original discussion because I
> thought of it was mostly related to IPython1 (though obviously I can't
> remember why I missed it) ;-)
> 
-- 
Glenn H. Tarbox, PhD

glenn at tarbox.org



From fperez.net at gmail.com  Thu Mar 20 13:47:36 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 20 Mar 2008 10:47:36 -0700
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
	<47e21ccf.0aaa660a.1579.1217@mx.google.com>
	<46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>
Message-ID: <db6b5ecc0803201047l7e2220fepb7ba2d3c4acb57cd@mail.gmail.com>

On Thu, Mar 20, 2008 at 1:33 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> On Thu, Mar 20, 2008 at 10:13 AM, Laurent Dufr?chou
>  <laurent.dufrechou at gmail.com> wrote:
>
>  >  With this you can have a really really good dev env for graphical app.
>  >  Moreover designing such a server will be raally good for people who want an
>  >  ipython console, because with this all the wx/QT/... console code will share
>  >  the same ipython object that will save lot of work.
>  >
>  >  Glenn, perhaps my idea is stupid, but do you thing we will have a lot of
>  >  performance related problem with this, do you see any issue as you've worked
>  >  a little on something like this??
>  >  Perhaps it is a too big thing for a simple Ipython console...
>
>  It really isn't, once we get the twisted reactor to be the "core" of
>  IPython0. There is lots of fun to be had with this.

It would certainly be great to have this done in ip0 as an optional
component: we could always tell our users: you have two options for
gui support:

 - the threads-based hack we have, which mostly but not always works,
but has no dependencies outside the stdlib and your own GUI toolkit.
 - a robust, solid, twisted-based one (if this really proves to be the
case in the long run).

The second option is one more dependency but if it proves to really do
the right thing, in the end I'm sure most users would run with it.

After both options have existed for a while and receive real-world
testing on all backends/platforms, we can see if we decide to
completely deprecate the first or keep it.

In addition, a twisted-based design is very much in line with the ip1
architecture, so I'd love to see this done for all the GUI backends
using twisted.  Does it have reactors for Qt3/4, Wx and GTK that work
on Windows, Linux and OSX?  That's our support requirement currently
(even if it's not perfect, all those do work and have real users).

cheers,

f


From vivainio at gmail.com  Thu Mar 20 14:30:27 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 20 Mar 2008 20:30:27 +0200
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <db6b5ecc0803201047l7e2220fepb7ba2d3c4acb57cd@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
	<47e21ccf.0aaa660a.1579.1217@mx.google.com>
	<46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>
	<db6b5ecc0803201047l7e2220fepb7ba2d3c4acb57cd@mail.gmail.com>
Message-ID: <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com>

On Thu, Mar 20, 2008 at 7:47 PM, Fernando Perez <fperez.net at gmail.com> wrote:

>  It would certainly be great to have this done in ip0 as an optional
>  component: we could always tell our users: you have two options for
>  gui support:

It's in trunk now. It's still a threads-based hack, in that ipython
mainloop runs in a thread, but now the mainloop is a twisted reactor.
See IPython/twshell.py. You can try it out by "python twshell.py".
kbdint and Exit seem to work.

I used the stuff from mtexp branch, so macros should work, and it
should survive reactor death (5 sec timeout as before). Now, I gotta
run, but someone might want to take a look at it.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From ellisonbg.net at gmail.com  Thu Mar 20 14:49:57 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Thu, 20 Mar 2008 12:49:57 -0600
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
	<47e21ccf.0aaa660a.1579.1217@mx.google.com>
	<46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>
	<db6b5ecc0803201047l7e2220fepb7ba2d3c4acb57cd@mail.gmail.com>
	<46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com>
Message-ID: <6ce0ac130803201149o4001e507w9b2274afa054be81@mail.gmail.com>

Noooooo ;-).  Please, don't start doing stuff like this without all of
us discussing the best direction.  The parallel computing stuff in
IPython1 is designed to work with IPython0 and uses a very carefully
crafted approach for integrating with the Twisted main loop.  Doing
something like this could easily break IPython1 for the many current
users - even if it is optional, I don't want it in IPython0.  More
discussion is needed first.  Sorry I don't have more time to
participate in the discussion right now - if you are interested in how
we are handling this in IPython1, please look at that code base.  What
ever ends up in IPython0 _must_ be compatible with that.

Brian

On Thu, Mar 20, 2008 at 12:30 PM, Ville M. Vainio <vivainio at gmail.com> wrote:
> On Thu, Mar 20, 2008 at 7:47 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>
>  >  It would certainly be great to have this done in ip0 as an optional
>  >  component: we could always tell our users: you have two options for
>  >  gui support:
>
>  It's in trunk now. It's still a threads-based hack, in that ipython
>  mainloop runs in a thread, but now the mainloop is a twisted reactor.
>  See IPython/twshell.py. You can try it out by "python twshell.py".
>  kbdint and Exit seem to work.
>
>  I used the stuff from mtexp branch, so macros should work, and it
>  should survive reactor death (5 sec timeout as before). Now, I gotta
>  run, but someone might want to take a look at it.
>
>
>  --
>  Ville M. Vainio - vivainio.googlepages.com
>  blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
>  _______________________________________________
>
>
> IPython-dev mailing list
>  IPython-dev at scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From ellisonbg.net at gmail.com  Thu Mar 20 18:15:14 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Thu, 20 Mar 2008 16:15:14 -0600
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
	<47e21ccf.0aaa660a.1579.1217@mx.google.com>
	<46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>
	<db6b5ecc0803201047l7e2220fepb7ba2d3c4acb57cd@mail.gmail.com>
	<46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com>
Message-ID: <6ce0ac130803201515j7366f715t3de24dc585de8cc8@mail.gmail.com>

>  >  It would certainly be great to have this done in ip0 as an optional
>  >  component: we could always tell our users: you have two options for
>  >  gui support:
>
>  It's in trunk now. It's still a threads-based hack, in that ipython
>  mainloop runs in a thread, but now the mainloop is a twisted reactor.
>  See IPython/twshell.py. You can try it out by "python twshell.py".
>  kbdint and Exit seem to work.
>
>  I used the stuff from mtexp branch, so macros should work, and it
>  should survive reactor death (5 sec timeout as before). Now, I gotta
>  run, but someone might want to take a look at it.

I should provide a few more details about my earlier objections to
having this in ipython0.

One of the big problems we have dealt with in IPython1 is that we
want/need to have networking clients that are based on Twisted.  Thus,
we need to have the Twisted reactor running in the IPython0 frontend.
But that is the easy part.  The challenge is that we need to put a
blocking interface on top of Twisted deferred returning functions and
methods.  This took us a _very_ long time to figure out how to do.
Here is what we are doing currently in ipython1 (when using it from
ipython0):

1) We simply start the twisted reactor in a thread.

2) When we need to call a method/function that returns a deferred, we
use blockingCallFromThread.  This uses a Queue to block the users
thread (where Ipython0 is running) until the deferred has fired in the
other thread (the twisted one).

There code for these things is here:

http://bazaar.launchpad.net/~ipython/ipython/ipython1-dev/annotate/ellisonbg%40gmail.com-20080311024040-ctltp13xxrh9iba7?file_id=twistedutil.py-20080302203151-17poomviavsf2zjj-141

Or if you have the ipython1-dev branch, it is in:

ipython/kernel/twistedutil.py

>From our testing so far, this seems to provide a fairly robust way of
running the reactor in ipython0.

...queue scary music...

But, I am not sure how all of this works with the current threaded
Ipython shells.

I think most of it works, but it probably fails to handle signals
properly.  Also, I am not sure how everything would behave if we run a
Twisted GUI reactor.

Here is another big problem lurking on the horizon.

Eventually, your average IPython session will be an IPython1 session
and will thus have the Twisted reactor running in its main thread.  We
want to be able to write networking clients that will run in that
context.  That itself is not a problem.  The problem is that we want
to be able to have blocking network clients.  This means somehow we
need to figure out a way of blocking on the result of a deferred.

With a single thread, this is impossible.  The only way I know of
doing this is to have two threads:

1) One running the twisted reactor

2) Another that runs the users code

Then user code can use blockingCallFromThread to block on a deferred.
While we don't yet have this implemented in Ipython1, it should not be
difficult.

The bigger complication comes in when you add a GUI event loop to the
picture.  Then you need to make sure that all three things (GUI event
loop, twisted reactor and user code that potentially needs to block on
a deferred) work together in a thread happy manner.

That is the main thing to keep in mind.  Sorry about my earlier rant.
I don't want to discourage people from playing around with this stuff.
 But, this stuff is extremely complicated and easy to break.  Please
just keep in mind that currently we are in this really odd state where
IPython1 needs to be able to work from within IPython0.

Cheers,

Brian


>
>  --
>  Ville M. Vainio - vivainio.googlepages.com
>  blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
>  _______________________________________________
>
>
> IPython-dev mailing list
>  IPython-dev at scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From laurent.dufrechou at gmail.com  Thu Mar 20 19:22:31 2008
From: laurent.dufrechou at gmail.com (=?windows-1250?Q?Laurent_Dufr=E9chou?=)
Date: Fri, 21 Mar 2008 00:22:31 +0100
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <6ce0ac130803201515j7366f715t3de24dc585de8cc8@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>	<47e21ccf.0aaa660a.1579.1217@mx.google.com>	<46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>	<db6b5ecc0803201047l7e2220fepb7ba2d3c4acb57cd@mail.gmail.com>	<46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com>
	<6ce0ac130803201515j7366f715t3de24dc585de8cc8@mail.gmail.com>
Message-ID: <47e2f1c4.0305560a.7758.5828@mx.google.com>

Thanks for the reply guys.

I must clarify my idea:
My main objective is to provide a set of classes that anybody can use in
QT/WX/whatelse to create easily some ipython console GUI widget
(QTIpython,WXIpython,TKIpython etc...)

My work is based on WX but can be easily used for QT or other.I'm using
Scintilla for the display so... can be also easy to be ported.
(will see this weekend with all the guy how I could interface with
pyreadline)

So,first I wrote a little piece of code to get ipython shell inside a wx
widget.
It was based on user keyboard interaction with no thread.
I used for this a piece of gtk console and IPInteractive Shell class where I
could launch some commands line by line.
Then arise the problem of infinite loop:
While 1:
	Pass
With this my widget and all my GUI was frozen, using a thread where I sent
line by line my command to an IPInteractive Shell solved this issue.
That's why I needed thread glenn ;)

A little Ascii drawing:

Thread wich feeds Ipython instance line by line  <--- communicate -->  WxApp
Main loop Thread

Now I've got this problem:
User want to  be able to use Wx Ipython console widget with for example QT4
support or WX support...
So I see no other idea to give them this class run in an other process where
they could launch any GUI unrelated to my widget GUI loop.
That's why I wanted xml-rpc and twisted support in ipython0.
(If you take a look at this it looks like that ipython1 should be the WAY to
do, but I don't know if ipython1 has interactive support like ipython0 and
ipython0 NEEDS GUI widgets ;) )

So perhaps twisted integration in ipython0 for that is not a clever idea,
perhaps a simpler xmlrpc server loop can be OK.
I really want to have GUI loop+XML/RPC loop+ipython interactivity in a class
that I could call to do all of this.

Guys,what is the best way? 

Ville:
I'll take a look at your file this week end, when gael and others guys in
france will have explained me all the bzr stuff :)

Glenn:

>So, your sketch using XML-RPC as an integration strategy makes sense if
you're inter-process but, I think, might be overkill in the same process.
Integrating multiple event loops is nasty because they >all "help" by
abstracting (hiding) the underlying OS mechanisms which emit events to the
application layer.  If we can get around this (as I did with the Qt reactor)
we're better off.

In fact xml-rpc should be used when guys want to play with GUI in Ipython
shell widget whithout burry my WXapp.
If it not a required thing than, xml-rpc is unusefull sure :), but I also
don't want to split my code...

>Furthermore, I wonder if your strategy will work in general even with
threads... it all depends on what the other event loop does with the
foundational wrapping of the event sources.... so, if you try, >for example,
to add your own socket and try and block on select, will that interfere with
Wx?

Well, for what I've tried on the current code, it works ;) I mean the
ipython instance with no GUI support and a wx loop controlling it (my
wxConsole widget)

>  If you had Wx and Qt in the same process, where would the window events
go?  None of these core engines >acknowledge the existence of any other
engines... and would likely behave very badly with the introduction of such
a virus in their process space.

That why I want to do the "xml-rpc/ipython0 in an other process" trick :)

>If you're in a different process, all this is an entirely different
discussion and is more about event IO... none of which requires threading...
Qt, Twisted, and Wx all provide very rich mechanisms to >register callbacks
for this kind of event handling.

Yep I discovered that, before my wx widget was doing some polling on on the
the ipython thread, now I'm using wx callback and no more polling is
necessary, but the thread is still necessary due to while 1: pass lockup. In
a different process n omore thread is necessary and that is what I want :)

Glenn, if I well understood, what you've done is QT support and twisted and
ipython inside the "same loop"?
It could be interesting to see if we could have some sort of unified class
to support the whole thing wx+twisted+ipython and qt+twisted+ipython.
We need to check also Brian warnings :)

Brian:
Do you have a better idea than twisted integration for my specific need
(that will less destroy ipython1)???

Laurent

No virus found in this outgoing message.
Checked by AVG. 
Version: 7.5.519 / Virus Database: 269.21.7/1335 - Release Date: 19/03/2008
09:54
 



From gael.varoquaux at normalesup.org  Fri Mar 21 02:59:09 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 21 Mar 2008 07:59:09 +0100
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <47e2f1c4.0305560a.7758.5828@mx.google.com>
References: <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
	<47e21ccf.0aaa660a.1579.1217@mx.google.com>
	<46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>
	<db6b5ecc0803201047l7e2220fepb7ba2d3c4acb57cd@mail.gmail.com>
	<46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com>
	<6ce0ac130803201515j7366f715t3de24dc585de8cc8@mail.gmail.com>
	<47e2f1c4.0305560a.7758.5828@mx.google.com>
Message-ID: <20080321065909.GC25208@phare.normalesup.org>

On Fri, Mar 21, 2008 at 12:22:31AM +0100, Laurent Dufr?chou wrote:
> Ville:
> I'll take a look at your file this week end, when gael and others guys in
> france will have explained me all the bzr stuff :)

You need to install bzr (under Linux it should be trivial, under windows,
it is just a Python-based program). Once you have it, I made a small note
for the nipy development, that could be useful to you:

http://bazaar.launchpad.net/~nipy-developers/nipy/trunk/annotate/cburns%40berkeley.edu-20080320145517-20ap9pirmqr1no0k?file_id=svn2bzr-006188684113f68f12549ab5c42381681e5b61fc

HTH,

Ga?l


From vivainio at gmail.com  Fri Mar 21 08:57:30 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 21 Mar 2008 14:57:30 +0200
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <6ce0ac130803201149o4001e507w9b2274afa054be81@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205966116.26877.104.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
	<47e21ccf.0aaa660a.1579.1217@mx.google.com>
	<46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>
	<db6b5ecc0803201047l7e2220fepb7ba2d3c4acb57cd@mail.gmail.com>
	<46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com>
	<6ce0ac130803201149o4001e507w9b2274afa054be81@mail.gmail.com>
Message-ID: <46cb515a0803210557u2d453a90h49adafe751a7518@mail.gmail.com>

On Thu, Mar 20, 2008 at 8:49 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:

>  we are handling this in IPython1, please look at that code base.  What
>  ever ends up in IPython0 _must_ be compatible with that.

The stuff in IPython0 is not in the core, so this is not a problem
(the same reason as explained before - we can't have a twisted
dependency).

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From ellisonbg.net at gmail.com  Fri Mar 21 09:24:37 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Fri, 21 Mar 2008 07:24:37 -0600
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <46cb515a0803210557u2d453a90h49adafe751a7518@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
	<47e21ccf.0aaa660a.1579.1217@mx.google.com>
	<46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>
	<db6b5ecc0803201047l7e2220fepb7ba2d3c4acb57cd@mail.gmail.com>
	<46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com>
	<6ce0ac130803201149o4001e507w9b2274afa054be81@mail.gmail.com>
	<46cb515a0803210557u2d453a90h49adafe751a7518@mail.gmail.com>
Message-ID: <6ce0ac130803210624v7ee1a7c6u2365ac23de076333@mail.gmail.com>

On Fri, Mar 21, 2008 at 6:57 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> On Thu, Mar 20, 2008 at 8:49 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
>
>  >  we are handling this in IPython1, please look at that code base.  What
>  >  ever ends up in IPython0 _must_ be compatible with that.
>
>  The stuff in IPython0 is not in the core, so this is not a problem
>  (the same reason as explained before - we can't have a twisted
>  dependency).

Absolutely true.  An optional Twisted dep for IPython0 would be just fine.

Brian

>  --
>
>
> Ville M. Vainio - vivainio.googlepages.com
>  blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
>


From ellisonbg.net at gmail.com  Fri Mar 21 09:28:02 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Fri, 21 Mar 2008 07:28:02 -0600
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <6ce0ac130803210624v7ee1a7c6u2365ac23de076333@mail.gmail.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
	<47e21ccf.0aaa660a.1579.1217@mx.google.com>
	<46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>
	<db6b5ecc0803201047l7e2220fepb7ba2d3c4acb57cd@mail.gmail.com>
	<46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com>
	<6ce0ac130803201149o4001e507w9b2274afa054be81@mail.gmail.com>
	<46cb515a0803210557u2d453a90h49adafe751a7518@mail.gmail.com>
	<6ce0ac130803210624v7ee1a7c6u2365ac23de076333@mail.gmail.com>
Message-ID: <6ce0ac130803210628s1bcdae41tf26a241aa2110174@mail.gmail.com>

>  Absolutely true.  An optional Twisted dep for IPython0 would be just fine.

I just realized that this may be confusing.

Just in terms of dependencies, having IPython0 depend on Twisted
(optionally of course) is not a problem.  But, how IPython0 _uses_
twisted (even though optional) should be compatible with the way
people are currently using Ipython1 for parallel computing - they
simply use it with IPython0.  That is, the any IPython0 Twisted usage
must not break the IPython0+IPython1 usage pattern that is most common
today.

Brian

>  Brian
>
>  >  --
>
>
> >
>  >
>  > Ville M. Vainio - vivainio.googlepages.com
>  >  blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
>  >
>


From ellisonbg.net at gmail.com  Fri Mar 21 16:09:59 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Fri, 21 Mar 2008 14:09:59 -0600
Subject: [IPython-dev] Twisted reactor + IPython0
In-Reply-To: <47e2f1c4.0305560a.7758.5828@mx.google.com>
References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com>
	<46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com>
	<1205991678.3359.8.camel@ghtmyth.hq.tarbox.org>
	<46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com>
	<47e21ccf.0aaa660a.1579.1217@mx.google.com>
	<46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com>
	<db6b5ecc0803201047l7e2220fepb7ba2d3c4acb57cd@mail.gmail.com>
	<46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com>
	<6ce0ac130803201515j7366f715t3de24dc585de8cc8@mail.gmail.com>
	<47e2f1c4.0305560a.7758.5828@mx.google.com>
Message-ID: <6ce0ac130803211309s1036581frba09fc9aa273cbb2@mail.gmail.com>

>  I must clarify my idea:
>  My main objective is to provide a set of classes that anybody can use in
>  QT/WX/whatelse to create easily some ipython console GUI widget
>  (QTIpython,WXIpython,TKIpython etc...)

Great!

>  My work is based on WX but can be easily used for QT or other.I'm using
>  Scintilla for the display so... can be also easy to be ported.
>  (will see this weekend with all the guy how I could interface with
>  pyreadline)
>
>  So,first I wrote a little piece of code to get ipython shell inside a wx
>  widget.
>  It was based on user keyboard interaction with no thread.
>  I used for this a piece of gtk console and IPInteractive Shell class where I
>  could launch some commands line by line.
>  Then arise the problem of infinite loop:
>  While 1:
>         Pass
>  With this my widget and all my GUI was frozen, using a thread where I sent
>  line by line my command to an IPInteractive Shell solved this issue.
>  That's why I needed thread glenn ;)
>
>  A little Ascii drawing:
>
>  Thread wich feeds Ipython instance line by line  <--- communicate -->  WxApp
>  Main loop Thread
>
>  Now I've got this problem:
>  User want to  be able to use Wx Ipython console widget with for example QT4
>  support or WX support...
>  So I see no other idea to give them this class run in an other process where
>  they could launch any GUI unrelated to my widget GUI loop.
>  That's why I wanted xml-rpc and twisted support in ipython0.
>  (If you take a look at this it looks like that ipython1 should be the WAY to
>  do, but I don't know if ipython1 has interactive support like ipython0 and
>  ipython0 NEEDS GUI widgets ;) )

>  So perhaps twisted integration in ipython0 for that is not a clever idea,
>  perhaps a simpler xmlrpc server loop can be OK.
>  I really want to have GUI loop+XML/RPC loop+ipython interactivity in a class
>  that I could call to do all of this.
>
>  Guys,what is the best way?

It really depends on your goals.  If you need something super quick
and are OK hacking together something with IPython0, then just do
that.  But just a warning.  You will end up hacking something together
that only half works and as IPython1 matures, you will end up
re-writing it to use IPython1.

IPython1 is being designed from the ground up to support this type of
thing.  Does IPython1 have interactive support - absolutely!!! -
overwise it would have the "I" in front.  You could start developing
such GUI widgets today using IPython1 and the result would be robust,
well designed code that will last for years.

Example:  Just last week, Barry Wark created a very nice Cocoa App for
IPython1 using PyObjC.  While the resulting app doesn't have all of
the features of IPython0 (magics, tab completion, shell commands,
etc.), those are the easy part.  The important and hard part is
getting the whole design right.  And that is already done in IPython1.
 Depending on how you want to design the GUI stuff, you might need to
learn a bit about twisted, but that is not too bad.

The key point is that if you go this route, you will end up having fun
coding things that are well designed and will stand the test of time,
rather than hacking around trying to square a circle.

We are more than willing to point you to the relevant code in IPython1.

>  Ville:
>  I'll take a look at your file this week end, when gael and others guys in
>  france will have explained me all the bzr stuff :)

Sounds fun!

>  Glenn:
>
>
>  >So, your sketch using XML-RPC as an integration strategy makes sense if
>  you're inter-process but, I think, might be overkill in the same process.
>  Integrating multiple event loops is nasty because they >all "help" by
>  abstracting (hiding) the underlying OS mechanisms which emit events to the
>  application layer.  If we can get around this (as I did with the Qt reactor)
>  we're better off.
>
>  In fact xml-rpc should be used when guys want to play with GUI in Ipython
>  shell widget whithout burry my WXapp.
>  If it not a required thing than, xml-rpc is unusefull sure :), but I also
>  don't want to split my code...
>
>
>  >Furthermore, I wonder if your strategy will work in general even with
>  threads... it all depends on what the other event loop does with the
>  foundational wrapping of the event sources.... so, if you try, >for example,
>  to add your own socket and try and block on select, will that interfere with
>  Wx?
>
>  Well, for what I've tried on the current code, it works ;) I mean the
>  ipython instance with no GUI support and a wx loop controlling it (my
>  wxConsole widget)
>
>
>  >  If you had Wx and Qt in the same process, where would the window events
>  go?  None of these core engines >acknowledge the existence of any other
>  engines... and would likely behave very badly with the introduction of such
>  a virus in their process space.

Yes, multiple GUIs in the same process is a bad idea.

>  That why I want to do the "xml-rpc/ipython0 in an other process" trick :)
>
>
>  >If you're in a different process, all this is an entirely different
>  discussion and is more about event IO... none of which requires threading...
>  Qt, Twisted, and Wx all provide very rich mechanisms to >register callbacks
>  for this kind of event handling.
>
>  Yep I discovered that, before my wx widget was doing some polling on on the
>  the ipython thread, now I'm using wx callback and no more polling is
>  necessary, but the thread is still necessary due to while 1: pass lockup. In
>  a different process n omore thread is necessary and that is what I want :)
>
>  Glenn, if I well understood, what you've done is QT support and twisted and
>  ipython inside the "same loop"?
>  It could be interesting to see if we could have some sort of unified class
>  to support the whole thing wx+twisted+ipython and qt+twisted+ipython.
>  We need to check also Brian warnings :)
>
>  Brian:
>  Do you have a better idea than twisted integration for my specific need
>  (that will less destroy ipython1)???

If you want to got this route (using twisted in ipython0) there is a good route:

1) Start the reactor in a thread

2) Use blockingCallFromThread to call Twisted using methods.
blockingCallFromThread allows you to block in one thread until a
deferred fires in the Twisted thread.

You can see how we are doing this by checking out ipython1 and looking in:

ipython1/kernel/twistedutil.py
ipython1/kernel/client.py
ipython1/kernel/multiengineclient.py


Cheers,

Brian

>  Laurent
>
>
>  No virus found in this outgoing message.
>  Checked by AVG.
>  Version: 7.5.519 / Virus Database: 269.21.7/1335 - Release Date: 19/03/2008
>  09:54
>
>
>
>
>  _______________________________________________
>  IPython-dev mailing list
>  IPython-dev at scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From ellisonbg.net at gmail.com  Fri Mar 21 16:25:41 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Fri, 21 Mar 2008 14:25:41 -0600
Subject: [IPython-dev] Discussion on Bruce Eckel's blog
In-Reply-To: <1206022514.3359.71.camel@ghtmyth.hq.tarbox.org>
References: <db6b5ecc0803200307t6e32eabar2351acaa625d7553@mail.gmail.com>
	<1206022514.3359.71.camel@ghtmyth.hq.tarbox.org>
Message-ID: <6ce0ac130803211325y42205594y24561c9cd4b57dec@mail.gmail.com>

>  Is he trying to discuss the issues of parallelism, threading, co-routine
>  abstractions, inter process-communication?  In the same brief blurb  he
>  discusses and mixes all four.  Makes mention of the GIL and how java
>  multithreading through co-routines (stackless???) are... what?  I got
>  confused even parsing the words.  none of these issues are related at
>  the level of the post.

>  Stack frames, threads, processes, IPC and netowrking are, clearly,
>  related but wildly different subjects... not to mention that none of
>  these have anything to do with solving parallel computing problems other
>  than we might want to use multiple computers and electricity...

I completely agree with this evaluation.  Parallel and distributed
computing is a very complex and rich field and throwing all of these
things together really confuses the problems and solutions.

>  And IPython1?  Sure, there's a notion of a redesign to embrace actual
>  parallel computation issues, and even a couplea examples of how one
>  might use asynchronous models to implement map-reduce (or scatter-gather
>  or whatever)
>
>  But the IPython1 effort is mostly about embracing the asynchronous model
>  (read Twisted) with a sugary sweet layer of IPython to make it a bit
>  less painful.  Unless I missed a bunch of documentation somewhere
>  (certainly possible given the state of the documentation), this hardly

>  rises to: "a powerful architecture for parallel computing,"

So, I must confess that this is language that I am responsible for, so
it is not something that Bruce can take the blame on.  But, I would
argue that the statement is true.  At the same time, I admit that we
(mostly myself) have done a really bad job of laying out why this is
the case.  This is evidenced by the fact that Parallel Processing and
Processing are being spoken of as the leading solutions in this area -
even though IPython1 is more mature and technically superior.  I
should say though that I really like both Parallel Processing and
Processing - I just think we are trying to accomplish something much
more ambitious with IPython1 and are having good successes with our
efforts.

>  IPython1 has a notion
>  that there should be an architecture and even some early examples... but
>  there's no new model, no power other than doing stuff correctly with
>  modern tools.

I like this thinking, even though I don't fully agree.  The ironic
thing is that "doing stuff correctly with modern tools" is extremely
difficult and uncommon when it comes to parallel and distributed
computing.  Where I disagree slightly is that I think you do get
something really powerful when you 1) have a well designed
architecture and 2) do things correctly with modern tools.

Cheers,

Brian


From steve at shrogers.com  Fri Mar 21 22:17:22 2008
From: steve at shrogers.com (Steven H. Rogers)
Date: Fri, 21 Mar 2008 20:17:22 -0600
Subject: [IPython-dev] Discussion on Bruce Eckel's blog
In-Reply-To: <6ce0ac130803211325y42205594y24561c9cd4b57dec@mail.gmail.com>
References: <db6b5ecc0803200307t6e32eabar2351acaa625d7553@mail.gmail.com>	<1206022514.3359.71.camel@ghtmyth.hq.tarbox.org>
	<6ce0ac130803211325y42205594y24561c9cd4b57dec@mail.gmail.com>
Message-ID: <47E46C32.1040006@shrogers.com>

Brian Granger wrote:
>>  Stack frames, threads, processes, IPC and netowrking are, clearly,
>>  related but wildly different subjects... not to mention that none of
>>  these have anything to do with solving parallel computing problems other
>>  than we might want to use multiple computers and electricity...
>>     
>
> I completely agree with this evaluation.  Parallel and distributed
> computing is a very complex and rich field and throwing all of these
> things together really confuses the problems and solutions.
> ...
>   
Bruce seems to be a bit confused, but I think this represents the state 
of most programmers w.r.t. concurrent, parallel, and distributed 
processing.  They want to continue programming the way they're 
accustomed to and just have it magically take advantage of parallel 
and/or distributed hardware.  They're not interested in, and don't 
really want to know about, the technical difficulties.
> ...  This is evidenced by the fact that Parallel Processing and
> Processing are being spoken of as the leading solutions in this area -
> even though IPython1 is more mature and technically superior.  I
> should say though that I really like both Parallel Processing and
> Processing - I just think we are trying to accomplish something much
> more ambitious with IPython1 and are having good successes with our
> efforts
Those successes need to be communicated.  I believe a PEP in in 
preparation to include some sort of parallel solution into the Python 
Standard Library. 

# Steve


From fperez.net at gmail.com  Sat Mar 22 10:45:43 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 22 Mar 2008 07:45:43 -0700
Subject: [IPython-dev] #ipython on IRC (freenode server)
Message-ID: <db6b5ecc0803220745r5e7e9885vc3c4de63ba9f939a@mail.gmail.com>

Hi all,

we've joined the (existing, but seldom used) #ipython channel on
freenode.  I know we've been doing a poor job of keeping the rest of
the wider user/developer community up to date with the development
plans and work on ipython, but we've also promised to do better.

This is the first (small) step in that direction: for the next two
days, Stefan van der Walt, Gael Varoquaux, Laurent Dufrechou (who
recently contributed the Wx shell) and myself are hanging out at
Gael's parent's house south of Paris, working on kickstarting the
integration work for ip0/ip1, building on top of all of the recent
flurry of activity.

We'll also gradually document better the development process, figure
out what to do about Trac and old tickets, and hopefully in a few
months we'll not only have more functioning code, but also a more
active community that really encourages *anyone* who is interested in
the project and willing to participate, to do so.

Cheers,

f


From ondrej at certik.cz  Sat Mar 22 23:41:33 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Sun, 23 Mar 2008 04:41:33 +0100
Subject: [IPython-dev] sphinx documentation for ipython1
Message-ID: <85b5c3130803222041mbb83d34x397c6dda646d0778@mail.gmail.com>

Hi,

I committed a nice sphinx based documentation to the ipython1-doc
branch. You can try it out for example by:

$ bzr branch http://bazaar.launchpad.net/~ipython/ipython/ipython1-doc
$ cd ipython1-doc/docs
$ make html
$ epiphany _build/html/index.html

(Use your favourite browser instead of epiphany).

The "make html" will download Sphinx from the net (so you need to have
an internet connection) and puts it into the sphinx directory in the
current dir. Subsequent invocations of "make html" will reuse it. I
suggest you to browse the history of my commits to see what exact
changes I had to do in order to make all this work.

The changes broke the pdf generation - this would have to be fixed,
but I suggest to get at least the html documentation right and
polished, only then try to fix the pdf generation.
Also the Sphinx framework should be able to generate pdf files too.

Unfortunately it took my almost the whole night to fix all things
(it's 4:40am), so I won't be able to help much with ipython on Sunday,
as I will have to work on other things. But I'll be available if you
need to explain anything regarding my patches.

Enjoy hacking at Gaels,
Ondrej


From gael.varoquaux at normalesup.org  Sun Mar 23 06:13:13 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 23 Mar 2008 11:13:13 +0100
Subject: [IPython-dev] sphinx documentation for ipython1
In-Reply-To: <85b5c3130803222041mbb83d34x397c6dda646d0778@mail.gmail.com>
References: <85b5c3130803222041mbb83d34x397c6dda646d0778@mail.gmail.com>
Message-ID: <20080323101313.GA14881@phare.normalesup.org>

On Sun, Mar 23, 2008 at 04:41:33AM +0100, Ondrej Certik wrote:
> I committed a nice sphinx based documentation to the ipython1-doc
> branch. You can try it out for example by:

> $ bzr branch http://bazaar.launchpad.net/~ipython/ipython/ipython1-doc
> $ cd ipython1-doc/docs
> $ make html
> $ epiphany _build/html/index.html

> (Use your favourite browser instead of epiphany).

Very sweet. Nice !

Ga?l


From fperez.net at gmail.com  Sun Mar 23 06:49:44 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 23 Mar 2008 03:49:44 -0700
Subject: [IPython-dev] sphinx documentation for ipython1
In-Reply-To: <85b5c3130803222041mbb83d34x397c6dda646d0778@mail.gmail.com>
References: <85b5c3130803222041mbb83d34x397c6dda646d0778@mail.gmail.com>
Message-ID: <db6b5ecc0803230349l67608002hd6ab54efc5995499@mail.gmail.com>

On Sat, Mar 22, 2008 at 8:41 PM, Ondrej Certik <ondrej at certik.cz> wrote:
> Hi,
>
>  I committed a nice sphinx based documentation to the ipython1-doc
>  branch. You can try it out for example by:
>
>  $ bzr branch http://bazaar.launchpad.net/~ipython/ipython/ipython1-doc
>  $ cd ipython1-doc/docs
>  $ make html
>  $ epiphany _build/html/index.html
>
>  (Use your favourite browser instead of epiphany).

Sweet!  The built-in search (jscript tricks, I presume) is absolutely
phenomenal.  I'm updating the cron job to pull from this so everyone
has access to these docs, we should merge this very soon into the dev
branch.

>  The "make html" will download Sphinx from the net (so you need to have
>  an internet connection) and puts it into the sphinx directory in the
>  current dir. Subsequent invocations of "make html" will reuse it. I
>  suggest you to browse the history of my commits to see what exact
>  changes I had to do in order to make all this work.
>
>  The changes broke the pdf generation - this would have to be fixed,
>  but I suggest to get at least the html documentation right and
>  polished, only then try to fix the pdf generation.
>  Also the Sphinx framework should be able to generate pdf files too.
>
>  Unfortunately it took my almost the whole night to fix all things
>  (it's 4:40am), so I won't be able to help much with ipython on Sunday,
>  as I will have to work on other things. But I'll be available if you
>  need to explain anything regarding my patches.

Thanks so much!  This is great, I hope you can get some rest, and
don't worry for now.  Let me know when you have a minute to look at
the xml2rst business for the other docs, since that would be the
remaining piece missing.

>
>  Enjoy hacking at Gaels,

We certainly are: the hospitality given by him and his parents has
been amazing.  Chalk one to great open source development sponsorship!

Cheers,

f


From fperez.net at gmail.com  Sun Mar 23 19:17:10 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 23 Mar 2008 16:17:10 -0700
Subject: [IPython-dev] Docs for ipython1
Message-ID: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>

Hi all,

just to ping those who might not have been hanging on IRC, thanks to
Ondrej's work (completing what was started on the previous meeting at
Enthought), we now have:

http://ipython.scipy.org/doc/ipython1/html/
http://ipython.scipy.org/doc/ipython1/ipython1.pdf

these are the docs for ipython1 in html and pdf, using sphinx, the
Python 2.6 official documentation system.  These docs are guaranteed
to be up to date, since they get built every night from the main bzr
branch.

As I recently said, I know we've done a poor job of recently
communicating our development effort and keeping everyone informed,
involved and welcome.  I hope that better docs and this weekend's
interaction on IRC are proof that we mean it when we say that we're
trying to do better.

I'd like to mention that while sprinting on IRC, when we decide to
jump in on skype for a direct discussion *everyone* is welcome!  Skype
allows fairly large conference calls, and we'd love to have others on
board.  We don't yet have funds for large face to face ipython
meetings (except when piggybacking on other projects), so irc+skype
will have to do.  But anyone is welcome to participate if we switch
over to skype briefly, so please do NOT interpret such switches on IRC
as a 'closing of the doors to the cognoscenti'.  It is just a
practical tool to talk rapidly about a given topic beyond what typing
on IRC allows.

Cheers,

f


From ondrej at certik.cz  Sun Mar 23 21:41:54 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 24 Mar 2008 02:41:54 +0100
Subject: [IPython-dev] Docs for ipython1
In-Reply-To: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
Message-ID: <85b5c3130803231841y45346051yb056a333c7e5a766@mail.gmail.com>

On Mon, Mar 24, 2008 at 12:17 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi all,
>
>  just to ping those who might not have been hanging on IRC, thanks to
>  Ondrej's work (completing what was started on the previous meeting at
>  Enthought), we now have:
>
>  http://ipython.scipy.org/doc/ipython1/html/
>  http://ipython.scipy.org/doc/ipython1/ipython1.pdf
>
>  these are the docs for ipython1 in html and pdf, using sphinx, the
>  Python 2.6 official documentation system.  These docs are guaranteed
>  to be up to date, since they get built every night from the main bzr
>  branch.

You meantioned that you'd like to have the search box - so I asked
upstream and Georg fixed it in the Sphinx svn, so if you download
the svn, you'll get a search box on every page.

Ondrej


From vivainio at gmail.com  Mon Mar 24 05:14:57 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 24 Mar 2008 11:14:57 +0200
Subject: [IPython-dev] IPython winpdb interface
In-Reply-To: <47E6E860.30702@winpdb.org>
References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com>
	<47E6E860.30702@winpdb.org>
Message-ID: <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com>

[CC-ing to ipython-dev, about adding winpdb support to IPython]

On Mon, Mar 24, 2008 at 1:31 AM, Nir <nir at winpdb.org> wrote:

>  Lets start with solving your initial scenario.
>  Can you describe how is such an ipython session supposed to look like?

If you now execute

%run -d test.py

IPython now sets the breakpoint at start of test.py and launches it in
pdb, running the test.py in ipython process.

I'd like to add a new magic command %wpdb (in an extension, say
ipy_winpdb), so you can do

%wpdb test.py

which will do the same thing, i.e. launch test.py in the ipython
process, with rpdb2 breakpoint at start of the script. The rpdb2
should only be started once in the IPython process.

Likewise, if the ipy_winpdb extension is loaded and post-mortem
debugging is enabled, hitting an exception would break rpdb2 at the
exception.

All of this is pretty easy in ipython side, we only need  API support
in rpdb2 module to create a breakpoint at first valid position in a
file on file system.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Mon Mar 24 05:30:24 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 24 Mar 2008 11:30:24 +0200
Subject: [IPython-dev] Docs for ipython1
In-Reply-To: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
Message-ID: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>

On Mon, Mar 24, 2008 at 1:17 AM, Fernando Perez <fperez.net at gmail.com> wrote:

>  http://ipython.scipy.org/doc/ipython1/html/
>  http://ipython.scipy.org/doc/ipython1/ipython1.pdf
>
>  these are the docs for ipython1 in html and pdf, using sphinx, the
>  Python 2.6 official documentation system.  These docs are guaranteed

And aren't they pretty!

There still seems to be some reST artifacts in code examples e.g. here:

http://ipython.scipy.org/doc/ipython1/html/parallel_task.html#quickstart-task-farming

Ondrej, any idea when you could have something like this up for
IPython0 as well?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Mon Mar 24 05:35:50 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 24 Mar 2008 02:35:50 -0700
Subject: [IPython-dev] Sprint over, offline...
Message-ID: <db6b5ecc0803240235n1a42399by7915d7133722fceb@mail.gmail.com>

Hi all,

I just wanted to briefly thank Gael, AND his family, for the great
hospitality during the sprint.  We got a lot done, and I hope this
will kickstart a lot of things that had been stalling for  a while.

I have to run now and I'll be offline until the end of the week, and I
think so will Gael and Stefan, but we'll do a little recap later.

Thanks to everyone who participated via irc/skype, and see you all soon!

Cheers,

f


From ondrej at certik.cz  Mon Mar 24 08:50:34 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 24 Mar 2008 13:50:34 +0100
Subject: [IPython-dev] Docs for ipython1
In-Reply-To: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>
Message-ID: <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>

On Mon, Mar 24, 2008 at 10:30 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> On Mon, Mar 24, 2008 at 1:17 AM, Fernando Perez <fperez.net at gmail.com> wrote:
>
>  >  http://ipython.scipy.org/doc/ipython1/html/
>  >  http://ipython.scipy.org/doc/ipython1/ipython1.pdf
>  >
>  >  these are the docs for ipython1 in html and pdf, using sphinx, the
>  >  Python 2.6 official documentation system.  These docs are guaranteed
>
>  And aren't they pretty!
>
>  There still seems to be some reST artifacts in code examples e.g. here:
>
>  http://ipython.scipy.org/doc/ipython1/html/parallel_task.html#quickstart-task-farming

We had a similar problem in sympy with different artefacts. Please
report them to Georg on the doc-sig mailinglist. He fixed our problems
in about 2 hours.

>  Ondrej, any idea when you could have something like this up for
>  IPython0 as well?

The xml docs need to be converted, but I won't have time for this
anytime soon unfortunately. But I just updated the ipython1 docs to
the latest (fixed) Sphinx, so now the search box appears on every
page.

Ondrej


From ellisonbg.net at gmail.com  Mon Mar 24 12:11:58 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Mon, 24 Mar 2008 10:11:58 -0600
Subject: [IPython-dev] Docs for ipython1
In-Reply-To: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>
Message-ID: <6ce0ac130803240911h7a27e4beh28ba75f78d4d4d2d@mail.gmail.com>

>  There still seems to be some reST artifacts in code examples e.g. here:
>
>  http://ipython.scipy.org/doc/ipython1/html/parallel_task.html#quickstart-task-farming

Where are the artifacts exactly?  I looked over that section and
couldn't see any obvious ones.

Brian


From vivainio at gmail.com  Mon Mar 24 12:16:22 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 24 Mar 2008 18:16:22 +0200
Subject: [IPython-dev] Docs for ipython1
In-Reply-To: <6ce0ac130803240911h7a27e4beh28ba75f78d4d4d2d@mail.gmail.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>
	<6ce0ac130803240911h7a27e4beh28ba75f78d4d4d2d@mail.gmail.com>
Message-ID: <46cb515a0803240916r52701d26jb52efe529e9117ef@mail.gmail.com>

On Mon, Mar 24, 2008 at 6:11 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
> >  There still seems to be some reST artifacts in code examples e.g. here:
>  >
>  >  http://ipython.scipy.org/doc/ipython1/html/parallel_task.html#quickstart-task-farming
>
>  Where are the artifacts exactly?  I looked over that section and
>  couldn't see any obvious ones.

Stuff like this:

In [8]: tr = ``Task``_results[73]

In [9]: tr
Out[9]: ``TaskResult``[ID:73]:{'a':73}


-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From ellisonbg.net at gmail.com  Mon Mar 24 12:42:25 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Mon, 24 Mar 2008 10:42:25 -0600
Subject: [IPython-dev] Docs for ipython1
In-Reply-To: <46cb515a0803240916r52701d26jb52efe529e9117ef@mail.gmail.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>
	<6ce0ac130803240911h7a27e4beh28ba75f78d4d4d2d@mail.gmail.com>
	<46cb515a0803240916r52701d26jb52efe529e9117ef@mail.gmail.com>
Message-ID: <6ce0ac130803240942n589d7d82g8ce630a81ee21cba@mail.gmail.com>

Ooops, I will follow this up.  Thanks

Brian

On Mon, Mar 24, 2008 at 10:16 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
>
> On Mon, Mar 24, 2008 at 6:11 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
>  > >  There still seems to be some reST artifacts in code examples e.g. here:
>  >  >
>  >  >  http://ipython.scipy.org/doc/ipython1/html/parallel_task.html#quickstart-task-farming
>  >
>  >  Where are the artifacts exactly?  I looked over that section and
>  >  couldn't see any obvious ones.
>
>  Stuff like this:
>
>  In [8]: tr = ``Task``_results[73]
>
>  In [9]: tr
>  Out[9]: ``TaskResult``[ID:73]:{'a':73}
>
>
>
>
>  --
>  Ville M. Vainio - vivainio.googlepages.com
>  blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
>


From laurent.dufrechou at gmail.com  Mon Mar 24 12:42:36 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Mon, 24 Mar 2008 17:42:36 +0100
Subject: [IPython-dev] IPython winpdb interface
In-Reply-To: <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com>
References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com>	<47E6E860.30702@winpdb.org>
	<46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com>
Message-ID: <47e7da01.0407560a.53ef.0285@mx.google.com>

Hi ville,

That's a GOOD idea :)
I'm used to debug my script with winpdb crossplatform debugger and I'm quite
happy to listen this :)

Does the %wpdb will:
- launch the winpdb GUI or
- only the server and we will have to launch the winpdb gui and attach to
the server manually or 
- usage is only via rdpb server without winpdb GUI?

What are your plan about the usage scenario??

-----Message d'origine-----
De?: ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De
la part de Ville M. Vainio
Envoy??: lundi 24 mars 2008 10:15
??: Nir; ipython-dev Mailing list
Objet?: Re: [IPython-dev] IPython winpdb interface

[CC-ing to ipython-dev, about adding winpdb support to IPython]

On Mon, Mar 24, 2008 at 1:31 AM, Nir <nir at winpdb.org> wrote:

>  Lets start with solving your initial scenario.
>  Can you describe how is such an ipython session supposed to look like?

If you now execute

%run -d test.py

IPython now sets the breakpoint at start of test.py and launches it in
pdb, running the test.py in ipython process.

I'd like to add a new magic command %wpdb (in an extension, say
ipy_winpdb), so you can do

%wpdb test.py

which will do the same thing, i.e. launch test.py in the ipython
process, with rpdb2 breakpoint at start of the script. The rpdb2
should only be started once in the IPython process.

Likewise, if the ipy_winpdb extension is loaded and post-mortem
debugging is enabled, hitting an exception would break rpdb2 at the
exception.

All of this is pretty easy in ipython side, we only need  API support
in rpdb2 module to create a breakpoint at first valid position in a
file on file system.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev

No virus found in this incoming message.
Checked by AVG. 
Version: 7.5.519 / Virus Database: 269.21.8/1340 - Release Date: 23/03/2008
18:50
 

No virus found in this outgoing message.
Checked by AVG. 
Version: 7.5.519 / Virus Database: 269.21.8/1340 - Release Date: 23/03/2008
18:50
 



From fperez.net at gmail.com  Mon Mar 24 18:58:56 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 24 Mar 2008 15:58:56 -0700
Subject: [IPython-dev] Docs for ipython1
In-Reply-To: <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>
	<85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>
Message-ID: <db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>

On Mon, Mar 24, 2008 at 5:50 AM, Ondrej Certik <ondrej at certik.cz> wrote:

>  The xml docs need to be converted, but I won't have time for this
>  anytime soon unfortunately. But I just updated the ipython1 docs to
>  the latest (fixed) Sphinx, so now the search box appears on every
>  page.

That's great (the search box)!  I just ran the update by hand and now
the search box is now always visible on the side.  This is great!!!

And don't worry about ip0 for now, if you can help us later that's
great, but you've already done a lot!

Cheers,

f


From laurent.dufrechou at gmail.com  Mon Mar 24 19:14:43 2008
From: laurent.dufrechou at gmail.com (=?windows-1250?Q?Laurent_Dufr=E9chou?=)
Date: Tue, 25 Mar 2008 00:14:43 +0100
Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk
In-Reply-To: <db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>	<85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>
	<db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
Message-ID: <47e835ea.0c07560a.5adf.ffffb9c5@mx.google.com>

Hi all,
Where can I put some info on how to install bzr under Vista?
I've installed it with the .exe that is a big error and nobody must do that.
So I would like to warn newcomers how to set up bzr + bzr gui tools without
too much pain...

I had installed bzr via the .exe install that is completely uncompatible
with the olive-gtk gui (!)
I had to deinstall bzr and install the bzr python installer,(setup.py
install as administrator).
Now bzr viz and all other things works.

Here is an interesting link:
http://bazaar-vcs.org/MariusKruger#head-e234ddc518938afb140793d47320a2bee0b6
c8aa
(a bit old, but you can easily find newer packages)

and
http://bazaar-vcs.org/Download
(DO not use the standalone installer or will not have a working plugin
directory!)

Cheers,
Laurent

No virus found in this outgoing message.
Checked by AVG. 
Version: 7.5.519 / Virus Database: 269.21.8/1340 - Release Date: 23/03/2008
18:50
 



From fperez.net at gmail.com  Mon Mar 24 22:13:14 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 24 Mar 2008 19:13:14 -0700
Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk
In-Reply-To: <47e835ea.0c07560a.5adf.ffffb9c5@mx.google.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>
	<85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>
	<db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
	<47e835ea.0c07560a.5adf.ffffb9c5@mx.google.com>
Message-ID: <db6b5ecc0803241913sf681490wfe015b59cfeeca2f@mail.gmail.com>

Hi Laurent,

On Mon, Mar 24, 2008 at 4:14 PM, Laurent Dufr?chou
<laurent.dufrechou at gmail.com> wrote:
> Hi all,
>  Where can I put some info on how to install bzr under Vista?
>  I've installed it with the .exe that is a big error and nobody must do that.
>  So I would like to warn newcomers how to set up bzr + bzr gui tools without
>  too much pain...

That would be fantastic.   Ideally, I'd like to have more of these
docs in proper reST under source control, so I'd prefer it if you
contribute it in the docs themselves.  The right file for this would
probably be

http://ipython.scipy.org/doc/ipython1/html/development.html#version-control

which is actually in this file (lovely bzr links, if anyone knows how
to do better, let me know):

http://bazaar.launchpad.net/~ipython/ipython/ipython1-dev/annotate/ondrej%40certik.cz-20080324124519-t9fsj9phqqzue1x1?file_id=development.txt-20080302203151-17poomviavsf2zjj-18

Eventually we should have a way to have users easily edit via the wiki
these files, so that any user can contribute easily to the rest
sources without even having to send a patch, just edit a wiki page and
we get a patch automatically.  Emanuelle (Gael's super-hacker
girlfriend, who also participatedin the sprint!) has written some
tools to start building some such functionality, and Stefan may push
that further for numpy later, so I'm sure eventually we'll get all of
that.

In the meantime, just edit the source documents providing this
information in the proper place.  If you have any doubts and would
like a review, feel free to work on it on a private branch on your
+junk/ launchpad scratch area, so that others can look at it and help
you out.  Make a branch where these are the *only* changes so it's
easy for someone else to pull them, review then and give you quick
feedback without any danger of pulling changes to anything else.  Also
keep each commit reasonably small, so it's easy later to cherrypick if
needed.

We currently are nowhere near the necessary manpower to have a formal
policy of patch review before commits (such as Sage uses, for
example), but I think it's a good idea to get a bit of review when
you're becoming familiar with new parts of the tree you hadn't gone
into before.  We're also trying to be much more strict with
documentation thoroughness and testing in ipython1 than we ever were
in ip0.

Cheers,

f


From vivainio at gmail.com  Tue Mar 25 04:17:33 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 25 Mar 2008 10:17:33 +0200
Subject: [IPython-dev] Docs for ipython1
In-Reply-To: <db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>
	<85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>
	<db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
Message-ID: <46cb515a0803250117i48b3212dr9c334598a3dc2a7f@mail.gmail.com>

On Tue, Mar 25, 2008 at 12:58 AM, Fernando Perez <fperez.net at gmail.com> wrote:

>  And don't worry about ip0 for now, if you can help us later that's
>  great, but you've already done a lot!

I'm doing the ip0 thing. There is already ipython.rst in doc/, but I
need to fix the sphinx setup. It already creates the full document,
but there are some warnings to clear up.

We should have a pretty IPython0 document up soon, and I'll also make
it a bit more up to date before 0.8.3.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Tue Mar 25 08:10:45 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 25 Mar 2008 05:10:45 -0700
Subject: [IPython-dev] Docs for ipython1
In-Reply-To: <46cb515a0803250117i48b3212dr9c334598a3dc2a7f@mail.gmail.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>
	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>
	<85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>
	<db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
	<46cb515a0803250117i48b3212dr9c334598a3dc2a7f@mail.gmail.com>
Message-ID: <db6b5ecc0803250510v1d8271a5s5e9a76b2e5efe498@mail.gmail.com>

On Tue, Mar 25, 2008 at 1:17 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> On Tue, Mar 25, 2008 at 12:58 AM, Fernando Perez <fperez.net at gmail.com> wrote:
>
>  >  And don't worry about ip0 for now, if you can help us later that's
>  >  great, but you've already done a lot!
>
>  I'm doing the ip0 thing. There is already ipython.rst in doc/, but I
>  need to fix the sphinx setup. It already creates the full document,
>  but there are some warnings to clear up.
>
>  We should have a pretty IPython0 document up soon, and I'll also make
>  it a bit more up to date before 0.8.3.

Great!

Cheers,

f


From ondrej at certik.cz  Tue Mar 25 09:48:16 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Tue, 25 Mar 2008 14:48:16 +0100
Subject: [IPython-dev] IPython winpdb interface
In-Reply-To: <47e7da01.0407560a.53ef.0285@mx.google.com>
References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com>
	<47E6E860.30702@winpdb.org>
	<46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com>
	<47e7da01.0407560a.53ef.0285@mx.google.com>
Message-ID: <85b5c3130803250648u47b0e3a6qeceb76417c32b48e@mail.gmail.com>

>
>  -----Message d'origine-----
>  De : ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De
>  la part de Ville M. Vainio
>  Envoy? : lundi 24 mars 2008 10:15
>  ? : Nir; ipython-dev Mailing list
>  Objet : Re: [IPython-dev] IPython winpdb interface
>
>
>
>  [CC-ing to ipython-dev, about adding winpdb support to IPython]
>
>  On Mon, Mar 24, 2008 at 1:31 AM, Nir <nir at winpdb.org> wrote:
>
>  >  Lets start with solving your initial scenario.
>  >  Can you describe how is such an ipython session supposed to look like?
>
>  If you now execute
>
>  %run -d test.py
>
>  IPython now sets the breakpoint at start of test.py and launches it in
>  pdb, running the test.py in ipython process.
>
>  I'd like to add a new magic command %wpdb (in an extension, say
>  ipy_winpdb), so you can do
>
>  %wpdb test.py
>
>  which will do the same thing, i.e. launch test.py in the ipython
>  process, with rpdb2 breakpoint at start of the script. The rpdb2
>  should only be started once in the IPython process.
>
>  Likewise, if the ipy_winpdb extension is loaded and post-mortem
>  debugging is enabled, hitting an exception would break rpdb2 at the
>  exception.
>
>  All of this is pretty easy in ipython side, we only need  API support
>  in rpdb2 module to create a breakpoint at first valid position in a
>  file on file system.

That's a very good idea indeed. I am also using winpdb almost
exclusively for any debugging.

Ondrej

From vivainio at gmail.com  Tue Mar 25 11:35:35 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 25 Mar 2008 17:35:35 +0200
Subject: [IPython-dev] IPython winpdb interface
In-Reply-To: <47e7da01.0407560a.53ef.0285@mx.google.com>
References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com>
	<47E6E860.30702@winpdb.org>
	<46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com>
	<47e7da01.0407560a.53ef.0285@mx.google.com>
Message-ID: <46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com>

On Mon, Mar 24, 2008 at 6:42 PM, Laurent Dufr?chou
<laurent.dufrechou at gmail.com> wrote:

> Hi ville,
>
>  That's a GOOD idea :)

I think it will be handy too (pdb is a bit dated, usagewise). Now it's
up to Nir to create some kind of rpdb2 api we can use.

>  Does the %wpdb will:
>  - launch the winpdb GUI or
>  - only the server and we will have to launch the winpdb gui and attach to
>  the server manually or
>  - usage is only via rdpb server without winpdb GUI?

We will only launch the rpdb2 server and let the user launch winpdb
manually. The idea is to have rpdb2 running inside IPython process
constantly, taking care of all the debugging needs... we can store the
password to _ip.db, so it does not need to be entered every time
IPython is launched.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From laurent.dufrechou at gmail.com  Tue Mar 25 11:57:36 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Tue, 25 Mar 2008 16:57:36 +0100
Subject: [IPython-dev] IPython winpdb interface
In-Reply-To: <46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com>
References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com>	
	<47E6E860.30702@winpdb.org>	
	<46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com>	
	<47e7da01.0407560a.53ef.0285@mx.google.com>
	<46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com>
Message-ID: <47e920fb.0405560a.1927.ffffd4d4@mx.google.com>

Ok I see, perhaps Nir will confirm, but if you manage to make rpdb server
run inside ipython, I think we will be able to launch the GUI and set
breakpoint and do step in/out etc...(this is what you say no?)
And I'm quite sure that it ever exist an API to manage it (if you want to
control it via ipython) because Nir said me a long time ago that PIDA was
using it like this...(if I remember well)...

In which bzr branch do you plan to play with this, mtexp or stable one??



-----Message d'origine-----
De?: Ville M. Vainio [mailto:vivainio at gmail.com] 
Envoy??: mardi 25 mars 2008 16:36
??: Laurent Dufr?chou
Cc?: Nir; ipython-dev Mailing list
Objet?: Re: [IPython-dev] IPython winpdb interface

On Mon, Mar 24, 2008 at 6:42 PM, Laurent Dufr?chou
<laurent.dufrechou at gmail.com> wrote:

> Hi ville,
>
>  That's a GOOD idea :)

I think it will be handy too (pdb is a bit dated, usagewise). Now it's
up to Nir to create some kind of rpdb2 api we can use.

>  Does the %wpdb will:
>  - launch the winpdb GUI or
>  - only the server and we will have to launch the winpdb gui and attach to
>  the server manually or
>  - usage is only via rdpb server without winpdb GUI?

We will only launch the rpdb2 server and let the user launch winpdb
manually. The idea is to have rpdb2 running inside IPython process
constantly, taking care of all the debugging needs... we can store the
password to _ip.db, so it does not need to be entered every time
IPython is launched.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'



From vivainio at gmail.com  Tue Mar 25 12:38:23 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 25 Mar 2008 18:38:23 +0200
Subject: [IPython-dev] IPython winpdb interface
In-Reply-To: <47e920fb.0405560a.1927.ffffd4d4@mx.google.com>
References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com>
	<47E6E860.30702@winpdb.org>
	<46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com>
	<47e7da01.0407560a.53ef.0285@mx.google.com>
	<46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com>
	<47e920fb.0405560a.1927.ffffd4d4@mx.google.com>
Message-ID: <46cb515a0803250938h5ecfc531hb8933bf18a4b579b@mail.gmail.com>

On Tue, Mar 25, 2008 at 5:57 PM, Laurent Dufr?chou
<laurent.dufrechou at gmail.com> wrote:

>  And I'm quite sure that it ever exist an API to manage it (if you want to
>  control it via ipython) because Nir said me a long time ago that PIDA was
>  using it like this...(if I remember well)...

If I understood correctly. no such API exist yet.

>  In which bzr branch do you plan to play with this, mtexp or stable one??

Depends on time we get an API. This will end up post-0.8.3.

BTW, mtexp is dead, it has already been integrated to trunk.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Tue Mar 25 15:42:11 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 25 Mar 2008 21:42:11 +0200
Subject: [IPython-dev] IPython winpdb interface
In-Reply-To: <47E939BA.4080104@winpdb.org>
References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com>
	<47E6E860.30702@winpdb.org>
	<46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com>
	<47e7da01.0407560a.53ef.0285@mx.google.com>
	<46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com>
	<47E939BA.4080104@winpdb.org>
Message-ID: <46cb515a0803251242g44663a5ch35bdaaa3c9193799@mail.gmail.com>

On Tue, Mar 25, 2008 at 7:43 PM, Nir <nir at winpdb.org> wrote:

>  To do that type in ipython:
>  import rpdb2; rpdb2.start_embedded_debugger(<password>)

Yes, that is not a problem.

>  If I understand Ville, he is interested in usability similar to ipydb.
>  This can be arranged.

I am interested in setting breakpoints programmatically - this is
needed for %run -d like functionality.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From bialix at ukr.net  Tue Mar 25 16:07:47 2008
From: bialix at ukr.net (Alexander Belchenko)
Date: Tue, 25 Mar 2008 22:07:47 +0200
Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk
In-Reply-To: <47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>	<85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>	<db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
	<47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com>
Message-ID: <47E95B93.2050004@ukr.net>

Laurent Dufr?chou ?????:
> Hi all,
> Where can I put some info on how to install bzr under Vista?

It will be nice if you put the copy of your instruction to http://bazaar-vcs.org.
It's a wiki in the end.

There is already instructions how to install it:
http://bazaar-vcs.org/WindowsInstall

Can you tell me is this instructions is wrong or not good enough?

> I've installed it with the .exe that is a big error and nobody must do that.

It's shame for me personally to hear this.

> So I would like to warn newcomers how to set up bzr + bzr gui tools without
> too much pain...

QBzr GUI is compatible with bzr.exe. Of course, it's not the GTK one.

> I had installed bzr via the .exe install that is completely uncompatible
> with the olive-gtk gui (!)

IMO, it's not true. Here is package of bzr-gtk compatible with bzr.exe:
http://d5190871.u44.websitesource.net/bzr/

There is bzr-gtk-0.93.0 release only, but I don't follow bzr-gtk development,
so I'm not sure what's latest release is.

> I had to deinstall bzr and install the bzr python installer,(setup.py
> install as administrator).
> Now bzr viz and all other things works.
> 
> Here is an interesting link:
> http://bazaar-vcs.org/MariusKruger#head-e234ddc518938afb140793d47320a2bee0b6
> c8aa
> (a bit old, but you can easily find newer packages)
> 
> and
> http://bazaar-vcs.org/Download

> (DO not use the standalone installer or will not have a working plugin
> directory!)

I don't understand this warning. Can you explain it a bit more?



From vivainio at gmail.com  Tue Mar 25 16:06:16 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 25 Mar 2008 22:06:16 +0200
Subject: [IPython-dev] IPython winpdb interface
In-Reply-To: <47E9596F.7030304@winpdb.org>
References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com>
	<47E6E860.30702@winpdb.org>
	<46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com>
	<47e7da01.0407560a.53ef.0285@mx.google.com>
	<46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com>
	<47E939BA.4080104@winpdb.org>
	<46cb515a0803251242g44663a5ch35bdaaa3c9193799@mail.gmail.com>
	<47E9596F.7030304@winpdb.org>
Message-ID: <46cb515a0803251306m5fcb412amfcf693584f893a46@mail.gmail.com>

On Tue, Mar 25, 2008 at 9:58 PM, Nir <nir at winpdb.org> wrote:

>  > I am interested in setting breakpoints programmatically - this is
>  > needed for %run -d like functionality.
>  >
>  Do you intend to have a debugger console?

Not really. Being able to set the initial brekapoint is enough.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From laurent.dufrechou at gmail.com  Tue Mar 25 17:10:34 2008
From: laurent.dufrechou at gmail.com (=?UTF-8?Q?Laurent_Dufr=C3=A9chou?=)
Date: Tue, 25 Mar 2008 22:10:34 +0100
Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk
In-Reply-To: <47E95B93.2050004@ukr.net>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>	<85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>	<db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
	<47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com>
	<47E95B93.2050004@ukr.net>
Message-ID: <47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com>

Hi Alexander,
You are the maintainer of the .exe version??
In fact I've felt in some problem installing the .exe.
Sorry for my previous email, it was more a warning on my usage case than a 'these guys have done a bad job'!
So let me explain my problems (and perhaps you will tell that I've done somethng bad!), if not let's find solutions to improve things :)

I've used bazar first installing bazar with the .exe install. (1.3 version)
It added bzr here in the context menu when you right click on a directory.
Since the install the context seemed sluggish, AND bugguy.
(perhaps it was a problem with my vista but uninstalling it resolved the problem.)
(I can retry if you want just to chek this) By bugguy I mean, that selecting an option in right click context menu made the action and made the menu appears an other time (!)

The instruction was good but:
GTK+ Runtime link is obsolete.(will modify it with one I found, it was really hard to found, well I was bad on this point...).
http://sourceforge.net/project/showfiles.php?group_id=71914

Bazaar Gui didn't worked at all with the bzr 1.3.0 .exe install.
What I saw was:
It created a directory in c:/program files/bazaar/
And the bazaar GUI install itself in c:/python25/lib/site-package/bazaar/plugins
But nothing else was in this directory the site-package/bazar one.
So the olive gtk didn't work at all.
Using http://bazaar-vcs.org/MariusKruger#head-e234ddc518938afb140793d47320a2bee0b6c8aa I uninstalled the .exe version and installed a recent python install version (setup install.py) and then olive-gtk worked.

The .Bat file works fine. I've done a .ico if you want it to add it somewhere to have a beautifull shortcut.

How and last problem I encoutered, don't know why bit I had to add:
libglade-2.0-0.dll
libxml2.dll
rename zlib1.dll into zlib.dll
in C:\Program Files\GTK2-Runtime\lib
To make olive-gtk work.
I must have missed something there...

On your reply:
> http://d5190871.u44.websitesource.net/bzr/
Ok I've not tried this olive-gtk.exe perhaps mixing bzr.exe with olive-gtk.gz install was a bad idea, but the wiki don't warn about this.

> It's shame for me personally to hear this.
Sorry.

>QBzr GUI is compatible with bzr.exe. Of course, it's not the GTK one.
Ok will try this!

Regards,
Laurent

-----Message d'origine-----
De : Alexander Belchenko [mailto:bialix at ukr.net] 
Envoy? : mardi 25 mars 2008 21:08
? : Laurent Dufr?chou
Cc : 'ipython user list'; 'ipython-dev Mailing list'
Objet : Re: Bzr install under vista + bzr olive-gtk

Laurent Dufr?chou ?????:
> Hi all,
> Where can I put some info on how to install bzr under Vista?

It will be nice if you put the copy of your instruction to http://bazaar-vcs.org.
It's a wiki in the end.

There is already instructions how to install it:
http://bazaar-vcs.org/WindowsInstall

Can you tell me is this instructions is wrong or not good enough?

> I've installed it with the .exe that is a big error and nobody must do that.

It's shame for me personally to hear this.

> So I would like to warn newcomers how to set up bzr + bzr gui tools without
> too much pain...

QBzr GUI is compatible with bzr.exe. Of course, it's not the GTK one.

> I had installed bzr via the .exe install that is completely uncompatible
> with the olive-gtk gui (!)

IMO, it's not true. Here is package of bzr-gtk compatible with bzr.exe:
http://d5190871.u44.websitesource.net/bzr/

There is bzr-gtk-0.93.0 release only, but I don't follow bzr-gtk development,
so I'm not sure what's latest release is.

> I had to deinstall bzr and install the bzr python installer,(setup.py
> install as administrator).
> Now bzr viz and all other things works.
> 
> Here is an interesting link:
> http://bazaar-vcs.org/MariusKruger#head-e234ddc518938afb140793d47320a2bee0b6
> c8aa
> (a bit old, but you can easily find newer packages)
> 
> and
> http://bazaar-vcs.org/Download

> (DO not use the standalone installer or will not have a working plugin
> directory!)

I don't understand this warning. Can you explain it a bit more?



From bialix at ukr.net  Tue Mar 25 18:23:27 2008
From: bialix at ukr.net (Alexander Belchenko)
Date: Wed, 26 Mar 2008 00:23:27 +0200
Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk
In-Reply-To: <47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>	<85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>	<db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
	<47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com>
	<47E95B93.2050004@ukr.net>
	<47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com>
Message-ID: <47E97B5F.4090901@ukr.net>

Laurent Dufr?chou ?????:
> Hi Alexander,
Bonjour/Bonsoir,

> You are the maintainer of the .exe version??
I build all 3 installers for windows.

> In fact I've felt in some problem installing the .exe.
> Sorry for my previous email, it was more a warning on my usage case than a 'these guys have done a bad job'!
OK.

> So let me explain my problems (and perhaps you will tell that I've done somethng bad!), if not let's find solutions to improve things :)
> 
> I've used bazar first installing bazar with the .exe install. (1.3 version)
> It added bzr here in the context menu when you right click on a directory.
> Since the install the context seemed sluggish, AND bugguy.

Thank you to remind me about this misfeature. I made context menu after looking at cygwin's
implementation but did something wrong, and in all recent releases it disabled by default.
I'm leaning to remove it at all in next release.

> (perhaps it was a problem with my vista but uninstalling it resolved the problem.)
> (I can retry if you want just to chek this) By bugguy I mean, that selecting an option in right click context menu made the action and made the menu appears an other time (!)

> 
> The instruction was good but:
> GTK+ Runtime link is obsolete.(will modify it with one I found, it was really hard to found, well I was bad on this point...).
> http://sourceforge.net/project/showfiles.php?group_id=71914

Sorry, I'm not the author of the page http://bazaar-vcs.org/WindowsInstall.
You need only register on bazaar-vcs.org to get access to editing this page.
Feel free to fix broken links.

> Bazaar Gui didn't worked at all with the bzr 1.3.0 .exe install.

There is no 'Bazaar Gui'. There is bzr-gtk plugin, that is not the part of bzr core,
and therefore is not the part of standalone bzr.exe.
Plain bzr-gtk is not working because bzr.exe does not contains inside any required
PyGTK libraries.

> What I saw was:
> It created a directory in c:/program files/bazaar/
> And the bazaar GUI install itself in c:/python25/lib/site-package/bazaar/plugins
You're using wrong installer of bzr-gtk. You need installer especially created to work with bzr.exe.
Such installer should have PyGTK/GTK libs inside as well.
I'm not maintainer of installer for bzr-gtk. Because I use QBzr.

> But nothing else was in this directory the site-package/bazar one.
> So the olive gtk didn't work at all.
> Using http://bazaar-vcs.org/MariusKruger#head-e234ddc518938afb140793d47320a2bee0b6c8aa I uninstalled the .exe version and installed a recent python install version (setup install.py) and then olive-gtk worked.
> 
> The .Bat file works fine. I've done a .ico if you want it to add it somewhere to have a beautifull shortcut.
> 
> How and last problem I encoutered, don't know why bit I had to add:
> libglade-2.0-0.dll
> libxml2.dll
> rename zlib1.dll into zlib.dll
> in C:\Program Files\GTK2-Runtime\lib
> To make olive-gtk work.
> I must have missed something there...

Sorry, IMHO GTK toolkit is too alien in Windows environment.
 From all GTK universe I'm using GIMP, but only because I'm too lazy to look for better free non-GTK 
tool.

> 
> On your reply:
>> http://d5190871.u44.websitesource.net/bzr/
> Ok I've not tried this olive-gtk.exe perhaps mixing bzr.exe with olive-gtk.gz install was a bad idea, but the wiki don't warn about this.

I mean this one: http://d5190871.u44.websitesource.net/bzr/bzr-gtk-0.93.0-setup-1.exe

 From description and its content it seems like it's designed to work with bzr.exe, but I'm not sure.
At least it should install 'gtk' plugin to C:\Program Files\Bazaar\plugins\

> 
>> QBzr GUI is compatible with bzr.exe. Of course, it's not the GTK one.
> Ok will try this!

New version will be released soon:
http://permalink.gmane.org/gmane.comp.version-control.bazaar-ng.general/39104




From laurent.dufrechou at gmail.com  Tue Mar 25 18:55:46 2008
From: laurent.dufrechou at gmail.com (=?UTF-8?Q?Laurent_Dufr=C3=A9chou?=)
Date: Tue, 25 Mar 2008 23:55:46 +0100
Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk
In-Reply-To: <47E97B5F.4090901@ukr.net>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>	<85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>	<db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
	<47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com>
	<47E95B93.2050004@ukr.net>
	<47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com>
	<47E97B5F.4090901@ukr.net>
Message-ID: <47e98300.0aa1660a.2a5a.2731@mx.google.com>

Ok alanxander, i've updated the wiki.
I've also installed QBzr 0.8:
https://launchpad.net/qbzr/trunk/0.8.0/+download/qbzr-0.8.0.win32.exe

Files have been installed in bzr/plugins but there is no script to launch.(Qbzr or something like this?)
Do I Miss something?




From fperez.net at gmail.com  Tue Mar 25 19:19:07 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 25 Mar 2008 16:19:07 -0700
Subject: [IPython-dev] IPython winpdb interface
In-Reply-To: <46cb515a0803250938h5ecfc531hb8933bf18a4b579b@mail.gmail.com>
References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com>
	<47E6E860.30702@winpdb.org>
	<46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com>
	<47e7da01.0407560a.53ef.0285@mx.google.com>
	<46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com>
	<47e920fb.0405560a.1927.ffffd4d4@mx.google.com>
	<46cb515a0803250938h5ecfc531hb8933bf18a4b579b@mail.gmail.com>
Message-ID: <db6b5ecc0803251619g1a5ea511jf8e210d02590323e@mail.gmail.com>

On Tue, Mar 25, 2008 at 9:38 AM, Ville M. Vainio <vivainio at gmail.com> wrote:

>  >  In which bzr branch do you plan to play with this, mtexp or stable one??
>
>  Depends on time we get an API. This will end up post-0.8.3.
>
>  BTW, mtexp is dead, it has already been integrated to trunk.

Then it should be removed from launchpad, to reduce confusion.  Let's
keep a habit of killing branches as soon as their intended purpose is
finished.  Also, let's try to use all  the labelling features of
launchpad to properly indicate what a branch is meant for.  Here's an
example of what I mean (I used both the title and the details field to
clarify the intent of that branch to potential collaborators):

https://code.launchpad.net/~nipy-developers/nipy/fff2

Cheers,

f


From ondrej at certik.cz  Tue Mar 25 20:19:02 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 26 Mar 2008 01:19:02 +0100
Subject: [IPython-dev] mercurial now has free hosting too
Message-ID: <85b5c3130803251719pde17151x96761bb8cd2b4f7e@mail.gmail.com>

Hi,

since there was so much discussion whether bzr or hg, Mercurial has
now free hosting too:

http://freehg.org/

also Mercurial 1.0 was finally released yesterday.

Bzr has Launchpad, that's one of the (main) reasons ipython is
investigating it, so I am still learning how to use bzr, but it's not
really so much different. The only little annoying thing I discovered
so far is that it feels slower than hg, on regular tasks, like "bzr
st", "bzr pull", "bzr up", etc. Only a little bit, but still it
creates the feeling in me, that something is missing -- it's like if
you are used to a fast car and then you get into a slower car - even
though it's fast enough to drive you to the shop, you still are
missing something, at least I am. :)

But generally I wanted to say, that I think bzr is a good choice too.

Ondrej


From vivainio at gmail.com  Wed Mar 26 05:36:18 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 26 Mar 2008 11:36:18 +0200
Subject: [IPython-dev] mercurial now has free hosting too
In-Reply-To: <85b5c3130803251719pde17151x96761bb8cd2b4f7e@mail.gmail.com>
References: <85b5c3130803251719pde17151x96761bb8cd2b4f7e@mail.gmail.com>
Message-ID: <46cb515a0803260236r2a4685eeg7f2bacda6d043f81@mail.gmail.com>

On Wed, Mar 26, 2008 at 2:19 AM, Ondrej Certik <ondrej at certik.cz> wrote:

>  since there was so much discussion whether bzr or hg, Mercurial has
>  now free hosting too:

I think the killer issue here is Launchpad. We get pretty much
everything for free, and it's something that we can expect to stay
around in the long haul (and will continue to improve). The slight
performance disadvantage of bzr is a minor concern, esp. for the
project with the size of IPython.

Both hg and bzr as-such (if we forget the hosting options etc) are
"good enough" right now, and both are being improved. As it stands, I
think we should stick with LP + bzr for now - we can re-evaluate the
situation a year or so down the line, if need be.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From gael.varoquaux at normalesup.org  Wed Mar 26 05:40:47 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Wed, 26 Mar 2008 10:40:47 +0100
Subject: [IPython-dev] mercurial now has free hosting too
In-Reply-To: <46cb515a0803260236r2a4685eeg7f2bacda6d043f81@mail.gmail.com>
References: <85b5c3130803251719pde17151x96761bb8cd2b4f7e@mail.gmail.com>
	<46cb515a0803260236r2a4685eeg7f2bacda6d043f81@mail.gmail.com>
Message-ID: <20080326094047.GB7186@phare.normalesup.org>

On Wed, Mar 26, 2008 at 11:36:18AM +0200, Ville M. Vainio wrote:
> I think the killer issue here is Launchpad. We get pretty much
> everything for free, and it's something that we can expect to stay
> around in the long haul (and will continue to improve). The slight
> performance disadvantage of bzr is a minor concern, esp. for the
> project with the size of IPython.

> Both hg and bzr as-such (if we forget the hosting options etc) are
> "good enough" right now, and both are being improved. As it stands, I
> think we should stick with LP + bzr for now - we can re-evaluate the
> situation a year or so down the line, if need be.

I have the very exact same gut fealing than you.

I have watch developpers switch to DVCS lately, and I must say it
requires a certain change in working habits. Let us wait for people to
get used to these new tools before discussing which one to you. And don't
get me wrong, I love DVCS, I just don't want to lose developper because
of it.

Ga?l



From jorgen.stenarson at bostream.nu  Wed Mar 26 14:09:26 2008
From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Wed, 26 Mar 2008 19:09:26 +0100
Subject: [IPython-dev] pyreadline
Message-ID: <47EA9156.2050603@bostream.nu>

Hi all,

I have been thinking about the future of pyreadline considering the work 
  on ipython1.

  * ANSI color escapes. In trunk ansi color escapes are used to handle 
color in prompts. This is convenient(for the programmer) when the 
terminal understands these escapes. But on windows this is not the case, 
I imagine a few of the guis proposed for ipython1 would also have 
trouble with them. In pyreadline we have code for parsing this. So it 
would probably make sense to have this functionality available for 
ipython1 as well. From a users point of view I think we should consider 
figuring out a way that is easier to understand and expand for the 
future. We could include ways to use boldface, italics, underline etc.

  * I know Gael is looking into using pyreadline for some gui interface 
this means there will be even more dependency of ipython on pyreadline. 
Perhaps this even means we should add pyreadline to ipython1.

  * Completions. Parts of the tab-completion code in ipython could 
probably be made to work in pyreadline directly making it useful at a 
normal pythonprompt as well.


There is a ton of work I would like to do in cleaning up the codebase 
and improving the test coverage. Going with nose here just as in ipython 
makes sense. Converting the documentation to rest format using sphinx 
looks like a good idea as well.

J?rgen








From laurent.dufrechou at gmail.com  Wed Mar 26 16:47:27 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Wed, 26 Mar 2008 21:47:27 +0100
Subject: [IPython-dev] pyreadline
In-Reply-To: <47EA9156.2050603@bostream.nu>
References: <47EA9156.2050603@bostream.nu>
Message-ID: <47eab66f.25e2660a.156e.0565@mx.google.com>

Hi jorg?n,

Ga?l explained me a little what was a readline. Now I better understand how
it interface with a gui.
(Please any GUI programmer correct me if I'm wrong, especially for cocoa or
QT)

* ANSI color escapes:
In a gui we use a widget that has methods to color the text it display,
depending on the widget used, the methods are way different.
When I interfaced wx to ipython, I sended a string to ipython shell and it
sent back (via readline interface?) an ansi color escaped string.
I had to parse it and process it to make it appears the way it was expected.
>From my point of view I don't expect readline to manage to handle my widget
display.
I mean, send me the data the way I can process it and I will be happy.
Readline must be (I think) GUI agnostic.
Now we can discuss how you can communicate the data. It would be really cool
to be able to select an output object. For example, we could select to set
readline output as escape character OR for example (if we want to handle
bolt etc easily) as HTML output.Why html? Because some widget on WX can get
HTML and display it as is.
We can imagine also XML output or whatelse a GUI guy would need.

*Completion:
If ipython1 manage it by itself, I'm not sure readline should manage it too.
Or does it provide more functions?

*History
>From what I understood from Gael, readline is the object that manage
history.
What I needed when I wrote Wx shell for ip0, was:
getHistoryBack(),getHistoryWidth(),getHistory() etc... in fact function that
can provide me a way to get and manage history strings.
So if we could define a simple interface to readline that could be good.

One important point, gael tells me that usually readline get all character
received from a frontend and it handle it.
I think, (perhaps I'm wrong) that from a GUI designer point of view, I would
greatly prefer to send to readline full string instead of character. Key
event handling is tighly managed by gui itself, so I think it is a little
complex to send it to readline with readline being GUI agnostic...
Well I think we will have to discuss this point. QT/Cocoa guys, what do you
think about this? Perhaps the solution is to create a wxreadline that will
be a thin wrapper between wx key event and readline processing?

Don't have a real solution there...

Be careful, I was speaking from a GUI designer point of view, from a shell
frontend point of view I understand the way that readline process char by
char the data...
So that why, if you explain the advantage of calling readline each time a
char is processed, I will be willing to try to use it like a shell frontend
designer would.(to have a unified interface)

Laurent

-----Message d'origine-----
De?: ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De
la part de J?rgen Stenarson
Envoy??: mercredi 26 mars 2008 19:09
??: IPython-dev List
Objet?: [IPython-dev] pyreadline

Hi all,

I have been thinking about the future of pyreadline considering the work 
  on ipython1.

  * ANSI color escapes. In trunk ansi color escapes are used to handle 
color in prompts. This is convenient(for the programmer) when the 
terminal understands these escapes. But on windows this is not the case, 
I imagine a few of the guis proposed for ipython1 would also have 
trouble with them. In pyreadline we have code for parsing this. So it 
would probably make sense to have this functionality available for 
ipython1 as well. From a users point of view I think we should consider 
figuring out a way that is easier to understand and expand for the 
future. We could include ways to use boldface, italics, underline etc.

  * I know Gael is looking into using pyreadline for some gui interface 
this means there will be even more dependency of ipython on pyreadline. 
Perhaps this even means we should add pyreadline to ipython1.

  * Completions. Parts of the tab-completion code in ipython could 
probably be made to work in pyreadline directly making it useful at a 
normal pythonprompt as well.


There is a ton of work I would like to do in cleaning up the codebase 
and improving the test coverage. Going with nose here just as in ipython 
makes sense. Converting the documentation to rest format using sphinx 
looks like a good idea as well.

J?rgen






_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev



From gael.varoquaux at normalesup.org  Wed Mar 26 16:56:45 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Wed, 26 Mar 2008 21:56:45 +0100
Subject: [IPython-dev] pyreadline
In-Reply-To: <47eab66f.25e2660a.156e.0565@mx.google.com>
References: <47EA9156.2050603@bostream.nu>
	<47eab66f.25e2660a.156e.0565@mx.google.com>
Message-ID: <20080326205645.GC24201@phare.normalesup.org>

On Wed, Mar 26, 2008 at 09:47:27PM +0100, Laurent Dufr?chou wrote:
> One important point, gael tells me that usually readline get all
> character
> received from a frontend and it handle it.
> I think, (perhaps I'm wrong) that from a GUI designer point of view, I
> would
> greatly prefer to send to readline full string instead of character.
> Key
> event handling is tighly managed by gui itself, so I think it is a
> little
> complex to send it to readline with readline being GUI agnostic...

OK, but how to you choose how a key must behave? You need one single
object that knows the configuration of keys (is pageup history, or
nothing?), and what if I want to switch to VI mode, you don't want to
implement this in all the frontends.

Sending a string defeats the goal of readline: readline is what
translates keystrokes to a command string. Readline knows that if I use
"ctr-left", it should jump the cursor one word to the left. Yes, it is
not easy, but that's what readline is about, IMHO.

Ga?l


From laurent.dufrechou at gmail.com  Wed Mar 26 20:34:39 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Thu, 27 Mar 2008 01:34:39 +0100
Subject: [IPython-dev] pyreadline (Hey cocoa/QT/wx ipython GUI what do
	you think about this?)
In-Reply-To: <20080326205645.GC24201@phare.normalesup.org>
References: <47EA9156.2050603@bostream.nu>	<47eab66f.25e2660a.156e.0565@mx.google.com>
	<20080326205645.GC24201@phare.normalesup.org>
Message-ID: <47eaebb1.0c58560a.2b21.16d9@mx.google.com>

Hi ga?l,

>OK, but how to you choose how a key must behave? You need one single
>object that knows the configuration of keys (is pageup history, or
>nothing?), and what if I want to switch to VI mode, you don't want to
>implement this in all the frontends.

When you use a widget, like scintilla one that I used, all the classical key
where already handled by the text widget itself. I add to overload some to
better match ipython shell behavior. Like pushing 'up', does not go up but
call history back.
So if you want VI mode , just design your ipython widget around a VI aware
text widget, that's all... 

>Sending a string defeats the goal of readline: readline is what
>translates keystrokes to a command string. Readline knows that if I use
>"ctr-left", it should jump the cursor one word to the left. Yes, it is
>not easy, but that's what readline is about, IMHO.

Ok I've deleted the beginning of my email:
In fact the problem can be solved 'easily'.
For WX the widget developer will ahev something like this:

Class MyTextAreaWidget(object):

    def __init__(self):
	  ...
        Self.readline = Readline(ipython_instance)
        ...
	  self.readline.MoveCursorLeft = self.MoveCursorLeft
        self.readline.GotoStartOfLine = self.MoveToPrompt
#------------------------ Key Handler ------------------------------------
    def keyPress(self, event):
        '''
        Key press callback with plenty of shell goodness, like history,
        autocompletions, etc.
        '''
        
	  #readline shortcircuit
        if event.GetKeyCode() == ord('C'):
     		if event.Modifiers == wx.MOD_CONTROL:
                #we raise an exception inside the IPython thread container
                self.IP.ce.raise_exc(KeyboardInterrupt)
                return
	  #readline key handling
	  self.readline.ProcessKey(event.GetKeycode())
	  
        #post key processing
	  if (self.readline.has_processed_key == False):
		 self.postProcessKey()

Developer must have the possibility to let or not readline to handle the
keys.
If he don't want to use readline key handling, he must have access to
function that can make ipython execute a string, and that should be ok.

Am I clear in my explanation? Readline only process key but does not manage
widget directly. User will to hook it's widget function to readline one.
With this approach I'm OK. So we will be able to use a text area renderer, a
scintilla renderer or even an html renderer...
This will not be a problem for readline, has it will only redirect key event
to good widget functions via a unified interface.

Laurent



From gael.varoquaux at normalesup.org  Thu Mar 27 04:55:10 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 27 Mar 2008 09:55:10 +0100
Subject: [IPython-dev] pyreadline (Hey cocoa/QT/wx ipython GUI what
	do	you think about this?)
In-Reply-To: <47eaebb1.0c58560a.2b21.16d9@mx.google.com>
References: <47EA9156.2050603@bostream.nu>
	<47eab66f.25e2660a.156e.0565@mx.google.com>
	<20080326205645.GC24201@phare.normalesup.org>
	<47eaebb1.0c58560a.2b21.16d9@mx.google.com>
Message-ID: <20080327085510.GB3109@phare.normalesup.org>

On Thu, Mar 27, 2008 at 01:34:39AM +0100, Laurent Dufr?chou wrote:
> When you use a widget, like scintilla one that I used, all the
> classical key
> where already handled by the text widget itself. I add to overload some
> to
> better match ipython shell behavior. Like pushing 'up', does not go up
> but
> call history back.
> So if you want VI mode , just design your ipython widget around a VI
> aware
> text widget, that's all...

If you head in this direction you are going to repeat the functionnality
in each front end. As a result you will have more code to maintain,
and inconsistent behavior between widgets.

> Developer must have the possibility to let or not readline to handle
> the
> keys.
> If he don't want to use readline key handling, he must have access to
> function that can make ipython execute a string, and that should be ok.

I just don't see why. The fonction call overhead is not the end of the
world. It is true that finding a decent interface between the readline
and the front end is hard, but it will make future development easier,
and the code easier to read. Anyhow, this is all theory for now, and
their is a lot of work before we actually face these problems.

> Readline only process key but does not manage widget directly.

Well, it gives back high-level instructions on the widget describing how
it behaves. Finding this set of high-level instructions is the key to
having a good decoupling between front ends and ipython.

Ga?l


From walter at livinglogic.de  Thu Mar 27 04:57:42 2008
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu, 27 Mar 2008 09:57:42 +0100
Subject: [IPython-dev] pyreadline
In-Reply-To: <47EA9156.2050603@bostream.nu>
References: <47EA9156.2050603@bostream.nu>
Message-ID: <47EB6186.7070308@livinglogic.de>

J?rgen Stenarson wrote:

> Hi all,
> 
> I have been thinking about the future of pyreadline considering the work 
>   on ipython1.
> 
>   * ANSI color escapes. In trunk ansi color escapes are used to handle 
> color in prompts. This is convenient(for the programmer) when the 
> terminal understands these escapes. But on windows this is not the case, 
> I imagine a few of the guis proposed for ipython1 would also have 
> trouble with them. In pyreadline we have code for parsing this. So it 
> would probably make sense to have this functionality available for 
> ipython1 as well. From a users point of view I think we should consider 
> figuring out a way that is easier to understand and expand for the 
> future. We could include ways to use boldface, italics, underline etc.

What I did for the ipipe stuff is that the output for an object comes 
from a generator that looks like this:

def repr_list(obj):
     yield (normal, '[')
     for (i, child) in enumerate(obj):
         if i:
             yield (normal, ', ')
         for (color, part) in specialrepr(child):
             yield (color, part)
     yield (normal, '[')

i.e. the generator produces tuples with a string and a color object for 
that string. The advantage of a generator is that you can cancel the 
generator once the screen is filled, so you don't have to produce output
that you are not displaying anyway.

Currently in ipipe this color object is a wrapper for curses/ANSI 
colors, i.e. it knows about 8 foreground colors, 8 background colors, 
bold and underline.

I have a half finished version lying around, where forground and 
background color are three floats (i.e. Color(0., 0., 0.) is black, 
Color(1., 1., 1.) is white). For terminal and curses output this gets 
mapped to the nearest of the 16 ANSI colors (it also supports 256 color 
terminals).

For this to be usable we probably would need theme support, because for 
terminal output you might very well have a black background, for GUIs 
you almost never have.

Servus,
    Walter


From bialix at ukr.net  Fri Mar 28 03:56:49 2008
From: bialix at ukr.net (Alexander Belchenko)
Date: Fri, 28 Mar 2008 09:56:49 +0200
Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk
In-Reply-To: <47e98300.0aa1660a.2a5a.2731@mx.google.com>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>	<85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>	<db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
	<47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com>
	<47E95B93.2050004@ukr.net>
	<47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com>
	<47E97B5F.4090901@ukr.net>
	<47e98300.0aa1660a.2a5a.2731@mx.google.com>
Message-ID: <47ECA4C1.9080203@ukr.net>

Laurent Dufr?chou ?????:
> Ok alanxander, i've updated the wiki.
> I've also installed QBzr 0.8:
> https://launchpad.net/qbzr/trunk/0.8.0/+download/qbzr-0.8.0.win32.exe
> 
> Files have been installed in bzr/plugins but there is no script to launch.(Qbzr or something like this?)
> Do I Miss something?

QBzr is like bzr-gtk (not Olive): it provides a set of commands similar to basic bzr commands,
but with GUI interface. QBzr 0.8 provides: qdiff, qlog, qcommit, qannotate, qbrowse, qconfig.
You need to cd to your branch and launch them from command line.

May be it's not very convenient if you're like Olive program, but currently there is no QBzr
standalone program yet.


From laurent.dufrechou at gmail.com  Fri Mar 28 05:00:34 2008
From: laurent.dufrechou at gmail.com (=?utf-8?Q?Laurent_Dufr=C3=A9chou?=)
Date: Fri, 28 Mar 2008 10:00:34 +0100
Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk
In-Reply-To: <47ECA4C1.9080203@ukr.net>
References: <db6b5ecc0803231617s3e5687a2vc62a76595c01ba83@mail.gmail.com>	<46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com>	<85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com>	<db6b5ecc0803241558v5c8a25e0h3fcbfad6eeef448f@mail.gmail.com>
	<47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com>
	<47E95B93.2050004@ukr.net>
	<47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com>
	<47E97B5F.4090901@ukr.net>
	<47e98300.0aa1660a.2a5a.2731@mx.google.com>
	<47ECA4C1.9080203@ukr.net>
Message-ID: <47ecb3cb.0437560a.7752.2810@mx.google.com>

Ok, i found how to use it!
bzr qannotate myfile.py etc...
thx

-----Message d'origine-----
De : Alexander Belchenko [mailto:bialix at ukr.net] 
Envoy? : vendredi 28 mars 2008 08:57
? : Laurent Dufr?chou
Cc : 'ipython user list'; 'ipython-dev Mailing list'
Objet : Re: Bzr install under vista + bzr olive-gtk

Laurent Dufr?chou ?????:
> Ok alanxander, i've updated the wiki.
> I've also installed QBzr 0.8:
> https://launchpad.net/qbzr/trunk/0.8.0/+download/qbzr-0.8.0.win32.exe
> 
> Files have been installed in bzr/plugins but there is no script to launch.(Qbzr or something like this?)
> Do I Miss something?

QBzr is like bzr-gtk (not Olive): it provides a set of commands similar to basic bzr commands,
but with GUI interface. QBzr 0.8 provides: qdiff, qlog, qcommit, qannotate, qbrowse, qconfig.
You need to cd to your branch and launch them from command line.

May be it's not very convenient if you're like Olive program, but currently there is no QBzr
standalone program yet.



From glenn at tarbox.org  Fri Mar 28 15:29:30 2008
From: glenn at tarbox.org (Glenn H Tarbox, PhD)
Date: Fri, 28 Mar 2008 12:29:30 -0700
Subject: [IPython-dev] Twisted 8...
Message-ID: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org>

Ipython1 no longer compiles with the latest Twisted trunk because of
Twisted's new numbering scheme.  Fails in the version check.

I'm only aware of one significant IPython1 refactoring issue relating to
the blockingThread capability now in the Twisted mainline. (probably not
refactoring, just an adjustment to use Twisted's call rather than
whatever version IPython1 was using in anticipation of this new twisted
feature.)

-glenn

-- 
Glenn H. Tarbox, PhD    | Don?t worry about people stealing your ideas. If your ideas
206-494-0819            | are any good, you?ll have to ram them down people?s throats
glenn at tarbox.org        | ?Howard Aiken, IBM engineer




From ellisonbg.net at gmail.com  Fri Mar 28 16:00:13 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Fri, 28 Mar 2008 14:00:13 -0600
Subject: [IPython-dev] Twisted 8...
In-Reply-To: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org>
References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org>
Message-ID: <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>

Glenn,

I am plannig to get IPython1 working with Twisted 8.0 soon.  As you
mention, there should only be a few things we have to do.  How do
people feel about requiring the new version of Twisted?  It is a
significant release, and Twisted 2.5.0 had been out for a very long
time.

Brian

On Fri, Mar 28, 2008 at 1:29 PM, Glenn H Tarbox, PhD <glenn at tarbox.org> wrote:
> Ipython1 no longer compiles with the latest Twisted trunk because of
>  Twisted's new numbering scheme.  Fails in the version check.
>
>  I'm only aware of one significant IPython1 refactoring issue relating to
>  the blockingThread capability now in the Twisted mainline. (probably not
>  refactoring, just an adjustment to use Twisted's call rather than
>  whatever version IPython1 was using in anticipation of this new twisted
>  feature.)
>
>  -glenn
>
>  --
>  Glenn H. Tarbox, PhD    | Don't worry about people stealing your ideas. If your ideas
>  206-494-0819            | are any good, you'll have to ram them down people's throats
>  glenn at tarbox.org        | ?Howard Aiken, IBM engineer
>
>
>  _______________________________________________
>  IPython-dev mailing list
>  IPython-dev at scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From steve at shrogers.com  Fri Mar 28 17:51:11 2008
From: steve at shrogers.com (Steven H. Rogers)
Date: Fri, 28 Mar 2008 15:51:11 -0600 (MDT)
Subject: [IPython-dev] Twisted 8...
In-Reply-To: <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>
References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org>
	<6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>
Message-ID: <11946.192.55.12.36.1206741071.squirrel@mail2.webfaction.com>

On Fri, March 28, 2008 14:00, Brian Granger wrote:
> Glenn,
>
> I am plannig to get IPython1 working with Twisted 8.0 soon.  As you
> mention, there should only be a few things we have to do.  How do
> people feel about requiring the new version of Twisted?  It is a
> significant release, and Twisted 2.5.0 had been out for a very long
> time.
>
> Brian
>

+1 for requiring Twisted 8.0+

# Steve


From laurent.dufrechou at gmail.com  Fri Mar 28 20:52:30 2008
From: laurent.dufrechou at gmail.com (=?us-ascii?Q?Laurent_Dufrechou?=)
Date: Sat, 29 Mar 2008 01:52:30 +0100
Subject: [IPython-dev] How to create a new ticket?
Message-ID: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com>

Hi guys,

 

I've found a little feature request :

'a'.+TAB does not autocomplete.

 

How can I put a ticket inside trac?? I've tried to register but I need a
scipy user/password.

 

Laurent

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080329/90f2c179/attachment.html>

From vivainio at gmail.com  Sat Mar 29 03:28:56 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 29 Mar 2008 09:28:56 +0200
Subject: [IPython-dev] How to create a new ticket?
In-Reply-To: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com>
References: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com>
Message-ID: <46cb515a0803290028r43c66c41xd9131a7d9d19aed1@mail.gmail.com>

On Sat, Mar 29, 2008 at 2:52 AM, Laurent Dufrechou
<laurent.dufrechou at gmail.com> wrote:

> How can I put a ticket inside trac?? I've tried to register but I need a
> scipy user/password?

Try putting it on launchpad - it's about time we get started using the
"bugs" section :-)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From laurent.dufrechou at gmail.com  Sat Mar 29 05:26:45 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Sat, 29 Mar 2008 10:26:45 +0100
Subject: [IPython-dev] How to create a new ticket?
In-Reply-To: <46cb515a0803290028r43c66c41xd9131a7d9d19aed1@mail.gmail.com>
References: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com>
	<46cb515a0803290028r43c66c41xd9131a7d9d19aed1@mail.gmail.com>
Message-ID: <47ee0b73.0b38560a.66cc.0e88@mx.google.com>

Hum, i've gone to launchad, ipython project then bugs.
And no bug tracker is defined.
I think it's not defined for the moment :)
Do I miss something?

-----Message d'origine-----
De?: Ville M. Vainio [mailto:vivainio at gmail.com] 
Envoy??: samedi 29 mars 2008 08:29
??: Laurent Dufrechou
Cc?: ipython-dev Mailing list
Objet?: Re: [IPython-dev] How to create a new ticket?

On Sat, Mar 29, 2008 at 2:52 AM, Laurent Dufrechou
<laurent.dufrechou at gmail.com> wrote:

> How can I put a ticket inside trac?? I've tried to register but I need a
> scipy user/password


Try putting it on launchpad - it's about time we get started using the
"bugs" section :-)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'



From laurent.dufrechou at gmail.com  Sat Mar 29 07:02:52 2008
From: laurent.dufrechou at gmail.com (=?us-ascii?Q?Laurent_Dufrechou?=)
Date: Sat, 29 Mar 2008 12:02:52 +0100
Subject: [IPython-dev] Help! TR: TR: Ipython plugin
Message-ID: <47ee21fb.05a0660a.4a91.3ff1@mx.google.com>

Hi guys,

I've submitted a pre-plugin for editra as an alternative to pyshell ;)

All is working except some quirk with ctrl+c (editra interact with it) and
enter under mac osx..

(I had never tried under mac osx.)

So here are my two help request:

-Can somebody tell if it exist a macosx emulator??? Or is there someone
willing to help me debug this remotely? (else trying to correct the code by
itself with my help or perhaps via a vnc connection?)

Here is what cody saw:

2) The plugin loads and shows on Mac OSX but there must be some problem with
the key handling because hitting 'Enter' only inserts a new line in the
control and doesn't execute the command. This problem doesn't exist when
running under wxGTK.

 

-Cody is asking if it is possible to embed Ipython in the plugin:

It might also be interesting to try and embed IPython itself into the plugin
so nobody has to worry about dependencies. If IPython is pure python (not a
C extension) this should be as easy as copying the IPython directory into
your plugins __init__.py path and then including it the setup.py's data
files.

 

Does Ipython as C dependency I'm not aware of???

 

Cheers,

Laurent

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080329/0e1ebdb6/attachment.html>

From vivainio at gmail.com  Sat Mar 29 07:05:45 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 29 Mar 2008 13:05:45 +0200
Subject: [IPython-dev] How to create a new ticket?
In-Reply-To: <47ee0b73.0b38560a.66cc.0e88@mx.google.com>
References: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com>
	<46cb515a0803290028r43c66c41xd9131a7d9d19aed1@mail.gmail.com>
	<47ee0b73.0b38560a.66cc.0e88@mx.google.com>
Message-ID: <46cb515a0803290405x6ab3b518g22e8107514f19cd9@mail.gmail.com>

On Sat, Mar 29, 2008 at 11:26 AM, Laurent Dufr?chou
<laurent.dufrechou at gmail.com> wrote:
> Hum, i've gone to launchad, ipython project then bugs.
>  And no bug tracker is defined.
>  I think it's not defined for the moment :)
>  Do I miss something?

No, you didn't, I had to enable the bug tracking.

Try again :-)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From laurent.dufrechou at gmail.com  Sat Mar 29 07:29:04 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Sat, 29 Mar 2008 12:29:04 +0100
Subject: [IPython-dev] How to create a new ticket?
In-Reply-To: <46cb515a0803290405x6ab3b518g22e8107514f19cd9@mail.gmail.com>
References: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com>	
	<46cb515a0803290028r43c66c41xd9131a7d9d19aed1@mail.gmail.com>	
	<47ee0b73.0b38560a.66cc.0e88@mx.google.com>
	<46cb515a0803290405x6ab3b518g22e8107514f19cd9@mail.gmail.com>
Message-ID: <47ee281f.0ab6660a.7304.61f3@mx.google.com>

Ok now we can report bug throught launchpad!
You can now find a fix to my bug report :p

By the way, how about old trac bug list? Will it be lost?

-----Message d'origine-----
De?: Ville M. Vainio [mailto:vivainio at gmail.com] 
Envoy??: samedi 29 mars 2008 12:06
??: Laurent Dufr?chou
Cc?: ipython-dev Mailing list
Objet?: Re: [IPython-dev] How to create a new ticket?

On Sat, Mar 29, 2008 at 11:26 AM, Laurent Dufr?chou
<laurent.dufrechou at gmail.com> wrote:
> Hum, i've gone to launchad, ipython project then bugs.
>  And no bug tracker is defined.
>  I think it's not defined for the moment :)
>  Do I miss something?

No, you didn't, I had to enable the bug tracking.

Try again :-)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'



From barrywark at gmail.com  Sat Mar 29 13:51:19 2008
From: barrywark at gmail.com (Barry Wark)
Date: Sat, 29 Mar 2008 10:51:19 -0700
Subject: [IPython-dev] Twisted 8...
In-Reply-To: <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>
References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org>
	<6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>
Message-ID: <cd7634ce0803291051p4324c6f7jfec78cf444bca642@mail.gmail.com>

+1

On Fri, Mar 28, 2008 at 1:00 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
> Glenn,
>
>  I am plannig to get IPython1 working with Twisted 8.0 soon.  As you
>  mention, there should only be a few things we have to do.  How do
>  people feel about requiring the new version of Twisted?  It is a
>  significant release, and Twisted 2.5.0 had been out for a very long
>  time.
>
>  Brian
>
>
>
>  On Fri, Mar 28, 2008 at 1:29 PM, Glenn H Tarbox, PhD <glenn at tarbox.org> wrote:
>  > Ipython1 no longer compiles with the latest Twisted trunk because of
>  >  Twisted's new numbering scheme.  Fails in the version check.
>  >
>  >  I'm only aware of one significant IPython1 refactoring issue relating to
>  >  the blockingThread capability now in the Twisted mainline. (probably not
>  >  refactoring, just an adjustment to use Twisted's call rather than
>  >  whatever version IPython1 was using in anticipation of this new twisted
>  >  feature.)
>  >
>  >  -glenn
>  >
>  >  --
>  >  Glenn H. Tarbox, PhD    | Don't worry about people stealing your ideas. If your ideas
>  >  206-494-0819            | are any good, you'll have to ram them down people's throats
>  >  glenn at tarbox.org        | ?Howard Aiken, IBM engineer
>  >
>  >
>  >  _______________________________________________
>  >  IPython-dev mailing list
>  >  IPython-dev at scipy.org
>  >  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>  >
>  _______________________________________________
>  IPython-dev mailing list
>  IPython-dev at scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From ondrej at certik.cz  Mon Mar 31 10:04:13 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 31 Mar 2008 16:04:13 +0200
Subject: [IPython-dev] GSoC project - a notebook like GUI
Message-ID: <85b5c3130803310704g794d9085y84d037273bb42e09@mail.gmail.com>

Hi,

we got an interesting proposal for the Google Summer of Code (GSoC) in SymPy:

http://groups.google.com/group/sympy/browse_thread/thread/f4a28675d34e0ee6

and an interesting discussion has taken place in the thread.

With ipython1 and it's frontends in mind, I'd like to ask about your
opinions of the GSoC proposal. Can maybe some work be done together
with ipython1?

Ondrej


From bwaters at nrao.edu  Mon Mar 31 13:48:27 2008
From: bwaters at nrao.edu (Boyd Waters)
Date: Mon, 31 Mar 2008 11:48:27 -0600
Subject: [IPython-dev] Twisted 8...
In-Reply-To: <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>
References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org>
	<6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>
Message-ID: <A9EAF3FC-0FCE-4C6B-9BD4-DFA2E688E2BC@nrao.edu>


On Mar 28, 2008, at 2:00 PM, Brian Granger wrote:
>  How do people feel about requiring the new version of Twisted?

+1

no guts, no glory




From fperez.net at gmail.com  Mon Mar 31 20:15:58 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 31 Mar 2008 17:15:58 -0700
Subject: [IPython-dev] Twisted 8...
In-Reply-To: <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>
References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org>
	<6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>
Message-ID: <db6b5ecc0803311715u3bb0593ch172222b9db25ca91@mail.gmail.com>

On Fri, Mar 28, 2008 at 1:00 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:

>  I am plannig to get IPython1 working with Twisted 8.0 soon.  As you
>  mention, there should only be a few things we have to do.  How do
>  people feel about requiring the new version of Twisted?  It is a
>  significant release, and Twisted 2.5.0 had been out for a very long
>  time.

Just curious: what *benefits* do we obtain from using twisted 8.0?
It's nice that today, all of ipython1's depenedencies are satisfied
out of the box on current linux distros.  But even the
not-yet-released ubuntu has twisted 8, they are still on 2.5:

http://packages.ubuntu.com/hardy/python-twisted-core

I'm not fundamentally opposed to requiring 8.0 if we actually need it,
but otherwise I'd prefer to find a way (if possible, I haven't looked
at the code) to work with both 2.5 and 8.0. I hope the twisted devs
didn't make so many backwards incompatible changes as to make that
impossible...

Cheers,

f


From gael.varoquaux at normalesup.org  Mon Mar 31 20:18:18 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 1 Apr 2008 02:18:18 +0200
Subject: [IPython-dev] Twisted 8...
In-Reply-To: <db6b5ecc0803311715u3bb0593ch172222b9db25ca91@mail.gmail.com>
References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org>
	<6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>
	<db6b5ecc0803311715u3bb0593ch172222b9db25ca91@mail.gmail.com>
Message-ID: <20080401001817.GM26101@phare.normalesup.org>

On Mon, Mar 31, 2008 at 05:15:58PM -0700, Fernando Perez wrote:
> I'm not fundamentally opposed to requiring 8.0 if we actually need it,
> but otherwise I'd prefer to find a way (if possible, I haven't looked
> at the code) to work with both 2.5 and 8.0.

+1. I have enough work like this without dealing with installing software
myself.

Cheers,

Ga?l


From barrywark at gmail.com  Mon Mar 31 21:18:35 2008
From: barrywark at gmail.com (Barry Wark)
Date: Mon, 31 Mar 2008 18:18:35 -0700
Subject: [IPython-dev] Twisted 8...
In-Reply-To: <db6b5ecc0803311715u3bb0593ch172222b9db25ca91@mail.gmail.com>
References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org>
	<6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>
	<db6b5ecc0803311715u3bb0593ch172222b9db25ca91@mail.gmail.com>
Message-ID: <cd7634ce0803311818n21d9b738t873a39ca1dea497a@mail.gmail.com>

Being installable via easy_install is a huge win (really a
requirement) on OS X 10.5. Leopard comes with a system Twisted 2.4 and
installing a newer version (such as the 2.5 required by ipython1)
requires mucking with the PYTHONPATH (not a mac user-friendly task).
Even worse, PYTHONPATH changes are not propagated via sudo (for
obvious reasons), so running code that requires twisted 2.5 as root is
impossible, even with the PYTHONPATH modification. Installation via
easy_install fixes these issues (the easy_install'd  Twisted is placed
before the system Twisted on the sys.path by setuptools). With
setuptools/easy_install support, it's possible to automate deployment
of pre-compiled eggs.

There's currently a branch of Twisted 2.5 that is installable via
setuptools/easy_install, but the official release isn't.

So, making Twisted and ipython1 a reality of OS X will require
setuptools/easy_install support. So, +1 from OS X land.

Barry


On Mon, Mar 31, 2008 at 5:15 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Fri, Mar 28, 2008 at 1:00 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
>
>
> >  I am plannig to get IPython1 working with Twisted 8.0 soon.  As you
>  >  mention, there should only be a few things we have to do.  How do
>  >  people feel about requiring the new version of Twisted?  It is a
>  >  significant release, and Twisted 2.5.0 had been out for a very long
>  >  time.
>
>  Just curious: what *benefits* do we obtain from using twisted 8.0?
>  It's nice that today, all of ipython1's depenedencies are satisfied
>  out of the box on current linux distros.  But even the
>  not-yet-released ubuntu has twisted 8, they are still on 2.5:
>
>  http://packages.ubuntu.com/hardy/python-twisted-core
>
>  I'm not fundamentally opposed to requiring 8.0 if we actually need it,
>  but otherwise I'd prefer to find a way (if possible, I haven't looked
>  at the code) to work with both 2.5 and 8.0. I hope the twisted devs
>  didn't make so many backwards incompatible changes as to make that
>  impossible...
>
>  Cheers,
>
>
>
>  f
>  _______________________________________________
>  IPython-dev mailing list
>  IPython-dev at scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From fperez.net at gmail.com  Mon Mar 31 21:48:25 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 31 Mar 2008 18:48:25 -0700
Subject: [IPython-dev] Twisted 8...
In-Reply-To: <cd7634ce0803311818n21d9b738t873a39ca1dea497a@mail.gmail.com>
References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org>
	<6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com>
	<db6b5ecc0803311715u3bb0593ch172222b9db25ca91@mail.gmail.com>
	<cd7634ce0803311818n21d9b738t873a39ca1dea497a@mail.gmail.com>
Message-ID: <db6b5ecc0803311848o6c10ee02g96589d9a79b6a480@mail.gmail.com>

On Mon, Mar 31, 2008 at 6:18 PM, Barry Wark <barrywark at gmail.com> wrote:
> Being installable via easy_install is a huge win (really a
>  requirement) on OS X 10.5.

Thanks for the info, easy_install support is certainly a benefit.
That doesn't outweigh the cost of losing out-of-the-box installability
on all linuxes, I'm afraid (it's not our fault that OSX is such a mess
to develop for when outside of the Apple garden), so we should find
out what's needed to make the code run with both 2.5 and 8.0. I'm now
praying that they didn't make it impossible...

Brian, you know that code far better than I do.  Any ideas on how hard
that would be?

Cheers,

f