From ellisonbg at gmail.com  Mon May  2 23:11:46 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 2 May 2011 20:11:46 -0700
Subject: [IPython-dev] ipython in a background thread
In-Reply-To: <840FA957-B8F0-481D-8BC1-7B26921D90B1@yale.edu>
References: <840FA957-B8F0-481D-8BC1-7B26921D90B1@yale.edu>
Message-ID: <BANLkTimF0xnu1Z1g3FjSRfM2ofHm7Ozt4w@mail.gmail.com>

I would check out the multiprocess Qt GUI that we have in master now.

Cheers,

Brian

On Fri, Apr 29, 2011 at 10:03 AM, Zachary Pincus
<zachary.pincus at yale.edu> wrote:
> Hello all,
>
> I find that ipython, specifically ipython running in a good terminal (as opposed to embedded in a GUI window), provides a really great interface (e.g. in my case, a microscope and camera hardware). Given this use, it's been important to also have a GUI thread running so I can throw acquired images rapidly up onto a GL canvas or something. For a while, I've used a hacked-up pyglet runloop that would work in a background thread, and set up a simple message-passing system.
>
> This got quite kluged quite fast though, and it turns out this approach won't work at all with cocoa GUIs on OS X, which seem to require running on thread-0 (a pity). I know people on this list have looked and thought about these issues a lot, so I'd be curious what seems like the best approach in general? (There used to be some code in IPython for doing this with various window toolkits, but I don't see that in the 0.10.2's codebase anymore...)
>
> - I could run the GUI from thread-0 and try message-passing to IPython on another thread. Does this work well at all, or is it a huge kluge to get all the readline etc. features working right?
>
> - I could run the GUI in a separate process entirely, which would force a much cleaner API, but I'm not sure if pumping images at video-rate could be done cleanly. Maybe with shared memmapped arrays? (I've seen some recipes for this sort of thing on the numpy list lately.)
>
> - Any other possibilities or thoughts?
>
> Thanks a lot,
> Zach
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From zachary.pincus at yale.edu  Tue May  3 07:36:59 2011
From: zachary.pincus at yale.edu (Zachary Pincus)
Date: Tue, 3 May 2011 07:36:59 -0400
Subject: [IPython-dev] ipython in a background thread
In-Reply-To: <BANLkTimF0xnu1Z1g3FjSRfM2ofHm7Ozt4w@mail.gmail.com>
References: <840FA957-B8F0-481D-8BC1-7B26921D90B1@yale.edu>
	<BANLkTimF0xnu1Z1g3FjSRfM2ofHm7Ozt4w@mail.gmail.com>
Message-ID: <640923AE-FBEE-4D68-894C-976FC20BF81B@yale.edu>

> I would check out the multiprocess Qt GUI that we have in master now.

I was looking at that -- unless I'm wrong, that throws up a window and emulates a terminal (or rather, runs ipython) inside of it? I was hoping to avoid that sort of approach, actually, and use a dedicated terminal emulator. However, if folks with more experience with this sort of thing than I have decided that the latter approach really isn't workable, I suppose the former will have to do.

Zach



> Cheers,
> 
> Brian
> 
> On Fri, Apr 29, 2011 at 10:03 AM, Zachary Pincus
> <zachary.pincus at yale.edu> wrote:
>> Hello all,
>> 
>> I find that ipython, specifically ipython running in a good terminal (as opposed to embedded in a GUI window), provides a really great interface (e.g. in my case, a microscope and camera hardware). Given this use, it's been important to also have a GUI thread running so I can throw acquired images rapidly up onto a GL canvas or something. For a while, I've used a hacked-up pyglet runloop that would work in a background thread, and set up a simple message-passing system.
>> 
>> This got quite kluged quite fast though, and it turns out this approach won't work at all with cocoa GUIs on OS X, which seem to require running on thread-0 (a pity). I know people on this list have looked and thought about these issues a lot, so I'd be curious what seems like the best approach in general? (There used to be some code in IPython for doing this with various window toolkits, but I don't see that in the 0.10.2's codebase anymore...)
>> 
>> - I could run the GUI from thread-0 and try message-passing to IPython on another thread. Does this work well at all, or is it a huge kluge to get all the readline etc. features working right?
>> 
>> - I could run the GUI in a separate process entirely, which would force a much cleaner API, but I'm not sure if pumping images at video-rate could be done cleanly. Maybe with shared memmapped arrays? (I've seen some recipes for this sort of thing on the numpy list lately.)
>> 
>> - Any other possibilities or thoughts?
>> 
>> Thanks a lot,
>> Zach
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
> 
> 
> 
> -- 
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com



From takowl at gmail.com  Tue May  3 08:07:49 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 3 May 2011 13:07:49 +0100
Subject: [IPython-dev] ipython in a background thread
In-Reply-To: <640923AE-FBEE-4D68-894C-976FC20BF81B@yale.edu>
References: <840FA957-B8F0-481D-8BC1-7B26921D90B1@yale.edu>
	<BANLkTimF0xnu1Z1g3FjSRfM2ofHm7Ozt4w@mail.gmail.com>
	<640923AE-FBEE-4D68-894C-976FC20BF81B@yale.edu>
Message-ID: <BANLkTi=AAX4WjqUQz7=+5gjveJqvphgLeA@mail.gmail.com>

On 3 May 2011 12:36, Zachary Pincus <zachary.pincus at yale.edu> wrote:

> I was looking at that -- unless I'm wrong, that throws up a window and
> emulates a terminal (or rather, runs ipython) inside of it? I was hoping to
> avoid that sort of approach, actually, and use a dedicated terminal
> emulator. However, if folks with more experience with this sort of thing
> than I have decided that the latter approach really isn't workable, I
> suppose the former will have to do.
>

You can have the terminal running alongside GUI toolkits, and as far as I
know the code should still be in 0.10.2 (I don't know where, but it's under
IPython/lib in the dev version). I believe we do this by joining the
terminal and GUI event loops, rather than threading, but I've not looked
much at the relevant code. There isn't an implementation for Pyglet yet.

With the Qt GUI, it's possible to display images inline, rather than just
text. I don't know if it's what you're after, but it might be worth looking
at.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110503/bef8d303/attachment.html>

From takowl at gmail.com  Tue May  3 08:34:15 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 3 May 2011 13:34:15 +0100
Subject: [IPython-dev] ipython in a background thread
In-Reply-To: <BANLkTi=jEorj--48dTp160FMHq7LHaY05A@mail.gmail.com>
References: <840FA957-B8F0-481D-8BC1-7B26921D90B1@yale.edu>
	<BANLkTimF0xnu1Z1g3FjSRfM2ofHm7Ozt4w@mail.gmail.com>
	<640923AE-FBEE-4D68-894C-976FC20BF81B@yale.edu>
	<BANLkTi=AAX4WjqUQz7=+5gjveJqvphgLeA@mail.gmail.com>
	<BANLkTi=jEorj--48dTp160FMHq7LHaY05A@mail.gmail.com>
Message-ID: <BANLkTikvk4rRivjn-wQrdH_zocVG3Xci3g@mail.gmail.com>

On 3 May 2011 13:10, Carl Joseph Younger <carl.input at gmail.com> wrote:

> Is is possible to animate the images or just display a still?
>

I'm not sure. I believe we're using Qt webkit, but perhaps someone else on
the list can give more details.

Brian's also got an HTML notebook interface in the works, but it's not in
master yet.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110503/07552a53/attachment.html>

From dave.hirschfeld at gmail.com  Thu May  5 12:18:32 2011
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Thu, 5 May 2011 16:18:32 +0000 (UTC)
Subject: [IPython-dev] zmq/pyzmq Version Incompatibilities
Message-ID: <loom.20110505T180121-595@post.gmane.org>

Not sure if this is an appropriate forum for pyzmq problems but since it's
a dependency for the parallel stuff I thought I'd post here.

This is more of a heads-up in case people weren't aware of possible problems
running off the latest github checkouts. Running stock zmq 2.1.4 with the pyzmq
2.1.4 branch works except for the (benign?) error:

======================================================================
FAIL: test_create (zmq.tests.test_socket.TestSocket)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\dev\bin\Python26\Lib\site-packages\zmq\tests\test_socket.py", 
    line 47, in test_create
    self.assertRaisesErrno(zmq.EPROTONOSUPPORT, s.bind, 'ftl://a')
  File "C:\dev\bin\Python26\Lib\site-packages\zmq\tests\__init__.py", 
    line 104, in assertRaisesErrno
    got '%s'" % (zmq.ZMQError(errno), zmq.ZMQError(e.errno)))
AssertionError: wrong error raised,
                expected 'Unknown error' got 'Protocol not supported'

----------------------------------------------------------------------


* With trunk pyzmq and trunk zmq I get the following error:

C:\dev\src\pyzmq>python setup.py configure --zmq=C:\dev\src\zeromq-trunk
running configure
******************************************
Configure: Autodetecting ZMQ settings...
    Custom ZMQ dir:       C:\dev\src\zeromq-trunk
c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox
/MD /W3 /GS- /DNDEBUG -IC:\dev\src\zeromq-trunk\include -Izmq\utils -Izmq\core
-Izmq\devices /Tcdetect\vers.c /Fodetect\vers.obj
vers.c
c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /nologo
/INCREMENTAL:NO /LIBPATH:C:\dev\src\zeromq-trunk\lib libzmq.lib detect\vers.obj
/OUT:detect\vers.exe /MANIFESTFILE:detect\vers.exe.manifest
C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\mt.exe -nologo -manifest
detect\vers.exe.manifest -outputresource:detect\vers.exe;1
    ZMQ version detected: 3.0.0
******************************************

C:\dev\src\pyzmq>python setup.py build
running build
running build_py
...
running build_ext
building 'zmq.core.constants' extension
creating build\temp.win32-2.6
creating build\temp.win32-2.6\Release
creating build\temp.win32-2.6\Release\zmq
creating build\temp.win32-2.6\Release\zmq\core
c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox
/MD /W3 /GS- /DNDEBUG -IC:\dev\src\zeromq-trunk\include -Izmq\utils -Izmq\core
-Izmq\devices -IC:\dev\bin\Python26\include -IC:\dev\bin\Python26\PC
/Tczmq\core\constants.c /Fobuild\temp.win32-2.6\Release\zmq\core\constants.obj
constants.c
zmq\core\constants.c(704) : 
    error C2065: 'ZMQ_NOBLOCK' : undeclared identifier
zmq\core\constants.c(824) : 
    error C2065: 'ZMQ_UPSTREAM' : undeclared identifier
zmq\core\constants.c(836) : 
    error C2065: 'ZMQ_DOWNSTREAM' : undeclared identifier
zmq\core\constants.c(916) : 
    error C2065: 'ZMQ_HWM' : undeclared identifier
zmq\core\constants.c(928) : 
    error C2065: 'ZMQ_SWAP' : undeclared identifier
zmq\core\constants.c(1012) : 
    error C2065: 'ZMQ_MCAST_LOOP' : undeclared identifier
zmq\core\constants.c(1108) : 
    error C2065: 'ZMQ_STREAMER' : undeclared identifier
zmq\core\constants.c(1120) : 
    error C2065: 'ZMQ_FORWARDER' : undeclared identifier
zmq\core\constants.c(1132) : 
    error C2065: 'ZMQ_QUEUE' : undeclared identifier
error: command '"c:\Program Files (x86)\Microsoft Visual Studio
9.0\VC\BIN\cl.exe"' failed with exit status 2

C:\dev\src\pyzmq>


* With trunk pyzmq and 2.1.6 zmq, pyzmq compiles but the tests segfault:

C:\dev\bin\Python26\Lib\site-packages\zmq\tests>python
Python 2.6.5 (r265:79096, Mar 19 2010, 21:48:26)
[MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
->>> import zmq
->>> zmq.pyzmq_version()
'2.1dev'
->>> zmq.zmq_version()
'2.1.6'
->>>

C:\dev\bin\Python26\Lib\site-packages\zmq\tests>nosetests -v
test_fail_init (zmq.tests.test_context.TestContext) ... ok
test_init (zmq.tests.test_context.TestContext) ... ok
test_instance (zmq.tests.test_context.TestContext) ... ok
test_term (zmq.tests.test_context.TestContext) ... ok
test_term_hang (zmq.tests.test_context.TestContext) ... ok
test_device_attributes (zmq.tests.test_device.TestDevice) ... ok
test_device_types (zmq.tests.test_device.TestDevice) ... ok
test_single_socket_forwarder_bind (zmq.tests.test_device.TestDevice) ...
Assertion failed: Permission denied (..\..\..\src\err.cpp:228)

* With trunk pyzmq and 2.1.4 zmq pyzmq won't import:

->>> import zmq
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\dev\bin\Python26\lib\site-packages\zmq\__init__.py", 
    line 35, in <module>
    from zmq import core, devices
  File "C:\dev\bin\Python26\lib\site-packages\zmq\core\__init__.py", 
    line 26, in <module>
    from zmq.core import (constants, error, message, context,
ImportError: DLL load failed: The specified procedure could not be found.
->>>

HTH,
Dave



From benjaminrk at gmail.com  Thu May  5 18:09:37 2011
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 5 May 2011 15:09:37 -0700
Subject: [IPython-dev] zmq/pyzmq Version Incompatibilities
In-Reply-To: <loom.20110505T180121-595@post.gmane.org>
References: <loom.20110505T180121-595@post.gmane.org>
Message-ID: <BANLkTin3jYanvmHubsMiYozAqUVVLyRcog@mail.gmail.com>

Hello,

Thanks for the thorough reports!

Are you using VS-2010?  I understand there are various issues when you
use VS-2010 (MSVC 10) to compile a library, but 2008 (MSVC 9, and the
one used for all official Pythons so far) for Python extensions.  We
have one known issue caused by this, and it is the errno mismatch you
see with 2.1.4+2.1.4, but it could be the cause of many of your
problems.

Note that libzmq has 3 repos right now on GitHub.  zeromq2-1 can be
considered 'maint', zeromq2-2 is the current dev, and libzmq (version
3.0) is the API-breaking "Don't worry about changing everything"
(think  Python 3) development repo.

I should also note that I have no build issues, and 100% of tests
passing on my Windows VM for pzmq-master with: zeromq-2.1.4, 2.1.6,
2.1dev (2.1.7), and 2.2dev (2.2.0).

My VM is Windows 7 x86_64, official Python 2.7.1 from MSI, MSVC 9 (VS
2008 express).

On Thu, May 5, 2011 at 09:18, Dave Hirschfeld <dave.hirschfeld at gmail.com> wrote:
> Not sure if this is an appropriate forum for pyzmq problems but since it's
> a dependency for the parallel stuff I thought I'd post here.
>
> This is more of a heads-up in case people weren't aware of possible problems
> running off the latest github checkouts. Running stock zmq 2.1.4 with the pyzmq
> 2.1.4 branch works except for the (benign?) error:

This error is Benign *unless* you are trying to handle ZMQ errors
yourself.  This errno mismatch only seems to occur when libzmq is
built with VS-2010.  It has no effect on IPython's use of libzmq.

>
> ======================================================================
> FAIL: test_create (zmq.tests.test_socket.TestSocket)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> ?File "C:\dev\bin\Python26\Lib\site-packages\zmq\tests\test_socket.py",
> ? ?line 47, in test_create
> ? ?self.assertRaisesErrno(zmq.EPROTONOSUPPORT, s.bind, 'ftl://a')
> ?File "C:\dev\bin\Python26\Lib\site-packages\zmq\tests\__init__.py",
> ? ?line 104, in assertRaisesErrno
> ? ?got '%s'" % (zmq.ZMQError(errno), zmq.ZMQError(e.errno)))
> AssertionError: wrong error raised,
> ? ? ? ? ? ? ? ?expected 'Unknown error' got 'Protocol not supported'
>
> ----------------------------------------------------------------------
>
>
> * With trunk pyzmq and trunk zmq I get the following error:

See above for multiply defined 'trunk' for zeromq.  We do not yet have
a branch that tracks 3.0 development (libzmq on github)

>
> C:\dev\src\pyzmq>python setup.py configure --zmq=C:\dev\src\zeromq-trunk
> running configure
> ******************************************
> Configure: Autodetecting ZMQ settings...
> ? ?Custom ZMQ dir: ? ? ? C:\dev\src\zeromq-trunk
> c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox
> /MD /W3 /GS- /DNDEBUG -IC:\dev\src\zeromq-trunk\include -Izmq\utils -Izmq\core
> -Izmq\devices /Tcdetect\vers.c /Fodetect\vers.obj
> vers.c
> c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /nologo
> /INCREMENTAL:NO /LIBPATH:C:\dev\src\zeromq-trunk\lib libzmq.lib detect\vers.obj
> /OUT:detect\vers.exe /MANIFESTFILE:detect\vers.exe.manifest
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\mt.exe -nologo -manifest
> detect\vers.exe.manifest -outputresource:detect\vers.exe;1
> ? ?ZMQ version detected: 3.0.0
> ******************************************
>
> C:\dev\src\pyzmq>python setup.py build
> running build
> running build_py
> ...
> running build_ext
> building 'zmq.core.constants' extension
> creating build\temp.win32-2.6
> creating build\temp.win32-2.6\Release
> creating build\temp.win32-2.6\Release\zmq
> creating build\temp.win32-2.6\Release\zmq\core
> c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox
> /MD /W3 /GS- /DNDEBUG -IC:\dev\src\zeromq-trunk\include -Izmq\utils -Izmq\core
> -Izmq\devices -IC:\dev\bin\Python26\include -IC:\dev\bin\Python26\PC
> /Tczmq\core\constants.c /Fobuild\temp.win32-2.6\Release\zmq\core\constants.obj
> constants.c
> zmq\core\constants.c(704) :
> ? ?error C2065: 'ZMQ_NOBLOCK' : undeclared identifier
> zmq\core\constants.c(824) :
> ? ?error C2065: 'ZMQ_UPSTREAM' : undeclared identifier
> zmq\core\constants.c(836) :
> ? ?error C2065: 'ZMQ_DOWNSTREAM' : undeclared identifier
> zmq\core\constants.c(916) :
> ? ?error C2065: 'ZMQ_HWM' : undeclared identifier
> zmq\core\constants.c(928) :
> ? ?error C2065: 'ZMQ_SWAP' : undeclared identifier
> zmq\core\constants.c(1012) :
> ? ?error C2065: 'ZMQ_MCAST_LOOP' : undeclared identifier
> zmq\core\constants.c(1108) :
> ? ?error C2065: 'ZMQ_STREAMER' : undeclared identifier
> zmq\core\constants.c(1120) :
> ? ?error C2065: 'ZMQ_FORWARDER' : undeclared identifier
> zmq\core\constants.c(1132) :
> ? ?error C2065: 'ZMQ_QUEUE' : undeclared identifier
> error: command '"c:\Program Files (x86)\Microsoft Visual Studio
> 9.0\VC\BIN\cl.exe"' failed with exit status 2
>
> C:\dev\src\pyzmq>
>
>
> * With trunk pyzmq and 2.1.6 zmq, pyzmq compiles but the tests segfault:

Permission denied is not a segfault.  I cannot reproduce this, pyzmq
master passes all tests with 2.1.6.  It's possible this is due to
using MSVC10 with libzmq, and MSVC 9 with pyzmq.  If your checkout was
before yesterday, this could have simply been a port conflict in the
test (recently fixed).

>
> C:\dev\bin\Python26\Lib\site-packages\zmq\tests>python
> Python 2.6.5 (r265:79096, Mar 19 2010, 21:48:26)
> [MSC v.1500 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> ->>> import zmq
> ->>> zmq.pyzmq_version()
> '2.1dev'
> ->>> zmq.zmq_version()
> '2.1.6'
> ->>>
>
> C:\dev\bin\Python26\Lib\site-packages\zmq\tests>nosetests -v
> test_fail_init (zmq.tests.test_context.TestContext) ... ok
> test_init (zmq.tests.test_context.TestContext) ... ok
> test_instance (zmq.tests.test_context.TestContext) ... ok
> test_term (zmq.tests.test_context.TestContext) ... ok
> test_term_hang (zmq.tests.test_context.TestContext) ... ok
> test_device_attributes (zmq.tests.test_device.TestDevice) ... ok
> test_device_types (zmq.tests.test_device.TestDevice) ... ok
> test_single_socket_forwarder_bind (zmq.tests.test_device.TestDevice) ...
> Assertion failed: Permission denied (..\..\..\src\err.cpp:228)
>
> * With trunk pyzmq and 2.1.4 zmq pyzmq won't import:
>
> ->>> import zmq
> Traceback (most recent call last):
> ?File "<stdin>", line 1, in <module>
> ?File "C:\dev\bin\Python26\lib\site-packages\zmq\__init__.py",
> ? ?line 35, in <module>
> ? ?from zmq import core, devices
> ?File "C:\dev\bin\Python26\lib\site-packages\zmq\core\__init__.py",
> ? ?line 26, in <module>
> ? ?from zmq.core import (constants, error, message, context,
> ImportError: DLL load failed: The specified procedure could not be found.

Do you have a different libzmq.dll on your PATH?  Did you make sure
pyzmq was clean when you rebuilt it against each new libzmq?

I can run pyzmq master with libzmq 2.1.4 (built with either VS 2010 or
2008), with no problems aside from the errno mismatch on VS2010).

I hope we can resolve these, because pzymq is known to work on Windows
in all of these combinations (excluding libzmq-3.0).

-MinRK

> ->>>
>
> HTH,
> Dave
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From dave.hirschfeld at gmail.com  Tue May 10 10:56:05 2011
From: dave.hirschfeld at gmail.com (David Hirschfeld)
Date: Tue, 10 May 2011 15:56:05 +0100
Subject: [IPython-dev] zmq/pyzmq Version Incompatibilities
In-Reply-To: <BANLkTin3jYanvmHubsMiYozAqUVVLyRcog@mail.gmail.com>
References: <loom.20110505T180121-595@post.gmane.org>
	<BANLkTin3jYanvmHubsMiYozAqUVVLyRcog@mail.gmail.com>
Message-ID: <BANLkTimoz8waVcSB5Gn71VUcps0mtG17cQ@mail.gmail.com>

On Thu, May 5, 2011 at 11:09 PM, MinRK <benjaminrk at gmail.com> wrote:
>
> Are you using VS-2010? ?I understand there are various issues when you
> use VS-2010 (MSVC 10) to compile a library, but 2008 (MSVC 9, and the
> one used for all official Pythons so far) for Python extensions. ?We
> have one known issue caused by this, and it is the errno mismatch you
> see with 2.1.4+2.1.4, but it could be the cause of many of your
> problems.
>
>
> I should also note that I have no build issues, and 100% of tests
> passing on my Windows VM for pzmq-master with: zeromq-2.1.4, 2.1.6,
> 2.1dev (2.1.7), and 2.2dev (2.2.0).
>
> My VM is Windows 7 x86_64, official Python 2.7.1 from MSI, MSVC 9 (VS
> 2008 express).
>
>
> Do you have a different libzmq.dll on your PATH? ?Did you make sure
> pyzmq was clean when you rebuilt it against each new libzmq?
>
> I can run pyzmq master with libzmq 2.1.4 (built with either VS 2010 or
> 2008), with no problems aside from the errno mismatch on VS2010).
>
> I hope we can resolve these, because pzymq is known to work on Windows
> in all of these combinations (excluding libzmq-3.0).
>
> -MinRK
>

Hi Min,
I'm on Win7 x64 running Python 2.6.5 (r265:79096, Mar 19 2010,
21:48:26) [MSC v.1500 32 bit (Intel)] on win32. I had previously used
VS2010 to compile zmq/pyzmq but to eliminate that in the following
I've used VS2008. I'm using zmq-2.2 master and pyzmq master and I made
sure to delete the build directory beforehand so it should be a clean
install. There's no other libzmq.dll in my path.

The results of the zmq/pyzmq build are attached - everything seems
fine at first glance however when running the tests I get the
"Permission denied" error occurring at random times:

C:\dev\bin\Python26\Lib\site-packages\zmq\tests>nosetests -v
<snip>
test_no_events (zmq.tests.test_poll.TestPoll) ... Assertion failed:
Permission denied (..\..\..\src\err.cpp:234)

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

C:\dev\bin\Python26\Lib\site-packages\zmq\tests>

C:\dev\bin\Python26\Lib\site-packages\zmq\tests>nosetests -v
<snip>
test_single_socket_forwarder_bind (zmq.tests.test_device.TestDevice)
... Assertion failed: Permission denied (..\..\..\src\err.cpp:234)

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

C:\dev\bin\Python26\Lib\site-packages\zmq\tests>

C:\dev\bin\Python26\Lib\site-packages\zmq\tests>nosetests -v
<snip>
test_monitor (zmq.tests.test_monqueue.TestMonitoredQueue) ...
Assertion failed: Permission denied (..\..\..\src\err.cpp:234)

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

C:\dev\bin\Python26\Lib\site-packages\zmq\tests>


I'm not sure if segfault is the right technical term but it does crash
the python interpreter, bringing up a "python.exe has stopped working"
dialog box.

It seems like a tricky one to debug and could easily be something
specific to my environment. Hopefully this helps narrow it down a bit
though...

-Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110510/454e2ec1/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pyzmq-build.log
Type: application/octet-stream
Size: 25472 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110510/454e2ec1/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pyzmq-configure.log
Type: application/octet-stream
Size: 793 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110510/454e2ec1/attachment-0001.obj>

From fperez.net at gmail.com  Thu May 12 09:56:44 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 12 May 2011 08:56:44 -0500
Subject: [IPython-dev] Heads-up on htmlnotebook branch
Message-ID: <BANLkTimJKyzc+XUvuwT5n+Ve20JHJfSzyw@mail.gmail.com>

Hi folks,

a couple of minor notes on the htmlnb branch...

- I'm seeing this traceback when I have the qt console open as well:

Traceback (most recent call last):
  File "/home/fperez/ipython/ipython/IPython/frontend/qt/base_frontend_mixin.py",
line 102, in _dispatch
    handler(msg)
  File "/home/fperez/ipython/ipython/IPython/frontend/qt/console/rich_ipython_widget.py",
line 98, in _handle_display_data
    if not self._hidden and self._is_from_this_session(msg):
  File "/home/fperez/ipython/ipython/IPython/frontend/qt/base_frontend_mixin.py",
line 109, in _is_from_this_session
    return msg['parent_header']['session'] == session
KeyError: 'session'

- I tried to merge htmlnotebook into master (locally, obviously :) to
see if it would go away, but right now there's a small conflict in
setupbase.py.  So we probably want to update htmlnotebook to merge
cleanly.

I won't get to this for the next couple of days, I'm giving a couple
of talks in Austin about it though (and demoing it :) so I'll report
on anything else I see.

I gave one at the Austin Python Users Group last night that was very
well received, and hopefully we'll be getting some more pull requests
soon from the local hackers :)

Cheers,

f


From ellisonbg at gmail.com  Thu May 12 12:49:39 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 12 May 2011 09:49:39 -0700
Subject: [IPython-dev] Heads-up on htmlnotebook branch
In-Reply-To: <BANLkTimJKyzc+XUvuwT5n+Ve20JHJfSzyw@mail.gmail.com>
References: <BANLkTimJKyzc+XUvuwT5n+Ve20JHJfSzyw@mail.gmail.com>
Message-ID: <BANLkTinqvi9USdtexknTMre1ysFb-POKvA@mail.gmail.com>

Fernando,

I think may be a regression in the qt app.  Can you try multiple qt
frontends with no htmlnotebook?

Brian

On Thu, May 12, 2011 at 6:56 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi folks,
>
> a couple of minor notes on the htmlnb branch...
>
> - I'm seeing this traceback when I have the qt console open as well:
>
> Traceback (most recent call last):
> ?File "/home/fperez/ipython/ipython/IPython/frontend/qt/base_frontend_mixin.py",
> line 102, in _dispatch
> ? ?handler(msg)
> ?File "/home/fperez/ipython/ipython/IPython/frontend/qt/console/rich_ipython_widget.py",
> line 98, in _handle_display_data
> ? ?if not self._hidden and self._is_from_this_session(msg):
> ?File "/home/fperez/ipython/ipython/IPython/frontend/qt/base_frontend_mixin.py",
> line 109, in _is_from_this_session
> ? ?return msg['parent_header']['session'] == session
> KeyError: 'session'
>
> - I tried to merge htmlnotebook into master (locally, obviously :) to
> see if it would go away, but right now there's a small conflict in
> setupbase.py. ?So we probably want to update htmlnotebook to merge
> cleanly.
>
> I won't get to this for the next couple of days, I'm giving a couple
> of talks in Austin about it though (and demoing it :) so I'll report
> on anything else I see.
>
> I gave one at the Austin Python Users Group last night that was very
> well received, and hopefully we'll be getting some more pull requests
> soon from the local hackers :)
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From joonpyro at gmail.com  Fri May 13 16:48:37 2011
From: joonpyro at gmail.com (Joon Ro)
Date: Fri, 13 May 2011 15:48:37 -0500
Subject: [IPython-dev] run filename autocompletion "The kernel heartbeat has
 been inactive ... " error
Message-ID: <op.vvftfbmudm0f0t@linux-64fk.site>

Hi,

First of all let me say the qtconsole is awesome. Thanks for the brilliant  
work.

I don't know if this is a bug, but when I press tab to autocomplete during  
the input of the argument of the run magic command:

In [2]: %run 'work[TAB]

ipython-qtconsole hangs for a while, and then gives me the following error:

The kernel heartbeat has been inactive for 3.00 seconds. Do you want to  
restart the kernel? You may first want to check the network connection.

And then if I press yes it tries to restart the python shell but it hangs  
after the initial messages (In [1] never shows up)

In other cases (for example, with cd) the autocompletion of directories  
works well.

I'm using openSUSE 11.4 64bit.

I am not sure if this is place to report these problems, so if not please  
let me know.

Joon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110513/d6fe9be2/attachment.html>

From epatters at enthought.com  Fri May 13 17:38:55 2011
From: epatters at enthought.com (Evan Patterson)
Date: Fri, 13 May 2011 16:38:55 -0500
Subject: [IPython-dev] run filename autocompletion "The kernel heartbeat
	has been inactive ... " error
In-Reply-To: <op.vvftfbmudm0f0t@linux-64fk.site>
References: <op.vvftfbmudm0f0t@linux-64fk.site>
Message-ID: <069E9B21-354D-431D-869B-810644BA352F@enthought.com>

Joon,

Thanks for the report. You have indeed found a bug, although I was not able to immediately reproduce it myself. 

Could you please create an issue for this on the IPython issue tracker on GitHub? Also, if you run the Qt console from a terminal, you should see some output from the kernel.  If there is a traceback after you execute the bad command, please attach it to the issue.

Thanks,
Evan

On May 13, 2011, at 3:48 PM, Joon Ro wrote:

> Hi,
> 
> First of all let me say the qtconsole is awesome. Thanks for the brilliant work.
> 
> I don't know if this is a bug, but when I press tab to autocomplete during the input of the argument of the run magic command: 
> 
> In [2]: %run 'work[TAB]
> 
> ipython-qtconsole hangs for a while, and then gives me the following error:
> 
> The kernel heartbeat has been inactive for 3.00 seconds. Do you want to restart the kernel? You may first want to check the network connection.
> 
> And then if I press yes it tries to restart the python shell but it hangs after the initial messages (In [1] never shows up)
> 
> In other cases (for example, with cd) the autocompletion of directories works well. 
> 
> I'm using openSUSE 11.4 64bit.
> 
> I am not sure if this is place to report these problems, so if not please let me know.
> 
> Joon
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110513/5300a148/attachment.html>

From joonpyro at gmail.com  Fri May 13 20:01:58 2011
From: joonpyro at gmail.com (Joon Ro)
Date: Fri, 13 May 2011 19:01:58 -0500
Subject: [IPython-dev] run filename autocompletion "The kernel heartbeat
 has been inactive ... " error
In-Reply-To: <069E9B21-354D-431D-869B-810644BA352F@enthought.com>
References: <op.vvftfbmudm0f0t@linux-64fk.site>
	<069E9B21-354D-431D-869B-810644BA352F@enthought.com>
Message-ID: <op.vvf2dkyedm0f0t@linux-64fk.site>

Thanks for the reply Evan.

As you said I ran ipython-qtconsole --pylab inline in the terminal, and  
actually that did not happen.
It happens only when I run it as an application - I made an application  
item with command: ipython-qtconsole --pylab inline (I'm using KDE)

Now I tried it with different options, and when I check "Run in terminal"  
it does not happen.

So it only happens when I run it as an application, with "Run in terminal"  
unchecked.

I will create an issue for this.

Thank you,
Joon



On Fri, 13 May 2011 16:38:55 -0500, Evan Patterson  
<epatters at enthought.com> wrote:

> Joon,
>
> Thanks for the report. You have indeed found a bug, although I was not  
> able to immediately reproduce it myself.
> Could you please create an issue for this on the IPython issue tracker  
> on GitHub? Also, if you run the Qt console from a terminal, you should  
> see some output from the kernel.  If there >is a traceback after you  
> execute the bad command, please attach it to the issue.
>
> Thanks,
> Evan
>
> On May 13, 2011, at 3:48 PM, Joon Ro wrote:
>
>>>> Hi,
>>
>> First of all let me say the qtconsole is awesome. Thanks for the  
>> brilliant work.
>>
>> I don't know if this is a bug, but when I press tab to autocomplete  
>> during the input of the argument of the run magic command:
>> In [2]: %run 'work[TAB]
>>
>> ipython-qtconsole hangs for a while, and then gives me the following  
>> error:
>>
>> The kernel heartbeat has been inactive for 3.00 seconds. Do you want to  
>> restart the kernel? You may first want to check the network connection.
>>
>> And then if I press yes it tries to restart the python shell but it  
>> hangs after the initial messages (In [1] never shows up)
>>
>> In other cases (for example, with cd) the autocompletion of directories  
>> works well.
>> I'm using openSUSE 11.4 64bit.
>>
>> I am not sure if this is place to report these problems, so if not  
>> please let me know.
>>
>> Joon
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110513/8aae0c8e/attachment.html>

From benjaminrk at gmail.com  Mon May 16 01:10:13 2011
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 15 May 2011 22:10:13 -0700
Subject: [IPython-dev] zmq/pyzmq Version Incompatibilities
In-Reply-To: <BANLkTimoz8waVcSB5Gn71VUcps0mtG17cQ@mail.gmail.com>
References: <loom.20110505T180121-595@post.gmane.org>
	<BANLkTin3jYanvmHubsMiYozAqUVVLyRcog@mail.gmail.com>
	<BANLkTimoz8waVcSB5Gn71VUcps0mtG17cQ@mail.gmail.com>
Message-ID: <BANLkTimfy-YyaOTQtQjeQWt80KNnLAJZfQ@mail.gmail.com>

My guess is that these are port conflict issues.  I believe Permission
Denied is what happens when you try to bind to an in-use port, but
maybe that's not accurate.

I don't know if this would change anything, but do you get any
different output if you do the Windows 'run as administrator' trick?
(I've never done such a thing with a command-line script).

On Tue, May 10, 2011 at 07:56, David Hirschfeld
<dave.hirschfeld at gmail.com> wrote:
> On Thu, May 5, 2011 at 11:09 PM, MinRK <benjaminrk at gmail.com> wrote:
>>
>> Are you using VS-2010? ?I understand there are various issues when you
>> use VS-2010 (MSVC 10) to compile a library, but 2008 (MSVC 9, and the
>> one used for all official Pythons so far) for Python extensions. ?We
>> have one known issue caused by this, and it is the errno mismatch you
>> see with 2.1.4+2.1.4, but it could be the cause of many of your
>> problems.
>>
>>
>> I should also note that I have no build issues, and 100% of tests
>> passing on my Windows VM for pzmq-master with: zeromq-2.1.4, 2.1.6,
>> 2.1dev (2.1.7), and 2.2dev (2.2.0).
>>
>> My VM is Windows 7 x86_64, official Python 2.7.1 from MSI, MSVC 9 (VS
>> 2008 express).
>>
>>
>> Do you have a different libzmq.dll on your PATH? ?Did you make sure
>> pyzmq was clean when you rebuilt it against each new libzmq?
>>
>> I can run pyzmq master with libzmq 2.1.4 (built with either VS 2010 or
>> 2008), with no problems aside from the errno mismatch on VS2010).
>>
>> I hope we can resolve these, because pzymq is known to work on Windows
>> in all of these combinations (excluding libzmq-3.0).
>>
>> -MinRK
>>
>
> Hi Min,
> I'm on Win7 x64 running Python 2.6.5 (r265:79096, Mar 19 2010,
> 21:48:26) [MSC v.1500 32 bit (Intel)] on win32. I had previously used
> VS2010 to compile zmq/pyzmq but to eliminate that in the following
> I've used VS2008. I'm using zmq-2.2 master and pyzmq master and I made
> sure to delete the build directory beforehand so it should be a clean
> install. There's no other libzmq.dll in my path.
>
> The results of the zmq/pyzmq build are attached - everything seems
> fine at first glance however when running the tests I get the
> "Permission denied" error occurring at random times:
>
> C:\dev\bin\Python26\Lib\site-packages\zmq\tests>nosetests -v
> <snip>
> test_no_events (zmq.tests.test_poll.TestPoll) ... Assertion failed:
> Permission denied (..\..\..\src\err.cpp:234)
>
> This application has requested the Runtime to terminate it in an unusual way.
> Please contact the application's support team for more information.
>
> C:\dev\bin\Python26\Lib\site-packages\zmq\tests>
>
> C:\dev\bin\Python26\Lib\site-packages\zmq\tests>nosetests -v
> <snip>
> test_single_socket_forwarder_bind (zmq.tests.test_device.TestDevice)
> ... Assertion failed: Permission denied (..\..\..\src\err.cpp:234)
>
> This application has requested the Runtime to terminate it in an unusual way.
> Please contact the application's support team for more information.
>
> C:\dev\bin\Python26\Lib\site-packages\zmq\tests>
>
> C:\dev\bin\Python26\Lib\site-packages\zmq\tests>nosetests -v
> <snip>
> test_monitor (zmq.tests.test_monqueue.TestMonitoredQueue) ...
> Assertion failed: Permission denied (..\..\..\src\err.cpp:234)
>
> This application has requested the Runtime to terminate it in an unusual way.
> Please contact the application's support team for more information.
>
> C:\dev\bin\Python26\Lib\site-packages\zmq\tests>
>
>
> I'm not sure if segfault is the right technical term but it does crash
> the python interpreter, bringing up a "python.exe has stopped working"
> dialog box.
>
> It seems like a tricky one to debug and could easily be something
> specific to my environment. Hopefully this helps narrow it down a bit
> though...
>
> -Dave
>


From robert.kern at gmail.com  Tue May 17 18:48:59 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 17 May 2011 17:48:59 -0500
Subject: [IPython-dev] Package-specific extensions: an idea
Message-ID: <iquu0u$qlm$1@dough.gmane.org>

There's a general problem for writing package-specific extensions to IPython. 
Namely, you want to be able to write and enable an extension to IPython that 
provides special behavior for particular objects, say numpy arrays, without 
requiring that a particular package be imported at startup. You can often get by 
in ad hoc ways. For example, in the pretty-printing code, I added the ability to 
register deferred type-checking. For magic functions, you can usually just do 
local imports inside the magics.

However, I've had an idea for a general solution.

Let's say you want to make an extension that does something special with numpy 
arrays. Perhaps it alters the ? introspection to print out array metadata like 
shape and dtype instead of the ndarray docstring when you use ? on arrays. You 
don't want it to modify anything until numpy is imported. You also don't want to 
import numpy at startup every time. You could implement a deferred isinstance() 
check the way I do it in pretty.py, but that's a complicated procedure that 
would need to be replicated everywhere.

Instead, we could have an API for our configuration for deferred extensions, 
namely, a list of (module_name, extension_name) pairs. IPython would keep a 
registry mapping these module names to the extension names. IPython's execution 
loop can be modified to check sys.modules after every execution and activate 
every extension whose requisite modules appear in sys.modules. Then it removes 
those extensions from the list.

So if my extension is named ipy_numpy_oinspect.py, then I'd have something like 
the following in my ipython_config.py file:

c.Global.deferred_extensions = [
     # (module_to_wait_for, extension)
     ('numpy', 'ipy_numpy_oinspect'),
]

And somewhere in run_cell() after the run_ast() call:

     # Check for any deferred extensions.
     # FIXME: probably wrap this up in a method.
     loaded = []
     for modname, extname in self.extension_manager.deferred_extensions:
         # Explicitly check for is not None since sometimes a missing module
         # will get a None entry to let the import mechanism fail faster.
         if sys.modules.get(modname, None) is not None:
             self.extension_manager.load_extension(extname)
             loaded.append((modname, extname))
     for x in loaded:
         self.extension_manager.deferred_extensions.remove(x)

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From ipython at mspacek.mm.st  Fri May 20 16:58:57 2011
From: ipython at mspacek.mm.st (Martin Spacek)
Date: Fri, 20 May 2011 13:58:57 -0700
Subject: [IPython-dev] replacement for appstart_qt4(app) ?
Message-ID: <ir6kmi$qmi$1@dough.gmane.org>

Motivated by ipython-qtconsole, I've taken the plunge and switched from 0.10 to 
git master. Apologies if this post is more appropriate for ipython-user.

I have an app that used appstart_qt4(app) in its __name__ == '__main__' block to 
automatically drop into the IPython debugger on any error. This is still 
referenced in:

docs/examples/lib/gui-qt.py

appstart_qt4() is now gone from git master, and I can't seem to find any mention 
of what to use in its place. get_app_qt4() and start_event_loop_qt4() in 
IPython.lib.guisupport seem like good candidates, but AFAICT they don't 
automatically drop you into the ipdb on an error.

I did however find this sys.excepthook method, which seems to work:

http://ipython.scipy.org/doc/manual/html/interactive/reference.html#automatic-invocation-of-pdb-on-exceptions

I want to both allow the user to drop into ipython at any point in time, as well 
as automatically drop into it on any error. After some messing around, I put 
together this minimal working example for git master:


import sys
from PyQt4 import QtCore, QtGui
from IPython import embed
from IPython.core import ultratb

def set_excepthook():
     """Drops us into IPython's debugger on any error"""
     sys.excepthook = ultratb.FormattedTB(mode='Verbose', call_pdb=1)

class SimpleWindow(QtGui.QWidget):
     def __init__(self, parent=None):
         QtGui.QWidget.__init__(self, parent)
         self.setGeometry(300, 300, 300, 200)
         self.setWindowTitle('embed and ipdb test')
         shellButton = QtGui.QPushButton('drop into shell', self)
         shellButton.setGeometry(10, 10, 100, 35)
         self.connect(shellButton, QtCore.SIGNAL('clicked()'),
                      self.shell)
         raiseErrorButton = QtGui.QPushButton('raise Error', self)
         raiseErrorButton.setGeometry(10, 50, 100, 35)
         self.connect(raiseErrorButton, QtCore.SIGNAL('clicked()'),
                      self.raise_error)

     def shell(self):
         embed(display_banner=False) # "self" is accessible from the shell
         # embed() seems to override the excepthook, need to reset it:
         set_excepthook()

     def raise_error(self):
         raise RuntimeError

if __name__ == "__main__":
     # prevents "The event loop is already running" errors:
     QtCore.pyqtRemoveInputHook()
     set_excepthook()
     app = QtGui.QApplication(sys.argv)
     window = SimpleWindow()
     window.show()
     sys.exit(app.exec_())


Hitting the shell button drops you into ipython, and hitting the raise Error 
button drops you into ipdb to inspect the error. I've noticed the embed() call 
clears the sys.excepthook setting, so I have to constantly reset it afterwards. 
Seems a bit messy. Is there a better way to do this?

What I'd ultimately want is a second Qt window running ipython-qtconsole in 
tandem with the main window, so the user never has to hit the "shell" button to 
stop the Qt loop just to inspect all of the existing objects. I used to have 
this kind of functionality in wxPython with pycrust/pyshell. Is such a thing 
currently possible with IPython in Qt?

Cheers,

Martin



From efiring at hawaii.edu  Sat May 21 17:49:26 2011
From: efiring at hawaii.edu (Eric Firing)
Date: Sat, 21 May 2011 11:49:26 -1000
Subject: [IPython-dev] tiny patch for sphinx version change
Message-ID: <4DD83366.6000808@hawaii.edu>

I updated to ipython master.  When attempting to build the docs I ran 
into a problem associated with changes in Sphinx between version 0.6.7 
and 1.x.  The attached patch allows the build to proceed with the latter 
and should still work with the former, although I have not tested that.

Eric
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: sphinx_version_patch
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110521/abf5b540/attachment.ksh>

From pivanov314 at gmail.com  Tue May 24 04:20:46 2011
From: pivanov314 at gmail.com (Paul Ivanov)
Date: Tue, 24 May 2011 01:20:46 -0700
Subject: [IPython-dev] UI related: TermKit (a webkit-based terminal)
Message-ID: <20110524082046.GE2795@ykcyc>

Hey gang,

thought I'd point you to this project, which looks like it has a
bit of overlap and capitalizes on some of the capabilities
we now have with the new Qt console stuff in ipython trunk:

a nice writeup:
http://acko.net/blog/on-termkit

code:
https://github.com/unconed/TermKit

-- 
Paul Ivanov
314 address only used for lists,  off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110524/ebd37a25/attachment.sig>

From dave.hirschfeld at gmail.com  Tue May 24 06:17:09 2011
From: dave.hirschfeld at gmail.com (David Hirschfeld)
Date: Tue, 24 May 2011 11:17:09 +0100
Subject: [IPython-dev] zmq/pyzmq Version Incompatibilities
In-Reply-To: <BANLkTimfy-YyaOTQtQjeQWt80KNnLAJZfQ@mail.gmail.com>
References: <loom.20110505T180121-595@post.gmane.org>
	<BANLkTin3jYanvmHubsMiYozAqUVVLyRcog@mail.gmail.com>
	<BANLkTimoz8waVcSB5Gn71VUcps0mtG17cQ@mail.gmail.com>
	<BANLkTimfy-YyaOTQtQjeQWt80KNnLAJZfQ@mail.gmail.com>
Message-ID: <BANLkTikhZSo-pa=B0DXBLse4Zvsm+-oQdw@mail.gmail.com>

On Mon, May 16, 2011 at 6:10 AM, MinRK <benjaminrk at gmail.com> wrote:
> My guess is that these are port conflict issues. ?I believe Permission
> Denied is what happens when you try to bind to an in-use port, but
> maybe that's not accurate.
>
> I don't know if this would change anything, but do you get any
> different output if you do the Windows 'run as administrator' trick?
> (I've never done such a thing with a command-line script).
>

I am an administrator on my machine. I did try specifically opening a
terminal as admin and running nosetests from there with the same
results.

If I open up the debugger on the exception I get the attached call
stack where it can be seen that it's trying to bind to
tcp://127.0.0.1:64385

If I use netstat to check what ports are being taken there's no
mention of 127.0.0.1:64385

C:\Users\dhirschfeld>netstat -ao | grep LISTENING
  TCP    0.0.0.0:135            DavePC:0             LISTENING       160
  TCP    0.0.0.0:445            DavePC:0             LISTENING       4
  TCP    0.0.0.0:3389           DavePC:0             LISTENING       1292
  TCP    0.0.0.0:8009           DavePC:0             LISTENING       7848
  TCP    0.0.0.0:8081           DavePC:0             LISTENING       2588
  TCP    0.0.0.0:9535           DavePC:0             LISTENING       1820
  TCP    0.0.0.0:9593           DavePC:0             LISTENING       1804
  TCP    0.0.0.0:9594           DavePC:0             LISTENING       1804
  TCP    0.0.0.0:9595           DavePC:0             LISTENING       1804
  TCP    0.0.0.0:33354          DavePC:0             LISTENING       1692
  TCP    0.0.0.0:49152          DavePC:0             LISTENING       692
  TCP    0.0.0.0:49153          DavePC:0             LISTENING       592
  TCP    0.0.0.0:49154          DavePC:0             LISTENING       940
  TCP    0.0.0.0:53638          DavePC:0             LISTENING       1524
  TCP    0.0.0.0:63961          DavePC:0             LISTENING       816
  TCP    0.0.0.0:64084          DavePC:0             LISTENING       780
  TCP    0.0.0.0:64089          DavePC:0             LISTENING       4904
  TCP    127.0.0.1:1434         DavePC:0             LISTENING       2224
  TCP    127.0.0.1:9592         DavePC:0             LISTENING       1804
  TCP    127.0.0.1:21584        DavePC:0             LISTENING       1072
  TCP    127.0.0.1:55376        DavePC:0             LISTENING       6364
  TCP    127.0.0.1:58862        DavePC:0             LISTENING       6364
  TCP    127.0.0.1:58901        DavePC:0             LISTENING       6364
  TCP    127.0.0.1:63963        DavePC:0             LISTENING       1692
  TCP    127.0.0.1:63964        DavePC:0             LISTENING       1804
  TCP    [::]:135               DavePC:0             LISTENING       160
  TCP    [::]:445               DavePC:0             LISTENING       4
  TCP    [::]:3389              DavePC:0             LISTENING       1292
  TCP    [::]:49152             DavePC:0             LISTENING       692
  TCP    [::]:49153             DavePC:0             LISTENING       592
  TCP    [::]:49154             DavePC:0             LISTENING       940
  TCP    [::]:53638             DavePC:0             LISTENING       1524
  TCP    [::]:63961             DavePC:0             LISTENING       816
  TCP    [::]:64084             DavePC:0             LISTENING       780
  TCP    [::]:64089             DavePC:0             LISTENING       4904
  TCP    [::1]:1434             DavePC:0             LISTENING       2224

C:\Users\dhirschfeld>netstat -ao | grep 64385

C:\Users\dhirschfeld>

...so I'm stumped, sorry. I think I'll have to revert to 2.1.4 in the meantime.


In case it helps in the Windows Event Viewer I get an Application
Error with the following details:

Faulting application name: python.exe, version: 0.0.0.0, time stamp: 0x4ba3e443
Faulting module name: MSVCR90D.dll, version: 9.0.30729.4148, time
stamp: 0x4a594c7b
Exception code: 0x80000003
Fault offset: 0x0003c355
Faulting process id: 0x2724
Faulting application start time: 0x01cc19fa9c5ac48d
Faulting application path: C:\dev\bin\Python26\python.exe
Faulting module path:
C:\WINDOWS\WinSxS\x86_microsoft.vc90.debugcrt_1fc8b3b9a1e18e3b_9.0.30729.4148_none_2a4cbfc25558bcd3\MSVCR90D.dll
Report Id: df3ef44d-85ed-11e0-8d2a-d48564bd9523


-Dave
-------------- next part --------------
A non-text attachment was scrubbed...
Name: call_stack.png
Type: image/png
Size: 30004 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110524/e34be96d/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: terminal_run_under_debug.png
Type: image/png
Size: 49142 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110524/e34be96d/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: call_stack2.png
Type: image/png
Size: 26969 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110524/e34be96d/attachment-0002.png>

From takowl at gmail.com  Tue May 24 06:18:58 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 24 May 2011 11:18:58 +0100
Subject: [IPython-dev] UI related: TermKit (a webkit-based terminal)
In-Reply-To: <20110524082046.GE2795@ykcyc>
References: <20110524082046.GE2795@ykcyc>
Message-ID: <BANLkTinSWniB6tezNKeD885mYVmqTj8VhQ@mail.gmail.com>

On 24 May 2011 09:20, Paul Ivanov <pivanov314 at gmail.com> wrote:

> Hey gang,
>
> thought I'd point you to this project, which looks like it has a
> bit of overlap and capitalizes on some of the capabilities
> we now have with the new Qt console stuff in ipython trunk:
>

I've left a comment on his blog about what we're doing, as well. He's
certainly had similar ideas to those behind the new interactive IPython
frontends. I like the idea.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110524/52bdb65b/attachment.html>

From andresete.chaos at gmail.com  Tue May 24 22:00:43 2011
From: andresete.chaos at gmail.com (=?UTF-8?Q?Omar_Andr=C3=A9s_Zapata_Mesa?=)
Date: Tue, 24 May 2011 21:00:43 -0500
Subject: [IPython-dev] Zmq Terminal.
Message-ID: <BANLkTin0hygka63OyA6C+O3cMq1qJFQCKw@mail.gmail.com>

Hi all.
Thanks Thomas. Sorry by my english!! I am studying and I undertake to
improve.
I am working in implemend Crt+C but I am having a rare problem.
Explain.
I am trying to capture the  exception when raw_input is getting the code,
see code
       try:
           print()
           self._splitter.push(raw_input(' In[%i]:
'%self.prompt_count+self.code))
           while self._splitter.push_accepts_more():
              self.code = raw_input(' .....: '+'
'*self._splitter.indent_spaces)
              self._splitter.push('
'*self._splitter.indent_spaces+self.code)
           self._execute(self._splitter.source,False)
           self._splitter.reset()
       except KeyboardInterrupt:
           print('\nKeyboardInterrupt\n')
           pass

but it produce an error in the ipkernel module

In[1]:
KeyboardInterrupt


 In[1]:
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
/home/omazapa/MyProjects/GSoC/ipython/IPython/frontend/zmqterminal/<string>
in <module>()

/home/omazapa/MyProjects/GSoC/ipython/IPython/zmq/ipkernel.pyc in main()
    678                                 shell=kernel.shell)
    679
--> 680     start_kernel(namespace, kernel)
    681
    682

/home/omazapa/MyProjects/GSoC/ipython/IPython/zmq/entry_point.pyc in
start_kernel(namespace=Namespace(colors=None, hb=39548,
ip='127.0.0.1',...ue, pub=34056, pylab=None, req=58834, xrep=59134),
kernel=<IPython.zmq.ipkernel.Kernel object>)
    147
    148     # Start the kernel mainloop.
--> 149     kernel.start()
    150
    151

/home/omazapa/MyProjects/GSoC/ipython/IPython/zmq/ipkernel.pyc in
start(self=<IPython.zmq.ipkernel.Kernel object>)
    168         """
    169         while True:
--> 170             time.sleep(self._poll_interval)
        global stdin = undefined
        global stdout = undefined
        global stderr = undefined
        global optional = undefined
        global default = undefined
        global None = undefined
    171             self.do_one_iteration()
    172

KeyboardInterrupt:


then the connection is lost.

the idea is capture the exception  KeyboardInterrupt in two differents times
1) in raw_input
2) when sub socket is is printing outputs to send signal SIGINT using kernel
manager's method self.km.interrupt_kernel()

The capture of second option is working but the first not for these unkown
error

you can see the code applying the attached diff file to apply in frontend.py
file

I prefer dont push this little change yet to the repo, before get some
opinion.


Very thanks Thomas You're a big help.



-- 
Omar Andres Zapata Mesa
User Linux  #490962
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110524/8759c1f3/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: frontend.diff
Type: text/x-patch
Size: 1022 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110524/8759c1f3/attachment.bin>

From takowl at gmail.com  Wed May 25 06:29:41 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 25 May 2011 11:29:41 +0100
Subject: [IPython-dev] Zmq Terminal.
In-Reply-To: <BANLkTin0hygka63OyA6C+O3cMq1qJFQCKw@mail.gmail.com>
References: <BANLkTin0hygka63OyA6C+O3cMq1qJFQCKw@mail.gmail.com>
Message-ID: <BANLkTikzgZ9axQv-ONbEsyGBqWDwc2P_Mw@mail.gmail.com>

2011/5/25 Omar Andr?s Zapata Mesa <andresete.chaos at gmail.com>

> but it produce an error in the ipkernel module


I think the problem is that both the frontend and the kernel get to see the
Ctrl-C, so there's a KeyboardInterrupt raised in each. I don't know much
about how processes fit together, but a bit of fiddling suggests that
setting independent to True when you start the kernel (add it to the kwargs
dict on line 242) seems to work. I think that might have other effects,
though - reading the code, it looks like the kernel will live on if the
frontend crashes, which may not be what you want.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/4433a09b/attachment.html>

From takowl at gmail.com  Wed May 25 10:45:35 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 25 May 2011 15:45:35 +0100
Subject: [IPython-dev] [IPython-User] ipython (0.10.1) -wthread and
	-pylab bug in ubuntu natty
In-Reply-To: <201105251553.22500.vincefn@users.sourceforge.net>
References: <201105251553.22500.vincefn@users.sourceforge.net>
Message-ID: <BANLkTinBUo2QYFJLbim=8neaS9u3YX-Dqg@mail.gmail.com>

That reminds me - if I select the GTK backend in matplotlibrc, I can
replicate the blocking behaviour in trunk as well, with:

x = randn(1000)
hist(x)   # No window appears
show()   # Blocks even after closing the window

Is this some quirk of my setup, or can anyone else replicate that? This is
matplotlib 0.99.3.

Thomas

On 25 May 2011 14:53, Vincent Favre-Nicolin
<vincefn at users.sourceforge.net>wrote:

> Apologies for cross-posting - if you are not using ubuntu, don't read
> ahead.
>
>  If you are using ipython combined with '-wthread' (for mayavi's mlab) or
> '-pylab' (for matplotlib), there is currently a bug in Ubunty 11.04 (natty)
> with ipython 0.10.1, which prevents the correct thread handling:
>  e.g. with mayavi's mlab, it is necessary to use 'show()' to interact with
> the window, which is not very practical (and is blocking). And for
> '-pylab',
> the GTK and wx (Aggg) backend do not work...
>
>  The bug is known and already corrected in ipython 0.10.2:
> https://github.com/ipython/ipython/issues/185
>
>  So the remaining issue is to get the new version as an update in ubuntu
> 11.04/natty !
> => this is why I'm writing this email, so that other people concerned can
> vote
> for the bug and, if maintainers read this, if they can push this as stable-
> release-update (SRU)...
>
>  The bug to vote for is @:
> https://bugs.launchpad.net/bugs/777420
>
>  Incidentally, it is easy to install ipython 0.10.2 'manually', as it is
> already available for ubuntu oneiric (11.10):
>
> wget http://launchpadlibrarian.net/70857487/ipython_0.10.2-1_all.deb
> sudo dpkg -i ipython_0.10.2-1_all.deb
>
>
>                Vincent
> --
> Vincent Favre-Nicolin
>
> CEA / INAC                              http://inac.cea.fr
> Universit? Joseph Fourier        http://www.ujf-grenoble.fr
>
> http://vincefn.net
> ObjCryst & Fox : http://objcryst.sourceforge.net
> _______________________________________________
> IPython-User mailing list
> IPython-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/1a76f95e/attachment.html>

From epatters at enthought.com  Wed May 25 11:06:20 2011
From: epatters at enthought.com (Evan Patterson)
Date: Wed, 25 May 2011 10:06:20 -0500
Subject: [IPython-dev] Zmq Terminal.
In-Reply-To: <BANLkTikzgZ9axQv-ONbEsyGBqWDwc2P_Mw@mail.gmail.com>
References: <BANLkTin0hygka63OyA6C+O3cMq1qJFQCKw@mail.gmail.com>
	<BANLkTikzgZ9axQv-ONbEsyGBqWDwc2P_Mw@mail.gmail.com>
Message-ID: <E1A2F8FB-F70F-4D20-85F4-82B2A98EBD62@enthought.com>


On May 25, 2011, at 5:29 AM, Thomas Kluyver wrote:

> 2011/5/25 Omar Andr?s Zapata Mesa <andresete.chaos at gmail.com>
> but it produce an error in the ipkernel module
> 
> I think the problem is that both the frontend and the kernel get to see the Ctrl-C, so there's a KeyboardInterrupt raised in each. I don't know much about how processes fit together, but a bit of fiddling suggests that setting independent to True when you start the kernel (add it to the kwargs dict on line 242) seems to work. I think that might have other effects, though - reading the code, it looks like the kernel will live on if the frontend crashes, which may not be what you want.

Right, if you set 'independent' the kernel process will survive if the frontend process terminates, which usually isn't what you want.

Evan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/a29eb76e/attachment.html>

From takowl at gmail.com  Wed May 25 11:54:08 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 25 May 2011 16:54:08 +0100
Subject: [IPython-dev] Zmq Terminal.
In-Reply-To: <E1A2F8FB-F70F-4D20-85F4-82B2A98EBD62@enthought.com>
References: <BANLkTin0hygka63OyA6C+O3cMq1qJFQCKw@mail.gmail.com>
	<BANLkTikzgZ9axQv-ONbEsyGBqWDwc2P_Mw@mail.gmail.com>
	<E1A2F8FB-F70F-4D20-85F4-82B2A98EBD62@enthought.com>
Message-ID: <BANLkTi=DJoJ1A=Xkv9Yu1E2CJ_b7kgS=hw@mail.gmail.com>

2011/5/25 Evan Patterson <epatters at enthought.com>

> Right, if you set 'independent' the kernel process will survive if the
> frontend process terminates, which usually isn't what you want.


I thought as much. I *think* the critical bit is calling os.setsid() as we
launch the kernel. If we do that as well as specifying --parent, will
ParentPollerUnix still be able to kill the kernel when the frontend dies? A
quick test suggests that it does, but I don't know much about these things.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/96e71174/attachment.html>

From epatters at enthought.com  Wed May 25 12:20:45 2011
From: epatters at enthought.com (Evan Patterson)
Date: Wed, 25 May 2011 11:20:45 -0500
Subject: [IPython-dev] Zmq Terminal.
In-Reply-To: <BANLkTi=DJoJ1A=Xkv9Yu1E2CJ_b7kgS=hw@mail.gmail.com>
References: <BANLkTin0hygka63OyA6C+O3cMq1qJFQCKw@mail.gmail.com>
	<BANLkTikzgZ9axQv-ONbEsyGBqWDwc2P_Mw@mail.gmail.com>
	<E1A2F8FB-F70F-4D20-85F4-82B2A98EBD62@enthought.com>
	<BANLkTi=DJoJ1A=Xkv9Yu1E2CJ_b7kgS=hw@mail.gmail.com>
Message-ID: <EEB1E5E5-D2B5-47F6-8CC9-157FB65E6838@enthought.com>


On May 25, 2011, at 10:54 AM, Thomas Kluyver wrote:

> 2011/5/25 Evan Patterson <epatters at enthought.com>
> Right, if you set 'independent' the kernel process will survive if the frontend process terminates, which usually isn't what you want.
> 
> I thought as much. I *think* the critical bit is calling os.setsid() as we launch the kernel. If we do that as well as specifying --parent, will ParentPollerUnix still be able to kill the kernel when the frontend dies? A quick test suggests that it does, but I don't know much about these things.

I think that ParentPollerUnix would still be able to kill the kernel reliably, but I'm hardly an expert on the subtleties of Unix process management.

I am wondering why this should be necessary at all, though. Shouldn't the kernel process always trap Ctrl+C signals at some point to prevent them from hosing the kernel? I've actually had a few problems with this myself. Sometimes when I interrupt the kernel from the Qt console, the whole kernel goes down.

Evan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/4aa874d0/attachment.html>

From takowl at gmail.com  Wed May 25 12:47:25 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 25 May 2011 17:47:25 +0100
Subject: [IPython-dev] Zmq Terminal.
In-Reply-To: <EEB1E5E5-D2B5-47F6-8CC9-157FB65E6838@enthought.com>
References: <BANLkTin0hygka63OyA6C+O3cMq1qJFQCKw@mail.gmail.com>
	<BANLkTikzgZ9axQv-ONbEsyGBqWDwc2P_Mw@mail.gmail.com>
	<E1A2F8FB-F70F-4D20-85F4-82B2A98EBD62@enthought.com>
	<BANLkTi=DJoJ1A=Xkv9Yu1E2CJ_b7kgS=hw@mail.gmail.com>
	<EEB1E5E5-D2B5-47F6-8CC9-157FB65E6838@enthought.com>
Message-ID: <BANLkTimaceyaX_tGnbk4R2dxbb2gGRVwGw@mail.gmail.com>

On 25 May 2011 17:20, Evan Patterson <epatters at enthought.com> wrote:

> I am wondering why this should be necessary at all, though. Shouldn't the
> kernel process always trap Ctrl+C signals at some point to prevent them from
> hosing the kernel? I've actually had a few problems with this myself.
> Sometimes when I interrupt the kernel from the Qt console, the whole kernel
> goes down.


It's when the frontend is accepting input - so Ctrl-C should just give you a
clean prompt, without sending anything to the kernel. We could catch the
exception in the kernel and ignore, but it doesn't feel like the right way
to do it. I'd prefer to arrange it so that the signal from the user hitting
Ctrl-C doesn't reach the kernel at all.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/894bd56b/attachment.html>

From pivanov314 at gmail.com  Wed May 25 13:02:29 2011
From: pivanov314 at gmail.com (Paul Ivanov)
Date: Wed, 25 May 2011 10:02:29 -0700
Subject: [IPython-dev] [IPython-User] ipython (0.10.1) -wthread and
 -pylab bug in ubuntu natty
In-Reply-To: <BANLkTinBUo2QYFJLbim=8neaS9u3YX-Dqg@mail.gmail.com>
References: <201105251553.22500.vincefn@users.sourceforge.net>
	<BANLkTinBUo2QYFJLbim=8neaS9u3YX-Dqg@mail.gmail.com>
Message-ID: <20110525170229.GM2795@ykcyc>


Hi Thomas, 

Thomas Kluyver, on 2011-05-25 15:45,  wrote:
> That reminds me - if I select the GTK backend in matplotlibrc, I can
> replicate the blocking behaviour in trunk as well, with:
> 
> x = randn(1000)
> hist(x)   # No window appears
> show()   # Blocks even after closing the window
> 
> Is this some quirk of my setup, or can anyone else replicate that? This is
> matplotlib 0.99.3.

That's a pretty old version of matplotlib for someone running
IPython trunk ;)

It works for me on ~month old mpl trunk, and ipython trunk.

I remember noting this kind of behavior when testing and fixing
the GTK blocking behavior that ended up in the 0.10.2 release.
IIRC, it occurred when you would try to use one backend for
matplotlib, but another is implied by the parameters IPython was
invoked with. In a comment on ipython/issues/185, I wrote

    ...users will see this issue if they use e.g "ipython -gthread"
    but their matplotlib is set to some other backend by default
    (say "WX").

> On 25 May 2011 14:53, Vincent Favre-Nicolin
> <vincefn at users.sourceforge.net> wrote:
> >  If you are using ipython combined with '-wthread' (for mayavi's mlab) or
> > '-pylab' (for matplotlib), there is currently a bug in Ubunty 11.04 (natty)
> > with ipython 0.10.1, which prevents the correct thread handling:
> >  e.g. with mayavi's mlab, it is necessary to use 'show()' to interact with
> > the window, which is not very practical (and is blocking). And for
> > '-pylab',
> > the GTK and wx (Aggg) backend do not work...

In the example Vincent provided above, if you invoked ipython
0.10.2 with -wthread, but have matplotlib use GTK, I believe you
will still see the blocking issue. (at least for a similar
vintage matplotlib, it might not be an issue for ~month old mpl
trunk, at least I couldn't replicate it there briefly just now).

best,
-- 
Paul Ivanov
314 address only used for lists,  off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/a40a910b/attachment.sig>

From epatters at enthought.com  Wed May 25 13:07:01 2011
From: epatters at enthought.com (Evan Patterson)
Date: Wed, 25 May 2011 12:07:01 -0500
Subject: [IPython-dev] Zmq Terminal.
In-Reply-To: <BANLkTimaceyaX_tGnbk4R2dxbb2gGRVwGw@mail.gmail.com>
References: <BANLkTin0hygka63OyA6C+O3cMq1qJFQCKw@mail.gmail.com>
	<BANLkTikzgZ9axQv-ONbEsyGBqWDwc2P_Mw@mail.gmail.com>
	<E1A2F8FB-F70F-4D20-85F4-82B2A98EBD62@enthought.com>
	<BANLkTi=DJoJ1A=Xkv9Yu1E2CJ_b7kgS=hw@mail.gmail.com>
	<EEB1E5E5-D2B5-47F6-8CC9-157FB65E6838@enthought.com>
	<BANLkTimaceyaX_tGnbk4R2dxbb2gGRVwGw@mail.gmail.com>
Message-ID: <E72CB4A1-BE71-4CF8-97F2-5794C3125AEB@enthought.com>


On May 25, 2011, at 11:47 AM, Thomas Kluyver wrote:

> On 25 May 2011 17:20, Evan Patterson <epatters at enthought.com> wrote:
> I am wondering why this should be necessary at all, though. Shouldn't the kernel process always trap Ctrl+C signals at some point to prevent them from hosing the kernel? I've actually had a few problems with this myself. Sometimes when I interrupt the kernel from the Qt console, the whole kernel goes down.
> 
> It's when the frontend is accepting input - so Ctrl-C should just give you a clean prompt, without sending anything to the kernel. We could catch the exception in the kernel and ignore, but it doesn't feel like the right way to do it. I'd prefer to arrange it so that the signal from the user hitting Ctrl-C doesn't reach the kernel at all.

The way interrupts work in the Qt console (in Unix) is precisely that, though: the frontend process sends a SIGINT signal to the kernel. The expectation is that the kernel will handle the signal in a reasonable way. We could try to devise some other scheme, like I did for Windows, but it would be difficult to get right. The kernel should just do the right thing.

Evan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/cffeec64/attachment.html>

From takowl at gmail.com  Wed May 25 13:30:43 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 25 May 2011 18:30:43 +0100
Subject: [IPython-dev] [IPython-User] ipython (0.10.1) -wthread and
 -pylab bug in ubuntu natty
In-Reply-To: <20110525170229.GM2795@ykcyc>
References: <201105251553.22500.vincefn@users.sourceforge.net>
	<BANLkTinBUo2QYFJLbim=8neaS9u3YX-Dqg@mail.gmail.com>
	<20110525170229.GM2795@ykcyc>
Message-ID: <BANLkTimipAJpdHcBkFNbdyRiSKqr9SBMRA@mail.gmail.com>

On 25 May 2011 18:02, Paul Ivanov <pivanov314 at gmail.com> wrote:

> That's a pretty old version of matplotlib for someone running
> IPython trunk ;)
>

I'm not a heavy matplotlib user, in fact ;). I tried easy_installing a newer
version in the virtualenv, but it wasn't compiling for some reason. I'll try
it again.


>
> It works for me on ~month old mpl trunk, and ipython trunk.
>
> I remember noting this kind of behavior when testing and fixing
> the GTK blocking behavior that ended up in the 0.10.2 release.
> IIRC, it occurred when you would try to use one backend for
> matplotlib, but another is implied by the parameters IPython was
> invoked with. In a comment on ipython/issues/185, I wrote
>
>    ...users will see this issue if they use e.g "ipython -gthread"
>    but their matplotlib is set to some other backend by default
>    (say "WX").
>

This was just with "ipython -pylab", so I assume it should be able to sort
out getting the correct backend.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/2807913e/attachment.html>

From epatters at enthought.com  Wed May 25 14:05:34 2011
From: epatters at enthought.com (Evan Patterson)
Date: Wed, 25 May 2011 13:05:34 -0500
Subject: [IPython-dev] Zmq Terminal.
In-Reply-To: <BANLkTinR-q6XprcnKPansW0i67tmO4bwXA@mail.gmail.com>
References: <BANLkTin0hygka63OyA6C+O3cMq1qJFQCKw@mail.gmail.com>
	<BANLkTikzgZ9axQv-ONbEsyGBqWDwc2P_Mw@mail.gmail.com>
	<E1A2F8FB-F70F-4D20-85F4-82B2A98EBD62@enthought.com>
	<BANLkTi=DJoJ1A=Xkv9Yu1E2CJ_b7kgS=hw@mail.gmail.com>
	<EEB1E5E5-D2B5-47F6-8CC9-157FB65E6838@enthought.com>
	<BANLkTimaceyaX_tGnbk4R2dxbb2gGRVwGw@mail.gmail.com>
	<E72CB4A1-BE71-4CF8-97F2-5794C3125AEB@enthought.com>
	<BANLkTinR-q6XprcnKPansW0i67tmO4bwXA@mail.gmail.com>
Message-ID: <558D4A18-DADA-4B78-A52D-5AC0608D36A1@enthought.com>

On May 25, 2011, at 12:24 PM, Thomas Kluyver wrote:

> On 25 May 2011 18:07, Evan Patterson <epatters at enthought.com> wrote:
> The way interrupts work in the Qt console (in Unix) is precisely that, though: the frontend process sends a SIGINT signal to the kernel. The expectation is that the kernel will handle the signal in a reasonable way. We could try to devise some other scheme, like I did for Windows, but it would be difficult to get right. The kernel should just do the right thing.
> 
> That only applies while something is being executed, though (looking at the code, it checks self._executing). We can do the equivalent and call interrupt_kernel from the terminal frontend. The problem is that the kernel gets the SIGINT straight from the user, wherever it is in the code. In particular, if the user presses Ctrl-C at an input prompt, that shows up while the kernel is idling. We could catch it, but I feel that the right way to do it is to insulate the kernel from the user's keystrokes.

But this is not robust. Frontends use SIGINT signals to tell to the kernel that it should it interrupt its execution. There is no guarantee that they will be polite about it, i.e. only send SIGINT when the kernel is actually executing. In fact, in the general case (where multiple frontends are connected to the same kernel), there is no way to even know whether the kernel is executing.

I think it is good principle to ensure that the kernel is as robust as possible to misbehaving clients. This means that it should be prepared to handle SIGINT at any time.

Evan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/da14bdd1/attachment.html>

From takowl at gmail.com  Wed May 25 14:06:57 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 25 May 2011 19:06:57 +0100
Subject: [IPython-dev] [IPython-User] ipython (0.10.1) -wthread and
 -pylab bug in ubuntu natty
In-Reply-To: <BANLkTimipAJpdHcBkFNbdyRiSKqr9SBMRA@mail.gmail.com>
References: <201105251553.22500.vincefn@users.sourceforge.net>
	<BANLkTinBUo2QYFJLbim=8neaS9u3YX-Dqg@mail.gmail.com>
	<20110525170229.GM2795@ykcyc>
	<BANLkTimipAJpdHcBkFNbdyRiSKqr9SBMRA@mail.gmail.com>
Message-ID: <BANLkTimVrmms=S0z=cgCLqwkvqVcUChDmA@mail.gmail.com>

On 25 May 2011 18:30, Thomas Kluyver <takowl at gmail.com> wrote:

> I'm not a heavy matplotlib user, in fact ;). I tried easy_installing a
> newer version in the virtualenv, but it wasn't compiling for some reason.
> I'll try it again.


OK, I've got matplotlib 1.0.1 installed. Now if I specify backend : GTK, no
window appears, even when I call show(). It works as expected with GTKAgg,
but not with GTK.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/5bea85d8/attachment.html>

From epatters at enthought.com  Wed May 25 14:20:56 2011
From: epatters at enthought.com (Evan Patterson)
Date: Wed, 25 May 2011 13:20:56 -0500
Subject: [IPython-dev] Zmq Terminal.
In-Reply-To: <BANLkTi=JFunM1i3f3Le6f-TM4C-iARyMQQ@mail.gmail.com>
References: <BANLkTin0hygka63OyA6C+O3cMq1qJFQCKw@mail.gmail.com>
	<BANLkTikzgZ9axQv-ONbEsyGBqWDwc2P_Mw@mail.gmail.com>
	<E1A2F8FB-F70F-4D20-85F4-82B2A98EBD62@enthought.com>
	<BANLkTi=DJoJ1A=Xkv9Yu1E2CJ_b7kgS=hw@mail.gmail.com>
	<EEB1E5E5-D2B5-47F6-8CC9-157FB65E6838@enthought.com>
	<BANLkTimaceyaX_tGnbk4R2dxbb2gGRVwGw@mail.gmail.com>
	<E72CB4A1-BE71-4CF8-97F2-5794C3125AEB@enthought.com>
	<BANLkTinR-q6XprcnKPansW0i67tmO4bwXA@mail.gmail.com>
	<558D4A18-DADA-4B78-A52D-5AC0608D36A1@enthought.com>
	<BANLkTi=JFunM1i3f3Le6f-TM4C-iARyMQQ@mail.gmail.com>
Message-ID: <127B4FC6-B3C2-4A06-B5C2-FDBD03599BFF@enthought.com>

On May 25, 2011, at 1:12 PM, Thomas Kluyver wrote:

> On 25 May 2011 19:05, Evan Patterson <epatters at enthought.com> wrote:
> I think it is good principle to ensure that the kernel is as robust as possible to misbehaving clients. This means that it should be prepared to handle SIGINT at any time.
> 
> That makes sense, although I still think that we should also insulate the kernel from the user when the terminal contains the frontend too.

Now that I can get behind. The kernel shouldn't be exposed to random junk from the frontend terminal. os.setpid() may be the solution for that, but it's probably worth doing a little more investigation.

> 
> Shall I add a try/except KeyboardInterrupt inside the loop in IPython.zmq.ipkernel.Kernel.start()?

Yep, I think that's where it makes sense to have it.

Thanks,
Evan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/bc9d801a/attachment.html>

From pivanov314 at gmail.com  Wed May 25 14:41:28 2011
From: pivanov314 at gmail.com (Paul Ivanov)
Date: Wed, 25 May 2011 11:41:28 -0700
Subject: [IPython-dev] [IPython-User] ipython (0.10.1) -wthread and
 -pylab bug in ubuntu natty
In-Reply-To: <BANLkTimVrmms=S0z=cgCLqwkvqVcUChDmA@mail.gmail.com>
References: <201105251553.22500.vincefn@users.sourceforge.net>
	<BANLkTinBUo2QYFJLbim=8neaS9u3YX-Dqg@mail.gmail.com>
	<20110525170229.GM2795@ykcyc>
	<BANLkTimipAJpdHcBkFNbdyRiSKqr9SBMRA@mail.gmail.com>
	<BANLkTimVrmms=S0z=cgCLqwkvqVcUChDmA@mail.gmail.com>
Message-ID: <20110525184128.GO2795@ykcyc>

Thomas Kluyver, on 2011-05-25 19:06,  wrote:
> On 25 May 2011 18:30, Thomas Kluyver <takowl at gmail.com> wrote:
> 
> > I'm not a heavy matplotlib user, in fact ;). I tried easy_installing a
> > newer version in the virtualenv, but it wasn't compiling for some reason.
> > I'll try it again.
> 
> 
> OK, I've got matplotlib 1.0.1 installed. Now if I specify backend : GTK, no
> window appears, even when I call show(). It works as expected with GTKAgg,
> but not with GTK.

odd. I get the same with -pylab, but not without it. Actually --
plt.figure() does make a window come up, but it doesn't render
anything on there.

With plain GTK backend, I get a window after show (I started with
just -gthread), but the call to show blocks. Hitting ctrl-c after that
returns me to the ipython prompt, and the figure appears to still
update when I resize it, so I'm not sure why the call to show
blocked, given that the threading seems to work.

Actually, the above behavior is what I see even if I just launch
ipython without -gthread -- it even works in just vanilla python
shell, so it must be using the internal matplotlib threading
code.

-- 
Paul Ivanov
314 address only used for lists,  off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/3be3a646/attachment.sig>

From takowl at gmail.com  Wed May 25 14:42:02 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 25 May 2011 19:42:02 +0100
Subject: [IPython-dev] Zmq Terminal.
In-Reply-To: <127B4FC6-B3C2-4A06-B5C2-FDBD03599BFF@enthought.com>
References: <BANLkTin0hygka63OyA6C+O3cMq1qJFQCKw@mail.gmail.com>
	<BANLkTikzgZ9axQv-ONbEsyGBqWDwc2P_Mw@mail.gmail.com>
	<E1A2F8FB-F70F-4D20-85F4-82B2A98EBD62@enthought.com>
	<BANLkTi=DJoJ1A=Xkv9Yu1E2CJ_b7kgS=hw@mail.gmail.com>
	<EEB1E5E5-D2B5-47F6-8CC9-157FB65E6838@enthought.com>
	<BANLkTimaceyaX_tGnbk4R2dxbb2gGRVwGw@mail.gmail.com>
	<E72CB4A1-BE71-4CF8-97F2-5794C3125AEB@enthought.com>
	<BANLkTinR-q6XprcnKPansW0i67tmO4bwXA@mail.gmail.com>
	<558D4A18-DADA-4B78-A52D-5AC0608D36A1@enthought.com>
	<BANLkTi=JFunM1i3f3Le6f-TM4C-iARyMQQ@mail.gmail.com>
	<127B4FC6-B3C2-4A06-B5C2-FDBD03599BFF@enthought.com>
Message-ID: <BANLkTi=SYHCdsKYtaHpqfAZa9HaagwrDDg@mail.gmail.com>

On 25 May 2011 19:20, Evan Patterson <epatters at enthought.com> wrote:

> Shall I add a try/except KeyboardInterrupt inside the loop in
> IPython.zmq.ipkernel.Kernel.start()?
>
>
> Yep, I think that's where it makes sense to have it.
>

Ok, great. Pull request:

https://github.com/ipython/ipython/pull/472

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/025f78ae/attachment.html>

From takowl at gmail.com  Wed May 25 14:47:51 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 25 May 2011 19:47:51 +0100
Subject: [IPython-dev] [IPython-User] ipython (0.10.1) -wthread and
 -pylab bug in ubuntu natty
In-Reply-To: <20110525184128.GO2795@ykcyc>
References: <201105251553.22500.vincefn@users.sourceforge.net>
	<BANLkTinBUo2QYFJLbim=8neaS9u3YX-Dqg@mail.gmail.com>
	<20110525170229.GM2795@ykcyc>
	<BANLkTimipAJpdHcBkFNbdyRiSKqr9SBMRA@mail.gmail.com>
	<BANLkTimVrmms=S0z=cgCLqwkvqVcUChDmA@mail.gmail.com>
	<20110525184128.GO2795@ykcyc>
Message-ID: <BANLkTi=uxiAqGpEWDSbNDmp9qWN5XNnA9Q@mail.gmail.com>

On 25 May 2011 19:41, Paul Ivanov <pivanov314 at gmail.com> wrote:

> Actually, the above behavior is what I see even if I just launch
> ipython without -gthread -- it even works in just vanilla python
> shell, so it must be using the internal matplotlib threading
> code.
>

This matches what I see. Are IPython and matplotlib both trying to do clever
things with threads, and tripping over each others toes?

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110525/d01585d8/attachment.html>

From fperez.net at gmail.com  Wed May 25 15:23:00 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 25 May 2011 12:23:00 -0700
Subject: [IPython-dev] control is not returned to main program,
	when plotting
In-Reply-To: <20110114100508.GK13983@ykcyc>
References: <20110114094843.1b1361db@moon> <20110114100508.GK13983@ykcyc>
Message-ID: <BANLkTin8dyAm5yu_wChzVk4QMWCr5cYPCQ@mail.gmail.com>

On Fri, Jan 14, 2011 at 2:05 AM, Paul Ivanov <pivanov314 at gmail.com> wrote:
> Hi Thomas,
>
> this was originally reported in issue 185, and I submitted a pull
> request on Dec 28th [**cough** last year! ;) ] to fix
> this, coming to the same conclusion as you, which Mark Voorhies
> graciously tested and verified that it resolves the issue.
>
> https://github.com/ipython/ipython/pull/237
>
> In the pull request message, I came to the same conclusion as
> you:
>
> ?What isn't clear to me is the purpose of the function
> ?check_gtk inside Shell.py, which was added in the problematic
> ?commit. For some reason, it unconditionally returns mode
> ?'tkthread' if gtk was safely imported and hasattr
> ?'set_interactive', regardless of what mode was passed to it.
>
> Your patch is a subset of the lines to be removed by the pull
> request. I additionally got rid of the entire check_gtk function
> as it was no longer used anywhere.
>
> We just need someone with ipython privileges to put pull it into
> the 0.10.2 branch. Anyone lurking out there? :)

Just to note for the record that this did get merged in to 0.10.2
before release, so the problem is fixed in the released version.

Thanks to those who did the actual work!

Cheers,

f


From efiring at hawaii.edu  Wed May 25 15:42:28 2011
From: efiring at hawaii.edu (Eric Firing)
Date: Wed, 25 May 2011 09:42:28 -1000
Subject: [IPython-dev] [IPython-User] ipython (0.10.1) -wthread and
 -pylab bug in ubuntu natty
In-Reply-To: <BANLkTi=uxiAqGpEWDSbNDmp9qWN5XNnA9Q@mail.gmail.com>
References: <201105251553.22500.vincefn@users.sourceforge.net>
	<BANLkTinBUo2QYFJLbim=8neaS9u3YX-Dqg@mail.gmail.com>
	<20110525170229.GM2795@ykcyc>
	<BANLkTimipAJpdHcBkFNbdyRiSKqr9SBMRA@mail.gmail.com>
	<BANLkTimVrmms=S0z=cgCLqwkvqVcUChDmA@mail.gmail.com>
	<20110525184128.GO2795@ykcyc>
	<BANLkTi=uxiAqGpEWDSbNDmp9qWN5XNnA9Q@mail.gmail.com>
Message-ID: <4DDD5BA4.1020804@hawaii.edu>

On 05/25/2011 08:47 AM, Thomas Kluyver wrote:
> On 25 May 2011 19:41, Paul Ivanov <pivanov314 at gmail.com
> <mailto:pivanov314 at gmail.com>> wrote:
>
>     Actually, the above behavior is what I see even if I just launch
>     ipython without -gthread -- it even works in just vanilla python
>     shell, so it must be using the internal matplotlib threading
>     code.
>
>
> This matches what I see. Are IPython and matplotlib both trying to do
> clever things with threads, and tripping over each others toes?

No, mpl does not use threads.  Ipython versions <= 0.10.x uses threads 
for gui interaction.  The present development version does not, and is 
considered much more robust as a consequence.

Eric

>
> Thomas


From fperez.net at gmail.com  Wed May 25 17:13:03 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 25 May 2011 14:13:03 -0700
Subject: [IPython-dev] UI related: TermKit (a webkit-based terminal)
In-Reply-To: <BANLkTinSWniB6tezNKeD885mYVmqTj8VhQ@mail.gmail.com>
References: <20110524082046.GE2795@ykcyc>
	<BANLkTinSWniB6tezNKeD885mYVmqTj8VhQ@mail.gmail.com>
Message-ID: <BANLkTikOGpyohy3ro7ZP1Diwh8Jh+Epi7w@mail.gmail.com>

On Tue, May 24, 2011 at 3:18 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> I've left a comment on his blog about what we're doing, as well. He's
> certainly had similar ideas to those behind the new interactive IPython
> frontends. I like the idea.

Yup, I'd seen it recently too.  There's a ton of similarity in the
underlying approach, at the end of the day our display protocol and
what he's doing are almost the same thing.  His demos look pretty
cool, not in that I would want an everyday shell to work like that
precisely, but in highlighting what can be done with the tools.

Cheers,

f


From fperez.net at gmail.com  Wed May 25 21:24:26 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 25 May 2011 18:24:26 -0700
Subject: [IPython-dev] [IPython-User] ipython (0.10.1) -wthread and
 -pylab bug in ubuntu natty
In-Reply-To: <BANLkTimVrmms=S0z=cgCLqwkvqVcUChDmA@mail.gmail.com>
References: <201105251553.22500.vincefn@users.sourceforge.net>
	<BANLkTinBUo2QYFJLbim=8neaS9u3YX-Dqg@mail.gmail.com>
	<20110525170229.GM2795@ykcyc>
	<BANLkTimipAJpdHcBkFNbdyRiSKqr9SBMRA@mail.gmail.com>
	<BANLkTimVrmms=S0z=cgCLqwkvqVcUChDmA@mail.gmail.com>
Message-ID: <BANLkTimfDjei58MxwjFdO5F6g6tKfs6sFA@mail.gmail.com>

On Wed, May 25, 2011 at 11:06 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> OK, I've got matplotlib 1.0.1 installed. Now if I specify backend : GTK, no
> window appears, even when I call show(). It works as expected with GTKAgg,
> but not with GTK.
>
> Thomas

OK, that was just a plain bug, fixed here:

https://github.com/ipython/ipython/pull/474

None of us uses plain gtk, so we'd simply missed the fact that it had
never worked in 0.11 :)

Cheers,

f


From fperez.net at gmail.com  Wed May 25 23:26:39 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 25 May 2011 20:26:39 -0700
Subject: [IPython-dev] tiny patch for sphinx version change
In-Reply-To: <4DD83366.6000808@hawaii.edu>
References: <4DD83366.6000808@hawaii.edu>
Message-ID: <BANLkTi=wh=Y_6+AdsYZvpQ-u-Pep9yfVBQ@mail.gmail.com>

On Sat, May 21, 2011 at 2:49 PM, Eric Firing <efiring at hawaii.edu> wrote:
> I updated to ipython master. ?When attempting to build the docs I ran into a
> problem associated with changes in Sphinx between version 0.6.7 and 1.x.
> ?The attached patch allows the build to proceed with the latter and should
> still work with the former, although I have not tested that.

Thanks!  Committed (with due authorship):

https://github.com/ipython/ipython/commit/54f4afc320a579317694177f8ac0ef877876c333

Cheers,

f


From takowl at gmail.com  Thu May 26 08:56:04 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 26 May 2011 13:56:04 +0100
Subject: [IPython-dev] %edit magic in new frontends
Message-ID: <BANLkTi=nLzrbQvCUCGcv6HOANJpjBRqY3A@mail.gmail.com>

Bringing this discussion onto the list, it started on a pull request here:
https://github.com/ipython/ipython/pull/460#issuecomment-1214412

In the plain terminal frontend, calling %edit brings up an editor, typically
something inside the terminal like nano or vi, and blocks until the editor
closes. Then, depending on how it was called, it can execute the code that's
just been edited, redefine a macro if you were editing one, or take no
action. In addition, it may spit out the edited code as a string so you can
reuse it.

In the Qt console, calling %edit will normally open the target in a GUI
editor.* Editors that can open multiple documents may not block the call,
although apparently some of them have command-line flags for that behaviour.
So as it stands, %edit in the Qt console will show you a file you can change
and save, but won't take any further action on it. Then again, since the Qt
console has decent editing capabilities itself, you can use the %recall and
%loadpy macros to bring code to the next prompt, and edit it there before
running it.

* I say 'will' because in fact it currently does nothing by default, but
that should be sorted soon.

We should also consider how we will want %edit to work in other situations,
including the HTML notebook which is being prepared, and IPython as an
embedded widget in other software.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110526/84ae2029/attachment.html>

From fperez.net at gmail.com  Thu May 26 17:04:39 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 26 May 2011 14:04:39 -0700
Subject: [IPython-dev] %edit magic in new frontends
In-Reply-To: <BANLkTi=nLzrbQvCUCGcv6HOANJpjBRqY3A@mail.gmail.com>
References: <BANLkTi=nLzrbQvCUCGcv6HOANJpjBRqY3A@mail.gmail.com>
Message-ID: <BANLkTinB1q+UWY4Ask87jjC+oDBbBToaTg@mail.gmail.com>

On Thu, May 26, 2011 at 5:56 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> In the Qt console, calling %edit will normally open the target in a GUI
> editor.* Editors that can open multiple documents may not block the call,
> although apparently some of them have command-line flags for that behaviour.
> So as it stands, %edit in the Qt console will show you a file you can change
> and save, but won't take any further action on it. Then again, since the Qt
> console has decent editing capabilities itself, you can use the %recall and
> %loadpy macros to bring code to the next prompt, and edit it there before
> running it.

It seems to me that %edit should, in the gui console, by default use
the -x flag always.  This way it's useful as a mechanism to quickly
load the thing you want to work on, but you're still responsible for
saving and doing %run yourself.

That doesn't bother me too much, because I think that with the
multiline capabilities of the Qt console, many of the use cases of
%edit are now subsumed by simply recalling a previous cell with
up-arrow and editing it right there.  And once the amount of code in
the cell becomes large, it's probably best to just keep a persistent
editor open using the classic 'save, alt-tab, run' cycle.

> We should also consider how we will want %edit to work in other situations,
> including the HTML notebook which is being prepared, and IPython as an
> embedded widget in other software.

Applications that embed the console can always override %edit to
synchronize with their own mechanisms; Evan, does that sound
reasonable for your use cases?

As for the html notebook, it would be really cool if edit would open
the file in-browser, either using a full-tab Codemirror cell or
something like ymacs (http://www.ymacs.org/).  But the architectural
decisions for magics with client-specific behavior are a bit tricky,
since all code is actually executed in-kernel, and the same kernel can
be simultaneously serving more than one client...

Cheers,

f


From takowl at gmail.com  Thu May 26 17:14:30 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 26 May 2011 22:14:30 +0100
Subject: [IPython-dev] %edit magic in new frontends
In-Reply-To: <BANLkTinB1q+UWY4Ask87jjC+oDBbBToaTg@mail.gmail.com>
References: <BANLkTi=nLzrbQvCUCGcv6HOANJpjBRqY3A@mail.gmail.com>
	<BANLkTinB1q+UWY4Ask87jjC+oDBbBToaTg@mail.gmail.com>
Message-ID: <BANLkTinh6ei2oO8aNvX0+CKVOK0tCq+X0w@mail.gmail.com>

On 26 May 2011 22:04, Fernando Perez <fperez.net at gmail.com> wrote:

> That doesn't bother me too much, because I think that with the
> multiline capabilities of the Qt console, many of the use cases of
> %edit are now subsumed by simply recalling a previous cell with
> up-arrow and editing it right there.  And once the amount of code in
> the cell becomes large, it's probably best to just keep a persistent
> editor open using the classic 'save, alt-tab, run' cycle.
>

The one thing I don't think you can easily do is editing a macro and have it
redefined from the edited code. You'd have to do '%recall macro', edit, run,
'%macro 52'. But I guess if you've got a better multi-line cell environment,
macros are less important.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110526/d4316419/attachment.html>

From fperez.net at gmail.com  Thu May 26 17:40:09 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 26 May 2011 14:40:09 -0700
Subject: [IPython-dev] %edit magic in new frontends
In-Reply-To: <BANLkTinh6ei2oO8aNvX0+CKVOK0tCq+X0w@mail.gmail.com>
References: <BANLkTi=nLzrbQvCUCGcv6HOANJpjBRqY3A@mail.gmail.com>
	<BANLkTinB1q+UWY4Ask87jjC+oDBbBToaTg@mail.gmail.com>
	<BANLkTinh6ei2oO8aNvX0+CKVOK0tCq+X0w@mail.gmail.com>
Message-ID: <BANLkTi=HPEEAtViPZ+TEwBDvZq72vTuDRw@mail.gmail.com>

On Thu, May 26, 2011 at 2:14 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> The one thing I don't think you can easily do is editing a macro and have it
> redefined from the edited code. You'd have to do '%recall macro', edit, run,
> '%macro 52'. But I guess if you've got a better multi-line cell environment,
> macros are less important.

Yes.  I realize macros get trickier here, but when I first wrote the
macro system, I thought of it as a poor man's solution to ease some of
the pain of not having decent multiline editing.  So I think for now
it's not the end of the world if "%edit macro" isn't great in the Qt
console, since I imagine it will be so rarely (if at all) used.

And as you indicated, %recall provides pretty much that functionality
in-console (since I don't expect anyone to be writing 100-line
macros), so I think we're Ok there.

Cheers,

f


From epatters at enthought.com  Thu May 26 20:08:09 2011
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 26 May 2011 19:08:09 -0500
Subject: [IPython-dev] %edit magic in new frontends
In-Reply-To: <BANLkTinB1q+UWY4Ask87jjC+oDBbBToaTg@mail.gmail.com>
References: <BANLkTi=nLzrbQvCUCGcv6HOANJpjBRqY3A@mail.gmail.com>
	<BANLkTinB1q+UWY4Ask87jjC+oDBbBToaTg@mail.gmail.com>
Message-ID: <BANLkTi=CzQ0-w8QYH_UeGxzBR70C+1XU1A@mail.gmail.com>

On Thu, May 26, 2011 at 4:04 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> On Thu, May 26, 2011 at 5:56 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> > In the Qt console, calling %edit will normally open the target in a GUI
> > editor.* Editors that can open multiple documents may not block the call,
> > although apparently some of them have command-line flags for that
> behaviour.
> > So as it stands, %edit in the Qt console will show you a file you can
> change
> > and save, but won't take any further action on it. Then again, since the
> Qt
> > console has decent editing capabilities itself, you can use the %recall
> and
> > %loadpy macros to bring code to the next prompt, and edit it there before
> > running it.
>
> It seems to me that %edit should, in the gui console, by default use
> the -x flag always.  This way it's useful as a mechanism to quickly
> load the thing you want to work on, but you're still responsible for
> saving and doing %run yourself.
>
> That doesn't bother me too much, because I think that with the
> multiline capabilities of the Qt console, many of the use cases of
> %edit are now subsumed by simply recalling a previous cell with
> up-arrow and editing it right there.  And once the amount of code in
> the cell becomes large, it's probably best to just keep a persistent
> editor open using the classic 'save, alt-tab, run' cycle.
>
> > We should also consider how we will want %edit to work in other
> situations,
> > including the HTML notebook which is being prepared, and IPython as an
> > embedded widget in other software.
>
> Applications that embed the console can always override %edit to
> synchronize with their own mechanisms; Evan, does that sound
> reasonable for your use cases?
>
> As for the html notebook, it would be really cool if edit would open
> the file in-browser, either using a full-tab Codemirror cell or
> something like ymacs (http://www.ymacs.org/).  But the architectural
> decisions for magics with client-specific behavior are a bit tricky,
> since all code is actually executed in-kernel, and the same kernel can
> be simultaneously serving more than one client...
>
>
Yep, as long as %edit is still programmatically over-ridable, it's all good
to me.

BTW, the link to ymacs was neat. I had no idea there was such a thing.

Evan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110526/a7482518/attachment.html>

From fperez.net at gmail.com  Sat May 28 17:18:45 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 28 May 2011 14:18:45 -0700
Subject: [IPython-dev] ipython in a background thread
In-Reply-To: <840FA957-B8F0-481D-8BC1-7B26921D90B1@yale.edu>
References: <840FA957-B8F0-481D-8BC1-7B26921D90B1@yale.edu>
Message-ID: <BANLkTi=jFjLnhXZGYnzXSaz_V-Pq8gHd7w@mail.gmail.com>

Hi Zach,

On Fri, Apr 29, 2011 at 10:03 AM, Zachary Pincus
<zachary.pincus at yale.edu> wrote:
> I find that ipython, specifically ipython running in a good terminal (as opposed to embedded in a GUI window), provides a really great interface (e.g. in my case, a microscope and camera hardware). Given this use, it's been important to also have a GUI thread running so I can throw acquired images rapidly up onto a GL canvas or something. For a while, I've used a hacked-up pyglet runloop that would work in a background thread, and set up a simple message-passing system.
>
> This got quite kluged quite fast though, and it turns out this approach won't work at all with cocoa GUIs on OS X, which seem to require running on thread-0 (a pity). I know people on this list have looked and thought about these issues a lot, so I'd be curious what seems like the best approach in general? (There used to be some code in IPython for doing this with various window toolkits, but I don't see that in the 0.10.2's codebase anymore...)
>
> - I could run the GUI from thread-0 and try message-passing to IPython on another thread. Does this work well at all, or is it a huge kluge to get all the readline etc. features working right?
>

That's what we had always in the 0.10.x series, and it's horribly
brittle.  In master/0.11, we have compelely restructured how GUI event
loop integration is handled, to work instead with PyOSInputHook and
tie directly into the event loop of each toolkit.  There's already
support for Qt, WX and GTK (Tk works automatically because Python does
the same internally).  It should be possible to do something similar
for OpenGL/pyglet, I think.

Here are some pointers in the documentation:

http://ipython.github.com/ipython-doc/dev/interactive/reference.html#gui-event-loop-support-support

The implementation machinery lives here:

https://github.com/ipython/ipython/blob/master/IPython/lib/inputhook.py
https://github.com/ipython/ipython/blob/master/IPython/lib/inputhookwx.py
https://github.com/ipython/ipython/blob/master/IPython/lib/inputhookgtk.py

And these are the key parts of the code that use it:

./frontend/terminal/interactiveshell.py:
   27 : from IPython.lib.inputhook import enable_gui
./frontend/terminal/ipapp.py:
   39 : from IPython.lib import inputhook
  513 :             gui = inputhook.GUI_WX
  515 :             gui = inputhook.GUI_QT
  517 :             gui = inputhook.GUI_GTK
  531 :             activate = inputhook.enable_gui

It would be great to have similar GL integration as we have for the
others, so please let us know if any of this makes sense, and if you
make any progress we'd love to integrate it.

> - I could run the GUI in a separate process entirely, which would force a much cleaner API, but I'm not sure if pumping images at video-rate could be done cleanly. Maybe with shared memmapped arrays? (I've seen some recipes for this sort of thing on the numpy list lately.)
>

If the client is only going to consume the data read-only (or writing
to it doesn't matter anymore for the sender), then you may consider
our two-process architecture, used by the Qt console.  It will require
some more design, that's true, but you'll have something very robust
and flexible in general, with possibilities for multi-client
monitoring, remote monitoring, etc.  The pyzmq code we use has the
ability to do zero-copy sends of numpy arrays and is quite fast, so it
might be possible to obtain video frame rates.

Something like that would be a very cool application of this
architecture, let us know if you decide to go down that road and we
might be able to provide some advice.

Best,

f


From fperez.net at gmail.com  Sat May 28 21:18:19 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 28 May 2011 18:18:19 -0700
Subject: [IPython-dev] Continued pain with gui integration,
	Traits/Qt-specific, help welcome
Message-ID: <BANLkTi=bQi9YRBPFOqMgQKehr2=zimQZhQ@mail.gmail.com>

Hi folks,

I directly pinged Robert and Evan on the bug, as they are the most
likely to have the necessary expertise, but anyone willing to pitch in
on what can be going on with Ctrl-C and Traits/Qt code, as explained
here:

https://github.com/ipython/ipython/issues/8#issuecomment-1255696

would do us a great favor.  I have most of what's needed to close that
bug in its original form, but this crash that I only noticed today
while doing more interactive testing is truly nasty, since it affects
--as best I can tell-- *any* Traits/Qt code.

That means that as it stands today (at least on my system), ipython
0.11 is useless for using any Enthought/Traits app with the Qt backend
(but OK with the WX one).

Thanks for any help/info/etc.  In particular, if anyone can find the
examples given in the comment linked above work OK for them (meaning,
Ctrl-C does *not* crash ipython for them), then details on your
platform, Qt version and Enthought tools version/installation origin
(EPD, distro, pythonxy, manual build, etc) would be enormously
helpful.

Thanks!

f


From fperez.net at gmail.com  Sun May 29 02:41:38 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 28 May 2011 23:41:38 -0700
Subject: [IPython-dev] IPython 0.11@SciPy2011 (i.e. July)
Message-ID: <BANLkTik3-QGaYWVWuM=DCh08t+fxQZRohg@mail.gmail.com>

Hi all,

[ sorry for the -user crosspost, but we're looking at the homestretch
of a major release so the feedback of our more adventurous users would
be vastly welcome ]

I know we've talked about it before, but now it does finally need to
happen.  We actually need to make a release, and not of the 0.10.x
codebase :)

We have both a talk and a tutorial scheduled at SciPy'11 in mid-July
around IPython, so we should really show up with a fresh-off-the-oven
0.11 with all the goodies we've been working on lately.

The last few months have seen a massive amount of activity, and I
actually think we're in pretty decent shape overall.  This is a view
of the issues tagged for 0.11 with critical priority, which I think is
realistically all we can consider tackling:

https://github.com/ipython/ipython/issues?labels=prio-critical&sort=created&direction=desc&state=open&page=1&milestone=5

Things don't look too bad, I think:

#8: it has a nasty side, but I just begged for help earlier, and
hopefully we'll figure out what's going on with Traits/Qt.  The fact
that matplotlib works perfectly with the Qt backend gives me hope
there's a way out of this one.

#66: doc updating, no problem.

#79/175: it's a lot of work, but much progress has been made recently,
so I think we'll be fine.

#340: Brian, what's your thought on this one?

#351/369: I think these are almost the same issue.  Annoying, but doable.

#378: doesn't look insurmountable.

Honestly the only thing I'm really worried about is today's discovery
of the disaster we have with Traits/Qt in #8, I think the rest is
either not too bad or major (all the config work) but well tackled at
this point.  The Qt one worries me because it's a bit of a wildcard, I
simply have no clue what's going on, despite spending most of the day
on it, reading the Qt mailing lists, stracing processes, etc.  But
perhaps (hopefully) someone more knowledgeable than me in Qt matters
can help us there, and it won't turn out to be such a big deal.

If anyone sees any issue that you think should be promoted to
0.11/critical, please say so on this thread and we'll look into it.
We can't guarantee that we'll promote them to that status (we can't
delay the release forever, and there will be other releases), but we
will certainly have a look.

At this point, anyone who can help:

- testing the git master branch in your normal usage patterns, and
reporting any problems you encounter, from simple documentation
confusions to things that don't work.
- tackling any of the above bugs with a pull request
- helping review any of our open pull requests

will earn our undying gratitude.  When we're a little closer to
release we'll put test eggs/release candidates, but even if you aren't
using git, github offers nicely updated single-click downloads as
zip/tarballs without needing git on your part, at:

https://github.com/ipython/ipython/archives/master

Anything I've missed?  Since SciPy is mid-July, I'd like to shoot for
the actual release to happen early July, so people have a chance to
download/install it before the conference.  It will give us a chance
to get useful feedback there, from which we can choose how to focus
our energy at the sprint in Austin.

Best,

f


From takowl at gmail.com  Mon May 30 09:30:19 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 30 May 2011 14:30:19 +0100
Subject: [IPython-dev] Website
Message-ID: <BANLkTi=x5Qu+9w2mAhNy-_EuzPU7CNefEg@mail.gmail.com>

Hi all,

I think IPython could probably use a dedicated homepage. At the moment,
we've got the wiki (http://ipython.scipy.org/moin/) and the documentation
site (http://ipython.github.com/ipython-doc/), but I envisage something a
bit more polished as a first port of call. It would probably just have a
brief description of IPython and a set of links (downloads, documentation,
development, and so on). I suggest we use github pages to host it at
ipython.github.org.

I'll have a shot at this, but I'm not particularly a designer, so if anyone
else reckons they could do a good job of it, I'm happy to stand aside. Also,
if there's any particular theme or style we try to maintain that I should
incorporate, let me know.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110530/7cbb9750/attachment.html>

From zachary.pincus at yale.edu  Mon May 30 15:46:05 2011
From: zachary.pincus at yale.edu (Zachary Pincus)
Date: Mon, 30 May 2011 15:46:05 -0400
Subject: [IPython-dev] ipython in a background thread
In-Reply-To: <BANLkTi=jFjLnhXZGYnzXSaz_V-Pq8gHd7w@mail.gmail.com>
References: <840FA957-B8F0-481D-8BC1-7B26921D90B1@yale.edu>
	<BANLkTi=jFjLnhXZGYnzXSaz_V-Pq8gHd7w@mail.gmail.com>
Message-ID: <78AE0AB6-3ADB-4993-AFFF-05E632558ADC@yale.edu>

Thanks for the pointers into the code, Fernando! I saw that there were a lot of changes in 0.11, but I wasn't sure where to go to see what the new mechanics were. Most obliged.

Zach


On May 28, 2011, at 5:18 PM, Fernando Perez wrote:

> Hi Zach,
> 
> On Fri, Apr 29, 2011 at 10:03 AM, Zachary Pincus
> <zachary.pincus at yale.edu> wrote:
>> I find that ipython, specifically ipython running in a good terminal (as opposed to embedded in a GUI window), provides a really great interface (e.g. in my case, a microscope and camera hardware). Given this use, it's been important to also have a GUI thread running so I can throw acquired images rapidly up onto a GL canvas or something. For a while, I've used a hacked-up pyglet runloop that would work in a background thread, and set up a simple message-passing system.
>> 
>> This got quite kluged quite fast though, and it turns out this approach won't work at all with cocoa GUIs on OS X, which seem to require running on thread-0 (a pity). I know people on this list have looked and thought about these issues a lot, so I'd be curious what seems like the best approach in general? (There used to be some code in IPython for doing this with various window toolkits, but I don't see that in the 0.10.2's codebase anymore...)
>> 
>> - I could run the GUI from thread-0 and try message-passing to IPython on another thread. Does this work well at all, or is it a huge kluge to get all the readline etc. features working right?
>> 
> 
> That's what we had always in the 0.10.x series, and it's horribly
> brittle.  In master/0.11, we have compelely restructured how GUI event
> loop integration is handled, to work instead with PyOSInputHook and
> tie directly into the event loop of each toolkit.  There's already
> support for Qt, WX and GTK (Tk works automatically because Python does
> the same internally).  It should be possible to do something similar
> for OpenGL/pyglet, I think.
> 
> Here are some pointers in the documentation:
> 
> http://ipython.github.com/ipython-doc/dev/interactive/reference.html#gui-event-loop-support-support
> 
> The implementation machinery lives here:
> 
> https://github.com/ipython/ipython/blob/master/IPython/lib/inputhook.py
> https://github.com/ipython/ipython/blob/master/IPython/lib/inputhookwx.py
> https://github.com/ipython/ipython/blob/master/IPython/lib/inputhookgtk.py
> 
> And these are the key parts of the code that use it:
> 
> ./frontend/terminal/interactiveshell.py:
>   27 : from IPython.lib.inputhook import enable_gui
> ./frontend/terminal/ipapp.py:
>   39 : from IPython.lib import inputhook
>  513 :             gui = inputhook.GUI_WX
>  515 :             gui = inputhook.GUI_QT
>  517 :             gui = inputhook.GUI_GTK
>  531 :             activate = inputhook.enable_gui
> 
> It would be great to have similar GL integration as we have for the
> others, so please let us know if any of this makes sense, and if you
> make any progress we'd love to integrate it.
> 
>> - I could run the GUI in a separate process entirely, which would force a much cleaner API, but I'm not sure if pumping images at video-rate could be done cleanly. Maybe with shared memmapped arrays? (I've seen some recipes for this sort of thing on the numpy list lately.)
>> 
> 
> If the client is only going to consume the data read-only (or writing
> to it doesn't matter anymore for the sender), then you may consider
> our two-process architecture, used by the Qt console.  It will require
> some more design, that's true, but you'll have something very robust
> and flexible in general, with possibilities for multi-client
> monitoring, remote monitoring, etc.  The pyzmq code we use has the
> ability to do zero-copy sends of numpy arrays and is quite fast, so it
> might be possible to obtain video frame rates.
> 
> Something like that would be a very cool application of this
> architecture, let us know if you decide to go down that road and we
> might be able to provide some advice.
> 
> Best,
> 
> f



From jason-sage at creativetrax.com  Tue May 31 13:42:49 2011
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 31 May 2011 12:42:49 -0500
Subject: [IPython-dev] pyzmq authentication
Message-ID: <4DE52899.90402@creativetrax.com>

In the docs for ipython, there are big warnings about the insecurity of 
using ZMQ.  Has anyone thought about adding authentication support to 
the pyzmq bindings, similar to what is done in the multiprocessing 
Connection objects?

http://docs.python.org/dev/library/multiprocessing#authentication-keys

(see also the examples in 
http://docs.python.org/dev/library/multiprocessing#multiprocessing.connection.Listener)

I realize that it is not in the design scope of ZMQ itself to implement 
security, but the python bindings might be the right level for at least 
authentication.

Thanks,

Jason

--
Jason Grout


From benjaminrk at gmail.com  Tue May 31 13:57:33 2011
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 31 May 2011 10:57:33 -0700
Subject: [IPython-dev] pyzmq authentication
In-Reply-To: <4DE52899.90402@creativetrax.com>
References: <4DE52899.90402@creativetrax.com>
Message-ID: <BANLkTi=W6v2HaAGboRFgXwf3KMxrPpFNVw@mail.gmail.com>

We did briefly have an encrypted socket, but the zeromq community
rightly opposed that rather vehemently, largely because we aren't
security experts, and the illusion of security provided by a poor
implementation is really *less* secure than having no security at all.

Our answer with IPython is that SSH provides our security.  Typically
the Controller listens on localhost, and the best way to connect to it
from another machine is with an SSH tunnel (IPython does help create
the tunnels) rather than listening on a public port.  We do provide a
small level of additional security by including an authentication key
in all messages that is checked when receiving to determine if the
sender is authorized to make a request.

That said, the SSH tunnel tools (IPython.external.ssh) don't depend on
anything else in IPython, so we have thought about moving them to
pyzmq itself.

-MinRK

On Tue, May 31, 2011 at 10:42, Jason Grout <jason-sage at creativetrax.com> wrote:
> In the docs for ipython, there are big warnings about the insecurity of
> using ZMQ. ?Has anyone thought about adding authentication support to
> the pyzmq bindings, similar to what is done in the multiprocessing
> Connection objects?
>
> http://docs.python.org/dev/library/multiprocessing#authentication-keys
>
> (see also the examples in
> http://docs.python.org/dev/library/multiprocessing#multiprocessing.connection.Listener)
>
> I realize that it is not in the design scope of ZMQ itself to implement
> security, but the python bindings might be the right level for at least
> authentication.
>
> Thanks,
>
> Jason
>
> --
> Jason Grout
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From jason-sage at creativetrax.com  Tue May 31 14:13:23 2011
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 31 May 2011 13:13:23 -0500
Subject: [IPython-dev] pyzmq authentication
In-Reply-To: <BANLkTi=W6v2HaAGboRFgXwf3KMxrPpFNVw@mail.gmail.com>
References: <4DE52899.90402@creativetrax.com>
	<BANLkTi=W6v2HaAGboRFgXwf3KMxrPpFNVw@mail.gmail.com>
Message-ID: <4DE52FC3.1080408@creativetrax.com>

On 5/31/11 12:57 PM, MinRK wrote:
> We did briefly have an encrypted socket, but the zeromq community
> rightly opposed that rather vehemently, largely because we aren't
> security experts, and the illusion of security provided by a poor
> implementation is really *less* secure than having no security at all.
>
> Our answer with IPython is that SSH provides our security.  Typically
> the Controller listens on localhost, and the best way to connect to it
> from another machine is with an SSH tunnel (IPython does help create
> the tunnels) rather than listening on a public port.  We do provide a
> small level of additional security by including an authentication key
> in all messages that is checked when receiving to determine if the
> sender is authorized to make a request.

If I understand things correctly, if I have several frontends running 
code on a single backend server (with multiple kernels---the sage 
notebook is my usecase), then untrusted code from any of the kernels 
could connect to and mess with the other sessions, right?  Is it correct 
to say that any user could connect with any kernel running on the same 
server?

Thanks,

Jason


From benjaminrk at gmail.com  Tue May 31 14:45:33 2011
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 31 May 2011 11:45:33 -0700
Subject: [IPython-dev] pyzmq authentication
In-Reply-To: <4DE52FC3.1080408@creativetrax.com>
References: <4DE52899.90402@creativetrax.com>
	<BANLkTi=W6v2HaAGboRFgXwf3KMxrPpFNVw@mail.gmail.com>
	<4DE52FC3.1080408@creativetrax.com>
Message-ID: <BANLkTi=nXFDBrpQMC=9LYW6bfdVuWccKeg@mail.gmail.com>

On Tue, May 31, 2011 at 11:13, Jason Grout <jason-sage at creativetrax.com> wrote:
> On 5/31/11 12:57 PM, MinRK wrote:
>>
>> We did briefly have an encrypted socket, but the zeromq community
>> rightly opposed that rather vehemently, largely because we aren't
>> security experts, and the illusion of security provided by a poor
>> implementation is really *less* secure than having no security at all.
>>
>> Our answer with IPython is that SSH provides our security. ?Typically
>> the Controller listens on localhost, and the best way to connect to it
>> from another machine is with an SSH tunnel (IPython does help create
>> the tunnels) rather than listening on a public port. ?We do provide a
>> small level of additional security by including an authentication key
>> in all messages that is checked when receiving to determine if the
>> sender is authorized to make a request.
>
> If I understand things correctly, if I have several frontends running code
> on a single backend server (with multiple kernels---the sage notebook is my
> usecase), then untrusted code from any of the kernels could connect to and
> mess with the other sessions, right? ?Is it correct to say that any user
> could connect with any kernel running on the same server?

Oh, you are talking about the *non* parallel kernel.  Yes, that code
has exactly zero security - anyone with access to the sockets can
execute arbitrary code.  We really do need to replace
IPython.zmq.session with the one in the parallel code which does
include simple key checking, which should be per-kernel (or
per-cluster in the parallel code).

-MinRK

>
> Thanks,
>
> Jason
>


From takowl at gmail.com  Tue May 31 16:37:25 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 31 May 2011 21:37:25 +0100
Subject: [IPython-dev] Website
In-Reply-To: <BANLkTi=x5Qu+9w2mAhNy-_EuzPU7CNefEg@mail.gmail.com>
References: <BANLkTi=x5Qu+9w2mAhNy-_EuzPU7CNefEg@mail.gmail.com>
Message-ID: <BANLkTi=UD4rmL6v77+0RAj-6JnMr5VDGog@mail.gmail.com>

On 30 May 2011 14:30, Thomas Kluyver <takowl at gmail.com> wrote:

> I think IPython could probably use a dedicated homepage. At the moment,
> we've got the wiki (http://ipython.scipy.org/moin/) and the documentation
> site (http://ipython.github.com/ipython-doc/), but I envisage something a
> bit more polished as a first port of call. It would probably just have a
> brief description of IPython and a set of links (downloads, documentation,
> development, and so on). I suggest we use github pages to host it at
> ipython.github.org.
>
> I'll have a shot at this, but I'm not particularly a designer, so if anyone
> else reckons they could do a good job of it, I'm happy to stand aside. Also,
> if there's any particular theme or style we try to maintain that I should
> incorporate, let me know.
>

This (attached) is what I have at present. The main focus is on the
navigation box, which is supposed to be an easy route in to the main IPython
resources.

One thing I'm not sure about is whether to have a 'more links' section at
the bottom, for things like the talks & presentations section, or the list
of projects using IPython. Do we want to promote these from the homepage, or
is a link from the docs front page sufficient? If we do want to promote
them, I think we should make an effort to keep them up to date.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110531/408ff4a7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipython-website.zip
Type: application/zip
Size: 246254 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110531/408ff4a7/attachment.zip>

From ellisonbg at gmail.com  Tue May 31 23:07:15 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 31 May 2011 20:07:15 -0700
Subject: [IPython-dev] pyzmq authentication
In-Reply-To: <BANLkTi=nXFDBrpQMC=9LYW6bfdVuWccKeg@mail.gmail.com>
References: <4DE52899.90402@creativetrax.com>
	<BANLkTi=W6v2HaAGboRFgXwf3KMxrPpFNVw@mail.gmail.com>
	<4DE52FC3.1080408@creativetrax.com>
	<BANLkTi=nXFDBrpQMC=9LYW6bfdVuWccKeg@mail.gmail.com>
Message-ID: <BANLkTim-OChy-V+6A8Bqg6Co_p+PLjDD1A@mail.gmail.com>

Jason,

The other thing to consider is that the main frontend that will need
security is the html notebook.  For that, we do plan on having HTTPS
enabled by default.

Cheers,

Brian

On Tue, May 31, 2011 at 11:45 AM, MinRK <benjaminrk at gmail.com> wrote:
> On Tue, May 31, 2011 at 11:13, Jason Grout <jason-sage at creativetrax.com> wrote:
>> On 5/31/11 12:57 PM, MinRK wrote:
>>>
>>> We did briefly have an encrypted socket, but the zeromq community
>>> rightly opposed that rather vehemently, largely because we aren't
>>> security experts, and the illusion of security provided by a poor
>>> implementation is really *less* secure than having no security at all.
>>>
>>> Our answer with IPython is that SSH provides our security. ?Typically
>>> the Controller listens on localhost, and the best way to connect to it
>>> from another machine is with an SSH tunnel (IPython does help create
>>> the tunnels) rather than listening on a public port. ?We do provide a
>>> small level of additional security by including an authentication key
>>> in all messages that is checked when receiving to determine if the
>>> sender is authorized to make a request.
>>
>> If I understand things correctly, if I have several frontends running code
>> on a single backend server (with multiple kernels---the sage notebook is my
>> usecase), then untrusted code from any of the kernels could connect to and
>> mess with the other sessions, right? ?Is it correct to say that any user
>> could connect with any kernel running on the same server?
>
> Oh, you are talking about the *non* parallel kernel. ?Yes, that code
> has exactly zero security - anyone with access to the sockets can
> execute arbitrary code. ?We really do need to replace
> IPython.zmq.session with the one in the parallel code which does
> include simple key checking, which should be per-kernel (or
> per-cluster in the parallel code).
>
> -MinRK
>
>>
>> Thanks,
>>
>> Jason
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Tue May 31 23:33:01 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 31 May 2011 20:33:01 -0700
Subject: [IPython-dev] Website
In-Reply-To: <BANLkTi=x5Qu+9w2mAhNy-_EuzPU7CNefEg@mail.gmail.com>
References: <BANLkTi=x5Qu+9w2mAhNy-_EuzPU7CNefEg@mail.gmail.com>
Message-ID: <BANLkTimjar0=s6EfZ8eY6KrStC5PfVeRFQ@mail.gmail.com>

Thomas,

Yes, the website definitely needs a refresh.

Here are somethings we have talked about the following in the past:

* Moving away from ipython.scipy.org to simply ipython.org
* Using sphinx for the entire site as matplotlib does.
* Moving the wiki stuff over to github.

Cheers,

Brian

On Mon, May 30, 2011 at 6:30 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> Hi all,
>
> I think IPython could probably use a dedicated homepage. At the moment,
> we've got the wiki (http://ipython.scipy.org/moin/) and the documentation
> site (http://ipython.github.com/ipython-doc/), but I envisage something a
> bit more polished as a first port of call. It would probably just have a
> brief description of IPython and a set of links (downloads, documentation,
> development, and so on). I suggest we use github pages to host it at
> ipython.github.org.
>
> I'll have a shot at this, but I'm not particularly a designer, so if anyone
> else reckons they could do a good job of it, I'm happy to stand aside. Also,
> if there's any particular theme or style we try to maintain that I should
> incorporate, let me know.
>
> Thanks,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Tue May 31 23:37:57 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 31 May 2011 20:37:57 -0700
Subject: [IPython-dev] %edit magic in new frontends
In-Reply-To: <BANLkTinB1q+UWY4Ask87jjC+oDBbBToaTg@mail.gmail.com>
References: <BANLkTi=nLzrbQvCUCGcv6HOANJpjBRqY3A@mail.gmail.com>
	<BANLkTinB1q+UWY4Ask87jjC+oDBbBToaTg@mail.gmail.com>
Message-ID: <BANLkTinZZfUQbP==w2J-fWS1aWzM1t1m4w@mail.gmail.com>

I apologize that I can't participate more fully in this discussion
right now (last week of classes + finals).  A few thoughts though:

* The edit logic must be 100% frontend.  Otherwise different frontends
connected to the same kernel will get confused.
* I vote that the -x flag be removed completely.  Edit should just edit.

Cheers,

Brian

On Thu, May 26, 2011 at 2:04 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Thu, May 26, 2011 at 5:56 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>> In the Qt console, calling %edit will normally open the target in a GUI
>> editor.* Editors that can open multiple documents may not block the call,
>> although apparently some of them have command-line flags for that behaviour.
>> So as it stands, %edit in the Qt console will show you a file you can change
>> and save, but won't take any further action on it. Then again, since the Qt
>> console has decent editing capabilities itself, you can use the %recall and
>> %loadpy macros to bring code to the next prompt, and edit it there before
>> running it.
>
> It seems to me that %edit should, in the gui console, by default use
> the -x flag always. ?This way it's useful as a mechanism to quickly
> load the thing you want to work on, but you're still responsible for
> saving and doing %run yourself.
>
> That doesn't bother me too much, because I think that with the
> multiline capabilities of the Qt console, many of the use cases of
> %edit are now subsumed by simply recalling a previous cell with
> up-arrow and editing it right there. ?And once the amount of code in
> the cell becomes large, it's probably best to just keep a persistent
> editor open using the classic 'save, alt-tab, run' cycle.
>
>> We should also consider how we will want %edit to work in other situations,
>> including the HTML notebook which is being prepared, and IPython as an
>> embedded widget in other software.
>
> Applications that embed the console can always override %edit to
> synchronize with their own mechanisms; Evan, does that sound
> reasonable for your use cases?
>
> As for the html notebook, it would be really cool if edit would open
> the file in-browser, either using a full-tab Codemirror cell or
> something like ymacs (http://www.ymacs.org/). ?But the architectural
> decisions for magics with client-specific behavior are a bit tricky,
> since all code is actually executed in-kernel, and the same kernel can
> be simultaneously serving more than one client...
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Tue May 31 23:39:22 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 31 May 2011 20:39:22 -0700
Subject: [IPython-dev] %edit magic in new frontends
In-Reply-To: <BANLkTinB1q+UWY4Ask87jjC+oDBbBToaTg@mail.gmail.com>
References: <BANLkTi=nLzrbQvCUCGcv6HOANJpjBRqY3A@mail.gmail.com>
	<BANLkTinB1q+UWY4Ask87jjC+oDBbBToaTg@mail.gmail.com>
Message-ID: <BANLkTimk55Aid=O8PW2bs6S3+m8ov9oVUw@mail.gmail.com>

On Thu, May 26, 2011 at 2:04 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Thu, May 26, 2011 at 5:56 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>> In the Qt console, calling %edit will normally open the target in a GUI
>> editor.* Editors that can open multiple documents may not block the call,
>> although apparently some of them have command-line flags for that behaviour.
>> So as it stands, %edit in the Qt console will show you a file you can change
>> and save, but won't take any further action on it. Then again, since the Qt
>> console has decent editing capabilities itself, you can use the %recall and
>> %loadpy macros to bring code to the next prompt, and edit it there before
>> running it.
>
> It seems to me that %edit should, in the gui console, by default use
> the -x flag always. ?This way it's useful as a mechanism to quickly
> load the thing you want to work on, but you're still responsible for
> saving and doing %run yourself.

+1

I would go a step further and say -x should be i) removed entirely or
ii) made the default in all frontends.

> That doesn't bother me too much, because I think that with the
> multiline capabilities of the Qt console, many of the use cases of
> %edit are now subsumed by simply recalling a previous cell with
> up-arrow and editing it right there. ?And once the amount of code in
> the cell becomes large, it's probably best to just keep a persistent
> editor open using the classic 'save, alt-tab, run' cycle.
>
>> We should also consider how we will want %edit to work in other situations,
>> including the HTML notebook which is being prepared, and IPython as an
>> embedded widget in other software.
>
> Applications that embed the console can always override %edit to
> synchronize with their own mechanisms; Evan, does that sound
> reasonable for your use cases?
>
> As for the html notebook, it would be really cool if edit would open
> the file in-browser, either using a full-tab Codemirror cell or
> something like ymacs (http://www.ymacs.org/). ?But the architectural
> decisions for magics with client-specific behavior are a bit tricky,
> since all code is actually executed in-kernel, and the same kernel can
> be simultaneously serving more than one client...
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com