From rgbkrk at gmail.com  Fri Aug  1 00:23:18 2014
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Thu, 31 Jul 2014 23:23:18 -0500
Subject: [IPython-dev] CVE Request: Enforce use of HTTPS for MathJax in
	IPython
Message-ID: <CA+tbMaX-6SHyNHB9GFQuoo7zL8bXCzycY83=WN65JKzpyZw9jw@mail.gmail.com>

All,

We would like to request a CVE for a vulnerability in the IPython notebook,
reported today by Leopold Schabel on IPython's GitHub issue tracker at
https://github.com/ipython/ipython/issues/6246.

Email address of requester: security at ipython.org; rgbkrk at gmail.com
Software name: IPython notebook
Type of vulnerability: Use of insecure resources
Attack outcome: Remote execution
Patch/issue: https://github.com/ipython/ipython/pull/6249,
https://github.com/ipython/ipython/issues/6246
Affected versions: 0.12 ? version ? 2.1

Summary: When using the IPython notebook without encryption (i.e. running
the server on HTTP instead of HTTPS), mathjax is loaded over HTTP. An
attacker with fortuitous network position could execute code on a local
IPython notebook by modifying the mathjax javascript.

This issue was fixed in the git master branch (development branch for
upcoming v. 2.2) with commit cf793ebc4, on 7/31/2014:

https://github.com/ipython/ipython/commit/cf793ebc4f9e8483f104667e4c73748357fa8c56

Mitigations:
* Run the notebook with SSL (see
http://ipython.org/ipython-doc/2/notebook/public_server.html#securing-a-notebook-server
).
* Install mathjax
    from IPython.external.mathjax import install_mathjax
    install_mathjax()

Regards,

Kyle Kelley
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140731/b60923ec/attachment.html>

From rgbkrk at gmail.com  Fri Aug  1 16:57:10 2014
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Fri, 1 Aug 2014 15:57:10 -0500
Subject: [IPython-dev] Insecure loading of mathjax
Message-ID: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>

Hey all,

As reported in https://github.com/ipython/ipython/issues/6246, MathJax will
load over HTTP if using the notebook with an unecncrypted connection (e.g.
http://127.0.0.1:8888). Someone with an appropriate network position (the
router at your local internet cafe for example) could modify the mathjax
javascript before it gets to you, adding their own javascript. This would
result in being able to run code on your IPython kernel with just a little
bit of javascript (`IPython.notebook.kernel.execute(code)`).

This issue was fixed in the git master branch (development branch for
upcoming v. 2.2) with commit cf793ebc4, on 7/31/2014:

https://github.com/ipython/ipython/commit/cf793ebc4f9e8483f104667e4c73748357fa8c56

Mitigations:

* Run the notebook with SSL (see
http://ipython.org/ipython-doc/2/notebook/public_server.html#securing-a-notebook-server).
This will load MathJax over SSL via the MathJax CDN.

* Install mathjax locally

    from IPython.external.mathjax import install_mathjax
    install_mathjax()

We have requested a CVE for this via the oss-security mailing list.

--
Kyle Kelley (@rgbkrk; http://lambdaops.com)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140801/ee662e40/attachment.html>

From jtaylor.debian at googlemail.com  Fri Aug  1 17:13:39 2014
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Fri, 01 Aug 2014 23:13:39 +0200
Subject: [IPython-dev] Insecure loading of mathjax
In-Reply-To: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>
References: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>
Message-ID: <53DC0303.7060500@googlemail.com>

On 01.08.2014 22:57, Kyle Kelley wrote:
> Hey all,
> 
> As reported in https://github.com/ipython/ipython/issues/6246, MathJax
> will load over HTTP if using the notebook with an unecncrypted
> connection (e.g. http://127.0.0.1:8888). Someone with an appropriate
> network position (the router at your local internet cafe for example)
> could modify the mathjax javascript before it gets to you, adding their
> own javascript. This would result in being able to run code on your
> IPython kernel with just a little bit of javascript
> (`IPython.notebook.kernel.execute(code)`).
> 
> This issue was fixed in the git master branch (development branch for
> upcoming v. 2.2) with commit cf793ebc4, on 7/31/2014:
> 
> https://github.com/ipython/ipython/commit/cf793ebc4f9e8483f104667e4c73748357fa8c56
> 

Fwiw, I reported this already back in version 0.12 and the Debian
package has always been enforcing the local mathjax for that reason.

Is the mathjax cdn certificate still a shared between all users of
whatever hosting provide is behind is?
Back then this was the case for the https cdn mathjax used making it
quite pointless as any users of that hosting service (I think it was
amazon) could serve you a forged mathjax via valid https.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140801/e8a3a802/attachment.sig>

From takowl at gmail.com  Fri Aug  1 17:37:12 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 1 Aug 2014 14:37:12 -0700
Subject: [IPython-dev] Insecure loading of mathjax
In-Reply-To: <53DC0303.7060500@googlemail.com>
References: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>
	<53DC0303.7060500@googlemail.com>
Message-ID: <CAOvn4qhmURNa+O_crL1Y0wJaMtUG7j-R6FVen-P6S7COGT9MVQ@mail.gmail.com>

On 1 August 2014 14:13, Julian Taylor <jtaylor.debian at googlemail.com> wrote:

> Is the mathjax cdn certificate still a shared between all users of
> whatever hosting provide is behind is?
> Back then this was the case for the https cdn mathjax used making it
> quite pointless as any users of that hosting service (I think it was
> amazon) could serve you a forged mathjax via valid https.
>

Looking at the certificate details in my browser, it looks like that is
still an issue. It doesn't look like it's shared between all users of the
hosting service - there's a list of 30 or so domains that appear to share
it. I think that means that only someone who controlled one of those
domains could do a MITM attack, so it's a lot more secure than just loading
it over http, but still not properly secure. Gah, what's the point of HTTPS
if it gets used like this...

I am very much not a security expert, so take my assessment with a large
pinch of salt.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140801/b7738506/attachment.html>

From pi at berkeley.edu  Fri Aug  1 17:40:38 2014
From: pi at berkeley.edu (Paul Ivanov)
Date: Fri, 1 Aug 2014 14:40:38 -0700
Subject: [IPython-dev] Insecure loading of mathjax
In-Reply-To: <53DC0303.7060500@googlemail.com>
References: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>
	<53DC0303.7060500@googlemail.com>
Message-ID: <20140801214038.GD27567@HbI-OTOH.berkeley.edu>

Hi Julian, Kyle, and list,

I just wanted to publicly thank Kyle again for following through
with these and ensure that they get reported and communicated in
the right manner. None of the other other IPython developers have
any experience with disclosing security vulnerabilities to
appropriate channels, and Kyle has stepped up entirely in a
volunteer capacity to do this for the benefit of the community.

Thanks to you as well, Julian, for bringing that CDN certificate
issue to our attention. We need all the help we can get, and I
my immediate reaction to reading "...making it quite
pointless..." was that Kyle is getting the stick instead of a
carrot for following through and doing a better job than we would
have done without him (your point about reporting this back in
0.12 is an example of our previous lack of familiarity,
appreciation, and engagement with security related issues). 

If you have the time and interest, We'd love your help on the
security side of things (contact Kyle or me offlist), and I think
Kyle is striving to do a much more punctual disclosure of this
vulnerability in part because of your feedback on CVE-2014-3429.
I just want to make sure that we continue to have productive
interactions.

my sincerest appreciation to both of you,
-- 
                   _
                  / \
                A*   \^   -
             ,./   _.`\\ / \
            / ,--.S    \/   \
           /  `"~,_     \    \
     __o           ?
   _ \<,_         /:\
--(_)/-(_)----.../ | \
--------------.......J
Paul Ivanov
ipython and matplotlib core developer
http://pirsquared.org


From takowl at gmail.com  Fri Aug  1 17:55:09 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 1 Aug 2014 14:55:09 -0700
Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython
	Notebook on your own machine
Message-ID: <CAOvn4qjaZq5w-Y1hV5tJchWTNURcmzBf=V458-meR-fMSyAmkw@mail.gmail.com>

I've been working on a couple of tools to improve the experience of working
with IPython notebooks on your own computer.

First, nbopen, a tool to open a notebook in the nearest available server,
or start a new server if there isn't one available. It now integrates with
your file manager so you can double click on a notebook file to open it
(open source desktops only for now, PRs for other systems welcome):

https://github.com/takluyver/nbopen

However, there's no easy way to shut down notebook servers launched by
double clicking on notebooks, because there isn't a terminal where you can
press ctrl-c. So I also made nbmanager, a GUI app which discovers running
notebook servers and open notebooks, and lets you easily shut them down:

https://github.com/takluyver/nbmanager

nbopen will work on IPython 2, while nbmanager, which is more experimental,
requires you to be running IPython from git at the moment.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140801/d803d4f4/attachment.html>

From fperez.net at gmail.com  Fri Aug  1 17:57:45 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 1 Aug 2014 14:57:45 -0700
Subject: [IPython-dev] Insecure loading of mathjax
In-Reply-To: <20140801214038.GD27567@HbI-OTOH.berkeley.edu>
References: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>
	<53DC0303.7060500@googlemail.com>
	<20140801214038.GD27567@HbI-OTOH.berkeley.edu>
Message-ID: <CAHAreOrZbS8GX-6_a=4CZRcqtznCo6LFFrfweLcaY4Kw4M+JpQ@mail.gmail.com>

I was about to write something along the same lines, but Paul said it much
better than I. +1 to all of that...

Cheers,

f


On Fri, Aug 1, 2014 at 2:40 PM, Paul Ivanov <pi at berkeley.edu> wrote:

> Hi Julian, Kyle, and list,
>
> I just wanted to publicly thank Kyle again for following through
> with these and ensure that they get reported and communicated in
> the right manner. None of the other other IPython developers have
> any experience with disclosing security vulnerabilities to
> appropriate channels, and Kyle has stepped up entirely in a
> volunteer capacity to do this for the benefit of the community.
>
> Thanks to you as well, Julian, for bringing that CDN certificate
> issue to our attention. We need all the help we can get, and I
> my immediate reaction to reading "...making it quite
> pointless..." was that Kyle is getting the stick instead of a
> carrot for following through and doing a better job than we would
> have done without him (your point about reporting this back in
> 0.12 is an example of our previous lack of familiarity,
> appreciation, and engagement with security related issues).
>
> If you have the time and interest, We'd love your help on the
> security side of things (contact Kyle or me offlist), and I think
> Kyle is striving to do a much more punctual disclosure of this
> vulnerability in part because of your feedback on CVE-2014-3429.
> I just want to make sure that we continue to have productive
> interactions.
>
> my sincerest appreciation to both of you,
> --
>                    _
>                   / \
>                 A*   \^   -
>              ,./   _.`\\ / \
>             / ,--.S    \/   \
>            /  `"~,_     \    \
>      __o           ?
>    _ \<,_         /:\
> --(_)/-(_)----.../ | \
> --------------.......J
> Paul Ivanov
> ipython and matplotlib core developer
> http://pirsquared.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140801/b9afc6d3/attachment.html>

From jtaylor.debian at googlemail.com  Fri Aug  1 18:03:58 2014
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Sat, 02 Aug 2014 00:03:58 +0200
Subject: [IPython-dev] Insecure loading of mathjax
In-Reply-To: <20140801214038.GD27567@HbI-OTOH.berkeley.edu>
References: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>	<53DC0303.7060500@googlemail.com>
	<20140801214038.GD27567@HbI-OTOH.berkeley.edu>
Message-ID: <53DC0ECE.6030108@googlemail.com>

On 01.08.2014 23:40, Paul Ivanov wrote:
> Hi Julian, Kyle, and list,
> 
> I just wanted to publicly thank Kyle again for following through
> with these and ensure that they get reported and communicated in
> the right manner. None of the other other IPython developers have
> any experience with disclosing security vulnerabilities to
> appropriate channels, and Kyle has stepped up entirely in a
> volunteer capacity to do this for the benefit of the community.
> 
> Thanks to you as well, Julian, for bringing that CDN certificate
> issue to our attention. We need all the help we can get, and I
> my immediate reaction to reading "...making it quite
> pointless..." was that Kyle is getting the stick instead of a
> carrot for following through and doing a better job than we would
> have done without him (your point about reporting this back in
> 0.12 is an example of our previous lack of familiarity,
> appreciation, and engagement with security related issues). 

I think it is great that Kyle is communicating the issues and helping
fixing issues. I did not intend to give "the stick" with my statement I
was referring to the situation back then, where I myself could have
probably handled the situation much better.

I too do not know very much about network security and how best to
report issues, though I can possibly help out contacting the
distribution security teams if needed.

The shared certificate is probably not a huge problem as the number of
AltNames for the mathjax certificate is quite small, but some googling
showed that this is indeed an attack vector:
http://news.netcraft.com/archives/2013/10/07/phishers-using-cloudflare-for-ssl.html

But I don't know if this is relevant enough to require any more actions
from IPython.

> 
> If you have the time and interest, We'd love your help on the
> security side of things (contact Kyle or me offlist), and I think
> Kyle is striving to do a much more punctual disclosure of this
> vulnerability in part because of your feedback on CVE-2014-3429.
> I just want to make sure that we continue to have productive
> interactions.
> 
> my sincerest appreciation to both of you,
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140802/fbe104a5/attachment.sig>

From takowl at gmail.com  Fri Aug  1 18:26:20 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 1 Aug 2014 15:26:20 -0700
Subject: [IPython-dev] Insecure loading of mathjax
In-Reply-To: <53DC0ECE.6030108@googlemail.com>
References: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>
	<53DC0303.7060500@googlemail.com>
	<20140801214038.GD27567@HbI-OTOH.berkeley.edu>
	<53DC0ECE.6030108@googlemail.com>
Message-ID: <CAOvn4qgTz0j+ZgSerJS_NSNWvZ=WuLT3q+kRX-2n7aNZOAaG1Q@mail.gmail.com>

On 1 August 2014 15:03, Julian Taylor <jtaylor.debian at googlemail.com> wrote:

> The shared certificate is probably not a huge problem as the number of
> AltNames for the mathjax certificate is quite small, but some googling
> showed that this is indeed an attack vector:
>
> http://news.netcraft.com/archives/2013/10/07/phishers-using-cloudflare-for-ssl.html
>

That doesn't appear to show someone actually MITMing an https server using
the shared certificate. The concern there appears to be that cloudflare is
letting people use its https certificates without much/any validation,
making for more convincing phishing attacks. That wouldn't compromise a
request to a specific, known URL.

Amusingly, if predictably, it appears that these shared certificates are
needed because of 'a lack of support for SNI in Internet Explorer on
Windows XP'.

I'll put this on the agenda for the next dev meeting.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140801/9640aec3/attachment.html>

From lstagner at uci.edu  Fri Aug  1 18:36:46 2014
From: lstagner at uci.edu (Luke Stagner)
Date: Fri, 01 Aug 2014 15:36:46 -0700
Subject: [IPython-dev] IDL/GDL kernel
In-Reply-To: <CAOvn4qiquFYsO_V0n-DvL4Y+WtwkGdj-Qt7DXYpkQjhgNYRWug@mail.gmail.com>
References: <mailman.1976.1406767312.14095.ipython-dev@scipy.org>	<53D99C9D.4070900@uci.edu>	<CAOvn4qgV6m+uKufyYwdQhRsOhPh6VzA+BVUphZ7AxqFsmcFF_A@mail.gmail.com>	<53D9B7A0.5050309@uci.edu>	<CAOvn4qim71sK+u5t0xe+SOTh6KaBPWsYQv2WZ2RS2S_V_svv+w@mail.gmail.com>	<53D9C05D.1030204@uci.edu>
	<CAOvn4qiquFYsO_V0n-DvL4Y+WtwkGdj-Qt7DXYpkQjhgNYRWug@mail.gmail.com>
Message-ID: <53DC167E.3070901@uci.edu>

Thanks that did the trick.

-Luke

On 07/31/2014 03:55 PM, Thomas Kluyver wrote:
> On 30 July 2014 21:04, Luke Stagner <lstagner at uci.edu 
> <mailto:lstagner at uci.edu>> wrote:
>
>     Good news no more error. Bad news still no inline plot. I even set
>     it so that it reads some random png I have every time and still no go.
>
>
> Sorry, I got the message structure wrong when I gave you an example 
> without trying it. This time, here's an example that I have tested and 
> ensured that it works:
>
> with open('foo.png', 'rb') as f:
>
>     import base64
>
>     imgdata = base64.b64encode(f.read())
>
> self.send_response(self.iopub_socket, 'display_data', 
> {'data':{'image/png': imgdata.decode('ascii')}})
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140801/c7b560c0/attachment.html>

From jtaylor.debian at googlemail.com  Fri Aug  1 18:47:52 2014
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Sat, 02 Aug 2014 00:47:52 +0200
Subject: [IPython-dev] Insecure loading of mathjax
In-Reply-To: <CAOvn4qgTz0j+ZgSerJS_NSNWvZ=WuLT3q+kRX-2n7aNZOAaG1Q@mail.gmail.com>
References: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>	<53DC0303.7060500@googlemail.com>	<20140801214038.GD27567@HbI-OTOH.berkeley.edu>	<53DC0ECE.6030108@googlemail.com>
	<CAOvn4qgTz0j+ZgSerJS_NSNWvZ=WuLT3q+kRX-2n7aNZOAaG1Q@mail.gmail.com>
Message-ID: <53DC1918.60404@googlemail.com>

On 02.08.2014 00:26, Thomas Kluyver wrote:
> On 1 August 2014 15:03, Julian Taylor <jtaylor.debian at googlemail.com
> <mailto:jtaylor.debian at googlemail.com>> wrote:
> 
>     The shared certificate is probably not a huge problem as the number of
>     AltNames for the mathjax certificate is quite small, but some googling
>     showed that this is indeed an attack vector:
>     http://news.netcraft.com/archives/2013/10/07/phishers-using-cloudflare-for-ssl.html
> 
> 
> That doesn't appear to show someone actually MITMing an https server
> using the shared certificate. The concern there appears to be that
> cloudflare is letting people use its https certificates without much/any
> validation, making for more convincing phishing attacks. That wouldn't
> compromise a request to a specific, known URL.

Normally if you can do an MITM and someone redirects your request
mathjax.org to another domain the browser would give you a certificate
error as the certificate served would not be valid for the domain you
requested.
In this case if someone is able to do a MITM and controls one of the
domains that share the certificate with mathjax that person would be
able to redirect the request to mathjax.org to his domain and serve any
file. The browser would accept this as the certificate is valid for the
domain even though its a different one than the request was sent to.
Or would the browser complain about the change in any case?

But I guess this is unlikely to be feasible/worthwhile to attack ipython
users in practice.

> 
> Amusingly, if predictably, it appears that these shared certificates are
> needed because of 'a lack of support for SNI in Internet Explorer on
> Windows XP'.

also amusingly amazon cloudfront which (I think) mathjax used before
rackspace and cloudfront now supports SNI since march this year:
http://aws.amazon.com/blogs/aws/server-name-indication-sni-and-http-redirection-for-amazon-cloudfront/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140802/c8b08b89/attachment.sig>

From takowl at gmail.com  Fri Aug  1 21:20:42 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 1 Aug 2014 18:20:42 -0700
Subject: [IPython-dev] Insecure loading of mathjax
In-Reply-To: <53DC1918.60404@googlemail.com>
References: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>
	<53DC0303.7060500@googlemail.com>
	<20140801214038.GD27567@HbI-OTOH.berkeley.edu>
	<53DC0ECE.6030108@googlemail.com>
	<CAOvn4qgTz0j+ZgSerJS_NSNWvZ=WuLT3q+kRX-2n7aNZOAaG1Q@mail.gmail.com>
	<53DC1918.60404@googlemail.com>
Message-ID: <CAOvn4qhVYcyMSGtQS3=fPVz_Huu9=P5vnEsb7U1kTGKsBvPWEA@mail.gmail.com>

On 1 August 2014 15:47, Julian Taylor <jtaylor.debian at googlemail.com> wrote:

> In this case if someone is able to do a MITM and controls one of the
> domains that share the certificate with mathjax that person would be
> able to redirect the request to mathjax.org to his domain and serve any
> file. The browser would accept this as the certificate is valid for the
> domain even though its a different one than the request was sent to.
> Or would the browser complain about the change in any case?
>

I have opened an issue on Mathjax to work out if this is possible. It's
probably most concerning for IPython, but if it's possible, it potentially
affects any site loading Mathjax from the CDN.

https://github.com/mathjax/MathJax/issues/885

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140801/169c7534/attachment.html>

From rgbkrk at gmail.com  Fri Aug  1 21:54:29 2014
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Fri, 1 Aug 2014 20:54:29 -0500
Subject: [IPython-dev] Docker IPython
Message-ID: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>

Earlier this week we released an experimental base Docker image for the
IPython notebook:

https://registry.hub.docker.com/u/ipython/notebook/

As well as a Docker image for the notebook viewer (which will be used in
production soon):

https://registry.hub.docker.com/u/ipython/nbviewer/

Yay!

Several folks have reached out about these images, and I'd like to get a
conversation started amongst those that want to build from a common base.
There are, after all, a *lot* of ipython images up on Docker already.
Additionally, there is a LONG thread on the software carpentry mailing list

There are some open questions from me:

* Who should the user be inside the container?
  * Should they be able to install more packages?
  * In a chef cookbook and deployment setup, I just made the user named
ipynb.

* Is it ok to force HTTPS?
  * If we do, we're stuck with self signed certs or requiring them to
provide them on run
  * This isn't really the right way to compose applications with Docker and
probably makes things problematic for anyone building on top of ours

* Who is our target audience?
  * People deploying notebooks across their organization for users?
  * Students (and their teachers) wanting access to an easy analytics
environment?

* What would you see out of a base image? Where would the value be?

* Would people like to see a bleeding latest image that you can spin up to
see the current state of the IPython notebook by just running `docker run
ipython/notebook:bleeding-latest`?

* Do people want a base image that has a large portion of the scipy stack?

I'll stop with these and see where the discussion takes us.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140801/5e9535f2/attachment.html>

From bjoern.gruening at gmail.com  Sat Aug  2 02:03:38 2014
From: bjoern.gruening at gmail.com (=?UTF-8?B?QmrDtnJuIEdyw7xuaW5n?=)
Date: Sat, 02 Aug 2014 08:03:38 +0200
Subject: [IPython-dev] Announcement: Galaxy IPython Integration -
 interactive programming in Galaxy
Message-ID: <53DC7F3A.4000601@gmail.com>


Hi all,

we proudly present the first release of the Galaxy IPython project.

Galaxy IPython is a visualization plugin which should enable Galaxy 
users with coding skills to easily process their data in the most 
flexible way. With this plugin, it is possible to analyse and 
post-process data without downloading datasets or entire histories. One 
of our aims was to make Galaxy more attractive and accessible to 
bioinformaticians and programmers, and we hope that this project will 
build some bridges.

More information about Galaxy: http://galaxyproject.org/
Screencast: http://www.youtube.com/watch?v=jQDyTuYnn1k

Disclaimer: Even though the Ipython notebooks can be stored and reused, 
this plugin will break the Galaxy philosophy of reproducibility, I feel 
personally bad about that, but I also think it is a great opportunity to 
get more bioinformaticians into Galaxy, and to get Galaxy used more 
often as a teaching resource. By being able to teach not only about 
workflows but also about data analysis tasks often necessary with 
Bioinformatics, Galaxy will be significantly more useful in teaching 
environments.

A few features we have up and running:
* Use IPython directly in the main window or in the Scratchbook
* Completely encapsulated IPython environment with matplotlib, 
biopython, pandas and friends already installed.
* IPython runs completely self-contained within a docker container, 
separate from your Galaxy data
* Easy access to datasets from your current history via pre-defined 
IPython functions
* Manipulate and plot data as you like and export your new files back 
into your Galaxy history
* Save IPython Notebooks across analysis sessions in your Galaxy history 
with the click of a button.
* View saved IPython Notebooks directly in HTML format, or re-open them 
to continue your analysis.
* Self-closing and self-cleaning IPython docker container
Notebooks are secure, only accessible to the intended user


Please follow the installation instruction on our project page under:
https://github.com/bgruening/galaxy-ipython

The underlying IPython Notebook (+Galaxy sugar) is stored under:
https://github.com/bgruening/docker-ipython-notebook
https://registry.hub.docker.com/u/bgruening/docker-ipython-notebook/
You can also install a ipynb datatype:
https://github.com/bgruening/galaxytools/tree/master/datatypes/json
https://testtoolshed.g2.bx.psu.edu/view/iuc/datatyp_ipynb


Comments welcome!
Happy research!
Eric & Bj?rn


From rgbkrk at gmail.com  Sat Aug  2 11:23:16 2014
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Sat, 2 Aug 2014 10:23:16 -0500
Subject: [IPython-dev] Announcement: Galaxy IPython Integration -
 interactive programming in Galaxy
In-Reply-To: <53DC7F3A.4000601@gmail.com>
References: <53DC7F3A.4000601@gmail.com>
Message-ID: <CA+tbMaXSbVYYtKmbs+0mGJUfOt6_Gc-vHWzEpUQO1y+3b15Pgg@mail.gmail.com>

Bj?rn,

Thanks for sharing. I went ahead and created a Galaxy account. What's the
simplest path for trying out the IPython notebook integration within Galaxy?

-- Kyle


On Sat, Aug 2, 2014 at 1:03 AM, Bj?rn Gr?ning <bjoern.gruening at gmail.com>
wrote:

>
> Hi all,
>
> we proudly present the first release of the Galaxy IPython project.
>
> Galaxy IPython is a visualization plugin which should enable Galaxy
> users with coding skills to easily process their data in the most
> flexible way. With this plugin, it is possible to analyse and
> post-process data without downloading datasets or entire histories. One
> of our aims was to make Galaxy more attractive and accessible to
> bioinformaticians and programmers, and we hope that this project will
> build some bridges.
>
> More information about Galaxy: http://galaxyproject.org/
> Screencast: http://www.youtube.com/watch?v=jQDyTuYnn1k
>
> Disclaimer: Even though the Ipython notebooks can be stored and reused,
> this plugin will break the Galaxy philosophy of reproducibility, I feel
> personally bad about that, but I also think it is a great opportunity to
> get more bioinformaticians into Galaxy, and to get Galaxy used more
> often as a teaching resource. By being able to teach not only about
> workflows but also about data analysis tasks often necessary with
> Bioinformatics, Galaxy will be significantly more useful in teaching
> environments.
>
> A few features we have up and running:
> * Use IPython directly in the main window or in the Scratchbook
> * Completely encapsulated IPython environment with matplotlib,
> biopython, pandas and friends already installed.
> * IPython runs completely self-contained within a docker container,
> separate from your Galaxy data
> * Easy access to datasets from your current history via pre-defined
> IPython functions
> * Manipulate and plot data as you like and export your new files back
> into your Galaxy history
> * Save IPython Notebooks across analysis sessions in your Galaxy history
> with the click of a button.
> * View saved IPython Notebooks directly in HTML format, or re-open them
> to continue your analysis.
> * Self-closing and self-cleaning IPython docker container
> Notebooks are secure, only accessible to the intended user
>
>
> Please follow the installation instruction on our project page under:
> https://github.com/bgruening/galaxy-ipython
>
> The underlying IPython Notebook (+Galaxy sugar) is stored under:
> https://github.com/bgruening/docker-ipython-notebook
> https://registry.hub.docker.com/u/bgruening/docker-ipython-notebook/
> You can also install a ipynb datatype:
> https://github.com/bgruening/galaxytools/tree/master/datatypes/json
> https://testtoolshed.g2.bx.psu.edu/view/iuc/datatyp_ipynb
>
>
> Comments welcome!
> Happy research!
> Eric & Bj?rn
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140802/47a28873/attachment.html>

From bjoern.gruening at gmail.com  Sat Aug  2 11:34:11 2014
From: bjoern.gruening at gmail.com (=?windows-1252?Q?Bj=F6rn_Gr=FCning?=)
Date: Sat, 02 Aug 2014 17:34:11 +0200
Subject: [IPython-dev] Announcement: Galaxy IPython Integration -
 interactive programming in Galaxy
In-Reply-To: <CA+tbMaXSbVYYtKmbs+0mGJUfOt6_Gc-vHWzEpUQO1y+3b15Pgg@mail.gmail.com>
References: <53DC7F3A.4000601@gmail.com>
	<CA+tbMaXSbVYYtKmbs+0mGJUfOt6_Gc-vHWzEpUQO1y+3b15Pgg@mail.gmail.com>
Message-ID: <53DD04F3.2060505@gmail.com>

Hi Kyle,

currently you need to have an own Galaxy installation [1]. It's 
relatively simple and and mainly a hg clone and ./run.sh step.

If you have a running Galaxy instance you can follow the installation 
instruction under: https://github.com/bgruening/galaxy-ipython

It's simple as: git clone 
https://github.com/bgruening/galaxy-ipython.git 
$GALAX_DIR/config/plugins/viz/ipython

We are facing the problem that all our productive systems do not have 
access to docker, so we need a little bit of time to migrate our 
servers. Also, I would like to collect a few more comments and reviews 
before we can offer this feature to all Galaxy users.

Please note, that I'm in close contact with the main Galaxy devs but I'm 
not part of that gang, so I have nothing to decide weather or not that 
feature will be added to usegalaxy.org.

Thanks for your interest!
Bjoern

[1] https://wiki.galaxyproject.org/Admin/GetGalaxy

Am 02.08.2014 um 17:23 schrieb Kyle Kelley:
> Bj?rn,
>
> Thanks for sharing. I went ahead and created a Galaxy account. What's the
> simplest path for trying out the IPython notebook integration within Galaxy?
>
> -- Kyle
>
>
> On Sat, Aug 2, 2014 at 1:03 AM, Bj?rn Gr?ning <bjoern.gruening at gmail.com>
> wrote:
>
>>
>> Hi all,
>>
>> we proudly present the first release of the Galaxy IPython project.
>>
>> Galaxy IPython is a visualization plugin which should enable Galaxy
>> users with coding skills to easily process their data in the most
>> flexible way. With this plugin, it is possible to analyse and
>> post-process data without downloading datasets or entire histories. One
>> of our aims was to make Galaxy more attractive and accessible to
>> bioinformaticians and programmers, and we hope that this project will
>> build some bridges.
>>
>> More information about Galaxy: http://galaxyproject.org/
>> Screencast: http://www.youtube.com/watch?v=jQDyTuYnn1k
>>
>> Disclaimer: Even though the Ipython notebooks can be stored and reused,
>> this plugin will break the Galaxy philosophy of reproducibility, I feel
>> personally bad about that, but I also think it is a great opportunity to
>> get more bioinformaticians into Galaxy, and to get Galaxy used more
>> often as a teaching resource. By being able to teach not only about
>> workflows but also about data analysis tasks often necessary with
>> Bioinformatics, Galaxy will be significantly more useful in teaching
>> environments.
>>
>> A few features we have up and running:
>> * Use IPython directly in the main window or in the Scratchbook
>> * Completely encapsulated IPython environment with matplotlib,
>> biopython, pandas and friends already installed.
>> * IPython runs completely self-contained within a docker container,
>> separate from your Galaxy data
>> * Easy access to datasets from your current history via pre-defined
>> IPython functions
>> * Manipulate and plot data as you like and export your new files back
>> into your Galaxy history
>> * Save IPython Notebooks across analysis sessions in your Galaxy history
>> with the click of a button.
>> * View saved IPython Notebooks directly in HTML format, or re-open them
>> to continue your analysis.
>> * Self-closing and self-cleaning IPython docker container
>> Notebooks are secure, only accessible to the intended user
>>
>>
>> Please follow the installation instruction on our project page under:
>> https://github.com/bgruening/galaxy-ipython
>>
>> The underlying IPython Notebook (+Galaxy sugar) is stored under:
>> https://github.com/bgruening/docker-ipython-notebook
>> https://registry.hub.docker.com/u/bgruening/docker-ipython-notebook/
>> You can also install a ipynb datatype:
>> https://github.com/bgruening/galaxytools/tree/master/datatypes/json
>> https://testtoolshed.g2.bx.psu.edu/view/iuc/datatyp_ipynb
>>
>>
>> Comments welcome!
>> Happy research!
>> Eric & Bj?rn
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From jbarratt at serialized.net  Mon Aug  4 01:34:50 2014
From: jbarratt at serialized.net (Joshua Barratt)
Date: Sun, 3 Aug 2014 22:34:50 -0700
Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython
 Notebook on your own machine
In-Reply-To: <CAOvn4qjaZq5w-Y1hV5tJchWTNURcmzBf=V458-meR-fMSyAmkw@mail.gmail.com>
References: <CAOvn4qjaZq5w-Y1hV5tJchWTNURcmzBf=V458-meR-fMSyAmkw@mail.gmail.com>
Message-ID: <CAN4eD9AwsD3nW+PWCVSqZ65BsqcQ+Hc6H+_ZeoSvXx8+JGx8zw@mail.gmail.com>

Thanks for sharing those, very cool. I had been thinking about doing
something like nbopen, I'm happy to see that's something I can delete off
of the infinite to-do list.

I created something similar to nbmanager, but much more stripped down and
only allows viewing.

I called mine nblist <https://gist.github.com/jbarratt/ae8026493fedc79f122b>
.

        $ nblist
          http://127.0.0.1:8088 | /Users/me/work/notebooks
          http://127.0.0.1:8089 | /Users/me/work/otherproject


I also recently wrote something called nbgrep
<https://gist.github.com/jbarratt/fa1d3473048e5f856aeb> (blog post here
<http://serialized.net/2014/07/mining-your-ipython-notebooks-with-nbgrep/>)
which allows you to search the input cells of all your notebooks,
regardless of directory.

Unfortunately both of mine only work on OSX, though they could be modified
to be cross-platform.

Josh

On Fri, Aug 1, 2014 at 2:55 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> I've been working on a couple of tools to improve the experience of
> working with IPython notebooks on your own computer.
>
> First, nbopen, a tool to open a notebook in the nearest available server,
> or start a new server if there isn't one available. It now integrates with
> your file manager so you can double click on a notebook file to open it
> (open source desktops only for now, PRs for other systems welcome):
>
> https://github.com/takluyver/nbopen
>
> However, there's no easy way to shut down notebook servers launched by
> double clicking on notebooks, because there isn't a terminal where you can
> press ctrl-c. So I also made nbmanager, a GUI app which discovers running
> notebook servers and open notebooks, and lets you easily shut them down:
>
> https://github.com/takluyver/nbmanager
>
> nbopen will work on IPython 2, while nbmanager, which is more
> experimental, requires you to be running IPython from git at the moment.
>
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140803/0bf0be42/attachment.html>

From takowl at gmail.com  Mon Aug  4 01:49:02 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 3 Aug 2014 22:49:02 -0700
Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython
 Notebook on your own machine
In-Reply-To: <CAN4eD9AwsD3nW+PWCVSqZ65BsqcQ+Hc6H+_ZeoSvXx8+JGx8zw@mail.gmail.com>
References: <CAOvn4qjaZq5w-Y1hV5tJchWTNURcmzBf=V458-meR-fMSyAmkw@mail.gmail.com>
	<CAN4eD9AwsD3nW+PWCVSqZ65BsqcQ+Hc6H+_ZeoSvXx8+JGx8zw@mail.gmail.com>
Message-ID: <CAOvn4qhN8DBS+ADwxjbsik=UqdubPdorxcvULZsAKs5MWnr-hw@mail.gmail.com>

Neat, thanks for those. For nblist, instead of scanning processes and
ports, it may be easier to use our API function list_running_servers(),
especially for cross platform support:

http://ipython.org/ipython-doc/dev/api/generated/IPython.html.notebookapp.html#IPython.html.notebookapp.list_running_servers

nbmanager also includes a slightly higher level API around that, which is
completely independent of all the GUI code (though this requires IPython 3):

https://github.com/takluyver/nbmanager/blob/master/nbmanager/api.py

Best wishes,
Thomas


On 3 August 2014 22:34, Joshua Barratt <jbarratt at serialized.net> wrote:

> Thanks for sharing those, very cool. I had been thinking about doing
> something like nbopen, I'm happy to see that's something I can delete off
> of the infinite to-do list.
>
> I created something similar to nbmanager, but much more stripped down and
> only allows viewing.
>
> I called mine nblist
> <https://gist.github.com/jbarratt/ae8026493fedc79f122b>.
>
>         $ nblist
>           http://127.0.0.1:8088 | /Users/me/work/notebooks
>
>           http://127.0.0.1:8089 | /Users/me/work/otherproject
>
>
> I also recently wrote something called nbgrep
> <https://gist.github.com/jbarratt/fa1d3473048e5f856aeb> (blog post here
> <http://serialized.net/2014/07/mining-your-ipython-notebooks-with-nbgrep/>)
> which allows you to search the input cells of all your notebooks,
> regardless of directory.
>
> Unfortunately both of mine only work on OSX, though they could be modified
> to be cross-platform.
>
> Josh
>
> On Fri, Aug 1, 2014 at 2:55 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
>> I've been working on a couple of tools to improve the experience of
>> working with IPython notebooks on your own computer.
>>
>> First, nbopen, a tool to open a notebook in the nearest available server,
>> or start a new server if there isn't one available. It now integrates with
>> your file manager so you can double click on a notebook file to open it
>> (open source desktops only for now, PRs for other systems welcome):
>>
>> https://github.com/takluyver/nbopen
>>
>> However, there's no easy way to shut down notebook servers launched by
>> double clicking on notebooks, because there isn't a terminal where you can
>> press ctrl-c. So I also made nbmanager, a GUI app which discovers running
>> notebook servers and open notebooks, and lets you easily shut them down:
>>
>> https://github.com/takluyver/nbmanager
>>
>> nbopen will work on IPython 2, while nbmanager, which is more
>> experimental, requires you to be running IPython from git at the moment.
>>
>> Thomas
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140803/15c49975/attachment.html>

From jbarratt at serialized.net  Mon Aug  4 01:59:57 2014
From: jbarratt at serialized.net (Joshua Barratt)
Date: Sun, 3 Aug 2014 22:59:57 -0700
Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython
 Notebook on your own machine
In-Reply-To: <CAOvn4qhN8DBS+ADwxjbsik=UqdubPdorxcvULZsAKs5MWnr-hw@mail.gmail.com>
References: <CAOvn4qjaZq5w-Y1hV5tJchWTNURcmzBf=V458-meR-fMSyAmkw@mail.gmail.com>
	<CAN4eD9AwsD3nW+PWCVSqZ65BsqcQ+Hc6H+_ZeoSvXx8+JGx8zw@mail.gmail.com>
	<CAOvn4qhN8DBS+ADwxjbsik=UqdubPdorxcvULZsAKs5MWnr-hw@mail.gmail.com>
Message-ID: <CAN4eD9DLEKvLyZGhA6PiWDBMsCAHN2h9t5J8CcnAtAnTttYwTg@mail.gmail.com>

Very cool, I looked for something like that, but somehow missed it. That's
perfect and much less janky, thanks!


On Sun, Aug 3, 2014 at 10:49 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> Neat, thanks for those. For nblist, instead of scanning processes and
> ports, it may be easier to use our API function list_running_servers(),
> especially for cross platform support:
>
>
> http://ipython.org/ipython-doc/dev/api/generated/IPython.html.notebookapp.html#IPython.html.notebookapp.list_running_servers
>
> nbmanager also includes a slightly higher level API around that, which is
> completely independent of all the GUI code (though this requires IPython 3):
>
> https://github.com/takluyver/nbmanager/blob/master/nbmanager/api.py
>
> Best wishes,
> Thomas
>
>
> On 3 August 2014 22:34, Joshua Barratt <jbarratt at serialized.net> wrote:
>
>> Thanks for sharing those, very cool. I had been thinking about doing
>> something like nbopen, I'm happy to see that's something I can delete off
>> of the infinite to-do list.
>>
>> I created something similar to nbmanager, but much more stripped down and
>> only allows viewing.
>>
>> I called mine nblist
>> <https://gist.github.com/jbarratt/ae8026493fedc79f122b>.
>>
>>         $ nblist
>>           http://127.0.0.1:8088 | /Users/me/work/notebooks
>>
>>           http://127.0.0.1:8089 | /Users/me/work/otherproject
>>
>>
>> I also recently wrote something called nbgrep
>> <https://gist.github.com/jbarratt/fa1d3473048e5f856aeb> (blog post here
>> <http://serialized.net/2014/07/mining-your-ipython-notebooks-with-nbgrep/>)
>> which allows you to search the input cells of all your notebooks,
>> regardless of directory.
>>
>> Unfortunately both of mine only work on OSX, though they could be
>> modified to be cross-platform.
>>
>> Josh
>>
>> On Fri, Aug 1, 2014 at 2:55 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>
>>> I've been working on a couple of tools to improve the experience of
>>> working with IPython notebooks on your own computer.
>>>
>>> First, nbopen, a tool to open a notebook in the nearest available
>>> server, or start a new server if there isn't one available. It now
>>> integrates with your file manager so you can double click on a notebook
>>> file to open it (open source desktops only for now, PRs for other systems
>>> welcome):
>>>
>>> https://github.com/takluyver/nbopen
>>>
>>> However, there's no easy way to shut down notebook servers launched by
>>> double clicking on notebooks, because there isn't a terminal where you can
>>> press ctrl-c. So I also made nbmanager, a GUI app which discovers running
>>> notebook servers and open notebooks, and lets you easily shut them down:
>>>
>>> https://github.com/takluyver/nbmanager
>>>
>>> nbopen will work on IPython 2, while nbmanager, which is more
>>> experimental, requires you to be running IPython from git at the moment.
>>>
>>> Thomas
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140803/c853a4a4/attachment.html>

From alessandro.gagliardi at glassdoor.com  Mon Aug  4 14:18:35 2014
From: alessandro.gagliardi at glassdoor.com (Alessandro Gagliardi)
Date: Mon, 4 Aug 2014 18:18:35 +0000
Subject: [IPython-dev] Download a Python-generated file over the web through
 IPython Notebook
Message-ID: <D0051C89.E2B3%alessandro.gagliardi@glassdoor.com>

I?d like to allow users to generate a CSV in a notebook that is hosted remotely and download that file to their local machine, perhaps through a widget (or some other way that doesn?t require the command line).

Is this possible? The closest thing I can think of is pandas? to_clipboard method which could then be used to paste the data into a local spreadsheet, but that seems clumsy and prone to human error.

Thanks in advance!

Alessandro Gagliardi | senior data scientist | Glassdoor | alessandro at glassdoor.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140804/6ca9624b/attachment.html>

From takowl at gmail.com  Mon Aug  4 14:22:32 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 4 Aug 2014 11:22:32 -0700
Subject: [IPython-dev] Download a Python-generated file over the web
 through IPython Notebook
In-Reply-To: <D0051C89.E2B3%alessandro.gagliardi@glassdoor.com>
References: <D0051C89.E2B3%alessandro.gagliardi@glassdoor.com>
Message-ID: <CAOvn4qjx6ZF2JNwj7eAvEN4+cXLR00XAkUYud07VxCDkkfA++A@mail.gmail.com>

On 4 August 2014 11:18, Alessandro Gagliardi <
alessandro.gagliardi at glassdoor.com> wrote:

> I?d like to allow users to generate a CSV in a notebook that is hosted
> remotely and download that file to their local machine, perhaps through a
> widget (or some other way that doesn?t require the command line).
>
>  Is this possible? The closest thing I can think of is pandas?
> to_clipboard method which could then be used to paste the data into a local
> spreadsheet, but that seems clumsy and prone to human error.
>

It's perfectly possible so long as the kernel is running on the same
machine as the notebook server (which is the default configuration). Just
write the file, and then display an HTML link pointing to it using a
relative URL.

to_clipboard won't work, because that sets the clipboard on the machine
where the code is running, not on the user's machine.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140804/19f17dea/attachment.html>

From petr.wolf at gmail.com  Mon Aug  4 15:03:41 2014
From: petr.wolf at gmail.com (Petr Wolf)
Date: Mon, 4 Aug 2014 15:03:41 -0400
Subject: [IPython-dev] Errors in ipython startup muted by Notebook?
Message-ID: <CA+2k6+xk=jqxEjeoiaTUgJ2oJPEnxkbmiLPK1q5aFM=8pmKOpw@mail.gmail.com>

Hi,

as described on
http://stackoverflow.com/questions/24942378/how-to-propagate-errors-out-of-ipython-profile-to-a-notebook,
I wonder how to handle errors in IPython startup code.

While the IPython console prints any errors right upfront, the Notebook app
seems to silence them completely.

That can be confusing (as some startup commands may have been executed and
others may not) and counterproductive (as the information does not seem to
be logged either).

What is the best way to have errors propagated out of the profile, when
using nootebook?

Regards,
Petr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140804/e9c0a863/attachment.html>

From ocefpaf at gmail.com  Mon Aug  4 15:55:38 2014
From: ocefpaf at gmail.com (Filipe Pires Alvarenga Fernandes)
Date: Mon, 4 Aug 2014 16:55:38 -0300
Subject: [IPython-dev] Automatically saving notebooks as scripts has been
	removed
Message-ID: <CAH2VmmB45-1pEDkst7Dmq=CaswM7G_eKExzde+2mLOsEjLJr_A@mail.gmail.com>

Hello IPython devs,

I was trying the latest github checkout of the IPython source and got the
following message:

Automatically saving notebooks as scripts has been removed.
Use `ipython nbconvert --to python [notebook]` instead.

I am wondering why this had to be removed.  Security?

I find the automatic script saving very helpful.  My work-group usually
commits the `.ipynb` and the `.py` side-by-side to obtain a better diff.  I
know that we can workaround this with nbconvert, but doing that manually is
error prone, and having a script to do that will add another layer of
complexity that we did not have with the automatic conversion.

Well, if the automatic conversion had to go I understand, but if there is
no reason can you guys consider leaving it there?

Thanks,

-Filipe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140804/67a47c2f/attachment.html>

From takowl at gmail.com  Mon Aug  4 17:18:51 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 4 Aug 2014 14:18:51 -0700
Subject: [IPython-dev] Automatically saving notebooks as scripts has
	been removed
In-Reply-To: <CAH2VmmB45-1pEDkst7Dmq=CaswM7G_eKExzde+2mLOsEjLJr_A@mail.gmail.com>
References: <CAH2VmmB45-1pEDkst7Dmq=CaswM7G_eKExzde+2mLOsEjLJr_A@mail.gmail.com>
Message-ID: <CAOvn4qiq78Cb_fk7N_P5ZuLk80wEFC+SsZmx4RjV=4MnJbdr8A@mail.gmail.com>

I think we may have been a bit too hasty in getting rid of that. We've been
talking about it for a while amongst ourselves, because it doesn't fit
terribly neatly into our infrastructure, but we didn't formally deprecate
it, and I don't think the alternative approaches are quite as convenient
yet.

I've put this in the agenda for the dev meeting on Thursday.

Thanks,
Thomas


On 4 August 2014 12:55, Filipe Pires Alvarenga Fernandes <ocefpaf at gmail.com>
wrote:

> Hello IPython devs,
>
> I was trying the latest github checkout of the IPython source and got the
> following message:
>
> Automatically saving notebooks as scripts has been removed.
> Use `ipython nbconvert --to python [notebook]` instead.
>
> I am wondering why this had to be removed.  Security?
>
> I find the automatic script saving very helpful.  My work-group usually
> commits the `.ipynb` and the `.py` side-by-side to obtain a better diff.  I
> know that we can workaround this with nbconvert, but doing that manually is
> error prone, and having a script to do that will add another layer of
> complexity that we did not have with the automatic conversion.
>
> Well, if the automatic conversion had to go I understand, but if there is
> no reason can you guys consider leaving it there?
>
> Thanks,
>
> -Filipe
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140804/c9ce7294/attachment.html>

From ocefpaf at gmail.com  Mon Aug  4 17:39:11 2014
From: ocefpaf at gmail.com (Filipe Pires Alvarenga Fernandes)
Date: Mon, 4 Aug 2014 18:39:11 -0300
Subject: [IPython-dev] Automatically saving notebooks as scripts has
	been removed
In-Reply-To: <CAOvn4qiq78Cb_fk7N_P5ZuLk80wEFC+SsZmx4RjV=4MnJbdr8A@mail.gmail.com>
References: <CAH2VmmB45-1pEDkst7Dmq=CaswM7G_eKExzde+2mLOsEjLJr_A@mail.gmail.com>
	<CAOvn4qiq78Cb_fk7N_P5ZuLk80wEFC+SsZmx4RjV=4MnJbdr8A@mail.gmail.com>
Message-ID: <CAH2VmmD-HwghBY5GEGyZUn6BCZXc1+f6mnh=YAiM26URuEASsA@mail.gmail.com>

Got it.  Thanks for looking into this.


On Mon, Aug 4, 2014 at 6:18 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> I think we may have been a bit too hasty in getting rid of that. We've
> been talking about it for a while amongst ourselves, because it doesn't fit
> terribly neatly into our infrastructure, but we didn't formally deprecate
> it, and I don't think the alternative approaches are quite as convenient
> yet.
>
> I've put this in the agenda for the dev meeting on Thursday.
>
> Thanks,
> Thomas
>
>
> On 4 August 2014 12:55, Filipe Pires Alvarenga Fernandes <
> ocefpaf at gmail.com> wrote:
>
>> Hello IPython devs,
>>
>> I was trying the latest github checkout of the IPython source and got the
>> following message:
>>
>> Automatically saving notebooks as scripts has been removed.
>> Use `ipython nbconvert --to python [notebook]` instead.
>>
>> I am wondering why this had to be removed.  Security?
>>
>> I find the automatic script saving very helpful.  My work-group usually
>> commits the `.ipynb` and the `.py` side-by-side to obtain a better diff.  I
>> know that we can workaround this with nbconvert, but doing that manually is
>> error prone, and having a script to do that will add another layer of
>> complexity that we did not have with the automatic conversion.
>>
>> Well, if the automatic conversion had to go I understand, but if there is
>> no reason can you guys consider leaving it there?
>>
>> Thanks,
>>
>> -Filipe
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140804/bfbda134/attachment.html>

From tylere at google.com  Mon Aug  4 17:52:36 2014
From: tylere at google.com (Tyler Erickson)
Date: Mon, 4 Aug 2014 14:52:36 -0700
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
Message-ID: <CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>

Kyle,

I think having an official base Docker instance will be quite useful,
especially for running workshops.

I added comments for my particular use case below.

On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley <rgbkrk at gmail.com> wrote:

> Earlier this week we released an experimental base Docker image for the
> IPython notebook:
>
> https://registry.hub.docker.com/u/ipython/notebook/
>
> As well as a Docker image for the notebook viewer (which will be used in
> production soon):
>
> https://registry.hub.docker.com/u/ipython/nbviewer/
>
> Yay!
>
> Several folks have reached out about these images, and I'd like to get a
> conversation started amongst those that want to build from a common base.
> There are, after all, a *lot* of ipython images up on Docker already.
> Additionally, there is a LONG thread on the software carpentry mailing list
>
> There are some open questions from me:
>
> * Who should the user be inside the container?
>   * Should they be able to install more packages?
>

I don't think the user accessing the instance needs to be able to install
custom packages. (It would be nice this functionality was available, but it
is not critical for running workshops.)

However, administrators need to have the ability to create custom images
based off the docker base image that have additional packages.


>   * In a chef cookbook and deployment setup, I just made the user named
> ipynb.
>
> * Is it ok to force HTTPS?
>   * If we do, we're stuck with self signed certs or requiring them to
> provide them on run
>   * This isn't really the right way to compose applications with Docker
> and probably makes things problematic for anyone building on top of ours
>
> * Who is our target audience?
>   * People deploying notebooks across their organization for users?
>   * Students (and their teachers) wanting access to an easy analytics
> environment?
>

My interest is in providing an environment for external users of our
analysis API, so it somewhat similar to the latter description. Some are in
academia, but many are not.

* What would you see out of a base image? Where would the value be?
>
> * Would people like to see a bleeding latest image that you can spin up to
> see the current state of the IPython notebook by just running `docker run
> ipython/notebook:bleeding-latest`?
>
> * Do people want a base image that has a large portion of the scipy stack?
>

Yes, particularly matplotlib and pandas.

Cheers,
Tyler

I'll stop with these and see where the discussion takes us.
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140804/89cbfc7a/attachment.html>

From andy at payne.org  Mon Aug  4 18:41:15 2014
From: andy at payne.org (Andrew Payne)
Date: Mon, 4 Aug 2014 18:41:15 -0400
Subject: [IPython-dev] Download a Python-generated file over the web
 through IPython Notebook
In-Reply-To: <D0051C89.E2B3%alessandro.gagliardi@glassdoor.com>
References: <D0051C89.E2B3%alessandro.gagliardi@glassdoor.com>
Message-ID: <CABbuyuVKHi0Uh7X__kbL_OdraaGFFFJ8_bHccXuTG+zSTX5mxw@mail.gmail.com>

On Mon, Aug 4, 2014 at 2:18 PM, Alessandro Gagliardi <
alessandro.gagliardi at glassdoor.com> wrote:

>  I?d like to allow users to generate a CSV in a notebook that is hosted
> remotely and download that file to their local machine, perhaps through a
> widget (or some other way that doesn?t require the command line).
>

Another approach is to use a data URI
<https://developer.mozilla.org/en-US/docs/Web/HTTP/data_URIs>, where a link
is specified "by value" (e.g. the target file contents)  *vs* "by
reference" where the URL points to a server.

For example, this cell presents a link in the output area that downloads a
small CSV file:

%%html
<a href="data:text/csv,1,2,3,4" target="_blank">Click here to download</a>


Generalizing, you should be able to get from (say) a Pandas Dataframe to a
HTML() IPython display object with a "click to download" link in a few
lines of Python.  Or, subclass the dataframe so that all of your dataframe
objects offer a "click to download" option.

Pros:

   - Entirely self-contained, doesn't depend on how/where IPython notebooks
   are served
   - Doesn't require kernel and web server on same machine
   - [90% certain] A published notebook (e.g. nbconvert) will work exactly
   as expected

Cons:

   - Entire CSV content embedded in cell (may be an issue for large files)
   - There are corner cases of lacking or partial browser support (notably,
   old IE)
   - The file is always called "download", unless you do Javascript tricks
   (e.g. see:
   http://stackoverflow.com/questions/833015/does-execcommand-saveas-work-in-firefox/13696029#13696029
   )

Also note there is a new "download" attribute in <a> tags that makes this
even easier.  See:
https://developer.mozilla.org/en-US/docs/Web/HTML/Element/a
Unfortunately, the tag is not (yet) in all major browsers.

-andy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140804/332912d5/attachment.html>

From alessandro.gagliardi at glassdoor.com  Mon Aug  4 18:45:09 2014
From: alessandro.gagliardi at glassdoor.com (Alessandro Gagliardi)
Date: Mon, 4 Aug 2014 22:45:09 +0000
Subject: [IPython-dev] Download a Python-generated file over the web
 through IPython Notebook
In-Reply-To: <mailman.2684.1407188359.14095.ipython-dev@scipy.org>
Message-ID: <D0055ABE.E329%alessandro.gagliardi@glassdoor.com>

Worked like a charm. Thanks!

It's perfectly possible so long as the kernel is running on the same
machine as the notebook server (which is the default configuration). Just
write the file, and then display an HTML link pointing to it using a
relative URL.

to_clipboard won't work, because that sets the clipboard on the machine
where the code is running, not on the user's machine.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140804/1e27d574/attachment.html>

From andrew.gibiansky at gmail.com  Mon Aug  4 20:41:19 2014
From: andrew.gibiansky at gmail.com (Andrew Gibiansky)
Date: Mon, 4 Aug 2014 17:41:19 -0700
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
Message-ID: <CAF-LYKL6xAuPk4tUZONne9e7UKFaFahCnPHVh1CenQu-jwwzJA@mail.gmail.com>

In addition to a base image which supports IPython-for-Python, it'd be nice
to have a base image for IPython-for-Other-Kernels.

In my case, I would love to have an IPython docker base image that I can
extend to make an IHaskell docker image. For this use case, I would *not*
want the scipy stack, or in fact anything other than the core libraries
required to run IPython itself.

I suspect the same would go for IJulia, IRuby, and so on.

-- Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140804/ad594d8d/attachment.html>

From rgbkrk at gmail.com  Tue Aug  5 01:33:12 2014
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Tue, 5 Aug 2014 00:33:12 -0500
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
Message-ID: <CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>

Tyler,

Taking in everyones comments and my opinion as a not-quite-sysadmin:

Base image, ipython/notebook, will

* have just the ipython notebook essentials
* up to date (as of image build) version of pip (or conda... plead for one
or the other please!)
* be installed for all users (all users inside the container)
  * This means that any images inheriting from it can set up their own user
setup, install more packages
* default cmd that runs the notebook server with some default parameters
  * This can be overwritten by a Dockerfile that uses FROM ipython/notebook

My personal take on this is that we could have an ipy user, with notebooks
at /home/ipy/notebooks.

Derivative image, ipython/scipystack (please suggest other names), could be

* numpy
* scipy
* matplotlib
* pandas
* scikit-learn

For an IHaskell base image, I'd prefer the namespace to actually be
jupyter/ihaskell in the long run.

There may come a time when we want multiple kernels in one container. After
all, have you seen what you can do with the current master branch?




Including all the kernels (and stacks) will make a hefty docker image
though, so I won't progress down that path just yet.



On Mon, Aug 4, 2014 at 4:52 PM, Tyler Erickson <tylere at google.com> wrote:

> Kyle,
>
> I think having an official base Docker instance will be quite useful,
> especially for running workshops.
>
> I added comments for my particular use case below.
>
> On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley <rgbkrk at gmail.com> wrote:
>
>> Earlier this week we released an experimental base Docker image for the
>> IPython notebook:
>>
>> https://registry.hub.docker.com/u/ipython/notebook/
>>
>> As well as a Docker image for the notebook viewer (which will be used in
>> production soon):
>>
>> https://registry.hub.docker.com/u/ipython/nbviewer/
>>
>> Yay!
>>
>> Several folks have reached out about these images, and I'd like to get a
>> conversation started amongst those that want to build from a common base.
>> There are, after all, a *lot* of ipython images up on Docker already.
>> Additionally, there is a LONG thread on the software carpentry mailing list
>>
>> There are some open questions from me:
>>
>> * Who should the user be inside the container?
>>   * Should they be able to install more packages?
>>
>
> I don't think the user accessing the instance needs to be able to install
> custom packages. (It would be nice this functionality was available, but it
> is not critical for running workshops.)
>
> However, administrators need to have the ability to create custom images
> based off the docker base image that have additional packages.
>
>
>>   * In a chef cookbook and deployment setup, I just made the user named
>> ipynb.
>>
>> * Is it ok to force HTTPS?
>>   * If we do, we're stuck with self signed certs or requiring them to
>> provide them on run
>>   * This isn't really the right way to compose applications with Docker
>> and probably makes things problematic for anyone building on top of ours
>>
>> * Who is our target audience?
>>   * People deploying notebooks across their organization for users?
>>   * Students (and their teachers) wanting access to an easy analytics
>> environment?
>>
>
> My interest is in providing an environment for external users of our
> analysis API, so it somewhat similar to the latter description. Some are in
> academia, but many are not.
>
> * What would you see out of a base image? Where would the value be?
>>
>> * Would people like to see a bleeding latest image that you can spin up
>> to see the current state of the IPython notebook by just running `docker
>> run ipython/notebook:bleeding-latest`?
>>
>> * Do people want a base image that has a large portion of the scipy stack?
>>
>
> Yes, particularly matplotlib and pandas.
>
> Cheers,
> Tyler
>
> I'll stop with these and see where the discussion takes us.
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/c9f3e94f/attachment.html>

From tylere at google.com  Tue Aug  5 01:46:01 2014
From: tylere at google.com (Tyler Erickson)
Date: Mon, 4 Aug 2014 22:46:01 -0700
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
Message-ID: <CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>

On Mon, Aug 4, 2014 at 10:33 PM, Kyle Kelley <rgbkrk at gmail.com> wrote:

> Tyler,
>
> Taking in everyones comments and my opinion as a not-quite-sysadmin:
>
> Base image, ipython/notebook, will
>
> * have just the ipython notebook essentials
> * up to date (as of image build) version of pip (or conda... plead for one
> or the other please!)
>

conda, please :)


> * be installed for all users (all users inside the container)
>   * This means that any images inheriting from it can set up their own
> user setup, install more packages
>

Sounds fine to me.


> * default cmd that runs the notebook server with some default parameters
>   * This can be overwritten by a Dockerfile that uses FROM ipython/notebook
>
> My personal take on this is that we could have an ipy user, with notebooks
> at /home/ipy/notebooks.
>

Sounds reasonable. I have no strong opinion.


> Derivative image, ipython/scipystack (please suggest other names), could be
>
> * numpy
> * scipy
> * matplotlib
> * pandas
> * scikit-learn
>

I would be happy with that list.

Cheers,
Tyler



> For an IHaskell base image, I'd prefer the namespace to actually be
> jupyter/ihaskell in the long run.
>
> There may come a time when we want multiple kernels in one container.
> After all, have you seen what you can do with the current master branch?
>
>
>
>
> Including all the kernels (and stacks) will make a hefty docker image
> though, so I won't progress down that path just yet.
>
>
>
> On Mon, Aug 4, 2014 at 4:52 PM, Tyler Erickson <tylere at google.com> wrote:
>
>> Kyle,
>>
>> I think having an official base Docker instance will be quite useful,
>> especially for running workshops.
>>
>> I added comments for my particular use case below.
>>
>> On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley <rgbkrk at gmail.com> wrote:
>>
>>> Earlier this week we released an experimental base Docker image for the
>>> IPython notebook:
>>>
>>> https://registry.hub.docker.com/u/ipython/notebook/
>>>
>>> As well as a Docker image for the notebook viewer (which will be used in
>>> production soon):
>>>
>>> https://registry.hub.docker.com/u/ipython/nbviewer/
>>>
>>> Yay!
>>>
>>> Several folks have reached out about these images, and I'd like to get a
>>> conversation started amongst those that want to build from a common base.
>>> There are, after all, a *lot* of ipython images up on Docker already.
>>> Additionally, there is a LONG thread on the software carpentry mailing list
>>>
>>> There are some open questions from me:
>>>
>>> * Who should the user be inside the container?
>>>   * Should they be able to install more packages?
>>>
>>
>>  I don't think the user accessing the instance needs to be able to
>> install custom packages. (It would be nice this functionality was
>> available, but it is not critical for running workshops.)
>>
>> However, administrators need to have the ability to create custom images
>> based off the docker base image that have additional packages.
>>
>>
>>>   * In a chef cookbook and deployment setup, I just made the user named
>>> ipynb.
>>>
>>> * Is it ok to force HTTPS?
>>>   * If we do, we're stuck with self signed certs or requiring them to
>>> provide them on run
>>>   * This isn't really the right way to compose applications with Docker
>>> and probably makes things problematic for anyone building on top of ours
>>>
>>> * Who is our target audience?
>>>   * People deploying notebooks across their organization for users?
>>>   * Students (and their teachers) wanting access to an easy analytics
>>> environment?
>>>
>>
>> My interest is in providing an environment for external users of our
>> analysis API, so it somewhat similar to the latter description. Some are in
>> academia, but many are not.
>>
>> * What would you see out of a base image? Where would the value be?
>>>
>>> * Would people like to see a bleeding latest image that you can spin up
>>> to see the current state of the IPython notebook by just running `docker
>>> run ipython/notebook:bleeding-latest`?
>>>
>>> * Do people want a base image that has a large portion of the scipy
>>> stack?
>>>
>>
>> Yes, particularly matplotlib and pandas.
>>
>> Cheers,
>>  Tyler
>>
>> I'll stop with these and see where the discussion takes us.
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
>
> --
> Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140804/e6a00e17/attachment.html>

From wes.turner at gmail.com  Tue Aug  5 02:00:17 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Tue, 5 Aug 2014 01:00:17 -0500
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>
Message-ID: <CACfEFw8LZmsXuvuktB5zNJGucDdGtZ7f+oXmcwj2P_-wBH6RBQ@mail.gmail.com>

On Tue, Aug 5, 2014 at 12:46 AM, Tyler Erickson <tylere at google.com> wrote:

>
> On Mon, Aug 4, 2014 at 10:33 PM, Kyle Kelley <rgbkrk at gmail.com> wrote:
>
>> Tyler,
>>
>> Taking in everyones comments and my opinion as a not-quite-sysadmin:
>>
>> Base image, ipython/notebook, will
>>
>> * have just the ipython notebook essentials
>> * up to date (as of image build) version of pip (or conda... plead for
>> one or the other please!)
>>
>
> conda, please :)
>

+1. http://conda.pydata.org/docs/build.html


>
>
>> * be installed for all users (all users inside the container)
>>   * This means that any images inheriting from it can set up their own
>> user setup, install more packages
>>
>
> Sounds fine to me.
>
>
>> * default cmd that runs the notebook server with some default parameters
>>   * This can be overwritten by a Dockerfile that uses FROM
>> ipython/notebook
>>
>> My personal take on this is that we could have an ipy user, with
>> notebooks at /home/ipy/notebooks.
>>
>
> Sounds reasonable. I have no strong opinion.
>
>
>> Derivative image, ipython/scipystack (please suggest other names), could
>> be
>>
>> * numpy
>> * scipy
>> * matplotlib
>> * pandas
>> * scikit-learn
>>
>
> I would be happy with that list.
>

* [ ] pip (which version?)

* Supervisord? http://supervisord.org/
https://github.com/Supervisor/initscripts

* conda https://pypi.python.org/pypi/conda

* enpkg https://pypi.python.org/pypi/enstaller

* pyenv https://github.com/yyuu/pyenv

* tox, {... build packages }

* { ad infinitum }


>
> Cheers,
> Tyler
>
>
>
>> For an IHaskell base image, I'd prefer the namespace to actually be
>> jupyter/ihaskell in the long run.
>>
>> There may come a time when we want multiple kernels in one container.
>> After all, have you seen what you can do with the current master branch?
>>
>>
>>
>>
>> Including all the kernels (and stacks) will make a hefty docker image
>> though, so I won't progress down that path just yet.
>>
>>
>>
>> On Mon, Aug 4, 2014 at 4:52 PM, Tyler Erickson <tylere at google.com> wrote:
>>
>>> Kyle,
>>>
>>> I think having an official base Docker instance will be quite useful,
>>> especially for running workshops.
>>>
>>> I added comments for my particular use case below.
>>>
>>> On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley <rgbkrk at gmail.com> wrote:
>>>
>>>> Earlier this week we released an experimental base Docker image for the
>>>> IPython notebook:
>>>>
>>>> https://registry.hub.docker.com/u/ipython/notebook/
>>>>
>>>> As well as a Docker image for the notebook viewer (which will be used
>>>> in production soon):
>>>>
>>>> https://registry.hub.docker.com/u/ipython/nbviewer/
>>>>
>>>> Yay!
>>>>
>>>> Several folks have reached out about these images, and I'd like to get
>>>> a conversation started amongst those that want to build from a common base.
>>>> There are, after all, a *lot* of ipython images up on Docker already.
>>>> Additionally, there is a LONG thread on the software carpentry mailing list
>>>>
>>>> There are some open questions from me:
>>>>
>>>> * Who should the user be inside the container?
>>>>   * Should they be able to install more packages?
>>>>
>>>
>>>  I don't think the user accessing the instance needs to be able to
>>> install custom packages. (It would be nice this functionality was
>>> available, but it is not critical for running workshops.)
>>>
>>> However, administrators need to have the ability to create custom images
>>> based off the docker base image that have additional packages.
>>>
>>>
>>>>   * In a chef cookbook and deployment setup, I just made the user named
>>>> ipynb.
>>>>
>>>> * Is it ok to force HTTPS?
>>>>   * If we do, we're stuck with self signed certs or requiring them to
>>>> provide them on run
>>>>   * This isn't really the right way to compose applications with Docker
>>>> and probably makes things problematic for anyone building on top of ours
>>>>
>>>> * Who is our target audience?
>>>>   * People deploying notebooks across their organization for users?
>>>>   * Students (and their teachers) wanting access to an easy analytics
>>>> environment?
>>>>
>>>
>>> My interest is in providing an environment for external users of our
>>> analysis API, so it somewhat similar to the latter description. Some are in
>>> academia, but many are not.
>>>
>>> * What would you see out of a base image? Where would the value be?
>>>>
>>>> * Would people like to see a bleeding latest image that you can spin up
>>>> to see the current state of the IPython notebook by just running `docker
>>>> run ipython/notebook:bleeding-latest`?
>>>>
>>>> * Do people want a base image that has a large portion of the scipy
>>>> stack?
>>>>
>>>
>>> Yes, particularly matplotlib and pandas.
>>>
>>> Cheers,
>>>  Tyler
>>>
>>> I'll stop with these and see where the discussion takes us.
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>>
>> --
>> Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/4d50f578/attachment.html>

From rgbkrk at gmail.com  Tue Aug  5 11:53:24 2014
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Tue, 5 Aug 2014 10:53:24 -0500
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CACfEFw8LZmsXuvuktB5zNJGucDdGtZ7f+oXmcwj2P_-wBH6RBQ@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>
	<CACfEFw8LZmsXuvuktB5zNJGucDdGtZ7f+oXmcwj2P_-wBH6RBQ@mail.gmail.com>
Message-ID: <CA+tbMaUJO2qTi+dwysVEpk2f7jpKfpQKFTtXNe7LQ1QyE2+TcA@mail.gmail.com>

Wes,

Responses inline.


On Tue, Aug 5, 2014 at 1:00 AM, Wes Turner <wes.turner at gmail.com> wrote:

>
> On Tue, Aug 5, 2014 at 12:46 AM, Tyler Erickson <tylere at google.com> wrote:
>
>>
>> On Mon, Aug 4, 2014 at 10:33 PM, Kyle Kelley <rgbkrk at gmail.com> wrote:
>>
>>> Tyler,
>>>
>>> Taking in everyones comments and my opinion as a not-quite-sysadmin:
>>>
>>> Base image, ipython/notebook, will
>>>
>>> * have just the ipython notebook essentials
>>> * up to date (as of image build) version of pip (or conda... plead for
>>> one or the other please!)
>>>
>>
>> conda, please :)
>>
>
> +1. http://conda.pydata.org/docs/build.html
>
>
>>
>>
>>> * be installed for all users (all users inside the container)
>>>   * This means that any images inheriting from it can set up their own
>>> user setup, install more packages
>>>
>>
>> Sounds fine to me.
>>
>>
>>>  * default cmd that runs the notebook server with some default
>>> parameters
>>>   * This can be overwritten by a Dockerfile that uses FROM
>>> ipython/notebook
>>>
>>> My personal take on this is that we could have an ipy user, with
>>> notebooks at /home/ipy/notebooks.
>>>
>>
>> Sounds reasonable. I have no strong opinion.
>>
>>
>>>  Derivative image, ipython/scipystack (please suggest other names),
>>> could be
>>>
>>> * numpy
>>> * scipy
>>> * matplotlib
>>> * pandas
>>> * scikit-learn
>>>
>>
>> I would be happy with that list.
>>
>
> * [ ] pip (which version?)
>
> * Supervisord? http://supervisord.org/
> https://github.com/Supervisor/initscripts
>

Why? The process can be kept up inside the container and managed outside of
the container. Does this grant something else?


>
> * conda https://pypi.python.org/pypi/conda
>
> * enpkg https://pypi.python.org/pypi/enstaller
>

That's just making things more complicated. Would you be able to pip
install or conda install this if you really need it?


>
>
> * pyenv https://github.com/yyuu/pyenv
>

No thank you. Switching kernels will be done inside the notebook itself.


>
> * tox, {... build packages }
>

tox is out of scope to me. What is your reasoning and audience?


>
> * { ad infinitum }
>
>
>>
>> Cheers,
>> Tyler
>>
>>
>>
>>> For an IHaskell base image, I'd prefer the namespace to actually be
>>> jupyter/ihaskell in the long run.
>>>
>>> There may come a time when we want multiple kernels in one container.
>>> After all, have you seen what you can do with the current master branch?
>>>
>>>
>>>
>>>
>>> Including all the kernels (and stacks) will make a hefty docker image
>>> though, so I won't progress down that path just yet.
>>>
>>>
>>>
>>> On Mon, Aug 4, 2014 at 4:52 PM, Tyler Erickson <tylere at google.com>
>>> wrote:
>>>
>>>> Kyle,
>>>>
>>>> I think having an official base Docker instance will be quite useful,
>>>> especially for running workshops.
>>>>
>>>> I added comments for my particular use case below.
>>>>
>>>> On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley <rgbkrk at gmail.com> wrote:
>>>>
>>>>> Earlier this week we released an experimental base Docker image for
>>>>> the IPython notebook:
>>>>>
>>>>> https://registry.hub.docker.com/u/ipython/notebook/
>>>>>
>>>>> As well as a Docker image for the notebook viewer (which will be used
>>>>> in production soon):
>>>>>
>>>>> https://registry.hub.docker.com/u/ipython/nbviewer/
>>>>>
>>>>> Yay!
>>>>>
>>>>> Several folks have reached out about these images, and I'd like to get
>>>>> a conversation started amongst those that want to build from a common base.
>>>>> There are, after all, a *lot* of ipython images up on Docker already.
>>>>> Additionally, there is a LONG thread on the software carpentry mailing list
>>>>>
>>>>> There are some open questions from me:
>>>>>
>>>>> * Who should the user be inside the container?
>>>>>   * Should they be able to install more packages?
>>>>>
>>>>
>>>>  I don't think the user accessing the instance needs to be able to
>>>> install custom packages. (It would be nice this functionality was
>>>> available, but it is not critical for running workshops.)
>>>>
>>>> However, administrators need to have the ability to create custom
>>>> images based off the docker base image that have additional packages.
>>>>
>>>>
>>>>>   * In a chef cookbook and deployment setup, I just made the user
>>>>> named ipynb.
>>>>>
>>>>> * Is it ok to force HTTPS?
>>>>>   * If we do, we're stuck with self signed certs or requiring them to
>>>>> provide them on run
>>>>>   * This isn't really the right way to compose applications with
>>>>> Docker and probably makes things problematic for anyone building on top of
>>>>> ours
>>>>>
>>>>> * Who is our target audience?
>>>>>   * People deploying notebooks across their organization for users?
>>>>>   * Students (and their teachers) wanting access to an easy analytics
>>>>> environment?
>>>>>
>>>>
>>>> My interest is in providing an environment for external users of our
>>>> analysis API, so it somewhat similar to the latter description. Some are in
>>>> academia, but many are not.
>>>>
>>>> * What would you see out of a base image? Where would the value be?
>>>>>
>>>>> * Would people like to see a bleeding latest image that you can spin
>>>>> up to see the current state of the IPython notebook by just running `docker
>>>>> run ipython/notebook:bleeding-latest`?
>>>>>
>>>>> * Do people want a base image that has a large portion of the scipy
>>>>> stack?
>>>>>
>>>>
>>>> Yes, particularly matplotlib and pandas.
>>>>
>>>> Cheers,
>>>>  Tyler
>>>>
>>>> I'll stop with these and see where the discussion takes us.
>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/a2c21d03/attachment.html>

From andrew.gibiansky at gmail.com  Tue Aug  5 12:01:06 2014
From: andrew.gibiansky at gmail.com (Andrew Gibiansky)
Date: Tue, 5 Aug 2014 09:01:06 -0700
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CA+tbMaUJO2qTi+dwysVEpk2f7jpKfpQKFTtXNe7LQ1QyE2+TcA@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>
	<CACfEFw8LZmsXuvuktB5zNJGucDdGtZ7f+oXmcwj2P_-wBH6RBQ@mail.gmail.com>
	<CA+tbMaUJO2qTi+dwysVEpk2f7jpKfpQKFTtXNe7LQ1QyE2+TcA@mail.gmail.com>
Message-ID: <CAF-LYKLu_LTZ-LucXuRsNH2dpmjcS=Whs9oDsoHDDcBCn-x-8A@mail.gmail.com>

Pip please :) Conda is fairly non-standard in the Python ecosystem in my
experience; it's common for data analysis and IPython sort of things, but
pip is the more common packaging standard. Though pip is less powerful than
conda, it has the benefits that *every* Python developer knows its use, and
I think it's much more appropriate for the base IPython notebook docker
file.

Again, this is from the view that I want the IPython notebook docker build
as a *base* for something non-Python related; so I *just* want the bare
essentials required for IPython itself, and nothing more. Conda seems like
it is adding more than you really need, whereas pip does that. It may make
sense to have the Python-based derived notebook (the one that includes
scipy and so on) to use Conda, though. (Not sure how well pip and conda
mix.)

-- Andrew


On Tue, Aug 5, 2014 at 8:53 AM, Kyle Kelley <rgbkrk at gmail.com> wrote:

> Wes,
>
> Responses inline.
>
>
> On Tue, Aug 5, 2014 at 1:00 AM, Wes Turner <wes.turner at gmail.com> wrote:
>
>>
>> On Tue, Aug 5, 2014 at 12:46 AM, Tyler Erickson <tylere at google.com>
>> wrote:
>>
>>>
>>> On Mon, Aug 4, 2014 at 10:33 PM, Kyle Kelley <rgbkrk at gmail.com> wrote:
>>>
>>>> Tyler,
>>>>
>>>> Taking in everyones comments and my opinion as a not-quite-sysadmin:
>>>>
>>>> Base image, ipython/notebook, will
>>>>
>>>> * have just the ipython notebook essentials
>>>> * up to date (as of image build) version of pip (or conda... plead for
>>>> one or the other please!)
>>>>
>>>
>>> conda, please :)
>>>
>>
>> +1. http://conda.pydata.org/docs/build.html
>>
>>
>>>
>>>
>>>> * be installed for all users (all users inside the container)
>>>>   * This means that any images inheriting from it can set up their own
>>>> user setup, install more packages
>>>>
>>>
>>> Sounds fine to me.
>>>
>>>
>>>>  * default cmd that runs the notebook server with some default
>>>> parameters
>>>>   * This can be overwritten by a Dockerfile that uses FROM
>>>> ipython/notebook
>>>>
>>>> My personal take on this is that we could have an ipy user, with
>>>> notebooks at /home/ipy/notebooks.
>>>>
>>>
>>> Sounds reasonable. I have no strong opinion.
>>>
>>>
>>>>  Derivative image, ipython/scipystack (please suggest other names),
>>>> could be
>>>>
>>>> * numpy
>>>> * scipy
>>>> * matplotlib
>>>> * pandas
>>>> * scikit-learn
>>>>
>>>
>>> I would be happy with that list.
>>>
>>
>> * [ ] pip (which version?)
>>
>> * Supervisord? http://supervisord.org/
>> https://github.com/Supervisor/initscripts
>>
>
> Why? The process can be kept up inside the container and managed outside
> of the container. Does this grant something else?
>
>
>>
>> * conda https://pypi.python.org/pypi/conda
>>
>> * enpkg https://pypi.python.org/pypi/enstaller
>>
>
> That's just making things more complicated. Would you be able to pip
> install or conda install this if you really need it?
>
>
>>
>>
>> * pyenv https://github.com/yyuu/pyenv
>>
>
> No thank you. Switching kernels will be done inside the notebook itself.
>
>
>>
>> * tox, {... build packages }
>>
>
> tox is out of scope to me. What is your reasoning and audience?
>
>
>>
>> * { ad infinitum }
>>
>>
>>>
>>> Cheers,
>>> Tyler
>>>
>>>
>>>
>>>> For an IHaskell base image, I'd prefer the namespace to actually be
>>>> jupyter/ihaskell in the long run.
>>>>
>>>> There may come a time when we want multiple kernels in one container.
>>>> After all, have you seen what you can do with the current master branch?
>>>>
>>>>
>>>>
>>>>
>>>> Including all the kernels (and stacks) will make a hefty docker image
>>>> though, so I won't progress down that path just yet.
>>>>
>>>>
>>>>
>>>> On Mon, Aug 4, 2014 at 4:52 PM, Tyler Erickson <tylere at google.com>
>>>> wrote:
>>>>
>>>>> Kyle,
>>>>>
>>>>> I think having an official base Docker instance will be quite useful,
>>>>> especially for running workshops.
>>>>>
>>>>> I added comments for my particular use case below.
>>>>>
>>>>> On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley <rgbkrk at gmail.com> wrote:
>>>>>
>>>>>> Earlier this week we released an experimental base Docker image for
>>>>>> the IPython notebook:
>>>>>>
>>>>>> https://registry.hub.docker.com/u/ipython/notebook/
>>>>>>
>>>>>> As well as a Docker image for the notebook viewer (which will be used
>>>>>> in production soon):
>>>>>>
>>>>>> https://registry.hub.docker.com/u/ipython/nbviewer/
>>>>>>
>>>>>> Yay!
>>>>>>
>>>>>> Several folks have reached out about these images, and I'd like to
>>>>>> get a conversation started amongst those that want to build from a common
>>>>>> base. There are, after all, a *lot* of ipython images up on Docker already.
>>>>>> Additionally, there is a LONG thread on the software carpentry mailing list
>>>>>>
>>>>>> There are some open questions from me:
>>>>>>
>>>>>> * Who should the user be inside the container?
>>>>>>   * Should they be able to install more packages?
>>>>>>
>>>>>
>>>>>  I don't think the user accessing the instance needs to be able to
>>>>> install custom packages. (It would be nice this functionality was
>>>>> available, but it is not critical for running workshops.)
>>>>>
>>>>> However, administrators need to have the ability to create custom
>>>>> images based off the docker base image that have additional packages.
>>>>>
>>>>>
>>>>>>   * In a chef cookbook and deployment setup, I just made the user
>>>>>> named ipynb.
>>>>>>
>>>>>> * Is it ok to force HTTPS?
>>>>>>   * If we do, we're stuck with self signed certs or requiring them to
>>>>>> provide them on run
>>>>>>   * This isn't really the right way to compose applications with
>>>>>> Docker and probably makes things problematic for anyone building on top of
>>>>>> ours
>>>>>>
>>>>>> * Who is our target audience?
>>>>>>   * People deploying notebooks across their organization for users?
>>>>>>   * Students (and their teachers) wanting access to an easy analytics
>>>>>> environment?
>>>>>>
>>>>>
>>>>> My interest is in providing an environment for external users of our
>>>>> analysis API, so it somewhat similar to the latter description. Some are in
>>>>> academia, but many are not.
>>>>>
>>>>> * What would you see out of a base image? Where would the value be?
>>>>>>
>>>>>> * Would people like to see a bleeding latest image that you can spin
>>>>>> up to see the current state of the IPython notebook by just running `docker
>>>>>> run ipython/notebook:bleeding-latest`?
>>>>>>
>>>>>> * Do people want a base image that has a large portion of the scipy
>>>>>> stack?
>>>>>>
>>>>>
>>>>> Yes, particularly matplotlib and pandas.
>>>>>
>>>>> Cheers,
>>>>>  Tyler
>>>>>
>>>>> I'll stop with these and see where the discussion takes us.
>>>>>>
>>>>>> _______________________________________________
>>>>>> IPython-dev mailing list
>>>>>> IPython-dev at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com
>>>> )
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
>
> --
> Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/972075d5/attachment.html>

From jsw at fnal.gov  Tue Aug  5 12:11:38 2014
From: jsw at fnal.gov (Jon Wilson)
Date: Tue, 5 Aug 2014 11:11:38 -0500
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CAF-LYKLu_LTZ-LucXuRsNH2dpmjcS=Whs9oDsoHDDcBCn-x-8A@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>	<CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>	<CACfEFw8LZmsXuvuktB5zNJGucDdGtZ7f+oXmcwj2P_-wBH6RBQ@mail.gmail.com>	<CA+tbMaUJO2qTi+dwysVEpk2f7jpKfpQKFTtXNe7LQ1QyE2+TcA@mail.gmail.com>
	<CAF-LYKLu_LTZ-LucXuRsNH2dpmjcS=Whs9oDsoHDDcBCn-x-8A@mail.gmail.com>
Message-ID: <53E1023A.7030704@fnal.gov>

If any substantial fraction of your users will want 
scipy/numpy/matplotlib, I would (almost, see below) recommend conda. 
Conda was, as I understand it, created because pip left too many 
barriers in place against the use of scipy/numpy etc.  Specifically, 
experience indicated that many people who might otherwise have casually 
investigated scientific python tools did not do so because pip required 
them to have a proper FORTRAN development environment set up, and they 
did not wish to figure out how to do this.

Conda distributes binaries rather than exclusively source, which is an 
effective way around this sort of problem.

OTOH, a pure-python package that is hosted on PyPI (and therefore 
installable via pip) can (usually) be trivially made into a conda 
package via `conda skeleton pypi <package-name>`.  So making pip-style 
packages tends to get you conda packages for almost free.

pip +1

Regards,
Jon

On 08/05/2014 11:01 AM, Andrew Gibiansky wrote:
> Pip please :) Conda is fairly non-standard in the Python ecosystem in my
> experience; it's common for data analysis and IPython sort of things,
> but pip is the more common packaging standard. Though pip is less
> powerful than conda, it has the benefits that /every/ Python developer
> knows its use, and I think it's much more appropriate for the base
> IPython notebook docker file.
>
> Again, this is from the view that I want the IPython notebook docker
> build as a /base/ for something non-Python related; so I /just/ want the
> bare essentials required for IPython itself, and nothing more. Conda
> seems like it is adding more than you really need, whereas pip does
> that. It may make sense to have the Python-based derived notebook (the
> one that includes scipy and so on) to use Conda, though. (Not sure how
> well pip and conda mix.)
>
> -- Andrew
>
>


From bussonniermatthias at gmail.com  Tue Aug  5 13:36:02 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Tue, 5 Aug 2014 19:36:02 +0200
Subject: [IPython-dev] Docker IPython
In-Reply-To: <53E1023A.7030704@fnal.gov>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>	<CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>	<CACfEFw8LZmsXuvuktB5zNJGucDdGtZ7f+oXmcwj2P_-wBH6RBQ@mail.gmail.com>	<CA+tbMaUJO2qTi+dwysVEpk2f7jpKfpQKFTtXNe7LQ1QyE2+TcA@mail.gmail.com>
	<CAF-LYKLu_LTZ-LucXuRsNH2dpmjcS=Whs9oDsoHDDcBCn-x-8A@mail.gmail.com>
	<53E1023A.7030704@fnal.gov>
Message-ID: <1D91D3C5-7C9C-45BD-B2C3-4127A90CC7A4@gmail.com>


Le 5 ao?t 2014 ? 18:11, Jon Wilson <jsw at fnal.gov> a ?crit :

> If any substantial fraction of your users will want 
> scipy/numpy/matplotlib, I would (almost, see below) recommend conda. 

Adrew?s IHaskell users will definitively not want the scipy stack, they want Haskell things. 


> Conda was, as I understand it, created because pip left too many 
> barriers in place against the use of scipy/numpy etc.  Specifically, 
> experience indicated that many people who might otherwise have casually 
> investigated scientific python tools did not do so because pip required 
> them to have a proper FORTRAN development environment set up, and they 
> did not wish to figure out how to do this.
> 
> Conda distributes binaries rather than exclusively source, which is an 
> effective way around this sort of problem.
> 
> OTOH, a pure-python package that is hosted on PyPI (and therefore 
> installable via pip) can (usually) be trivially made into a conda 
> package via `conda skeleton pypi <package-name>`.  So making pip-style 
> packages tends to get you conda packages for almost free.

I suppose this ? easy ? way to make PyPi package from conda package explain 
why  continuum package are outdated by more that a year using pip :-) 

Even if conda seem great, I still feel sad that there is little effort to help fixing 
python packaging (no, replacing is not fixing), but I understand that starting 
from scratch might be easier. Installig SciPy was much more a pain even a few
month ago than now.

Keep also  in mind than miniconda will be smaller if you decide to use it.

I would also reming you that with 3.0, the IPython notebook can start many kernels in different languages, 
so nothing prevent you from installing the notebook using pip, and having a conda kernel.
(it is even possible with 2.x)

So +1 for pip which should be enough, or even maybe Julia Taylor PPA if compatible. 
? 
M



From rgbkrk at gmail.com  Tue Aug  5 14:09:14 2014
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Tue, 5 Aug 2014 13:09:14 -0500
Subject: [IPython-dev] Docker IPython
In-Reply-To: <1D91D3C5-7C9C-45BD-B2C3-4127A90CC7A4@gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>
	<CACfEFw8LZmsXuvuktB5zNJGucDdGtZ7f+oXmcwj2P_-wBH6RBQ@mail.gmail.com>
	<CA+tbMaUJO2qTi+dwysVEpk2f7jpKfpQKFTtXNe7LQ1QyE2+TcA@mail.gmail.com>
	<CAF-LYKLu_LTZ-LucXuRsNH2dpmjcS=Whs9oDsoHDDcBCn-x-8A@mail.gmail.com>
	<53E1023A.7030704@fnal.gov>
	<1D91D3C5-7C9C-45BD-B2C3-4127A90CC7A4@gmail.com>
Message-ID: <CA+tbMaXHBhzcp4A1GzMsEci_0bDUB2Q-=OcCMo9er2qBSS1Y1w@mail.gmail.com>

I'm definitely on the pip for base image camp. conda can be installed after
as part of a separate image. We also don't need to manage multiple
environments. It is one container after all. Derivatives can have
components.


On Tue, Aug 5, 2014 at 12:36 PM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

>
> Le 5 ao?t 2014 ? 18:11, Jon Wilson <jsw at fnal.gov> a ?crit :
>
> > If any substantial fraction of your users will want
> > scipy/numpy/matplotlib, I would (almost, see below) recommend conda.
>
> Adrew?s IHaskell users will definitively not want the scipy stack, they
> want Haskell things.
>
>
> > Conda was, as I understand it, created because pip left too many
> > barriers in place against the use of scipy/numpy etc.  Specifically,
> > experience indicated that many people who might otherwise have casually
> > investigated scientific python tools did not do so because pip required
> > them to have a proper FORTRAN development environment set up, and they
> > did not wish to figure out how to do this.
> >
> > Conda distributes binaries rather than exclusively source, which is an
> > effective way around this sort of problem.
> >
> > OTOH, a pure-python package that is hosted on PyPI (and therefore
> > installable via pip) can (usually) be trivially made into a conda
> > package via `conda skeleton pypi <package-name>`.  So making pip-style
> > packages tends to get you conda packages for almost free.
>
> I suppose this ? easy ? way to make PyPi package from conda package explain
> why  continuum package are outdated by more that a year using pip :-)
>
> Even if conda seem great, I still feel sad that there is little effort to
> help fixing
> python packaging (no, replacing is not fixing), but I understand that
> starting
> from scratch might be easier. Installig SciPy was much more a pain even a
> few
> month ago than now.
>
> Keep also  in mind than miniconda will be smaller if you decide to use it.
>
> I would also reming you that with 3.0, the IPython notebook can start many
> kernels in different languages,
> so nothing prevent you from installing the notebook using pip, and having
> a conda kernel.
> (it is even possible with 2.x)
>
> So +1 for pip which should be enough, or even maybe Julia Taylor PPA if
> compatible.
> ?
> M
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/08c42a30/attachment.html>

From takowl at gmail.com  Tue Aug  5 14:11:25 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 5 Aug 2014 11:11:25 -0700
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
Message-ID: <CAOvn4qjrxk9jXrkqZNujC9jdAJOirS23foaF9gCE5mpZqPcFFQ@mail.gmail.com>

On 4 August 2014 22:33, Kyle Kelley <rgbkrk at gmail.com> wrote:

> Derivative image, ipython/scipystack (please suggest other names), could be
>
> * numpy
> * scipy
> * matplotlib
> * pandas
> * scikit-learn
>

If you call it scipystack, please put Sympy and nose in as well - a long
and heated discussion last year settled on a core set of packages that
distributions describing themselves as shipping the Scipy Stack should
include:

http://www.scipy.org/stackspec.html

As for packaging systems, I'd lean towards conda - it seems more in line
with our Jupyter-ish idea of cross-language tools for scientific computing.
Also, while pip can now handle wheels, you can't put wheels for Linux on
PyPI, so installing extension modules into a docker container with pip is
still not very practical.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/4f374c62/attachment.html>

From tylere at google.com  Tue Aug  5 14:12:19 2014
From: tylere at google.com (Tyler Erickson)
Date: Tue, 5 Aug 2014 11:12:19 -0700
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CA+tbMaXHBhzcp4A1GzMsEci_0bDUB2Q-=OcCMo9er2qBSS1Y1w@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>
	<CACfEFw8LZmsXuvuktB5zNJGucDdGtZ7f+oXmcwj2P_-wBH6RBQ@mail.gmail.com>
	<CA+tbMaUJO2qTi+dwysVEpk2f7jpKfpQKFTtXNe7LQ1QyE2+TcA@mail.gmail.com>
	<CAF-LYKLu_LTZ-LucXuRsNH2dpmjcS=Whs9oDsoHDDcBCn-x-8A@mail.gmail.com>
	<53E1023A.7030704@fnal.gov>
	<1D91D3C5-7C9C-45BD-B2C3-4127A90CC7A4@gmail.com>
	<CA+tbMaXHBhzcp4A1GzMsEci_0bDUB2Q-=OcCMo9er2qBSS1Y1w@mail.gmail.com>
Message-ID: <CAKAkUogz+o0FyegQfrxfWq4pHKWZe9F-+CWsembTn3OiPUtM8w@mail.gmail.com>

I agree that pip is useful, because PyPI has packages that are not
available in conda, and often has more recent versions. But it doesn't need
to be an either/or choice, because conda plays well with pip (at least in
my experience). After installing conda, I usually use it to install pip
("conda install pip") and then pick and choose from both the conda repo and
PyPI.


On Tue, Aug 5, 2014 at 11:09 AM, Kyle Kelley <rgbkrk at gmail.com> wrote:

> I'm definitely on the pip for base image camp. conda can be installed
> after as part of a separate image. We also don't need to manage multiple
> environments. It is one container after all. Derivatives can have
> components.
>
>
> On Tue, Aug 5, 2014 at 12:36 PM, Matthias Bussonnier <
> bussonniermatthias at gmail.com> wrote:
>
>>
>> Le 5 ao?t 2014 ? 18:11, Jon Wilson <jsw at fnal.gov> a ?crit :
>>
>> > If any substantial fraction of your users will want
>> > scipy/numpy/matplotlib, I would (almost, see below) recommend conda.
>>
>> Adrew?s IHaskell users will definitively not want the scipy stack, they
>> want Haskell things.
>>
>>
>> > Conda was, as I understand it, created because pip left too many
>> > barriers in place against the use of scipy/numpy etc.  Specifically,
>> > experience indicated that many people who might otherwise have casually
>> > investigated scientific python tools did not do so because pip required
>> > them to have a proper FORTRAN development environment set up, and they
>> > did not wish to figure out how to do this.
>> >
>> > Conda distributes binaries rather than exclusively source, which is an
>> > effective way around this sort of problem.
>> >
>> > OTOH, a pure-python package that is hosted on PyPI (and therefore
>> > installable via pip) can (usually) be trivially made into a conda
>> > package via `conda skeleton pypi <package-name>`.  So making pip-style
>> > packages tends to get you conda packages for almost free.
>>
>> I suppose this ? easy ? way to make PyPi package from conda package
>> explain
>> why  continuum package are outdated by more that a year using pip :-)
>>
>> Even if conda seem great, I still feel sad that there is little effort to
>> help fixing
>> python packaging (no, replacing is not fixing), but I understand that
>> starting
>> from scratch might be easier. Installig SciPy was much more a pain even a
>> few
>> month ago than now.
>>
>> Keep also  in mind than miniconda will be smaller if you decide to use it.
>>
>> I would also reming you that with 3.0, the IPython notebook can start
>> many kernels in different languages,
>> so nothing prevent you from installing the notebook using pip, and having
>> a conda kernel.
>> (it is even possible with 2.x)
>>
>> So +1 for pip which should be enough, or even maybe Julia Taylor PPA if
>> compatible.
>> ?
>> M
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/8165bac9/attachment.html>

From fperez.net at gmail.com  Tue Aug  5 15:07:05 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 5 Aug 2014 12:07:05 -0700
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CAOvn4qjrxk9jXrkqZNujC9jdAJOirS23foaF9gCE5mpZqPcFFQ@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAOvn4qjrxk9jXrkqZNujC9jdAJOirS23foaF9gCE5mpZqPcFFQ@mail.gmail.com>
Message-ID: <CAHAreOo1FemdXuHLdEp2NPui-GVnqAJwgFQ_T25Kc3X45QXw_g@mail.gmail.com>

On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver <takowl at gmail.com> wrote:

> If you call it scipystack, please put Sympy and nose in as well - a long
> and heated discussion last year settled on a core set of packages that
> distributions describing themselves as shipping the Scipy Stack should
> include:
>
> http://www.scipy.org/stackspec.html
>

+1. Let's use the 'scipy stack' term as per the above.

I actually would vote for the 'extended' stack, that includes a few
packages that in the real world lots of people do use/need, like
sklearn/image. Here's a slightly outdated version of that:

https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends?slide=8

Today I'd probably drop Mayav/PyTables (too specialized) but add Seaborn
(lightweight, easy, plays beautifully with pandas).

That would make a very compelling image out of the box, with minimal
additional complexity (while there's extension code in those, none of them
are harder than scipy itself).

Cheers,

f

-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/8e0956f5/attachment.html>

From wes.turner at gmail.com  Tue Aug  5 16:46:56 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Tue, 5 Aug 2014 15:46:56 -0500
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CA+tbMaUJO2qTi+dwysVEpk2f7jpKfpQKFTtXNe7LQ1QyE2+TcA@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>
	<CACfEFw8LZmsXuvuktB5zNJGucDdGtZ7f+oXmcwj2P_-wBH6RBQ@mail.gmail.com>
	<CA+tbMaUJO2qTi+dwysVEpk2f7jpKfpQKFTtXNe7LQ1QyE2+TcA@mail.gmail.com>
Message-ID: <CACfEFw-B2Q4Jqu75Ha9FyZ+KwZgMgnwJ15qJ1mUmpQ5P7t6-+g@mail.gmail.com>

>> * Supervisord? http://supervisord.org/
>> https://github.com/Supervisor/initscripts
>
>
> Why? The process can be kept up inside the container and managed outside of
> the container. Does this grant something else?

* platform init (init.d, systemd)
* autorestart (not handled by init)
* log management ("supervisorctl tail ipnb")

* strip_ansi escape sequences from logs

Just a suggestion.

>
>>
>>
>> * conda https://pypi.python.org/pypi/conda
>>
>> * enpkg https://pypi.python.org/pypi/enstaller
>
>
> That's just making things more complicated. Would you be able to pip install
> or conda install this if you really need it?

One could install either conda or enpkg in addition to pip.

https://us.pycon.org/2014/sponsors/

>
>>
>>
>>
>> * pyenv https://github.com/yyuu/pyenv
>
>
> No thank you. Switching kernels will be done inside the notebook itself.

Pivot_root should not be possible in userspace, over SSH or TLS.

In the back of my mind, I don't know why I am implicitly adding
"IPython testing" to your list of Use Cases.

Is this a Python 2 or a Python 3 container? Are the OS packages still
out of date? Tox also helps with multiple versions of Python.

>
>>
>>
>> * tox, {... build packages }
>
>
> tox is out of scope to me. What is your reasoning and audience?

Testing, development

>
>>
>>
>> * { ad infinitum }
>>

I've got a first draft of an ipython-formula for saltstack (which also
depends on ZMQ and Jinja2) which should work with any sort of
virtualization.

Managing the TLS CA / server / browser certs with salt appears even
easier than https://bitbucket.org/westurner/certtool/src/23307af8e54be08786f1d301c55d266d8a8918ec/certtool
.

As usual, a hard part appears to be what to install with:

* system pkgs
* pip packages as root
* pip packages into ~/.local
* pip into a ~/env virtualenv
* conda into a conda environment

Setting download-cache in pip.conf may be helpful.

Is a "Multi-User IPython notebook server with a scipy stack" within
scope for this container?

* Are they all named 'ipy'?
* SSH authorized_keys?
* Web based SSH?
* NFS / Google Drive / git-annex / Dropbox / PyFS (FUSE) mounts for
notebook storage? A Docker volume?

Is it easier to manage resource quotas and bursting with each user in
a separate container?

While log-shipping is unique in every environment, ephemeral
containers present additional challenges.

... "Running a public IPython Notebook service for teaching in
university courses"
http://www.reddit.com/r/IPython/comments/1q3sql/running_a_public_ipython_notebook_service_for/#cdds3be


From aron at ahmadia.net  Tue Aug  5 17:33:27 2014
From: aron at ahmadia.net (Aron Ahmadia)
Date: Tue, 5 Aug 2014 17:33:27 -0400
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CACfEFw-B2Q4Jqu75Ha9FyZ+KwZgMgnwJ15qJ1mUmpQ5P7t6-+g@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>
	<CACfEFw8LZmsXuvuktB5zNJGucDdGtZ7f+oXmcwj2P_-wBH6RBQ@mail.gmail.com>
	<CA+tbMaUJO2qTi+dwysVEpk2f7jpKfpQKFTtXNe7LQ1QyE2+TcA@mail.gmail.com>
	<CACfEFw-B2Q4Jqu75Ha9FyZ+KwZgMgnwJ15qJ1mUmpQ5P7t6-+g@mail.gmail.com>
Message-ID: <CAPhiW4i_q6ODuiX6EorfHiNDa-b7Ly9zXjYQjVhCnuF6Bpxy-Q@mail.gmail.com>

Long thread, but I'm also in favor of conda over pip for the reasons
mentioned above, but primarily that it has a better chance of installing a
user's needed Python dependency than pip due to its ability to install
binary dependencies.

A
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/7752167e/attachment.html>

From damianavila at gmail.com  Tue Aug  5 17:50:47 2014
From: damianavila at gmail.com (=?UTF-8?Q?Dami=C3=A1n_Avila?=)
Date: Tue, 5 Aug 2014 18:50:47 -0300
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CAPhiW4i_q6ODuiX6EorfHiNDa-b7Ly9zXjYQjVhCnuF6Bpxy-Q@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAKAkUoiD-oUm6=JeaUUhzG2jvG8ECgek5wbS+BY2ems1GFRBkQ@mail.gmail.com>
	<CACfEFw8LZmsXuvuktB5zNJGucDdGtZ7f+oXmcwj2P_-wBH6RBQ@mail.gmail.com>
	<CA+tbMaUJO2qTi+dwysVEpk2f7jpKfpQKFTtXNe7LQ1QyE2+TcA@mail.gmail.com>
	<CACfEFw-B2Q4Jqu75Ha9FyZ+KwZgMgnwJ15qJ1mUmpQ5P7t6-+g@mail.gmail.com>
	<CAPhiW4i_q6ODuiX6EorfHiNDa-b7Ly9zXjYQjVhCnuF6Bpxy-Q@mail.gmail.com>
Message-ID: <CAH+mRR3vkb7y5WrpzVSPkOHmg2f7p5S+TtJGaCeLQkN_tYf9uA@mail.gmail.com>

Conflict of interest aside ;-) I think conda is a better option right
now... I share the thought exposed by Thomas...


2014-08-05 18:33 GMT-03:00 Aron Ahmadia <aron at ahmadia.net>:

> Long thread, but I'm also in favor of conda over pip for the reasons
> mentioned above, but primarily that it has a better chance of installing a
> user's needed Python dependency than pip due to its ability to install
> binary dependencies.
>
> A
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
*Dami?n*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/59654b18/attachment.html>

From takowl at gmail.com  Tue Aug  5 20:38:31 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 5 Aug 2014 17:38:31 -0700
Subject: [IPython-dev] Fwd: Gateway Computing Environments workshop at SC14
 - submissions due August 27
In-Reply-To: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net>
References: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net>
Message-ID: <CAOvn4qg8rt6DkE7p-4GZ5nttMxXt+N+s5+iogZ6Edtwjp=zsXA@mail.gmail.com>

Are any of us going to Supercomputing this year? The science gateway people
are having a workshop there, and it's a category that IPython roughly falls
into, if someone wants to go and talk to them.

Thomas

---------- Forwarded message ----------
From: Science Gateways Institute <info at sciencegateways.org>
Date: 4 August 2014 14:22
Subject: Gateway Computing Environments workshop at SC14 - submissions due
August 27
To: Thomas Kluyver <takowl at gmail.com>


         Workshop call for submissions    View this email in your browser
<http://us6.campaign-archive2.com/?u=52c619f5272025e56bf9829f5&id=1782a391d2&e=3b55774f99>

Please spread the word about our *Gateway Computing Environments workshop
at SC14, *http://sciencegateways.org/upcoming-events/gce14/
<http://sciencegateways.us6.list-manage1.com/track/click?u=52c619f5272025e56bf9829f5&id=1da30c2c90&e=3b55774f99>
.

*Our half-day workshop will be the morning of Friday, November 21* (nice
timing for a weekend in New Orleans). Proceedings will be published in the
ACM digital library and we will once again co-publish extended
contributions in a special journal issue of CCPE (
http://www.cc-pe.net/journalinfo/
<http://sciencegateways.us6.list-manage1.com/track/click?u=52c619f5272025e56bf9829f5&id=5566a5105d&e=3b55774f99>,
in partnership with the International Workshop on Science Gateways [IWSG]).

Charlie Catlett will be our keynote speaker. Charlie is doing some very
interesting work in data analysis with the City of Chicago?gateways for
cities, if you will?through his work as director of the Urban Center for
Computation and Data https://urbanccd.org/
<http://sciencegateways.us6.list-manage2.com/track/click?u=52c619f5272025e56bf9829f5&id=1c7939bb34&e=3b55774f99>
.

We will feature lightning talks, a poster session, and a panel in addition
to Charlie's talk. *Interested presenters should submit abstracts (2?4
pages) by August 27 *(submission details on our website
http://sciencegateways.org/upcoming-events/gce14/
<http://sciencegateways.us6.list-manage.com/track/click?u=52c619f5272025e56bf9829f5&id=3af9729e02&e=3b55774f99>).
We invite the submission of papers related to various aspects of science
gateways?their design, but also their use and impact in research and
education. Though the event will be at SC, this is very definitely not just
for gateways accessing supercomputers. Please help continue the tradition
of fine speakers and attendees at this workshop series.
           You are receiving this email because you asked to be informed
about the activities of the Science Gateways Institute or you have
participated in an event associated with the Institute. These updates will
be infrequent, so we hope that you'll continue to stay in touch with us!

*Copyright ? 2014 Science Gateways Institute, All rights reserved.*

*Our mailing address is:*
Science Gateways Institute
University of California at San Diego
9500 Gilman Dr
San Diego, CA 92093-0505

Add us to your address book
<http://sciencegateways.us6.list-manage2.com/vcard?u=52c619f5272025e56bf9829f5&id=4922a07d54>


unsubscribe from this list
<http://sciencegateways.us6.list-manage1.com/unsubscribe?u=52c619f5272025e56bf9829f5&id=4922a07d54&e=3b55774f99&c=1782a391d2>
update subscription preferences
<http://sciencegateways.us6.list-manage1.com/profile?u=52c619f5272025e56bf9829f5&id=4922a07d54&e=3b55774f99>


[image: Email Marketing Powered by MailChimp]
<http://www.mailchimp.com/monkey-rewards/?utm_source=freemium_newsletter&utm_medium=email&utm_campaign=monkey_rewards&aid=52c619f5272025e56bf9829f5&afl=1>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/823b8550/attachment.html>

From dave.hirschfeld at gmail.com  Tue Aug  5 21:40:56 2014
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Wed, 6 Aug 2014 01:40:56 +0000 (UTC)
Subject: [IPython-dev] Docker IPython
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAOvn4qjrxk9jXrkqZNujC9jdAJOirS23foaF9gCE5mpZqPcFFQ@mail.gmail.com>
	<CAHAreOo1FemdXuHLdEp2NPui-GVnqAJwgFQ_T25Kc3X45QXw_g@mail.gmail.com>
Message-ID: <loom.20140806T022919-466@post.gmane.org>

Fernando Perez <fperez.net <at> gmail.com> writes:

> 
> 
> 
> On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver <takowl <at> gmail.com> 
wrote:If you call it scipystack, please put Sympy and nose in as well - a 
long and heated discussion last year settled on a core set of packages 
that distributions describing themselves as shipping the Scipy Stack 
should include:http://www.scipy.org/stackspec.html
> 
> 
> +1. Let's use the 'scipy stack' term as per the above.
> 
> 
> 
> 
> I actually would vote for the 'extended' stack, that includes a few 
packages that in the real world lots of people do use/need, like 
sklearn/image. Here's a slightly outdated version of that:
> 
> 
> https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends?
slide=8
> 
> 
> Today I'd probably drop Mayav/PyTables (too specialized) but add 
Seaborn (lightweight, easy, plays beautifully with pandas).
> 
> That would make a very compelling image out of the box, with minimal 
additional complexity (while there's extension code in those, none of 
them are harder than scipy itself).
> 
> 
> 
> 
> Cheers,
> 
> f-- Fernando Perez ( <at> fperez_org; http://fperez.org)

PyTables could be handy as it's an optional dependency for pandas which 
IIUC allows you to store pandas objects in hdf5 via the HDFStore which is 
becoming a very common way to store pandas objects.

-Dave






From ellisonbg at gmail.com  Tue Aug  5 23:54:21 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 5 Aug 2014 20:54:21 -0700
Subject: [IPython-dev] Fwd: Gateway Computing Environments workshop at
 SC14 - submissions due August 27
In-Reply-To: <CAOvn4qg8rt6DkE7p-4GZ5nttMxXt+N+s5+iogZ6Edtwjp=zsXA@mail.gmail.com>
References: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net>
	<CAOvn4qg8rt6DkE7p-4GZ5nttMxXt+N+s5+iogZ6Edtwjp=zsXA@mail.gmail.com>
Message-ID: <CAH4pYpSzq7yWkg96u52uNjpTMJr6uyvSaW5Ag-f9BVnyhX8cDQ@mail.gmail.com>

I can't go myself, Brian


On Tue, Aug 5, 2014 at 5:38 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> Are any of us going to Supercomputing this year? The science gateway
> people are having a workshop there, and it's a category that IPython
> roughly falls into, if someone wants to go and talk to them.
>
> Thomas
>
> ---------- Forwarded message ----------
> From: Science Gateways Institute <info at sciencegateways.org>
> Date: 4 August 2014 14:22
> Subject: Gateway Computing Environments workshop at SC14 - submissions due
> August 27
> To: Thomas Kluyver <takowl at gmail.com>
>
>
>          Workshop call for submissions    View this email in your browser
> <http://us6.campaign-archive2.com/?u=52c619f5272025e56bf9829f5&id=1782a391d2&e=3b55774f99>
>
> Please spread the word about our *Gateway Computing Environments workshop
> at SC14, *http://sciencegateways.org/upcoming-events/gce14/
> <http://sciencegateways.us6.list-manage1.com/track/click?u=52c619f5272025e56bf9829f5&id=1da30c2c90&e=3b55774f99>
> .
>
> *Our half-day workshop will be the morning of Friday, November 21* (nice
> timing for a weekend in New Orleans). Proceedings will be published in the
> ACM digital library and we will once again co-publish extended
> contributions in a special journal issue of CCPE (
> http://www.cc-pe.net/journalinfo/
> <http://sciencegateways.us6.list-manage1.com/track/click?u=52c619f5272025e56bf9829f5&id=5566a5105d&e=3b55774f99>,
> in partnership with the International Workshop on Science Gateways [IWSG]).
>
> Charlie Catlett will be our keynote speaker. Charlie is doing some very
> interesting work in data analysis with the City of Chicago?gateways for
> cities, if you will?through his work as director of the Urban Center for
> Computation and Data https://urbanccd.org/
> <http://sciencegateways.us6.list-manage2.com/track/click?u=52c619f5272025e56bf9829f5&id=1c7939bb34&e=3b55774f99>
> .
>
> We will feature lightning talks, a poster session, and a panel in addition
> to Charlie's talk. *Interested presenters should submit abstracts (2?4
> pages) by August 27 *(submission details on our website
> http://sciencegateways.org/upcoming-events/gce14/
> <http://sciencegateways.us6.list-manage.com/track/click?u=52c619f5272025e56bf9829f5&id=3af9729e02&e=3b55774f99>).
> We invite the submission of papers related to various aspects of science
> gateways?their design, but also their use and impact in research and
> education. Though the event will be at SC, this is very definitely not just
> for gateways accessing supercomputers. Please help continue the tradition
> of fine speakers and attendees at this workshop series.
>            You are receiving this email because you asked to be informed
> about the activities of the Science Gateways Institute or you have
> participated in an event associated with the Institute. These updates will
> be infrequent, so we hope that you'll continue to stay in touch with us!
>
> *Copyright ? 2014 Science Gateways Institute, All rights reserved.*
>
> *Our mailing address is:*
> Science Gateways Institute
> University of California at San Diego
> 9500 Gilman Dr
> San Diego, CA 92093-0505
>
> Add us to your address book
> <http://sciencegateways.us6.list-manage2.com/vcard?u=52c619f5272025e56bf9829f5&id=4922a07d54>
>
>
> unsubscribe from this list
> <http://sciencegateways.us6.list-manage1.com/unsubscribe?u=52c619f5272025e56bf9829f5&id=4922a07d54&e=3b55774f99&c=1782a391d2>
> update subscription preferences
> <http://sciencegateways.us6.list-manage1.com/profile?u=52c619f5272025e56bf9829f5&id=4922a07d54&e=3b55774f99>
>
>
> [image: Email Marketing Powered by MailChimp]
> <http://www.mailchimp.com/monkey-rewards/?utm_source=freemium_newsletter&utm_medium=email&utm_campaign=monkey_rewards&aid=52c619f5272025e56bf9829f5&afl=1>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/237a592c/attachment.html>

From matthewturk at gmail.com  Wed Aug  6 00:05:19 2014
From: matthewturk at gmail.com (Matthew Turk)
Date: Tue, 5 Aug 2014 23:05:19 -0500
Subject: [IPython-dev] Fwd: Gateway Computing Environments workshop at
 SC14 - submissions due August 27
In-Reply-To: <CAOvn4qg8rt6DkE7p-4GZ5nttMxXt+N+s5+iogZ6Edtwjp=zsXA@mail.gmail.com>
References: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net>
	<CAOvn4qg8rt6DkE7p-4GZ5nttMxXt+N+s5+iogZ6Edtwjp=zsXA@mail.gmail.com>
Message-ID: <CALO3=5EXV-MiH5ZmnQzeASw5Ti2FbYY=0VUOLdCW9zZ-bQKp4A@mail.gmail.com>

For what it's worth, I think this would be a really excellent community to
connect with; SciGAP and the Science Gateway conceptualization project are
both really exciting projects. A few weeks ago IPython came up on their
mailing list and was roundly applauded.

I'll be at SC14 (the WSSSPE meeting will be great!) and while I wouldn't be
a good "IPython representative," I'd be happy to serve as an "IPython
advocate."

Matt
On Aug 5, 2014 7:40 PM, "Thomas Kluyver" <takowl at gmail.com> wrote:

> Are any of us going to Supercomputing this year? The science gateway
> people are having a workshop there, and it's a category that IPython
> roughly falls into, if someone wants to go and talk to them.
>
> Thomas
>
> ---------- Forwarded message ----------
> From: Science Gateways Institute <info at sciencegateways.org>
> Date: 4 August 2014 14:22
> Subject: Gateway Computing Environments workshop at SC14 - submissions due
> August 27
> To: Thomas Kluyver <takowl at gmail.com>
>
>
>          Workshop call for submissions    View this email in your browser
> <http://us6.campaign-archive2.com/?u=52c619f5272025e56bf9829f5&id=1782a391d2&e=3b55774f99>
>
> Please spread the word about our *Gateway Computing Environments workshop
> at SC14, *http://sciencegateways.org/upcoming-events/gce14/
> <http://sciencegateways.us6.list-manage1.com/track/click?u=52c619f5272025e56bf9829f5&id=1da30c2c90&e=3b55774f99>
> .
>
> *Our half-day workshop will be the morning of Friday, November 21* (nice
> timing for a weekend in New Orleans). Proceedings will be published in the
> ACM digital library and we will once again co-publish extended
> contributions in a special journal issue of CCPE (
> http://www.cc-pe.net/journalinfo/
> <http://sciencegateways.us6.list-manage1.com/track/click?u=52c619f5272025e56bf9829f5&id=5566a5105d&e=3b55774f99>,
> in partnership with the International Workshop on Science Gateways [IWSG]).
>
> Charlie Catlett will be our keynote speaker. Charlie is doing some very
> interesting work in data analysis with the City of Chicago?gateways for
> cities, if you will?through his work as director of the Urban Center for
> Computation and Data https://urbanccd.org/
> <http://sciencegateways.us6.list-manage2.com/track/click?u=52c619f5272025e56bf9829f5&id=1c7939bb34&e=3b55774f99>
> .
>
> We will feature lightning talks, a poster session, and a panel in addition
> to Charlie's talk. *Interested presenters should submit abstracts (2?4
> pages) by August 27 *(submission details on our website
> http://sciencegateways.org/upcoming-events/gce14/
> <http://sciencegateways.us6.list-manage.com/track/click?u=52c619f5272025e56bf9829f5&id=3af9729e02&e=3b55774f99>).
> We invite the submission of papers related to various aspects of science
> gateways?their design, but also their use and impact in research and
> education. Though the event will be at SC, this is very definitely not just
> for gateways accessing supercomputers. Please help continue the tradition
> of fine speakers and attendees at this workshop series.
>            You are receiving this email because you asked to be informed
> about the activities of the Science Gateways Institute or you have
> participated in an event associated with the Institute. These updates will
> be infrequent, so we hope that you'll continue to stay in touch with us!
>
> *Copyright ? 2014 Science Gateways Institute, All rights reserved.*
>
> *Our mailing address is:*
> Science Gateways Institute
> University of California at San Diego
> 9500 Gilman Dr
> San Diego, CA 92093-0505
>
> Add us to your address book
> <http://sciencegateways.us6.list-manage2.com/vcard?u=52c619f5272025e56bf9829f5&id=4922a07d54>
>
>
> unsubscribe from this list
> <http://sciencegateways.us6.list-manage1.com/unsubscribe?u=52c619f5272025e56bf9829f5&id=4922a07d54&e=3b55774f99&c=1782a391d2>
> update subscription preferences
> <http://sciencegateways.us6.list-manage1.com/profile?u=52c619f5272025e56bf9829f5&id=4922a07d54&e=3b55774f99>
>
>
> [image: Email Marketing Powered by MailChimp]
> <http://www.mailchimp.com/monkey-rewards/?utm_source=freemium_newsletter&utm_medium=email&utm_campaign=monkey_rewards&aid=52c619f5272025e56bf9829f5&afl=1>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/21780369/attachment.html>

From goxberry at gmail.com  Wed Aug  6 00:34:34 2014
From: goxberry at gmail.com (Geoff Oxberry)
Date: Tue, 5 Aug 2014 21:34:34 -0700
Subject: [IPython-dev] Docker IPython
In-Reply-To: <loom.20140806T022919-466@post.gmane.org>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAOvn4qjrxk9jXrkqZNujC9jdAJOirS23foaF9gCE5mpZqPcFFQ@mail.gmail.com>
	<CAHAreOo1FemdXuHLdEp2NPui-GVnqAJwgFQ_T25Kc3X45QXw_g@mail.gmail.com>
	<loom.20140806T022919-466@post.gmane.org>
Message-ID: <CAHC8rGtZ+gWHLZ6+5eMjmyQ4F3j0RFpNdgToxozq6q7mrg_gBQ@mail.gmail.com>

In case chiming in matters, I'm vehemently opposed to conda in a base
image, because I think conda would be a great package manager if it didn't
try to do too much (as of Anaconda 1.9, it was also managing Python
version, and managing/creating virtualenvs). If it's in an "extended"
derivative image, that's fine by me. I agree that pip is more standard, and
conda is good for installing things (and only good for installing things).


On Tue, Aug 5, 2014 at 6:40 PM, Dave Hirschfeld <dave.hirschfeld at gmail.com>
wrote:

> Fernando Perez <fperez.net <at> gmail.com> writes:
>
> >
> >
> >
> > On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver <takowl <at> gmail.com>
> wrote:If you call it scipystack, please put Sympy and nose in as well - a
> long and heated discussion last year settled on a core set of packages
> that distributions describing themselves as shipping the Scipy Stack
> should include:http://www.scipy.org/stackspec.html
> >
> >
> > +1. Let's use the 'scipy stack' term as per the above.
> >
> >
> >
> >
> > I actually would vote for the 'extended' stack, that includes a few
> packages that in the real world lots of people do use/need, like
> sklearn/image. Here's a slightly outdated version of that:
> >
> >
> > https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends?
> slide=8
> >
> >
> > Today I'd probably drop Mayav/PyTables (too specialized) but add
> Seaborn (lightweight, easy, plays beautifully with pandas).
> >
> > That would make a very compelling image out of the box, with minimal
> additional complexity (while there's extension code in those, none of
> them are harder than scipy itself).
> >
> >
> >
> >
> > Cheers,
> >
> > f-- Fernando Perez ( <at> fperez_org; http://fperez.org)
>
> PyTables could be handy as it's an optional dependency for pandas which
> IIUC allows you to store pandas objects in hdf5 via the HDFStore which is
> becoming a very common way to store pandas objects.
>
> -Dave
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Geoffrey Oxberry, Ph.D., E.I.T.
goxberry at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140805/49a061af/attachment.html>

From ellisonbg at gmail.com  Wed Aug  6 00:46:42 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 5 Aug 2014 21:46:42 -0700
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CAHC8rGtZ+gWHLZ6+5eMjmyQ4F3j0RFpNdgToxozq6q7mrg_gBQ@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAOvn4qjrxk9jXrkqZNujC9jdAJOirS23foaF9gCE5mpZqPcFFQ@mail.gmail.com>
	<CAHAreOo1FemdXuHLdEp2NPui-GVnqAJwgFQ_T25Kc3X45QXw_g@mail.gmail.com>
	<loom.20140806T022919-466@post.gmane.org>
	<CAHC8rGtZ+gWHLZ6+5eMjmyQ4F3j0RFpNdgToxozq6q7mrg_gBQ@mail.gmail.com>
Message-ID: <CAH4pYpQ3pbKp0M8KyucmX3aC6o0-8qA+LnubcZt-L8VvYkrG-A@mail.gmail.com>

I think a more useful way of approaching this would be to ask the
following: what layers of docker images are useful? Here are some
ideas:

Base: ipython and its deps, installed using system packages or pip
Base+SciPy: Base plus the extended scipy stack installed using system
packages or pip
Base+R kernel
Base+Julia kernel
...
Conda/Anaconda version: ipython and deps installed using conda. We
should talk to Continuum about this

Question, do docker images have ways of pulling from multiple parents?

Cheers,

Brian

On Tue, Aug 5, 2014 at 9:34 PM, Geoff Oxberry <goxberry at gmail.com> wrote:
> In case chiming in matters, I'm vehemently opposed to conda in a base image,
> because I think conda would be a great package manager if it didn't try to
> do too much (as of Anaconda 1.9, it was also managing Python version, and
> managing/creating virtualenvs). If it's in an "extended" derivative image,
> that's fine by me. I agree that pip is more standard, and conda is good for
> installing things (and only good for installing things).
>
>
> On Tue, Aug 5, 2014 at 6:40 PM, Dave Hirschfeld <dave.hirschfeld at gmail.com>
> wrote:
>>
>> Fernando Perez <fperez.net <at> gmail.com> writes:
>>
>> >
>> >
>> >
>> > On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver <takowl <at> gmail.com>
>> wrote:If you call it scipystack, please put Sympy and nose in as well - a
>> long and heated discussion last year settled on a core set of packages
>> that distributions describing themselves as shipping the Scipy Stack
>> should include:http://www.scipy.org/stackspec.html
>> >
>> >
>> > +1. Let's use the 'scipy stack' term as per the above.
>> >
>> >
>> >
>> >
>> > I actually would vote for the 'extended' stack, that includes a few
>> packages that in the real world lots of people do use/need, like
>> sklearn/image. Here's a slightly outdated version of that:
>> >
>> >
>> > https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends?
>> slide=8
>> >
>> >
>> > Today I'd probably drop Mayav/PyTables (too specialized) but add
>> Seaborn (lightweight, easy, plays beautifully with pandas).
>> >
>> > That would make a very compelling image out of the box, with minimal
>> additional complexity (while there's extension code in those, none of
>> them are harder than scipy itself).
>> >
>> >
>> >
>> >
>> > Cheers,
>> >
>> > f-- Fernando Perez ( <at> fperez_org; http://fperez.org)
>>
>> PyTables could be handy as it's an optional dependency for pandas which
>> IIUC allows you to store pandas objects in hdf5 via the HDFStore which is
>> becoming a very common way to store pandas objects.
>>
>> -Dave
>>
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> --
> Geoffrey Oxberry, Ph.D., E.I.T.
> goxberry at gmail.com
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From wes.turner at gmail.com  Wed Aug  6 01:24:26 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Wed, 6 Aug 2014 00:24:26 -0500
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CAH4pYpQ3pbKp0M8KyucmX3aC6o0-8qA+LnubcZt-L8VvYkrG-A@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAOvn4qjrxk9jXrkqZNujC9jdAJOirS23foaF9gCE5mpZqPcFFQ@mail.gmail.com>
	<CAHAreOo1FemdXuHLdEp2NPui-GVnqAJwgFQ_T25Kc3X45QXw_g@mail.gmail.com>
	<loom.20140806T022919-466@post.gmane.org>
	<CAHC8rGtZ+gWHLZ6+5eMjmyQ4F3j0RFpNdgToxozq6q7mrg_gBQ@mail.gmail.com>
	<CAH4pYpQ3pbKp0M8KyucmX3aC6o0-8qA+LnubcZt-L8VvYkrG-A@mail.gmail.com>
Message-ID: <CACfEFw-P67t4FUdZDO0mxh6cvMczHL3VqReJS4UA=h2xkroEnQ@mail.gmail.com>

> Question, do docker images have ways of pulling from multiple parents?

AFAIU, docker layers image filesystems. There is no notion of types or
interfaces identity.

* https://docs.docker.com/terms/layer/
* https://docs.docker.com/reference/builder/#from
* https://en.wikipedia.org/wiki/Union_mount
* http://www.projectatomic.io/docs/filesystems/ (a helpful overview)

> Base: ipython and its deps, installed using system packages or pip

* Pending a current PPA,
* I'd build with *current* pip.

> Base+SciPy: Base plus the extended scipy stack installed using system
> packages or pip

Package names:
* http://www.scipy.org/stackspec.html

Package versions:

* http://docs.continuum.io/anaconda/pkg-docs.html
* https://www.enthought.com/products/canopy/package-index/

> Base+R kernel
> Base+Julia kernel

AFAIK, these could be built as conda packages.

....

**MathJax**

Any chance we could get an `install_mathjax` in there?

Something like this:
https://gist.github.com/westurner/3265445#file-setup_pandas_notebook_deb-sh-L78

...

https://registry.hub.docker.com/u/ipython/notebook/dockerfile

--
Wes Turner


On Tue, Aug 5, 2014 at 11:46 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> I think a more useful way of approaching this would be to ask the
> following: what layers of docker images are useful? Here are some
> ideas:
>
> Base: ipython and its deps, installed using system packages or pip
> Base+SciPy: Base plus the extended scipy stack installed using system
> packages or pip
> Base+R kernel
> Base+Julia kernel
> ...
> Conda/Anaconda version: ipython and deps installed using conda. We
> should talk to Continuum about this
>
> Question, do docker images have ways of pulling from multiple parents?
>
> Cheers,
>
> Brian
>
> On Tue, Aug 5, 2014 at 9:34 PM, Geoff Oxberry <goxberry at gmail.com> wrote:
>> In case chiming in matters, I'm vehemently opposed to conda in a base image,
>> because I think conda would be a great package manager if it didn't try to
>> do too much (as of Anaconda 1.9, it was also managing Python version, and
>> managing/creating virtualenvs). If it's in an "extended" derivative image,
>> that's fine by me. I agree that pip is more standard, and conda is good for
>> installing things (and only good for installing things).
>>
>>
>> On Tue, Aug 5, 2014 at 6:40 PM, Dave Hirschfeld <dave.hirschfeld at gmail.com>
>> wrote:
>>>
>>> Fernando Perez <fperez.net <at> gmail.com> writes:
>>>
>>> >
>>> >
>>> >
>>> > On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver <takowl <at> gmail.com>
>>> wrote:If you call it scipystack, please put Sympy and nose in as well - a
>>> long and heated discussion last year settled on a core set of packages
>>> that distributions describing themselves as shipping the Scipy Stack
>>> should include:http://www.scipy.org/stackspec.html
>>> >
>>> >
>>> > +1. Let's use the 'scipy stack' term as per the above.
>>> >
>>> >
>>> >
>>> >
>>> > I actually would vote for the 'extended' stack, that includes a few
>>> packages that in the real world lots of people do use/need, like
>>> sklearn/image. Here's a slightly outdated version of that:
>>> >
>>> >
>>> > https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends?
>>> slide=8
>>> >
>>> >
>>> > Today I'd probably drop Mayav/PyTables (too specialized) but add
>>> Seaborn (lightweight, easy, plays beautifully with pandas).
>>> >
>>> > That would make a very compelling image out of the box, with minimal
>>> additional complexity (while there's extension code in those, none of
>>> them are harder than scipy itself).
>>> >
>>> >
>>> >
>>> >
>>> > Cheers,
>>> >
>>> > f-- Fernando Perez ( <at> fperez_org; http://fperez.org)
>>>
>>> PyTables could be handy as it's an optional dependency for pandas which
>>> IIUC allows you to store pandas objects in hdf5 via the HDFStore which is
>>> becoming a very common way to store pandas objects.
>>>
>>> -Dave
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>>
>> --
>> Geoffrey Oxberry, Ph.D., E.I.T.
>> goxberry at gmail.com
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From tarun.gaba7 at gmail.com  Wed Aug  6 02:13:22 2014
From: tarun.gaba7 at gmail.com (TARUN GABA)
Date: Wed, 6 Aug 2014 11:43:22 +0530
Subject: [IPython-dev] Expected date for next release
Message-ID: <CAHAono1ZpGJamwGr2yTizAUn+=eXJdQ6r6ZNFan_usrgfS+P7g@mail.gmail.com>

Hi

What is the expected date for the next IPython release?
Are the release dates available on web beforehand, or are they decided
some time before the actual release?

I am working on a package which has IPython (dev) as a dependency,
and I want to base the next release of the package in accordance
with the IPython release.


Thanks

Tarun Gaba
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/3fef453d/attachment.html>

From rgbkrk at gmail.com  Wed Aug  6 02:36:16 2014
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Wed, 6 Aug 2014 01:36:16 -0500
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CAH4pYpQ3pbKp0M8KyucmX3aC6o0-8qA+LnubcZt-L8VvYkrG-A@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAOvn4qjrxk9jXrkqZNujC9jdAJOirS23foaF9gCE5mpZqPcFFQ@mail.gmail.com>
	<CAHAreOo1FemdXuHLdEp2NPui-GVnqAJwgFQ_T25Kc3X45QXw_g@mail.gmail.com>
	<loom.20140806T022919-466@post.gmane.org>
	<CAHC8rGtZ+gWHLZ6+5eMjmyQ4F3j0RFpNdgToxozq6q7mrg_gBQ@mail.gmail.com>
	<CAH4pYpQ3pbKp0M8KyucmX3aC6o0-8qA+LnubcZt-L8VvYkrG-A@mail.gmail.com>
Message-ID: <CA+tbMaUMS4u457i-zGLBSCMBmTGzWmfa8-XToB+HW6RVexSciA@mail.gmail.com>

Brian, that is right in line with what I'd like to see. +1000

> Question, do docker images have ways of pulling from multiple parents?

Docker images can only have a single parent.


On Tue, Aug 5, 2014 at 11:46 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> I think a more useful way of approaching this would be to ask the
> following: what layers of docker images are useful? Here are some
> ideas:
>
> Base: ipython and its deps, installed using system packages or pip
> Base+SciPy: Base plus the extended scipy stack installed using system
> packages or pip
> Base+R kernel
> Base+Julia kernel
> ...
> Conda/Anaconda version: ipython and deps installed using conda. We
> should talk to Continuum about this
>
> Question, do docker images have ways of pulling from multiple parents?
>
> Cheers,
>
> Brian
>
> On Tue, Aug 5, 2014 at 9:34 PM, Geoff Oxberry <goxberry at gmail.com> wrote:
> > In case chiming in matters, I'm vehemently opposed to conda in a base
> image,
> > because I think conda would be a great package manager if it didn't try
> to
> > do too much (as of Anaconda 1.9, it was also managing Python version, and
> > managing/creating virtualenvs). If it's in an "extended" derivative
> image,
> > that's fine by me. I agree that pip is more standard, and conda is good
> for
> > installing things (and only good for installing things).
> >
> >
> > On Tue, Aug 5, 2014 at 6:40 PM, Dave Hirschfeld <
> dave.hirschfeld at gmail.com>
> > wrote:
> >>
> >> Fernando Perez <fperez.net <at> gmail.com> writes:
> >>
> >> >
> >> >
> >> >
> >> > On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver <takowl <at>
> gmail.com>
> >> wrote:If you call it scipystack, please put Sympy and nose in as well -
> a
> >> long and heated discussion last year settled on a core set of packages
> >> that distributions describing themselves as shipping the Scipy Stack
> >> should include:http://www.scipy.org/stackspec.html
> >> >
> >> >
> >> > +1. Let's use the 'scipy stack' term as per the above.
> >> >
> >> >
> >> >
> >> >
> >> > I actually would vote for the 'extended' stack, that includes a few
> >> packages that in the real world lots of people do use/need, like
> >> sklearn/image. Here's a slightly outdated version of that:
> >> >
> >> >
> >> >
> https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends?
> >> slide=8
> >> >
> >> >
> >> > Today I'd probably drop Mayav/PyTables (too specialized) but add
> >> Seaborn (lightweight, easy, plays beautifully with pandas).
> >> >
> >> > That would make a very compelling image out of the box, with minimal
> >> additional complexity (while there's extension code in those, none of
> >> them are harder than scipy itself).
> >> >
> >> >
> >> >
> >> >
> >> > Cheers,
> >> >
> >> > f-- Fernando Perez ( <at> fperez_org; http://fperez.org)
> >>
> >> PyTables could be handy as it's an optional dependency for pandas which
> >> IIUC allows you to store pandas objects in hdf5 via the HDFStore which
> is
> >> becoming a very common way to store pandas objects.
> >>
> >> -Dave
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> >
> >
> > --
> > Geoffrey Oxberry, Ph.D., E.I.T.
> > goxberry at gmail.com
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/ed3ed0e6/attachment.html>

From bjoern.gruening at gmail.com  Wed Aug  6 12:03:03 2014
From: bjoern.gruening at gmail.com (bjoern.gruening@googlemail.com)
Date: Wed, 6 Aug 2014 18:03:03 +0200
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CA+tbMaUMS4u457i-zGLBSCMBmTGzWmfa8-XToB+HW6RVexSciA@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAOvn4qjrxk9jXrkqZNujC9jdAJOirS23foaF9gCE5mpZqPcFFQ@mail.gmail.com>
	<CAHAreOo1FemdXuHLdEp2NPui-GVnqAJwgFQ_T25Kc3X45QXw_g@mail.gmail.com>
	<loom.20140806T022919-466@post.gmane.org>
	<CAHC8rGtZ+gWHLZ6+5eMjmyQ4F3j0RFpNdgToxozq6q7mrg_gBQ@mail.gmail.com>
	<CAH4pYpQ3pbKp0M8KyucmX3aC6o0-8qA+LnubcZt-L8VvYkrG-A@mail.gmail.com>
	<CA+tbMaUMS4u457i-zGLBSCMBmTGzWmfa8-XToB+HW6RVexSciA@mail.gmail.com>
Message-ID: <CAExpnpUbmmjH4M6AnmSf=eQrMCfi+80c2biXJhnYkTmTArVYEQ@mail.gmail.com>

Hi,

as one of the IPython-Galaxy [1] developers we had such a similar
discussion about our IPython-docker image.

I really think we need to define you target audience. Imho, for an IPython
base Image our target audience it quite small. It should deliver a stable
and maintained IPython container. So everyone can build on-top of it.
Later we can build a second container, derived from it, with a full
installed scientific stack for teaching and so on ...

If you include to many tools it will not be used by many developers as base
image because to many peoples will disagree with your decision.

I would vote for a real basic set ... even the scipy stack is really
questionable for a "base image". If we go with the scipy stack than I would
reference the scipy community:

http://www.scipy.org/stackspec.html

Keep it small but stable, care about updates and documentation and let
other grow on top of your image.

Cheers,
Bjoern

[1] https://github.com/bgruening/galaxy-ipython


2014-08-06 8:36 GMT+02:00 Kyle Kelley <rgbkrk at gmail.com>:

> Brian, that is right in line with what I'd like to see. +1000
>
> > Question, do docker images have ways of pulling from multiple parents?
>
> Docker images can only have a single parent.
>
>
> On Tue, Aug 5, 2014 at 11:46 PM, Brian Granger <ellisonbg at gmail.com>
> wrote:
>
>> I think a more useful way of approaching this would be to ask the
>> following: what layers of docker images are useful? Here are some
>> ideas:
>>
>> Base: ipython and its deps, installed using system packages or pip
>> Base+SciPy: Base plus the extended scipy stack installed using system
>> packages or pip
>> Base+R kernel
>> Base+Julia kernel
>> ...
>> Conda/Anaconda version: ipython and deps installed using conda. We
>> should talk to Continuum about this
>>
>> Question, do docker images have ways of pulling from multiple parents?
>>
>> Cheers,
>>
>> Brian
>>
>> On Tue, Aug 5, 2014 at 9:34 PM, Geoff Oxberry <goxberry at gmail.com> wrote:
>> > In case chiming in matters, I'm vehemently opposed to conda in a base
>> image,
>> > because I think conda would be a great package manager if it didn't try
>> to
>> > do too much (as of Anaconda 1.9, it was also managing Python version,
>> and
>> > managing/creating virtualenvs). If it's in an "extended" derivative
>> image,
>> > that's fine by me. I agree that pip is more standard, and conda is good
>> for
>> > installing things (and only good for installing things).
>> >
>> >
>> > On Tue, Aug 5, 2014 at 6:40 PM, Dave Hirschfeld <
>> dave.hirschfeld at gmail.com>
>> > wrote:
>> >>
>> >> Fernando Perez <fperez.net <at> gmail.com> writes:
>> >>
>> >> >
>> >> >
>> >> >
>> >> > On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver <takowl <at>
>> gmail.com>
>> >> wrote:If you call it scipystack, please put Sympy and nose in as well
>> - a
>> >> long and heated discussion last year settled on a core set of packages
>> >> that distributions describing themselves as shipping the Scipy Stack
>> >> should include:http://www.scipy.org/stackspec.html
>> >> >
>> >> >
>> >> > +1. Let's use the 'scipy stack' term as per the above.
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > I actually would vote for the 'extended' stack, that includes a few
>> >> packages that in the real world lots of people do use/need, like
>> >> sklearn/image. Here's a slightly outdated version of that:
>> >> >
>> >> >
>> >> >
>> https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends?
>> >> slide=8
>> >> >
>> >> >
>> >> > Today I'd probably drop Mayav/PyTables (too specialized) but add
>> >> Seaborn (lightweight, easy, plays beautifully with pandas).
>> >> >
>> >> > That would make a very compelling image out of the box, with minimal
>> >> additional complexity (while there's extension code in those, none of
>> >> them are harder than scipy itself).
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > Cheers,
>> >> >
>> >> > f-- Fernando Perez ( <at> fperez_org; http://fperez.org)
>> >>
>> >> PyTables could be handy as it's an optional dependency for pandas which
>> >> IIUC allows you to store pandas objects in hdf5 via the HDFStore which
>> is
>> >> becoming a very common way to store pandas objects.
>> >>
>> >> -Dave
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>> >
>> >
>> > --
>> > Geoffrey Oxberry, Ph.D., E.I.T.
>> > goxberry at gmail.com
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> @ellisonbg on Twitter and GitHub
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/6c1e4c49/attachment.html>

From takowl at gmail.com  Wed Aug  6 13:06:06 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 6 Aug 2014 10:06:06 -0700
Subject: [IPython-dev] Expected date for next release
In-Reply-To: <CAHAono1ZpGJamwGr2yTizAUn+=eXJdQ6r6ZNFan_usrgfS+P7g@mail.gmail.com>
References: <CAHAono1ZpGJamwGr2yTizAUn+=eXJdQ6r6ZNFan_usrgfS+P7g@mail.gmail.com>
Message-ID: <CAOvn4qipheW2Y8vfH1f9zubgadSe1dEEt91ijvgBXGko829iSw@mail.gmail.com>

Hi Tarun,

On 5 August 2014 23:13, TARUN GABA <tarun.gaba7 at gmail.com> wrote:

> What is the expected date for the next IPython release?
> Are the release dates available on web beforehand, or are they decided
> some time before the actual release?
>
> I am working on a package which has IPython (dev) as a dependency,
> and I want to base the next release of the package in accordance
> with the IPython release.
>

The next major release will be some time this autumn. We try to put these
on our roadmap, although it's not always up to date:
https://github.com/ipython/ipython/wiki/Roadmap:-IPython

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/eec853a1/attachment.html>

From fperez.net at gmail.com  Wed Aug  6 14:18:19 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 6 Aug 2014 11:18:19 -0700
Subject: [IPython-dev] Docker IPython
In-Reply-To: <CAH4pYpQ3pbKp0M8KyucmX3aC6o0-8qA+LnubcZt-L8VvYkrG-A@mail.gmail.com>
References: <CA+tbMaXkwDRsyFqp_sAvL5NoiykDxSPnCwk=OcMd8FeqeGU4=w@mail.gmail.com>
	<CAKAkUoizJ+64Kf6ZLAOVBh-uPvnu-sjEnBAYHnhx0CRWj5FSKw@mail.gmail.com>
	<CA+tbMaU=E7P+jNcaaz-2s4Yi26iDURso10BV3iNEmbPYDyvz7w@mail.gmail.com>
	<CAOvn4qjrxk9jXrkqZNujC9jdAJOirS23foaF9gCE5mpZqPcFFQ@mail.gmail.com>
	<CAHAreOo1FemdXuHLdEp2NPui-GVnqAJwgFQ_T25Kc3X45QXw_g@mail.gmail.com>
	<loom.20140806T022919-466@post.gmane.org>
	<CAHC8rGtZ+gWHLZ6+5eMjmyQ4F3j0RFpNdgToxozq6q7mrg_gBQ@mail.gmail.com>
	<CAH4pYpQ3pbKp0M8KyucmX3aC6o0-8qA+LnubcZt-L8VvYkrG-A@mail.gmail.com>
Message-ID: <CAHAreOqkbsokqPxgTNQ0kjVNjXGifRtXW9-WJYxrR=ZyQsij7w@mail.gmail.com>

On Tue, Aug 5, 2014 at 9:46 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Base: ipython and its deps, installed using system packages or pip
> Base+SciPy: Base plus the extended scipy stack installed using system
> packages or pip
> Base+R kernel
> Base+Julia kernel
> ...
> Conda/Anaconda version: ipython and deps installed using conda. We
> should talk to Continuum about this
>

+1

And my vote is that we (ipython/jupyter) only deal with the first one or
two at the most. We should offer an official base image, possibly with the
scipy stack (with apt/pip), and let the rest of the community work on the
derivative ones (by community I mean both folks like R/Julia/Galaxy teams
and Continuum/Enthought/any other company).

We can't over-stretch ourselves too much, and we want a good ecosystem of
well-targeted images. But it's useful to have a base one we maintain, if
nothing else so that we understand what problems may be common to all these
scenarios and we solve them once and correctly.

Cheers

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/0a743073/attachment.html>

From fperez.net at gmail.com  Wed Aug  6 14:20:47 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 6 Aug 2014 11:20:47 -0700
Subject: [IPython-dev] Fwd: Gateway Computing Environments workshop at
 SC14 - submissions due August 27
In-Reply-To: <CALO3=5EXV-MiH5ZmnQzeASw5Ti2FbYY=0VUOLdCW9zZ-bQKp4A@mail.gmail.com>
References: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net>
	<CAOvn4qg8rt6DkE7p-4GZ5nttMxXt+N+s5+iogZ6Edtwjp=zsXA@mail.gmail.com>
	<CALO3=5EXV-MiH5ZmnQzeASw5Ti2FbYY=0VUOLdCW9zZ-bQKp4A@mail.gmail.com>
Message-ID: <CAHAreOoOrw=PPqq=g5w6eeQ4CmDAyGNSfQX4a5wpT7vFe3-M5w@mail.gmail.com>

On Tue, Aug 5, 2014 at 9:05 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> I'll be at SC14 (the WSSSPE meeting will be great!) and while I wouldn't
> be a good "IPython representative," I'd be happy to serve as an "IPython
> advocate."
>
> We can deputize you ;)  I think you know the project well enough that I'm
sure you can do an excellent job conveying what part of the problem space
we try to address. Obviously don't feel pressured into doing yet one more
job, but it would be great if you can pitch in when questions about IPython
arise, and send any further discussions our way.

I can't go this year either, though I probably should. It's a
near-certainty that I'll attend SC'15 though.

Cheers,

f
-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/e3a8e22e/attachment.html>

From matthewturk at gmail.com  Wed Aug  6 14:33:17 2014
From: matthewturk at gmail.com (Matthew Turk)
Date: Wed, 6 Aug 2014 13:33:17 -0500
Subject: [IPython-dev] Fwd: Gateway Computing Environments workshop at
 SC14 - submissions due August 27
In-Reply-To: <CAHAreOoOrw=PPqq=g5w6eeQ4CmDAyGNSfQX4a5wpT7vFe3-M5w@mail.gmail.com>
References: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net>
	<CAOvn4qg8rt6DkE7p-4GZ5nttMxXt+N+s5+iogZ6Edtwjp=zsXA@mail.gmail.com>
	<CALO3=5EXV-MiH5ZmnQzeASw5Ti2FbYY=0VUOLdCW9zZ-bQKp4A@mail.gmail.com>
	<CAHAreOoOrw=PPqq=g5w6eeQ4CmDAyGNSfQX4a5wpT7vFe3-M5w@mail.gmail.com>
Message-ID: <CALO3=5GEJ6vjQL2+xb4AYCMDdpP3oWjFg_ybWawqQkFJaQYXEA@mail.gmail.com>

On Wed, Aug 6, 2014 at 1:20 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>
> On Tue, Aug 5, 2014 at 9:05 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>>
>> I'll be at SC14 (the WSSSPE meeting will be great!) and while I wouldn't
>> be a good "IPython representative," I'd be happy to serve as an "IPython
>> advocate."
>
> We can deputize you ;)  I think you know the project well enough that I'm
> sure you can do an excellent job conveying what part of the problem space we
> try to address. Obviously don't feel pressured into doing yet one more job,
> but it would be great if you can pitch in when questions about IPython
> arise, and send any further discussions our way.
>

Sounds good!

> I can't go this year either, though I probably should. It's a near-certainty
> that I'll attend SC'15 though.
>
> Cheers,
>
> f
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From benjaminrk at gmail.com  Wed Aug  6 17:08:58 2014
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 6 Aug 2014 14:08:58 -0700
Subject: [IPython-dev] Automatically saving notebooks as scripts has
	been removed
In-Reply-To: <CAH2VmmD-HwghBY5GEGyZUn6BCZXc1+f6mnh=YAiM26URuEASsA@mail.gmail.com>
References: <CAH2VmmB45-1pEDkst7Dmq=CaswM7G_eKExzde+2mLOsEjLJr_A@mail.gmail.com>
	<CAOvn4qiq78Cb_fk7N_P5ZuLk80wEFC+SsZmx4RjV=4MnJbdr8A@mail.gmail.com>
	<CAH2VmmD-HwghBY5GEGyZUn6BCZXc1+f6mnh=YAiM26URuEASsA@mail.gmail.com>
Message-ID: <CAHNn8BXZjn=vsVSjKFf2xSwOMr2JX6tYHVzBHKs-Jbm+Hn=Ohw@mail.gmail.com>

The main reason it is removed is that nbformat had a notion of ?save as
.py?, which is removed in IPython 3.0 because this functionality has moved
to nbconvert in IPython 1.0. We could, of course, let the appropriate nbconvert
--to python call be implied by --script instead of the removed Python API,
but I don?t think --script was a good idea in the first place.

So it?s not a question of whether to use nbconvert or not - you have to use
nbconvert to get .py files from .ipynb. It?s just a question of whether you
can *automatically* use nbconvert on save via --script. I think if we want
to support this kind of thing, more generic save/rename hooks are better
than a single special-case flag.

-MinRK
?


On Mon, Aug 4, 2014 at 2:39 PM, Filipe Pires Alvarenga Fernandes <
ocefpaf at gmail.com> wrote:

> Got it.  Thanks for looking into this.
>
>
>
> On Mon, Aug 4, 2014 at 6:18 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
>> I think we may have been a bit too hasty in getting rid of that. We've
>> been talking about it for a while amongst ourselves, because it doesn't fit
>> terribly neatly into our infrastructure, but we didn't formally deprecate
>> it, and I don't think the alternative approaches are quite as convenient
>> yet.
>>
>> I've put this in the agenda for the dev meeting on Thursday.
>>
>> Thanks,
>> Thomas
>>
>>
>> On 4 August 2014 12:55, Filipe Pires Alvarenga Fernandes <
>> ocefpaf at gmail.com> wrote:
>>
>>> Hello IPython devs,
>>>
>>> I was trying the latest github checkout of the IPython source and got
>>> the following message:
>>>
>>> Automatically saving notebooks as scripts has been removed.
>>> Use `ipython nbconvert --to python [notebook]` instead.
>>>
>>> I am wondering why this had to be removed.  Security?
>>>
>>> I find the automatic script saving very helpful.  My work-group usually
>>> commits the `.ipynb` and the `.py` side-by-side to obtain a better diff.  I
>>> know that we can workaround this with nbconvert, but doing that manually is
>>> error prone, and having a script to do that will add another layer of
>>> complexity that we did not have with the automatic conversion.
>>>
>>> Well, if the automatic conversion had to go I understand, but if there
>>> is no reason can you guys consider leaving it there?
>>>
>>> Thanks,
>>>
>>> -Filipe
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/f4895c0d/attachment.html>

From benjaminrk at gmail.com  Wed Aug  6 17:11:36 2014
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 6 Aug 2014 14:11:36 -0700
Subject: [IPython-dev] Errors in ipython startup muted by Notebook?
In-Reply-To: <CA+2k6+xk=jqxEjeoiaTUgJ2oJPEnxkbmiLPK1q5aFM=8pmKOpw@mail.gmail.com>
References: <CA+2k6+xk=jqxEjeoiaTUgJ2oJPEnxkbmiLPK1q5aFM=8pmKOpw@mail.gmail.com>
Message-ID: <CAHNn8BVK_HMdUUHXtHPM+pgM3PYcRUe6bYe+0GARmOuJLptv8w@mail.gmail.com>

Yes, output from the IPython kernel is associated with the input that
caused it via a notion of a parent. In the notebook, each cell causes a
particular set of output. If output is received and not associated with a
cell on the page, it is discarded. Output from startup files has no parent,
and is thus discarded by the notebook. If we had a place to display
orphaned output, we could do this.

We could also consider sending output to sys.__stderr__ when parent=None,
which would mean that kernel output with no parent would at least go to the
terminal in which the kernel was started.

-MinRK
?


On Mon, Aug 4, 2014 at 12:03 PM, Petr Wolf <petr.wolf at gmail.com> wrote:

> Hi,
>
> as described on
> http://stackoverflow.com/questions/24942378/how-to-propagate-errors-out-of-ipython-profile-to-a-notebook,
> I wonder how to handle errors in IPython startup code.
>
> While the IPython console prints any errors right upfront, the Notebook
> app seems to silence them completely.
>
> That can be confusing (as some startup commands may have been executed and
> others may not) and counterproductive (as the information does not seem to
> be logged either).
>
> What is the best way to have errors propagated out of the profile, when
> using nootebook?
>
> Regards,
> Petr
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/9b517440/attachment.html>

From takowl at gmail.com  Wed Aug  6 17:16:34 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 6 Aug 2014 14:16:34 -0700
Subject: [IPython-dev] Errors in ipython startup muted by Notebook?
In-Reply-To: <CAHNn8BVK_HMdUUHXtHPM+pgM3PYcRUe6bYe+0GARmOuJLptv8w@mail.gmail.com>
References: <CA+2k6+xk=jqxEjeoiaTUgJ2oJPEnxkbmiLPK1q5aFM=8pmKOpw@mail.gmail.com>
	<CAHNn8BVK_HMdUUHXtHPM+pgM3PYcRUe6bYe+0GARmOuJLptv8w@mail.gmail.com>
Message-ID: <CAOvn4qiQUxEWR3MCan_+h_FotFSzsAjzDboud2Ew5dO=+1kFRw@mail.gmail.com>

On 6 August 2014 14:11, MinRK <benjaminrk at gmail.com> wrote:

> We could also consider sending output to sys.__stderr__ when parent=None,
> which would mean that kernel output with no parent would at least go to the
> terminal in which the kernel was started.
>

Though we should also consider that not every kernel is launched from a
visible terminal - users on the multiuser server won't be starting from a
terminal, and now that I can double click on notebooks to open them, I'm
often running the notebook without a terminal as well.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/48a67544/attachment.html>

From fperez.net at gmail.com  Wed Aug  6 17:47:17 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 6 Aug 2014 14:47:17 -0700
Subject: [IPython-dev] Errors in ipython startup muted by Notebook?
In-Reply-To: <CAOvn4qiQUxEWR3MCan_+h_FotFSzsAjzDboud2Ew5dO=+1kFRw@mail.gmail.com>
References: <CA+2k6+xk=jqxEjeoiaTUgJ2oJPEnxkbmiLPK1q5aFM=8pmKOpw@mail.gmail.com>
	<CAHNn8BVK_HMdUUHXtHPM+pgM3PYcRUe6bYe+0GARmOuJLptv8w@mail.gmail.com>
	<CAOvn4qiQUxEWR3MCan_+h_FotFSzsAjzDboud2Ew5dO=+1kFRw@mail.gmail.com>
Message-ID: <CAHAreOpLEbJbW28bjE=49=cpTG+wN4mxzhbL7Wu=RZ2Uad_LkA@mail.gmail.com>

On Wed, Aug 6, 2014 at 2:16 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> Though we should also consider that not every kernel is launched from a
> visible terminal - users on the multiuser server won't be starting from a
> terminal, and now that I can double click on notebooks to open them, I'm
> often running the notebook without a terminal as well.
>

Can you see what these give in one of your fully headless kernels?

n [9]: sys.__stderr__.fileno()
Out[9]: 2

In [10]: sys.__stderr__.isatty()
Out[10]: True

In [11]: sys.__stderr__.closed
Out[11]: False

We should be able to detect when stderr isn't meaningfully attached to
anything, and at least redirect orphaned output to a dump log...

Cheers

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/7c69b605/attachment.html>

From takowl at gmail.com  Wed Aug  6 17:49:10 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 6 Aug 2014 14:49:10 -0700
Subject: [IPython-dev] Insecure loading of mathjax
In-Reply-To: <CAOvn4qhVYcyMSGtQS3=fPVz_Huu9=P5vnEsb7U1kTGKsBvPWEA@mail.gmail.com>
References: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>
	<53DC0303.7060500@googlemail.com>
	<20140801214038.GD27567@HbI-OTOH.berkeley.edu>
	<53DC0ECE.6030108@googlemail.com>
	<CAOvn4qgTz0j+ZgSerJS_NSNWvZ=WuLT3q+kRX-2n7aNZOAaG1Q@mail.gmail.com>
	<53DC1918.60404@googlemail.com>
	<CAOvn4qhVYcyMSGtQS3=fPVz_Huu9=P5vnEsb7U1kTGKsBvPWEA@mail.gmail.com>
Message-ID: <CAOvn4qj519LpYeMhZPqO7fn2u6L5tv0-dKdgsSpa-VZJdJL1UA@mail.gmail.com>

My discussions with Mathjax developers led me to post a question on
security.stackexchange, and I am now satisfied that what Cloudflare is
doing with SSL certificates, although it is somewhat out of the ordinary,
does not allow someone controlling one of those sites to MITM requests to
another of them. So if we load Mathjax over HTTPS, we are only trusting
mathjax.org and Cloudflare.

https://github.com/mathjax/MathJax/issues/885
http://security.stackexchange.com/q/64738/53098

Thanks,
Thomas


On 1 August 2014 18:20, Thomas Kluyver <takowl at gmail.com> wrote:

> On 1 August 2014 15:47, Julian Taylor <jtaylor.debian at googlemail.com>
> wrote:
>
>> In this case if someone is able to do a MITM and controls one of the
>> domains that share the certificate with mathjax that person would be
>> able to redirect the request to mathjax.org to his domain and serve any
>> file. The browser would accept this as the certificate is valid for the
>> domain even though its a different one than the request was sent to.
>> Or would the browser complain about the change in any case?
>>
>
> I have opened an issue on Mathjax to work out if this is possible. It's
> probably most concerning for IPython, but if it's possible, it potentially
> affects any site loading Mathjax from the CDN.
>
> https://github.com/mathjax/MathJax/issues/885
>
> Thomas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/3b406a38/attachment.html>

From fperez.net at gmail.com  Wed Aug  6 17:56:24 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 6 Aug 2014 14:56:24 -0700
Subject: [IPython-dev] Insecure loading of mathjax
In-Reply-To: <CAOvn4qj519LpYeMhZPqO7fn2u6L5tv0-dKdgsSpa-VZJdJL1UA@mail.gmail.com>
References: <CA+tbMaX4eO97dqFDf113wGSmbkELXVdpRX8GH=4rjbq+U39Ypg@mail.gmail.com>
	<53DC0303.7060500@googlemail.com>
	<20140801214038.GD27567@HbI-OTOH.berkeley.edu>
	<53DC0ECE.6030108@googlemail.com>
	<CAOvn4qgTz0j+ZgSerJS_NSNWvZ=WuLT3q+kRX-2n7aNZOAaG1Q@mail.gmail.com>
	<53DC1918.60404@googlemail.com>
	<CAOvn4qhVYcyMSGtQS3=fPVz_Huu9=P5vnEsb7U1kTGKsBvPWEA@mail.gmail.com>
	<CAOvn4qj519LpYeMhZPqO7fn2u6L5tv0-dKdgsSpa-VZJdJL1UA@mail.gmail.com>
Message-ID: <CAHAreOpn6hT-x65sr0m7sCt1XDGhWruCSkaBf2omoxK1sB98Fw@mail.gmail.com>

On Wed, Aug 6, 2014 at 2:49 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> My discussions with Mathjax developers led me to post a question on
> security.stackexchange, and I am now satisfied that what Cloudflare is
> doing with SSL certificates, although it is somewhat out of the ordinary,
> does not allow someone controlling one of those sites to MITM requests to
> another of them. So if we load Mathjax over HTTPS, we are only trusting
> mathjax.org and Cloudflare.
>
> https://github.com/mathjax/MathJax/issues/885
> http://security.stackexchange.com/q/64738/53098
>

Excellent and very informative. Thomas, many thanks for following this one
through to the end.  Glad it's not a problem, but it was very useful to
understand exactly how/why.

Cheers,

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/b4099d98/attachment.html>

From takowl at gmail.com  Wed Aug  6 19:25:16 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 6 Aug 2014 16:25:16 -0700
Subject: [IPython-dev] IPython 2.2 release, including security fix
Message-ID: <CAOvn4qjbxBfQXVK+5612xPF-M_jQ7KL5y27VOFvJZcYPyGincQ@mail.gmail.com>

Hi all,

We've just released IPython 2.2, which rolls up a few bugfixes that have
been backported since the last release. This includes a fix for a security
issue posted a few days ago, which could have allowed an attacker who could
intercept network traffic (e.g. on insecure wifi) to intercept a request
for Mathjax and send back malicious javascript which would run in the
notebook. The fix was simply to always load mathjax over HTTPS:
https://github.com/ipython/ipython/pull/6249

You can see the full list of issues and PRs for this release here:
http://ipython.org/ipython-doc/stable/whatsnew/github-stats-2.0.html#issues-closed-in-2-2-0

You can install this release in any of the usual ways.
http://ipython.org/install.html

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/e87f82ed/attachment.html>

From fperez.net at gmail.com  Wed Aug  6 19:32:54 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 6 Aug 2014 16:32:54 -0700
Subject: [IPython-dev] IPython 2.2 release, including security fix
In-Reply-To: <CAOvn4qjbxBfQXVK+5612xPF-M_jQ7KL5y27VOFvJZcYPyGincQ@mail.gmail.com>
References: <CAOvn4qjbxBfQXVK+5612xPF-M_jQ7KL5y27VOFvJZcYPyGincQ@mail.gmail.com>
Message-ID: <CAHAreOqLgmv7T2wg208gpe5QEyPtSESQE-+zQOz1fwd=DUMnnQ@mail.gmail.com>

Thanks everyone!!


On Wed, Aug 6, 2014 at 4:25 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> Hi all,
>
> We've just released IPython 2.2, which rolls up a few bugfixes that have
> been backported since the last release. This includes a fix for a security
> issue posted a few days ago, which could have allowed an attacker who could
> intercept network traffic (e.g. on insecure wifi) to intercept a request
> for Mathjax and send back malicious javascript which would run in the
> notebook. The fix was simply to always load mathjax over HTTPS:
> https://github.com/ipython/ipython/pull/6249
>
> You can see the full list of issues and PRs for this release here:
>
> http://ipython.org/ipython-doc/stable/whatsnew/github-stats-2.0.html#issues-closed-in-2-2-0
>
> You can install this release in any of the usual ways.
> http://ipython.org/install.html
>
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/27d5560b/attachment.html>

From takowl at gmail.com  Wed Aug  6 19:37:54 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 6 Aug 2014 16:37:54 -0700
Subject: [IPython-dev] Errors in ipython startup muted by Notebook?
In-Reply-To: <CAHAreOpLEbJbW28bjE=49=cpTG+wN4mxzhbL7Wu=RZ2Uad_LkA@mail.gmail.com>
References: <CA+2k6+xk=jqxEjeoiaTUgJ2oJPEnxkbmiLPK1q5aFM=8pmKOpw@mail.gmail.com>
	<CAHNn8BVK_HMdUUHXtHPM+pgM3PYcRUe6bYe+0GARmOuJLptv8w@mail.gmail.com>
	<CAOvn4qiQUxEWR3MCan_+h_FotFSzsAjzDboud2Ew5dO=+1kFRw@mail.gmail.com>
	<CAHAreOpLEbJbW28bjE=49=cpTG+wN4mxzhbL7Wu=RZ2Uad_LkA@mail.gmail.com>
Message-ID: <CAOvn4qjwm2eQreVhnGGxad94g9+Y6qKQfqQq752e9tXXENHxuA@mail.gmail.com>

On 6 August 2014 14:47, Fernando Perez <fperez.net at gmail.com> wrote:

> Can you see what these give in one of your fully headless kernels?


In [6]: sys.__stderr__.fileno()
Out[6]: 2

In [7]: sys.__stderr__.isatty()
Out[7]: False

In [8]: sys.__stderr__.closed
Out[8]: False

So the difference is that it's not a TTY, which makes sense. I guess the
process launcher pipes stdout and stderr to a log file somewhere, or just
to /dev/null.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/013fb1bf/attachment.html>

From fperez.net at gmail.com  Wed Aug  6 20:07:38 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 6 Aug 2014 17:07:38 -0700
Subject: [IPython-dev] Errors in ipython startup muted by Notebook?
In-Reply-To: <CAOvn4qjwm2eQreVhnGGxad94g9+Y6qKQfqQq752e9tXXENHxuA@mail.gmail.com>
References: <CA+2k6+xk=jqxEjeoiaTUgJ2oJPEnxkbmiLPK1q5aFM=8pmKOpw@mail.gmail.com>
	<CAHNn8BVK_HMdUUHXtHPM+pgM3PYcRUe6bYe+0GARmOuJLptv8w@mail.gmail.com>
	<CAOvn4qiQUxEWR3MCan_+h_FotFSzsAjzDboud2Ew5dO=+1kFRw@mail.gmail.com>
	<CAHAreOpLEbJbW28bjE=49=cpTG+wN4mxzhbL7Wu=RZ2Uad_LkA@mail.gmail.com>
	<CAOvn4qjwm2eQreVhnGGxad94g9+Y6qKQfqQq752e9tXXENHxuA@mail.gmail.com>
Message-ID: <CAHAreOrR8UjDPpCPSpn_oXG__8Ss5=gdLEJQS5MO18PD_PH=1A@mail.gmail.com>

On Wed, Aug 6, 2014 at 4:37 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> So the difference is that it's not a TTY, which makes sense. I guess the
> process launcher pipes stdout and stderr to a log file somewhere, or just
> to /dev/null.
>

OK, I've tested and isatty() returns True for a normally started notebook
and kernel (opened from a terminal). So that could be the way to test, and
in that case, to use a file-based logger for all orphaned output.

Otherwise, debugging this stuff will be insanely hard :)

-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/4e52a9a7/attachment.html>

From fperez.net at gmail.com  Wed Aug  6 21:37:38 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 6 Aug 2014 18:37:38 -0700
Subject: [IPython-dev] Announcement: Galaxy IPython Integration -
 interactive programming in Galaxy
In-Reply-To: <53DC7F3A.4000601@gmail.com>
References: <53DC7F3A.4000601@gmail.com>
Message-ID: <CAHAreOp_KfRWL_sWAY6zwcgT+xG16ys7zM6NcW3DHxRyW4tKEQ@mail.gmail.com>

Hi Bjorn,

first, many thanks for posting this!

On Fri, Aug 1, 2014 at 11:03 PM, Bj?rn Gr?ning <bjoern.gruening at gmail.com>
wrote:

> Disclaimer: Even though the Ipython notebooks can be stored and reused,
> this plugin will break the Galaxy philosophy of reproducibility, I feel
> personally bad about that,
>

Could you unpack that statement a little bit? We think of IPython as a tool
that can *help* reproducible research, so I'd really like to understand
better what you mean by the above...

Cheers

f

-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140806/2ddb5aa9/attachment.html>

From bjoern.gruening at gmail.com  Thu Aug  7 02:56:54 2014
From: bjoern.gruening at gmail.com (=?windows-1252?Q?Bj=F6rn_Gr=FCning?=)
Date: Thu, 07 Aug 2014 08:56:54 +0200
Subject: [IPython-dev] Announcement: Galaxy IPython Integration -
 interactive programming in Galaxy
In-Reply-To: <CAHAreOp_KfRWL_sWAY6zwcgT+xG16ys7zM6NcW3DHxRyW4tKEQ@mail.gmail.com>
References: <53DC7F3A.4000601@gmail.com>
	<CAHAreOp_KfRWL_sWAY6zwcgT+xG16ys7zM6NcW3DHxRyW4tKEQ@mail.gmail.com>
Message-ID: <53E32336.3010405@gmail.com>

Hi Fernando,

>> Disclaimer: Even though the Ipython notebooks can be stored and reused,
>> this plugin will break the Galaxy philosophy of reproducibility, I feel
>> personally bad about that,
>>
>
> Could you unpack that statement a little bit? We think of IPython as a tool
> that can *help* reproducible research, so I'd really like to understand
> better what you mean by the above...

In Galaxy land we are kind of extreme about reproducibility, which is 
good. With the IPython container we loose a little bit of this, but we 
gain a lot of flexibility for developers.

For example in Galaxy every dataset and tool is versioned. If you 
process your data with matplotlib-1.2.1 we are logging this and you can 
rerun your analysis with the same settings, versions and so on ...

If we are now using the IPython extension, we don't have such a fine 
grained support for packages. We could version the containers and 
restrict the container to install packages ... but I feel like the 
status quo is acceptable, since I assume our users are mostly 
bioinformaticians.

But ... we still maintain the ipynb file. The user can, at any time, 
save the file to the history, we will attach the ipynb as metadata to 
every dataset that is exported. So that you can use the Galaxy reload 
button and you will be back in IPython with all your code.

I think this is already a high level of reproducibility and I'm whining 
on a high level :)

Cheers,
Bjoern

> Cheers
>
> f
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From kb201a at gmail.com  Thu Aug  7 09:16:56 2014
From: kb201a at gmail.com (Keith)
Date: Thu, 7 Aug 2014 09:16:56 -0400
Subject: [IPython-dev] tornado app
Message-ID: <20140807131656.GN24532@hike.sph.harvard.edu>

Hi,

I have an existing tornado application and am thinking
about leveraging (moving it into) ipython notebook.
Currently the app defines its own request handlers and
websocket for communication with the client browser.

I was thinking that the app could generate a notebook
dynamically which would be deployed on the client's
browser.  Then communication to the server could take
place within the notebook's cells.  Assuming this
is possible (I'm new to ipython), are there hooks in
ipython notebook that would allow me to extend tornado
i.e. add my own routes and request handlers etc?

thanks,
Keith


From paddy at paddymullen.com  Thu Aug  7 11:17:53 2014
From: paddy at paddymullen.com (Paddy Mullen)
Date: Thu, 7 Aug 2014 11:17:53 -0400
Subject: [IPython-dev] Url routes for individual cells
Message-ID: <CAFpY3uf65FDRc2AzMKqxVP_fLZv+HZBdZa6Ur2rKLSWo3VvSQg@mail.gmail.com>

I would like to be able to "pop out" analysis cells into separate windows.
 It would be useful if each cell could get it's own url route.

with routes like
http://localhost:8888/notebooks/analysis.ipynb
<http://rx1-qat-stc.global.tudor.com:8888/notebooks/Tap_pl_historgram-Copy0.ipynb>
/cells/23
implementing the pop out feature would be easy.

I think it would help to have a concept of visual cell number vs executed
cell number.  I don't think the notebook has this right now.  It would be
annoying to have to pop out a new window every time you re execute the
original code cell.

I saw this pull request, it's not clear if this is doing something similar
https://github.com/ipython/ipython/pull/6148 IFrames in all places.

Any thoughts on this?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/81510d02/attachment.html>

From paddy at paddymullen.com  Thu Aug  7 11:24:53 2014
From: paddy at paddymullen.com (Paddy Mullen)
Date: Thu, 7 Aug 2014 11:24:53 -0400
Subject: [IPython-dev] Url route to download a notebook and open it
Message-ID: <CAFpY3ueFDvDSTzBY7wJf0ppfZg8i5zRB+LR14x0pDkA=snK=ZA@mail.gmail.com>

It would be useful to have an url route that downloaded a notebook and
opened it.

I could see a route like http://localhost:8888/download/
https://raw.githubusercontent.com/ipython/ipython/master/examples/Notebook/SymPy.ipynb

that would download the SymPy notebook and open it.

This would make it easy to link to generated notebooks.  The alternative
workflow now is to provide a download link, and have the user drag that
file into the notebook filebrowser.

I realize that this would be a security risk, so it would probably be a
feature that was best disabled by default.  A configurable whitelist of
allowable download domains could help a lot.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/a1d49623/attachment.html>

From paddy at paddymullen.com  Thu Aug  7 11:36:33 2014
From: paddy at paddymullen.com (Paddy Mullen)
Date: Thu, 7 Aug 2014 11:36:33 -0400
Subject: [IPython-dev] API to execute cells of notebook
Message-ID: <CAFpY3uc1sKK7g9662q2Kyt6yYdK9wN95iPB6ygWekyMqzs4GGQ@mail.gmail.com>

I have played with the new widgets in the IPython notebook and it's a very
useful tool.  One thing that I would like to do is have the top cell of a
notebook contain widgets that setup global variables, and have the rest of
the cells function as a report based on the global variables.  My idea is
that there would be  a "Run it" button that executes the rest of the cells
in the notebook.

I don't see a way via python to cause the current kernel to execute the
rest of the cells in the notebook.  I could hack something together with
javascript, but that seems problematic and brittle.

Thanks,
Paddy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/af5b6456/attachment.html>

From ellisonbg at gmail.com  Thu Aug  7 11:53:11 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 7 Aug 2014 08:53:11 -0700
Subject: [IPython-dev] tornado app
In-Reply-To: <20140807131656.GN24532@hike.sph.harvard.edu>
References: <20140807131656.GN24532@hike.sph.harvard.edu>
Message-ID: <CAH4pYpRGAewSjSx2Y2=FiS475gHbJm7buns1kyn3YckkE__Pmw@mail.gmail.com>

We do not plan on allowed others to add new handlers to *our* app. The
recommendation is that you write your own top level app that 1)
defines any custom handlers you need and 2) imports any existing
IPython handlers that you need. The reason for this is that the second
we allow custom handlers in our app, IPython is on a slippery slope to
become a full blown web app development framework.

On Thu, Aug 7, 2014 at 6:16 AM, Keith <kb201a at gmail.com> wrote:
> Hi,
>
> I have an existing tornado application and am thinking
> about leveraging (moving it into) ipython notebook.
> Currently the app defines its own request handlers and
> websocket for communication with the client browser.
>
> I was thinking that the app could generate a notebook
> dynamically which would be deployed on the client's
> browser.  Then communication to the server could take
> place within the notebook's cells.  Assuming this
> is possible (I'm new to ipython), are there hooks in
> ipython notebook that would allow me to extend tornado
> i.e. add my own routes and request handlers etc?
>
> thanks,
> Keith
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Thu Aug  7 12:02:04 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 7 Aug 2014 09:02:04 -0700
Subject: [IPython-dev] Automatically saving notebooks as scripts has
	been removed
In-Reply-To: <CAHNn8BXZjn=vsVSjKFf2xSwOMr2JX6tYHVzBHKs-Jbm+Hn=Ohw@mail.gmail.com>
References: <CAH2VmmB45-1pEDkst7Dmq=CaswM7G_eKExzde+2mLOsEjLJr_A@mail.gmail.com>
	<CAOvn4qiq78Cb_fk7N_P5ZuLk80wEFC+SsZmx4RjV=4MnJbdr8A@mail.gmail.com>
	<CAH2VmmD-HwghBY5GEGyZUn6BCZXc1+f6mnh=YAiM26URuEASsA@mail.gmail.com>
	<CAHNn8BXZjn=vsVSjKFf2xSwOMr2JX6tYHVzBHKs-Jbm+Hn=Ohw@mail.gmail.com>
Message-ID: <CAH4pYpQhpGwmN9eJcKPhuPcJC=929KgWaPJt3pOkyPv+TmOYcA@mail.gmail.com>

I agree with @minrk that the old `--script` is not something we want
to bring back and that we should figure out a good way of having that
particular conversion done in the notebook UI.

On Wed, Aug 6, 2014 at 2:08 PM, MinRK <benjaminrk at gmail.com> wrote:
> The main reason it is removed is that nbformat had a notion of ?save as
> .py?, which is removed in IPython 3.0 because this functionality has moved
> to nbconvert in IPython 1.0. We could, of course, let the appropriate
> nbconvert --to python call be implied by --script instead of the removed
> Python API, but I don?t think --script was a good idea in the first place.
>
> So it?s not a question of whether to use nbconvert or not - you have to use
> nbconvert to get .py files from .ipynb. It?s just a question of whether you
> can automatically use nbconvert on save via --script. I think if we want to
> support this kind of thing, more generic save/rename hooks are better than a
> single special-case flag.
>
> -MinRK
>
>
>
> On Mon, Aug 4, 2014 at 2:39 PM, Filipe Pires Alvarenga Fernandes
> <ocefpaf at gmail.com> wrote:
>>
>> Got it.  Thanks for looking into this.
>>
>>
>>
>> On Mon, Aug 4, 2014 at 6:18 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>>
>>> I think we may have been a bit too hasty in getting rid of that. We've
>>> been talking about it for a while amongst ourselves, because it doesn't fit
>>> terribly neatly into our infrastructure, but we didn't formally deprecate
>>> it, and I don't think the alternative approaches are quite as convenient
>>> yet.
>>>
>>> I've put this in the agenda for the dev meeting on Thursday.
>>>
>>> Thanks,
>>> Thomas
>>>
>>>
>>> On 4 August 2014 12:55, Filipe Pires Alvarenga Fernandes
>>> <ocefpaf at gmail.com> wrote:
>>>>
>>>> Hello IPython devs,
>>>>
>>>> I was trying the latest github checkout of the IPython source and got
>>>> the following message:
>>>>
>>>> Automatically saving notebooks as scripts has been removed.
>>>> Use `ipython nbconvert --to python [notebook]` instead.
>>>>
>>>> I am wondering why this had to be removed.  Security?
>>>>
>>>> I find the automatic script saving very helpful.  My work-group usually
>>>> commits the `.ipynb` and the `.py` side-by-side to obtain a better diff.  I
>>>> know that we can workaround this with nbconvert, but doing that manually is
>>>> error prone, and having a script to do that will add another layer of
>>>> complexity that we did not have with the automatic conversion.
>>>>
>>>> Well, if the automatic conversion had to go I understand, but if there
>>>> is no reason can you guys consider leaving it there?
>>>>
>>>> Thanks,
>>>>
>>>> -Filipe
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From andy at payne.org  Thu Aug  7 12:33:25 2014
From: andy at payne.org (Andrew Payne)
Date: Thu, 7 Aug 2014 12:33:25 -0400
Subject: [IPython-dev] API to execute cells of notebook
In-Reply-To: <CAFpY3uc1sKK7g9662q2Kyt6yYdK9wN95iPB6ygWekyMqzs4GGQ@mail.gmail.com>
References: <CAFpY3uc1sKK7g9662q2Kyt6yYdK9wN95iPB6ygWekyMqzs4GGQ@mail.gmail.com>
Message-ID: <CABbuyuW_Z3Y0Sgsd6UBvzgSeWKrfNMt9y1EXvTGdcuozBxxp2w@mail.gmail.com>

> I don't see a way via python to cause the current kernel to execute the
> rest of the cells in the notebook.  I could hack something together with
> javascript, but that seems problematic and brittle.
>

There are a number of existing Javascript  methods for executing cells,
including execute_cell_range(start, end) and execute_all_cells().

See:
https://github.com/ipython/ipython/blob/6ecbe721af987d743b7bfcec0448e4edace5f812/IPython/html/static/notebook/js/notebook.js#L1685

And this notebook includes a simple example of adding keyboard shortcut
using this method.  See:
http://nbviewer.ipython.org/github/minrk/pydata-sv-14/blob/master/What's%20New.ipynb

It's easy to generalize that to a Javascript button.

-andy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/9ff2f6c4/attachment.html>

From paddy at paddymullen.com  Thu Aug  7 12:58:01 2014
From: paddy at paddymullen.com (Paddy Mullen)
Date: Thu, 7 Aug 2014 12:58:01 -0400
Subject: [IPython-dev] API to execute cells of notebook
In-Reply-To: <CABbuyuW_Z3Y0Sgsd6UBvzgSeWKrfNMt9y1EXvTGdcuozBxxp2w@mail.gmail.com>
References: <CAFpY3uc1sKK7g9662q2Kyt6yYdK9wN95iPB6ygWekyMqzs4GGQ@mail.gmail.com>
	<CABbuyuW_Z3Y0Sgsd6UBvzgSeWKrfNMt9y1EXvTGdcuozBxxp2w@mail.gmail.com>
Message-ID: <CAFpY3ucr2Nx2T8nkkkN_=VZG0f_Sy0HAQUw6d2vEnFm+-nC-hw@mail.gmail.com>

Thanks,
I could do that, but it seems a bit out of place in the IPython widget
framework.

Imagine if cells had a hidden attribute that could be set in python.  Then
you could use the widget framework to implement a slideshow, or select
which type of report to show.  Being able to only access those apis through
javascript raises the bar for use.




On Thu, Aug 7, 2014 at 12:33 PM, Andrew Payne <andy at payne.org> wrote:

>
> I don't see a way via python to cause the current kernel to execute the
>> rest of the cells in the notebook.  I could hack something together with
>> javascript, but that seems problematic and brittle.
>>
>
> There are a number of existing Javascript  methods for executing cells,
> including execute_cell_range(start, end) and execute_all_cells().
>
> See:
> https://github.com/ipython/ipython/blob/6ecbe721af987d743b7bfcec0448e4edace5f812/IPython/html/static/notebook/js/notebook.js#L1685
>
> And this notebook includes a simple example of adding keyboard shortcut
> using this method.  See:
> http://nbviewer.ipython.org/github/minrk/pydata-sv-14/blob/master/What's%20New.ipynb
>
> It's easy to generalize that to a Javascript button.
>
> -andy
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/d6ab1e65/attachment.html>

From raymond.yee at gmail.com  Thu Aug  7 13:40:14 2014
From: raymond.yee at gmail.com (Raymond Yee)
Date: Thu, 07 Aug 2014 10:40:14 -0700
Subject: [IPython-dev] IPython 2.2 release, including security fix
In-Reply-To: <CAOvn4qjbxBfQXVK+5612xPF-M_jQ7KL5y27VOFvJZcYPyGincQ@mail.gmail.com>
References: <CAOvn4qjbxBfQXVK+5612xPF-M_jQ7KL5y27VOFvJZcYPyGincQ@mail.gmail.com>
Message-ID: <53E3B9FE.9090400@gmail.com>

Thanks for the security fixes of 2.2.   I'm using conda to install
IPython.  Anyone know when continuum might release IPython 2.2 via
conda?   Should I be contacting continuum to ask?

-Raymond

On 8/6/14 4:25 PM, Thomas Kluyver wrote:
> Hi all,
>
> We've just released IPython 2.2, which rolls up a few bugfixes that
> have been backported since the last release. This includes a fix for a
> security issue posted a few days ago, which could have allowed an
> attacker who could intercept network traffic (e.g. on insecure wifi)
> to intercept a request for Mathjax and send back malicious javascript
> which would run in the notebook. The fix was simply to always load
> mathjax over HTTPS:
> https://github.com/ipython/ipython/pull/6249
>
> You can see the full list of issues and PRs for this release here:
> http://ipython.org/ipython-doc/stable/whatsnew/github-stats-2.0.html#issues-closed-in-2-2-0
>
> You can install this release in any of the usual ways.
> http://ipython.org/install.html
>
> Thomas
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/90e0fe9a/attachment.html>

From andy at payne.org  Thu Aug  7 13:50:03 2014
From: andy at payne.org (Andrew Payne)
Date: Thu, 7 Aug 2014 13:50:03 -0400
Subject: [IPython-dev] API to execute cells of notebook
In-Reply-To: <CAFpY3ucr2Nx2T8nkkkN_=VZG0f_Sy0HAQUw6d2vEnFm+-nC-hw@mail.gmail.com>
References: <CAFpY3uc1sKK7g9662q2Kyt6yYdK9wN95iPB6ygWekyMqzs4GGQ@mail.gmail.com>
	<CABbuyuW_Z3Y0Sgsd6UBvzgSeWKrfNMt9y1EXvTGdcuozBxxp2w@mail.gmail.com>
	<CAFpY3ucr2Nx2T8nkkkN_=VZG0f_Sy0HAQUw6d2vEnFm+-nC-hw@mail.gmail.com>
Message-ID: <CABbuyuXqnNLCikSsWbaBkUACo-i9DJ2qOye2Xc4XuPOA1hcUmA@mail.gmail.com>

>
> I could do that, but it seems a bit out of place in the IPython widget
> framework.
>
> Imagine if cells had a hidden attribute that could be set in python.  Then
> you could use the widget framework to implement a slideshow, or select
> which type of report to show.  Being able to only access those apis through
> javascript raises the bar for use.
>

The challenge (I suspect) is the execution of a Web notebook is driven from
Javascript.  For example, the execute_all_cells() JS function I referred to
earlier eventually calls notebook.execute_cell(), which invokes the
execute() method on the cell.

For code cells, that generates an execute message to the kernel, see:
https://github.com/ipython/ipython/blob/master/IPython/html/static/notebook/js/codecell.js#L286

And the message:
http://ipython.org/ipython-doc/dev/development/messaging.html#execute

Note that the kernel execute message has the actual code fragment to run,
not a reference to a cell.  The current state of the notebook (e.g. all the
code) is in the browser not the kernel (e.g. you can edit 3 code cells and
then do "run all" and get the result you expect).  This means a "run all"
operation has to involve the browser notebook somehow.

I'm not defending this design or saying what you're looking for is
unreasonable.  I'm just noting that there may be an impedance mismatch with
way the kernel and notebooks are built (which likely derives from the
team's stated design goal to keep them independent).

And given that widgets are essentially a combination of kernel-side Python
and browser-side Javascript with a comm backbone to pass state updates and
messages, you could easily (I think) implement a widget version of what
you're looking for by taking the button widget and calling the
execute_all_cells() method on invocation.

I hope this is helpful,

-andy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/1056eb63/attachment.html>

From fperez.net at gmail.com  Thu Aug  7 17:27:16 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 7 Aug 2014 14:27:16 -0700
Subject: [IPython-dev] IPython 2.2 release, including security fix
In-Reply-To: <53E3B9FE.9090400@gmail.com>
References: <CAOvn4qjbxBfQXVK+5612xPF-M_jQ7KL5y27VOFvJZcYPyGincQ@mail.gmail.com>
	<53E3B9FE.9090400@gmail.com>
Message-ID: <CAHAreOpVh+OMvxd0AjscRQ503xkzjWhX5NW=BC7jPoZYKs61=w@mail.gmail.com>

On Thu, Aug 7, 2014 at 10:40 AM, Raymond Yee <raymond.yee at gmail.com> wrote:

> Thanks for the security fixes of 2.2.   I'm using conda to install
> IPython.  Anyone know when continuum might release IPython 2.2 via conda?
> Should I be contacting continuum to ask?


Some of their devs hang out here, but it's probably best to ask directly on
their mailing list.  We have zero control over their release schedule.

Cheers

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/ded6ea4c/attachment.html>

From damianavila at gmail.com  Thu Aug  7 17:30:50 2014
From: damianavila at gmail.com (=?UTF-8?Q?Dami=C3=A1n_Avila?=)
Date: Thu, 7 Aug 2014 18:30:50 -0300
Subject: [IPython-dev] IPython 2.2 release, including security fix
In-Reply-To: <CAHAreOpVh+OMvxd0AjscRQ503xkzjWhX5NW=BC7jPoZYKs61=w@mail.gmail.com>
References: <CAOvn4qjbxBfQXVK+5612xPF-M_jQ7KL5y27VOFvJZcYPyGincQ@mail.gmail.com>
	<53E3B9FE.9090400@gmail.com>
	<CAHAreOpVh+OMvxd0AjscRQ503xkzjWhX5NW=BC7jPoZYKs61=w@mail.gmail.com>
Message-ID: <CAH+mRR2RgrJ2tFCo1r30_cdBhAoaKx9_cGN96H+R=7RmSCsfCg@mail.gmail.com>

It will be available soon ;-)
El ago 7, 2014 6:27 PM, "Fernando Perez" <fperez.net at gmail.com> escribi?:

>
> On Thu, Aug 7, 2014 at 10:40 AM, Raymond Yee <raymond.yee at gmail.com>
> wrote:
>
>> Thanks for the security fixes of 2.2.   I'm using conda to install
>> IPython.  Anyone know when continuum might release IPython 2.2 via conda?
>> Should I be contacting continuum to ask?
>
>
> Some of their devs hang out here, but it's probably best to ask directly
> on their mailing list.  We have zero control over their release schedule.
>
> Cheers
>
> f
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/3c86d93a/attachment.html>

From fperez.net at gmail.com  Thu Aug  7 17:42:13 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 7 Aug 2014 14:42:13 -0700
Subject: [IPython-dev] tornado app
In-Reply-To: <CAH4pYpRGAewSjSx2Y2=FiS475gHbJm7buns1kyn3YckkE__Pmw@mail.gmail.com>
References: <20140807131656.GN24532@hike.sph.harvard.edu>
	<CAH4pYpRGAewSjSx2Y2=FiS475gHbJm7buns1kyn3YckkE__Pmw@mail.gmail.com>
Message-ID: <CAHAreOoiV1TiVLJHkpEYL4G3TtHC6Rif6jX4kvKQfAc1wVejsg@mail.gmail.com>

On Thu, Aug 7, 2014 at 8:53 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> The reason for this is that the second
> we allow custom handlers in our app, IPython is on a slippery slope to
> become a full blown web app development framework.
>

But, but... Our scope is well limited to *everything*!

;)


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/4d3f68af/attachment.html>

From fperez.net at gmail.com  Thu Aug  7 17:43:59 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 7 Aug 2014 14:43:59 -0700
Subject: [IPython-dev] tornado app
In-Reply-To: <CAHAreOoiV1TiVLJHkpEYL4G3TtHC6Rif6jX4kvKQfAc1wVejsg@mail.gmail.com>
References: <20140807131656.GN24532@hike.sph.harvard.edu>
	<CAH4pYpRGAewSjSx2Y2=FiS475gHbJm7buns1kyn3YckkE__Pmw@mail.gmail.com>
	<CAHAreOoiV1TiVLJHkpEYL4G3TtHC6Rif6jX4kvKQfAc1wVejsg@mail.gmail.com>
Message-ID: <CAHAreOo2CU8P8m1=krcJ1iDM1oNipEKbpm7e1_meX8u7SQM6Lg@mail.gmail.com>

Jokes aside, you may want to study a little bit how our new colaboratory
works:

https://github.com/jupyter/colaboratory/blob/master/colaboratory/colaboratory.py

That will give you an idea of how a custom web application is built using
the IPython machinery, but one that is NOT the default notebook server.

Cheers,

f


On Thu, Aug 7, 2014 at 2:42 PM, Fernando Perez <fperez.net at gmail.com> wrote:

>
> On Thu, Aug 7, 2014 at 8:53 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>
>> The reason for this is that the second
>> we allow custom handlers in our app, IPython is on a slippery slope to
>> become a full blown web app development framework.
>>
>
> But, but... Our scope is well limited to *everything*!
>
> ;)
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>



-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/ab6c5469/attachment.html>

From asmeurer at gmail.com  Thu Aug  7 17:52:06 2014
From: asmeurer at gmail.com (Aaron Meurer)
Date: Thu, 7 Aug 2014 16:52:06 -0500
Subject: [IPython-dev] IPython 2.2 release, including security fix
In-Reply-To: <53E3B9FE.9090400@gmail.com>
References: <CAOvn4qjbxBfQXVK+5612xPF-M_jQ7KL5y27VOFvJZcYPyGincQ@mail.gmail.com>
	<53E3B9FE.9090400@gmail.com>
Message-ID: <CAKgW=6Lm9kRh1NL15ZicDj-3R8H-RkzCwAPsm-Krsu-HGRzA7Q@mail.gmail.com>

Some time next week at the latest.

Aaron Meurer


On Thu, Aug 7, 2014 at 12:40 PM, Raymond Yee <raymond.yee at gmail.com> wrote:

>  Thanks for the security fixes of 2.2.   I'm using conda to install
> IPython.  Anyone know when continuum might release IPython 2.2 via conda?
> Should I be contacting continuum to ask?
>
> -Raymond
>
>
> On 8/6/14 4:25 PM, Thomas Kluyver wrote:
>
>   Hi all,
>
>  We've just released IPython 2.2, which rolls up a few bugfixes that have
> been backported since the last release. This includes a fix for a security
> issue posted a few days ago, which could have allowed an attacker who could
> intercept network traffic (e.g. on insecure wifi) to intercept a request
> for Mathjax and send back malicious javascript which would run in the
> notebook. The fix was simply to always load mathjax over HTTPS:
> https://github.com/ipython/ipython/pull/6249
>
>  You can see the full list of issues and PRs for this release here:
>
> http://ipython.org/ipython-doc/stable/whatsnew/github-stats-2.0.html#issues-closed-in-2-2-0
>
>  You can install this release in any of the usual ways.
> http://ipython.org/install.html
>
>  Thomas
>
>
> _______________________________________________
> IPython-dev mailing listIPython-dev at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/54ab1981/attachment.html>

From illanos at cal.berkeley.edu  Thu Aug  7 23:22:25 2014
From: illanos at cal.berkeley.edu (Imelda Llanos de Luna)
Date: Thu, 7 Aug 2014 20:22:25 -0700
Subject: [IPython-dev] requesting feedback: IPython-BusinessNeed draft report
Message-ID: <BLU184-W1083D5D09FE4284AE92D4EC7EE0@phx.gbl>

Hi,I'm a Business Analyst, volunteering my services.  I am performing analysis of the IPython project, and I have completed a DRAFT of the Business Need section of the report.  For the draft, I performed a document analysis to help write the draft, pulling written sections that existed on other documents.  Please review the draft, and provide feedback by Thurs (8/14).  I welcome it, and any other corrections and clarifications that can be provided.
Again, this is only one section of the report, solely focused on defining the business need at a high level. There are more sections left to write: http://nbviewer.ipython.org/github/ofcloudpeople/Business-Analysis/blob/IPythonProject/IPython-BusinessNeed.ipynb .  Note also: I had to close the gap between the time the project started in 2001 to the present.  I believe I was able to close it as written.  If yes , then any writing in the next sections can focus on present goals and objectives, requirements, etc.
Thank you.
Sincerely,Imelda


 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/efc1f9d0/attachment.html>

From zvoros at gmail.com  Fri Aug  8 00:46:17 2014
From: zvoros at gmail.com (=?UTF-8?B?Wm9sdMOhbiBWw7Zyw7Zz?=)
Date: Fri, 08 Aug 2014 06:46:17 +0200
Subject: [IPython-dev] mathjax equation number is incremented on cntr-enter
Message-ID: <53E45619.4040302@gmail.com>

Hi all,

I don't know, whether this is a known issue (a quick search did not turn 
up anything), in fact, I don't even know, whether this is an issue at 
all, but I believe, the markdown cell did behave differently in the 
past. In short, it seems to me that mathjax increments the equation 
number, whenever I press Cntr-Enter on a markdown cell. Here is my 
notebook, and here is the code that produces the problem:

\begin{equation}
x = y
\end{equation}

I haven't removed anything, yet, the equation is labelled (4). Can 
someone comment on this? In particular, whether this was intended? Is 
this, perhaps, related to the security problem that was floating around 
recently? I am running the latest code from master.

Cheers,

Zolt?n
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot from 2014-08-08 06:37:04.png
Type: image/png
Size: 15159 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140808/39923e33/attachment.png>

From fperez.net at gmail.com  Fri Aug  8 02:29:21 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 7 Aug 2014 23:29:21 -0700
Subject: [IPython-dev] requesting feedback: IPython-BusinessNeed draft
	report
In-Reply-To: <BLU184-W1083D5D09FE4284AE92D4EC7EE0@phx.gbl>
References: <BLU184-W1083D5D09FE4284AE92D4EC7EE0@phx.gbl>
Message-ID: <CAHAreOoPYLO6x4zqAHFG68vh3djnd2Wbdz5a1hw+NMMeise-7w@mail.gmail.com>

Hi all,

I should have done this prior, but better late than never.. :)

I just wanted to introduce Imelda, who contacted us a few days ago and volu

I hope this will nteered her time and skills to help out IPython.
 Yesterday met at Berkeley with the local part of the team as well as Katy
Huff from Scipy.

Imelda volunteered to help the project with writing from a different
perspective than what we typically write: not technical docs but
business/project analysis.  That's a skill set different from our typical
one, so I'm happy to see where this experiment can take us.

We decided a good starting point could be for her to take the view of a
business analyst and look at what the project looks like *from the
outside*, without much detailed guidance from us at all. I'm sure that will
be a sobering diagnostic of some of our limitations in terms of docs,
project presentation, website, etc.

Cheers

f


On Thu, Aug 7, 2014 at 8:22 PM, Imelda Llanos de Luna <
illanos at cal.berkeley.edu> wrote:

> Hi,
> I'm a Business Analyst, volunteering my services.  I am performing
> analysis of the IPython project, and I have completed a DRAFT of the
> Business Need section of the report.  For the draft, I performed a document
> analysis to help write the draft, pulling written sections that existed on
> other documents.  Please review the draft, and provide feedback by Thurs
> (8/14).  I welcome it, and any other corrections and clarifications that
> can be provided.
>
> Again, this is only one section of the report, solely focused on defining
> the business need at a high level. There are more sections left to write:
> http://nbviewer.ipython.org/github/ofcloudpeople/Business-Analysis/blob/IPythonProject/IPython-BusinessNeed.ipynb .
>  Note also: I had to close the gap between the time the project started
> in 2001 to the present.  I believe I was able to close it as written.  If
> yes , then any writing in the next sections can focus on present goals and
> objectives, requirements, etc.
>
> Thank you.
>
> Sincerely,
> Imelda
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140807/f51a140b/attachment.html>

From markbak at gmail.com  Fri Aug  8 09:27:39 2014
From: markbak at gmail.com (Mark Bakker)
Date: Fri, 8 Aug 2014 15:27:39 +0200
Subject: [IPython-dev] changing working directory from IPython server
Message-ID: <CAEX=yaa5fCqSBMaL9v5HFKTKDxY5AMo3+4z+pzWyo4V+RVcg0A@mail.gmail.com>

Hello List,

I know this has been discussed before, but it doesn't seem to be resolved.

Currently (v.2.1 on my machine) I can start an IPython notebook server
anywhere.
Then click and open an IPython Notebook anywhere else.
That IPython Notebook then gets copied to the directory where I started the
server.

Problem with that workflow is that that is rarely what you want it to do.

Is there any way, or are there any plans, to be able to change the working
directory from the Notebook server page (the one that opens when you type
'ipython notebook')? It would make my life, and that of my students, SO
much easier.

Thanks for all the great work.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140808/cc82c07e/attachment.html>

From ellisonbg at gmail.com  Fri Aug  8 09:48:59 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 8 Aug 2014 06:48:59 -0700
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAEX=yaa5fCqSBMaL9v5HFKTKDxY5AMo3+4z+pzWyo4V+RVcg0A@mail.gmail.com>
References: <CAEX=yaa5fCqSBMaL9v5HFKTKDxY5AMo3+4z+pzWyo4V+RVcg0A@mail.gmail.com>
Message-ID: <6A43CDB9-EB1A-4F21-87FF-A3136AC47749@gmail.com>

The notebook shouldn't behave as you describe in 2.1. Notebooks should be saved in the directory they are created. There is something fishy going on...

Sent from my iPhone

> On Aug 8, 2014, at 6:27 AM, Mark Bakker <markbak at gmail.com> wrote:
> 
> Hello List,
> 
> I know this has been discussed before, but it doesn't seem to be resolved.
> 
> Currently (v.2.1 on my machine) I can start an IPython notebook server anywhere.
> Then click and open an IPython Notebook anywhere else.
> That IPython Notebook then gets copied to the directory where I started the server.
> 
> Problem with that workflow is that that is rarely what you want it to do.
> 
> Is there any way, or are there any plans, to be able to change the working directory from the Notebook server page (the one that opens when you type 'ipython notebook')? It would make my life, and that of my students, SO much easier.
> 
> Thanks for all the great work.
> 
> Mark
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From markbak at gmail.com  Fri Aug  8 10:00:31 2014
From: markbak at gmail.com (Mark Bakker)
Date: Fri, 8 Aug 2014 16:00:31 +0200
Subject: [IPython-dev] changing working directory from IPython server
Message-ID: <CAEX=yabOwzm3qvey_csOBmxX2fcbX7gxiAfAKvd2EzMz=k811Q@mail.gmail.com>

Brian, I sure hope I am doing something wrong.
What I do is click on 'To import a notebook, drag the file onto the listing
below or *click here*. When I click there, a window pops up, I got to the
directory with my ipython file, click on it, and then it shows up in the
list of files, with next to it a blue Upload button. When I click on the
Upload button the notebook is copied to the directory where I started the
IPython Notebook.

But I don't want to copy the Notebook to that directory (also cause the
data files associated with the notebook are still in the old directory). Am
I doing something wrong?

Mark

Brian Granger <ellisonbg at gmail.com> wrote:

The notebook shouldn't behave as you describe in 2.1. Notebooks should be
saved in the directory they are created. There is something fishy going
on...

Sent from my iPhone

On Fri, Aug 8, 2014 at 3:27 PM, Mark Bakker <markbak at gmail.com> wrote:

> Hello List,
>
> I know this has been discussed before, but it doesn't seem to be resolved.
>
> Currently (v.2.1 on my machine) I can start an IPython notebook server
> anywhere.
> Then click and open an IPython Notebook anywhere else.
> That IPython Notebook then gets copied to the directory where I started
> the server.
>
> Problem with that workflow is that that is rarely what you want it to do.
>
> Is there any way, or are there any plans, to be able to change the working
> directory from the Notebook server page (the one that opens when you type
> 'ipython notebook')? It would make my life, and that of my students, SO
> much easier.
>
> Thanks for all the great work.
>
> Mark
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140808/5f5e5e25/attachment.html>

From markbak at gmail.com  Fri Aug  8 10:03:21 2014
From: markbak at gmail.com (Mark Bakker)
Date: Fri, 8 Aug 2014 16:03:21 +0200
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAEX=yabOwzm3qvey_csOBmxX2fcbX7gxiAfAKvd2EzMz=k811Q@mail.gmail.com>
References: <CAEX=yabOwzm3qvey_csOBmxX2fcbX7gxiAfAKvd2EzMz=k811Q@mail.gmail.com>
Message-ID: <CAEX=yaZrixOBfMaYQMZbi=_hJbC-e7fAdswDTRT+EXP-G=4iUw@mail.gmail.com>

Oops, I am running IPython 2.0.
Is this better in 2.1?
If so, then very sorry for the noise and have a nice weekend.
(If not: help!)
Mark


On Fri, Aug 8, 2014 at 4:00 PM, Mark Bakker <markbak at gmail.com> wrote:

> Brian, I sure hope I am doing something wrong.
> What I do is click on 'To import a notebook, drag the file onto the
> listing below or *click here*. When I click there, a window pops up, I got
> to the directory with my ipython file, click on it, and then it shows up in
> the list of files, with next to it a blue Upload button. When I click on
> the Upload button the notebook is copied to the directory where I started
> the IPython Notebook.
>
> But I don't want to copy the Notebook to that directory (also cause the
> data files associated with the notebook are still in the old directory). Am
> I doing something wrong?
>
> Mark
>
> Brian Granger <ellisonbg at gmail.com> wrote:
>
> The notebook shouldn't behave as you describe in 2.1. Notebooks should be
> saved in the directory they are created. There is something fishy going
> on...
>
> Sent from my iPhone
>
>
> On Fri, Aug 8, 2014 at 3:27 PM, Mark Bakker <markbak at gmail.com> wrote:
>
>> Hello List,
>>
>> I know this has been discussed before, but it doesn't seem to be resolved.
>>
>> Currently (v.2.1 on my machine) I can start an IPython notebook server
>> anywhere.
>> Then click and open an IPython Notebook anywhere else.
>> That IPython Notebook then gets copied to the directory where I started
>> the server.
>>
>> Problem with that workflow is that that is rarely what you want it to do.
>>
>> Is there any way, or are there any plans, to be able to change the
>> working directory from the Notebook server page (the one that opens when
>> you type 'ipython notebook')? It would make my life, and that of my
>> students, SO much easier.
>>
>> Thanks for all the great work.
>>
>> Mark
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140808/e683940d/attachment.html>

From ellisonbg at gmail.com  Fri Aug  8 10:39:30 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 8 Aug 2014 07:39:30 -0700
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAEX=yaZrixOBfMaYQMZbi=_hJbC-e7fAdswDTRT+EXP-G=4iUw@mail.gmail.com>
References: <CAEX=yabOwzm3qvey_csOBmxX2fcbX7gxiAfAKvd2EzMz=k811Q@mail.gmail.com>
	<CAEX=yaZrixOBfMaYQMZbi=_hJbC-e7fAdswDTRT+EXP-G=4iUw@mail.gmail.com>
Message-ID: <9127FEAE-5736-4EE1-A43F-2CCD3549DF64@gmail.com>

Ahh sorry, I missed you are uploading a notebook. That may be a bug, but I don't remember. Can you open a github issue for thus and we will test ourselves?

Sent from my iPhone

> On Aug 8, 2014, at 7:03 AM, Mark Bakker <markbak at gmail.com> wrote:
> 
> Oops, I am running IPython 2.0.
> Is this better in 2.1?
> If so, then very sorry for the noise and have a nice weekend.
> (If not: help!)
> Mark
> 
> 
>> On Fri, Aug 8, 2014 at 4:00 PM, Mark Bakker <markbak at gmail.com> wrote:
>> Brian, I sure hope I am doing something wrong. 
>> What I do is click on 'To import a notebook, drag the file onto the listing below or *click here*. When I click there, a window pops up, I got to the directory with my ipython file, click on it, and then it shows up in the list of files, with next to it a blue Upload button. When I click on the Upload button the notebook is copied to the directory where I started the IPython Notebook.
>> 
>> But I don't want to copy the Notebook to that directory (also cause the data files associated with the notebook are still in the old directory). Am I doing something wrong?
>> 
>> Mark
>> 
>> Brian Granger <ellisonbg at gmail.com> wrote:
>> 
>> The notebook shouldn't behave as you describe in 2.1. Notebooks should be saved in the directory they are created. There is something fishy going on... 
>> 
>> Sent from my iPhone 
>> 
>> 
>>> On Fri, Aug 8, 2014 at 3:27 PM, Mark Bakker <markbak at gmail.com> wrote:
>>> Hello List,
>>> 
>>> I know this has been discussed before, but it doesn't seem to be resolved.
>>> 
>>> Currently (v.2.1 on my machine) I can start an IPython notebook server anywhere.
>>> Then click and open an IPython Notebook anywhere else.
>>> That IPython Notebook then gets copied to the directory where I started the server.
>>> 
>>> Problem with that workflow is that that is rarely what you want it to do.
>>> 
>>> Is there any way, or are there any plans, to be able to change the working directory from the Notebook server page (the one that opens when you type 'ipython notebook')? It would make my life, and that of my students, SO much easier.
>>> 
>>> Thanks for all the great work.
>>> 
>>> Mark
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140808/3e33a8a5/attachment.html>

From ellisonbg at gmail.com  Fri Aug  8 10:47:45 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 8 Aug 2014 07:47:45 -0700
Subject: [IPython-dev] requesting feedback: IPython-BusinessNeed draft
	report
In-Reply-To: <CAHAreOoPYLO6x4zqAHFG68vh3djnd2Wbdz5a1hw+NMMeise-7w@mail.gmail.com>
References: <BLU184-W1083D5D09FE4284AE92D4EC7EE0@phx.gbl>
	<CAHAreOoPYLO6x4zqAHFG68vh3djnd2Wbdz5a1hw+NMMeise-7w@mail.gmail.com>
Message-ID: <7F928AF2-EF31-4A3B-910A-F54BE90CD91F@gmail.com>

Great, I will have a look!

Sent from my iPhone

> On Aug 7, 2014, at 11:29 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> 
> Hi all,
> 
> I should have done this prior, but better late than never.. :)
> 
> I just wanted to introduce Imelda, who contacted us a few days ago and volu
> 
> I hope this will nteered her time and skills to help out IPython.  Yesterday met at Berkeley with the local part of the team as well as Katy Huff from Scipy.
> 
> Imelda volunteered to help the project with writing from a different perspective than what we typically write: not technical docs but business/project analysis.  That's a skill set different from our typical one, so I'm happy to see where this experiment can take us.
> 
> We decided a good starting point could be for her to take the view of a business analyst and look at what the project looks like *from the outside*, without much detailed guidance from us at all. I'm sure that will be a sobering diagnostic of some of our limitations in terms of docs, project presentation, website, etc.
> 
> Cheers
> 
> f
> 
> 
>> On Thu, Aug 7, 2014 at 8:22 PM, Imelda Llanos de Luna <illanos at cal.berkeley.edu> wrote:
>> Hi,
>> I'm a Business Analyst, volunteering my services.  I am performing analysis of the IPython project, and I have completed a DRAFT of the Business Need section of the report.  For the draft, I performed a document analysis to help write the draft, pulling written sections that existed on other documents.  Please review the draft, and provide feedback by Thurs (8/14).  I welcome it, and any other corrections and clarifications that can be provided.
>> 
>> Again, this is only one section of the report, solely focused on defining the business need at a high level. There are more sections left to write: http://nbviewer.ipython.org/github/ofcloudpeople/Business-Analysis/blob/IPythonProject/IPython-BusinessNeed.ipynb .  Note also: I had to close the gap between the time the project started in 2001 to the present.  I believe I was able to close it as written.  If yes , then any writing in the next sections can focus on present goals and objectives, requirements, etc.
>> 
>> Thank you.
>> 
>> Sincerely,
>> Imelda
>> 
>> 
>> 
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> -- 
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140808/e9eb4227/attachment.html>

From takowl at gmail.com  Fri Aug  8 13:51:41 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 8 Aug 2014 10:51:41 -0700
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAEX=yabOwzm3qvey_csOBmxX2fcbX7gxiAfAKvd2EzMz=k811Q@mail.gmail.com>
References: <CAEX=yabOwzm3qvey_csOBmxX2fcbX7gxiAfAKvd2EzMz=k811Q@mail.gmail.com>
Message-ID: <CAOvn4qh_fyR_Ed6Nrv3y05hY-aMsGmFBEoutJeafHR2KteYJzA@mail.gmail.com>

On 8 August 2014 07:00, Mark Bakker <markbak at gmail.com> wrote:

> What I do is click on 'To import a notebook, drag the file onto the
> listing below or *click here*. When I click there, a window pops up, I got
> to the directory with my ipython file, click on it, and then it shows up in
> the list of files, with next to it a blue Upload button. When I click on
> the Upload button the notebook is copied to the directory where I started
> the IPython Notebook.
>
> But I don't want to copy the Notebook to that directory (also cause the
> data files associated with the notebook are still in the old directory). Am
> I doing something wrong?
>

That is by design, though I agree that it's rather strange. An unfortunate
result of building the notebook as a browser UI and a web server is that
some parts behave like a desktop app, and some behave like a web app. The
original notion of files was designed like a web app - you upload files to
the server, and it works with the uploaded copy, ignoring the file it
originally came from (if it wasn't running locally, it wouldn't be able to
access the original file).

So don't upload notebooks. Start the notebook in the directory where your
notebooks are, or in a parent directory of that.

My nbopen tool helps with this - it opens a notebook in an already running
server, or starts a new one if necessary.
https://github.com/takluyver/nbopen

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140808/2b73e366/attachment.html>

From erik.m.bray at gmail.com  Fri Aug  8 17:55:14 2014
From: erik.m.bray at gmail.com (Erik Bray)
Date: Fri, 8 Aug 2014 17:55:14 -0400
Subject: [IPython-dev] Automatically saving notebooks as scripts has
	been removed
In-Reply-To: <CAH4pYpQhpGwmN9eJcKPhuPcJC=929KgWaPJt3pOkyPv+TmOYcA@mail.gmail.com>
References: <CAH2VmmB45-1pEDkst7Dmq=CaswM7G_eKExzde+2mLOsEjLJr_A@mail.gmail.com>
	<CAOvn4qiq78Cb_fk7N_P5ZuLk80wEFC+SsZmx4RjV=4MnJbdr8A@mail.gmail.com>
	<CAH2VmmD-HwghBY5GEGyZUn6BCZXc1+f6mnh=YAiM26URuEASsA@mail.gmail.com>
	<CAHNn8BXZjn=vsVSjKFf2xSwOMr2JX6tYHVzBHKs-Jbm+Hn=Ohw@mail.gmail.com>
	<CAH4pYpQhpGwmN9eJcKPhuPcJC=929KgWaPJt3pOkyPv+TmOYcA@mail.gmail.com>
Message-ID: <CAOTD34b5dMaJMiehJ3CU1fVWCtcE3EbxQP0JZyQs8acAPD=2tg@mail.gmail.com>

On Thu, Aug 7, 2014 at 12:02 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> I agree with @minrk that the old `--script` is not something we want
> to bring back and that we should figure out a good way of having that
> particular conversion done in the notebook UI.

Would somebody please be willing to summarize the differences between

nbconvert --to python

and the aforementioned --script option?

I think that the format that is output from the notebook by "Save as
.py..." is very handy to have. And in fact I was just recently singing
its praises as a decent way to go between code in a notebook to an
importable Python module (and perhaps back again?)

So it's definitely a feature that should exist in some form I think.

Erik

> On Wed, Aug 6, 2014 at 2:08 PM, MinRK <benjaminrk at gmail.com> wrote:
>> The main reason it is removed is that nbformat had a notion of ?save as
>> .py?, which is removed in IPython 3.0 because this functionality has moved
>> to nbconvert in IPython 1.0. We could, of course, let the appropriate
>> nbconvert --to python call be implied by --script instead of the removed
>> Python API, but I don?t think --script was a good idea in the first place.
>>
>> So it?s not a question of whether to use nbconvert or not - you have to use
>> nbconvert to get .py files from .ipynb. It?s just a question of whether you
>> can automatically use nbconvert on save via --script. I think if we want to
>> support this kind of thing, more generic save/rename hooks are better than a
>> single special-case flag.
>>
>> -MinRK
>>
>>
>>
>> On Mon, Aug 4, 2014 at 2:39 PM, Filipe Pires Alvarenga Fernandes
>> <ocefpaf at gmail.com> wrote:
>>>
>>> Got it.  Thanks for looking into this.
>>>
>>>
>>>
>>> On Mon, Aug 4, 2014 at 6:18 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>>>
>>>> I think we may have been a bit too hasty in getting rid of that. We've
>>>> been talking about it for a while amongst ourselves, because it doesn't fit
>>>> terribly neatly into our infrastructure, but we didn't formally deprecate
>>>> it, and I don't think the alternative approaches are quite as convenient
>>>> yet.
>>>>
>>>> I've put this in the agenda for the dev meeting on Thursday.
>>>>
>>>> Thanks,
>>>> Thomas
>>>>
>>>>
>>>> On 4 August 2014 12:55, Filipe Pires Alvarenga Fernandes
>>>> <ocefpaf at gmail.com> wrote:
>>>>>
>>>>> Hello IPython devs,
>>>>>
>>>>> I was trying the latest github checkout of the IPython source and got
>>>>> the following message:
>>>>>
>>>>> Automatically saving notebooks as scripts has been removed.
>>>>> Use `ipython nbconvert --to python [notebook]` instead.
>>>>>
>>>>> I am wondering why this had to be removed.  Security?
>>>>>
>>>>> I find the automatic script saving very helpful.  My work-group usually
>>>>> commits the `.ipynb` and the `.py` side-by-side to obtain a better diff.  I
>>>>> know that we can workaround this with nbconvert, but doing that manually is
>>>>> error prone, and having a script to do that will add another layer of
>>>>> complexity that we did not have with the automatic conversion.
>>>>>
>>>>> Well, if the automatic conversion had to go I understand, but if there
>>>>> is no reason can you guys consider leaving it there?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Filipe
>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From takowl at gmail.com  Fri Aug  8 18:03:55 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 8 Aug 2014 15:03:55 -0700
Subject: [IPython-dev] Automatically saving notebooks as scripts has
	been removed
In-Reply-To: <CAOTD34b5dMaJMiehJ3CU1fVWCtcE3EbxQP0JZyQs8acAPD=2tg@mail.gmail.com>
References: <CAH2VmmB45-1pEDkst7Dmq=CaswM7G_eKExzde+2mLOsEjLJr_A@mail.gmail.com>
	<CAOvn4qiq78Cb_fk7N_P5ZuLk80wEFC+SsZmx4RjV=4MnJbdr8A@mail.gmail.com>
	<CAH2VmmD-HwghBY5GEGyZUn6BCZXc1+f6mnh=YAiM26URuEASsA@mail.gmail.com>
	<CAHNn8BXZjn=vsVSjKFf2xSwOMr2JX6tYHVzBHKs-Jbm+Hn=Ohw@mail.gmail.com>
	<CAH4pYpQhpGwmN9eJcKPhuPcJC=929KgWaPJt3pOkyPv+TmOYcA@mail.gmail.com>
	<CAOTD34b5dMaJMiehJ3CU1fVWCtcE3EbxQP0JZyQs8acAPD=2tg@mail.gmail.com>
Message-ID: <CAOvn4qhfJovvXTXxQkzPmpXtZ0uu6+D_Qha0BkqDyNiGTb6MZg@mail.gmail.com>

On 8 August 2014 14:55, Erik Bray <erik.m.bray at gmail.com> wrote:

>  Would somebody please be willing to summarize the differences between
>
> nbconvert --to python
>
> and the aforementioned --script option?


--script is an option to the notebook server, so that whenever you save a
notebook, you get a copy of the .py script as well. nbconvert has to be run
as a separate command when you want to do a conversion. Currently, --script
relies on a separate mechanism to write notebooks to .py files, but that
mechanism is being removed.

We'll reinstate --script for IPython 3, using the nbconvert machinery to
write notebooks, but it will be deprecated.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140808/b8a8cdce/attachment.html>

From ocefpaf at gmail.com  Fri Aug  8 18:25:10 2014
From: ocefpaf at gmail.com (Filipe Pires Alvarenga Fernandes)
Date: Fri, 8 Aug 2014 19:25:10 -0300
Subject: [IPython-dev] Automatically saving notebooks as scripts has
	been removed
In-Reply-To: <CAOvn4qhfJovvXTXxQkzPmpXtZ0uu6+D_Qha0BkqDyNiGTb6MZg@mail.gmail.com>
References: <CAH2VmmB45-1pEDkst7Dmq=CaswM7G_eKExzde+2mLOsEjLJr_A@mail.gmail.com>
	<CAOvn4qiq78Cb_fk7N_P5ZuLk80wEFC+SsZmx4RjV=4MnJbdr8A@mail.gmail.com>
	<CAH2VmmD-HwghBY5GEGyZUn6BCZXc1+f6mnh=YAiM26URuEASsA@mail.gmail.com>
	<CAHNn8BXZjn=vsVSjKFf2xSwOMr2JX6tYHVzBHKs-Jbm+Hn=Ohw@mail.gmail.com>
	<CAH4pYpQhpGwmN9eJcKPhuPcJC=929KgWaPJt3pOkyPv+TmOYcA@mail.gmail.com>
	<CAOTD34b5dMaJMiehJ3CU1fVWCtcE3EbxQP0JZyQs8acAPD=2tg@mail.gmail.com>
	<CAOvn4qhfJovvXTXxQkzPmpXtZ0uu6+D_Qha0BkqDyNiGTb6MZg@mail.gmail.com>
Message-ID: <CAH2VmmAtRQ8H1Yn4ya2xDaiQS_NJHywTcJNQizOTb57gpOEMvA@mail.gmail.com>

>
> We'll reinstate --script for IPython 3, using the nbconvert machinery to
> write notebooks, but it will be deprecated.
>

So the final decision is to remove...  I will try one last pitch here.

I believe that, even for people using the Jupyter with other languages
(Julia, Haskell, etc), the "--script" option is useful.  My group uses the
notebook for visualization and the "script" for diffs on github by
committing them together*.  Having the script updated/saved automatically
with the "--script" flag is a big help for our workflow.  Converting the
notebooks via the command line or using the UI would be regression for our
use case.

I am sure that someone will come up with a %magic or a github hook for
that, but this feature is already there and, IMHO, I do not see a reason
for it to go

*Or at least until a project like nbdiff takes off.

Thanks,

-Filipe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140808/b0a429a2/attachment.html>

From jklymak at gmail.com  Sun Aug 10 11:39:03 2014
From: jklymak at gmail.com (Jody Klymak)
Date: Sun, 10 Aug 2014 08:39:03 -0700
Subject: [IPython-dev] Preferred way to customize CSS for something that
	will be displayed on nbviewer?
In-Reply-To: <E8AA2F4F-9562-4091-B852-DA2F01E9543D@gmail.com>
References: <CAPhiW4htyT1W9Ux-wB5PRQ05+ng7GwaFnKbqN5z=04WS_7CEXw@mail.gmail.com>
	<E8AA2F4F-9562-4091-B852-DA2F01E9543D@gmail.com>
Message-ID: <8149380B-9E8B-4B25-8067-934AFCF5B1E2@gmail.com>

Hi all

Has this been updated yet?  I'd like to share ipython notebooks on nbviewer for teaching and would like to change the CSS to make the pages easier to read.  What is the current best practice? 

Thanks.   Jody


On Feb 20, 2014, at  2:54 AM, Matthias Bussonnier <bussonniermatthias at gmail.com> wrote:

> There should be a discussion on IPython user about that from a few weeks ago. 
> 
> We are thinking on metadata in notebook level that nbviewer could look at that allow to select a theme on nbviewer.
> 
> IPEP and code to do so have to be written.
> 
> Envoy? de mon iPhone
> 
> Le 20 f?vr. 2014 ? 04:58, Aron Ahmadia <aron at ahmadia.net> a ?crit :
> 
>> Let's say I've developed a snazzy CSS file and I want it so that an uploaded notebook being viewed on nbviewer uses my custom CSS.
>> 
>> I've seen this bit of code in the wild[1]:
>> 
>> from IPython.core.display import HTML
>> def css_styling():
>>     styles = open("../styles/custom.css", "r").read()
>>     return HTML(styles)
>> css_styling()
>> 
>> But this seems a bit awkward, since I'm effectively just piping the CSS file into the notebook's rendered HTML.  Does nbviewer know to look anywhere for CSS, or is this the best we have right now?
>> 
>> -A
>> 
>> [1] http://nbviewer.ipython.org/github/barbagroup/AeroPython/blob/master/lessons/00_Lesson00_QuickPythonIntro.ipynb
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140810/be90a84c/attachment.html>

From aron at ahmadia.net  Sun Aug 10 11:40:00 2014
From: aron at ahmadia.net (Aron Ahmadia)
Date: Sun, 10 Aug 2014 11:40:00 -0400
Subject: [IPython-dev] Preferred way to customize CSS for something that
 will be displayed on nbviewer?
In-Reply-To: <8149380B-9E8B-4B25-8067-934AFCF5B1E2@gmail.com>
References: <CAPhiW4htyT1W9Ux-wB5PRQ05+ng7GwaFnKbqN5z=04WS_7CEXw@mail.gmail.com>
	<E8AA2F4F-9562-4091-B852-DA2F01E9543D@gmail.com>
	<8149380B-9E8B-4B25-8067-934AFCF5B1E2@gmail.com>
Message-ID: <CAPhiW4iVZHbdMjfvaSVCeo9Za9aX6Ga3VG-A-vTLrSLmXSrSoA@mail.gmail.com>

I think the latest discussion has been here:
https://github.com/ipython/ipython/issues/5921


On Sun, Aug 10, 2014 at 11:39 AM, Jody Klymak <jklymak at gmail.com> wrote:

> Hi all
>
> Has this been updated yet?  I'd like to share ipython notebooks on
> nbviewer for teaching and would like to change the CSS to make the pages
> easier to read.  What is the current best practice?
>
> Thanks.   Jody
>
>
> On Feb 20, 2014, at  2:54 AM, Matthias Bussonnier <
> bussonniermatthias at gmail.com> wrote:
>
> There should be a discussion on IPython user about that from a few weeks
> ago.
>
> We are thinking on metadata in notebook level that nbviewer could look at
> that allow to select a theme on nbviewer.
>
> IPEP and code to do so have to be written.
>
> Envoy? de mon iPhone
>
> Le 20 f?vr. 2014 ? 04:58, Aron Ahmadia <aron at ahmadia.net> a ?crit :
>
> Let's say I've developed a snazzy CSS file and I want it so that an
> uploaded notebook being viewed on nbviewer uses my custom CSS.
>
> I've seen this bit of code in the wild[1]:
>
> from IPython.core.display import HTML
> def css_styling():
>     styles = open("../styles/custom.css", "r").read()
>     return HTML(styles)
> css_styling()
>
> But this seems a bit awkward, since I'm effectively just piping the CSS
> file into the notebook's rendered HTML.  Does nbviewer know to look
> anywhere for CSS, or is this the best we have right now?
>
> -A
>
> [1]
> http://nbviewer.ipython.org/github/barbagroup/AeroPython/blob/master/lessons/00_Lesson00_QuickPythonIntro.ipynb
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140810/c40d3a4d/attachment.html>

From takowl at gmail.com  Sun Aug 10 20:22:03 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 10 Aug 2014 17:22:03 -0700
Subject: [IPython-dev] Separating the notebook from the file manager+text
	editor+terminal
Message-ID: <CAOvn4qh=EMRByNJyFiQ=RrGrO7Xx2EQzg9VNWKh16Rz3tsQSLA@mail.gmail.com>

We've often said that, for people accessing the notebook on a remote
server, we need to provide certain basic utilities - like the dashboard,
which is now under pressure to evolve into a file manager, and an
in-browser terminal and text editor, both of which we intend to add to the
notebook.

I propose that we separate development and packaging of the notebook from
the project to assemble this trio. The new project would live under the
Jupyter org, and be called something like 'utility-apps'. I see several
advantages of this:

- A different crowd of developers may be attracted to work on that, without
having to know anything about the internals of IPython's machinery for
handling and executing code. For instance, I'm pushing back on turning the
dashboard into a full blown file manager, because I think that adds a lot
of extra complexity in a codebase that's already complex, but if it was a
separate project, it could grow into a file manager without being a problem.
- People running the notebook locally may want to install just the notebook
component, and not poor web-based clones of applications which they have
well developed equivalents for already installed.
- There are lots of potential users of the file manager/text
editor/terminal trio who probably wouldn't use the notebook: for instance,
it could be a basic admin interface to any server.
- It could become a base on which people could write other HTML
applications to meet their own needs. Following our usual MO, I wouldn't
try to design it like this straight away, but keep it in mind, and evolve
the APIs as things mature.

There are still plenty of questions to be answered. How would this new
component interface with the notebook server? What would the notebook look
like without the file manager installed? But these could certainly be
worked out, and I think they're worth working out in order to keep the
scope of the notebook project in check.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140810/7cd0352e/attachment.html>

From moritz.beber at gmail.com  Mon Aug 11 08:52:13 2014
From: moritz.beber at gmail.com (Moritz Beber)
Date: Mon, 11 Aug 2014 14:52:13 +0200
Subject: [IPython-dev] IPython.parallel slow push
Message-ID: <CAFOFTpQJK+04XSx18FAQg5FnUQMFRdrTVGBhoTNiVU0RA8vXmA@mail.gmail.com>

Dear all,

I often find myself in the situation where I use IPython.parallel to push a
bunch of data to the kernels (around 2 GB) and then do calculations over a
large parameter space on that data. This is convenient and simple enough to
do but copying the data takes a large amount of time (it scales with the
number of kernels).

Is this something to be avoided altogether, are there ways to speed it up?

I'd welcome any pointers.

Best,
Moritz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/70615b31/attachment.html>

From wes.turner at gmail.com  Mon Aug 11 09:06:46 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Mon, 11 Aug 2014 08:06:46 -0500
Subject: [IPython-dev] IPython.parallel slow push
In-Reply-To: <CAFOFTpQJK+04XSx18FAQg5FnUQMFRdrTVGBhoTNiVU0RA8vXmA@mail.gmail.com>
References: <CAFOFTpQJK+04XSx18FAQg5FnUQMFRdrTVGBhoTNiVU0RA8vXmA@mail.gmail.com>
Message-ID: <CACfEFw_rKcAj2L_RSMmaCNKpeu4+SjBMZSKpjPexV8u0ZYa45A@mail.gmail.com>

If I understand your use case correctly, each computation node needs a
copy of all 2GB of the same data?

For reference, how does the transfer rate compare to rsync-ing the
same data over SSH?

These may be helpful:

* https://en.wikipedia.org/wiki/Locality_of_reference
* https://en.wikipedia.org/wiki/Clustered_file_system#Examples_2
* https://en.wikipedia.org/wiki/MapReduce#Dataflow
* https://spark.apache.org/
* http://continuum.io/blog/blaze :

> Unlike NumPy, Blaze is designed to handle out-of-core computations on large datasets that exceed the system memory capacity, as well as on distributed and streaming data. Blaze is able to operate on datasets transparently as if they behaved like in-memory NumPy arrays.
>
> We aim to allow analysts and scientists to productively write robust and efficient code, without getting bogged down in the details of how to distribute computation, or worse, how to transport and convert data between databases, formats, proprietary data warehouses, and other silos.
--
Wes Turner


On Mon, Aug 11, 2014 at 7:52 AM, Moritz Beber <moritz.beber at gmail.com> wrote:
> Dear all,
>
> I often find myself in the situation where I use IPython.parallel to push a
> bunch of data to the kernels (around 2 GB) and then do calculations over a
> large parameter space on that data. This is convenient and simple enough to
> do but copying the data takes a large amount of time (it scales with the
> number of kernels).
>
> Is this something to be avoided altogether, are there ways to speed it up?
>
> I'd welcome any pointers.
>
> Best,
> Moritz
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From moritz.beber at gmail.com  Mon Aug 11 09:35:32 2014
From: moritz.beber at gmail.com (Moritz Beber)
Date: Mon, 11 Aug 2014 15:35:32 +0200
Subject: [IPython-dev] IPython.parallel slow push
In-Reply-To: <CACfEFw_rKcAj2L_RSMmaCNKpeu4+SjBMZSKpjPexV8u0ZYa45A@mail.gmail.com>
References: <CAFOFTpQJK+04XSx18FAQg5FnUQMFRdrTVGBhoTNiVU0RA8vXmA@mail.gmail.com>
	<CACfEFw_rKcAj2L_RSMmaCNKpeu4+SjBMZSKpjPexV8u0ZYa45A@mail.gmail.com>
Message-ID: <CAFOFTpRJu7acU64GPpCv8FU_3Nw1Dt9_28E-VgdQ0xcH_uGrgQ@mail.gmail.com>

Hi Wes,

Thank you for the quick response.


On Mon, Aug 11, 2014 at 3:06 PM, Wes Turner <wes.turner at gmail.com> wrote:

> If I understand your use case correctly, each computation node needs a
> copy of all 2GB of the same data?
>
> For reference, how does the transfer rate compare to rsync-ing the
> same data over SSH?
>

I should have been more clear, it's actually a local cluster, i.e., same
host just multiple local kernels. So I have one process that loads the
data, does some pre-processing and then creates a dictionary that it pushes
to the namespace of the other kernels. I understand that the data needs to
go through ZMQ but since it's on the same machine I expected it to be
faster (with 64 kernels it takes about 15 min). If desired I can come up
with a minimal notebook.

Cheers,
Moritz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/f87095d7/attachment.html>

From wes.turner at gmail.com  Mon Aug 11 09:56:16 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Mon, 11 Aug 2014 08:56:16 -0500
Subject: [IPython-dev] IPython.parallel slow push
In-Reply-To: <CAFOFTpRJu7acU64GPpCv8FU_3Nw1Dt9_28E-VgdQ0xcH_uGrgQ@mail.gmail.com>
References: <CAFOFTpQJK+04XSx18FAQg5FnUQMFRdrTVGBhoTNiVU0RA8vXmA@mail.gmail.com>
	<CACfEFw_rKcAj2L_RSMmaCNKpeu4+SjBMZSKpjPexV8u0ZYa45A@mail.gmail.com>
	<CAFOFTpRJu7acU64GPpCv8FU_3Nw1Dt9_28E-VgdQ0xcH_uGrgQ@mail.gmail.com>
Message-ID: <CACfEFw-ZDFQZjQ_Voabq=EpDZNB8cdFPjz=X69Jq5ZvFa0LiSg@mail.gmail.com>

On Mon, Aug 11, 2014 at 8:35 AM, Moritz Beber <moritz.beber at gmail.com> wrote:
> Hi Wes,
>
> Thank you for the quick response.
>
>
> On Mon, Aug 11, 2014 at 3:06 PM, Wes Turner <wes.turner at gmail.com> wrote:
>>
>> If I understand your use case correctly, each computation node needs a
>> copy of all 2GB of the same data?
>>
>> For reference, how does the transfer rate compare to rsync-ing the
>> same data over SSH?
>
>
> I should have been more clear, it's actually a local cluster, i.e., same
> host just multiple local kernels.

Got it. So the network is probably not the primary bottleneck.

> So I have one process that loads the data,
> does some pre-processing and then creates a dictionary that it pushes to the
> namespace of the other kernels.

A 2GB dictionary from RAM to { ... }.

With Celery, similar workflows can be modeled with a "Chord". [1]

> I understand that the data needs to go
> through ZMQ but since it's on the same machine I expected it to be faster
> (with 64 kernels it takes about 15 min).

TBH, I'm not too familiar with IPython.parallel.

This [2] seems to suggest that anything that isn't a buffer,
str/bytes, or numpy array is pickled and copied.

Would it be faster to ETL into something like HDF5 (e.g. w/
Pandas/PyTables) and just synchronize the dataset URI?

[1] http://celery.readthedocs.org/en/latest/userguide/canvas.html#chords
[2] http://ipython.org/ipython-doc/dev/parallel/parallel_details.html#what-is-sendable

> If desired I can come up with a
> minimal notebook.
>
> Cheers,
> Moritz
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


--
Wes Turner


From ellisonbg at gmail.com  Mon Aug 11 10:51:22 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 11 Aug 2014 07:51:22 -0700
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAOvn4qh_fyR_Ed6Nrv3y05hY-aMsGmFBEoutJeafHR2KteYJzA@mail.gmail.com>
References: <CAEX=yabOwzm3qvey_csOBmxX2fcbX7gxiAfAKvd2EzMz=k811Q@mail.gmail.com>
	<CAOvn4qh_fyR_Ed6Nrv3y05hY-aMsGmFBEoutJeafHR2KteYJzA@mail.gmail.com>
Message-ID: <CAH4pYpT5LhRkmSRo7md1bQr+2NB1ZA8YSVt3b64pT7yftG9XAA@mail.gmail.com>

I don't remember that we made the decision to have this behavior. It
is possibly I just don't remember, but today I would consider that to
be a bug. Mark, could you open a GitHub issue on this to trigger a
discussion.

Cheers,

Brian

On Fri, Aug 8, 2014 at 10:51 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> On 8 August 2014 07:00, Mark Bakker <markbak at gmail.com> wrote:
>>
>> What I do is click on 'To import a notebook, drag the file onto the
>> listing below or *click here*. When I click there, a window pops up, I got
>> to the directory with my ipython file, click on it, and then it shows up in
>> the list of files, with next to it a blue Upload button. When I click on the
>> Upload button the notebook is copied to the directory where I started the
>> IPython Notebook.
>>
>> But I don't want to copy the Notebook to that directory (also cause the
>> data files associated with the notebook are still in the old directory). Am
>> I doing something wrong?
>
>
> That is by design, though I agree that it's rather strange. An unfortunate
> result of building the notebook as a browser UI and a web server is that
> some parts behave like a desktop app, and some behave like a web app. The
> original notion of files was designed like a web app - you upload files to
> the server, and it works with the uploaded copy, ignoring the file it
> originally came from (if it wasn't running locally, it wouldn't be able to
> access the original file).
>
> So don't upload notebooks. Start the notebook in the directory where your
> notebooks are, or in a parent directory of that.
>
> My nbopen tool helps with this - it opens a notebook in an already running
> server, or starts a new one if necessary.
> https://github.com/takluyver/nbopen
>
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From takowl at gmail.com  Mon Aug 11 13:03:56 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 11 Aug 2014 10:03:56 -0700
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAH4pYpT5LhRkmSRo7md1bQr+2NB1ZA8YSVt3b64pT7yftG9XAA@mail.gmail.com>
References: <CAEX=yabOwzm3qvey_csOBmxX2fcbX7gxiAfAKvd2EzMz=k811Q@mail.gmail.com>
	<CAOvn4qh_fyR_Ed6Nrv3y05hY-aMsGmFBEoutJeafHR2KteYJzA@mail.gmail.com>
	<CAH4pYpT5LhRkmSRo7md1bQr+2NB1ZA8YSVt3b64pT7yftG9XAA@mail.gmail.com>
Message-ID: <CAOvn4qg+mopcLAnP=0F4CQE6JPV6FvRrzABtPh5-zc5+cZsJyA@mail.gmail.com>

On 11 August 2014 07:51, Brian Granger <ellisonbg at gmail.com> wrote:

> I don't remember that we made the decision to have this behavior. It
> is possibly I just don't remember, but today I would consider that to
> be a bug. Mark, could you open a GitHub issue on this to trigger a
> discussion.
>

I'm pretty sure that what Mark describes is how we expect it to work - when
you upload a file, the server doesn't know where in your filesystem that
data is coming from, so it can't switch to that directory and use the
original notebook there, which I think is what Mark wants. The disconnect
is that the UI implicitly encourages uploading, although that doesn't
really make sense when you're running the server as a local application.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/8940e4d7/attachment.html>

From ellisonbg at gmail.com  Mon Aug 11 13:07:54 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 11 Aug 2014 10:07:54 -0700
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAOvn4qg+mopcLAnP=0F4CQE6JPV6FvRrzABtPh5-zc5+cZsJyA@mail.gmail.com>
References: <CAEX=yabOwzm3qvey_csOBmxX2fcbX7gxiAfAKvd2EzMz=k811Q@mail.gmail.com>
	<CAOvn4qh_fyR_Ed6Nrv3y05hY-aMsGmFBEoutJeafHR2KteYJzA@mail.gmail.com>
	<CAH4pYpT5LhRkmSRo7md1bQr+2NB1ZA8YSVt3b64pT7yftG9XAA@mail.gmail.com>
	<CAOvn4qg+mopcLAnP=0F4CQE6JPV6FvRrzABtPh5-zc5+cZsJyA@mail.gmail.com>
Message-ID: <CAH4pYpTsm2Vykb27Zaa4kJibmvaN14nSgAg6uq0NBASoQCSXgA@mail.gmail.com>

Oh, I might have understood.

Mark can you clarify what exactly you are seeing and what you expect to see?

On Mon, Aug 11, 2014 at 10:03 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> On 11 August 2014 07:51, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> I don't remember that we made the decision to have this behavior. It
>> is possibly I just don't remember, but today I would consider that to
>> be a bug. Mark, could you open a GitHub issue on this to trigger a
>> discussion.
>
>
> I'm pretty sure that what Mark describes is how we expect it to work - when
> you upload a file, the server doesn't know where in your filesystem that
> data is coming from, so it can't switch to that directory and use the
> original notebook there, which I think is what Mark wants. The disconnect is
> that the UI implicitly encourages uploading, although that doesn't really
> make sense when you're running the server as a local application.
>
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From markbak at gmail.com  Mon Aug 11 16:11:15 2014
From: markbak at gmail.com (Mark Bakker)
Date: Mon, 11 Aug 2014 22:11:15 +0200
Subject: [IPython-dev] changing working directory from IPython server
Message-ID: <CAEX=yaZjLsszPKyCZHEMHYLiJcZ_vL5C_hmZgs-tG3vcTGK2hQ@mail.gmail.com>

Thanks for considering this issue.

The behavior now:
I type: ipython notebook to start a notebook.
At the dashboard,  I click on 'To import a notebook, drag the file onto the
listing below or *click here*. When I click there, a window pops up, I go
to the directory with my notebook file, click on it, and then it shows up
in the list of files, with next to it a blue Upload button. When I click on
the Upload button the notebook is copied to the directory where I started
the IPython Notebook.

The problem: The notebook gets copied to the directory where I started
ipython notebook. That is not very useful, as all associated data files are
still in the directory where the notebook originated.

Does that make it clear? Should I file a bug report?

Mark

ps. It seems that the Anaconda guys are also expecting the behavior of the
Notebook to change. They now have a useful 'start notebook' icon, but that
starts in 'Documents/IPython Notebooks. And that faces the same problem as
I described above.


From: Brian Granger <ellisonbg at gmail.com>
> Subject: Re: [IPython-dev] changing working directory from IPython
>         server
> To: IPython developers list <ipython-dev at scipy.org>
> Message-ID:
>         <
> CAH4pYpTsm2Vykb27Zaa4kJibmvaN14nSgAg6uq0NBASoQCSXgA at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Oh, I might have understood.
>
> Mark can you clarify what exactly you are seeing and what you expect to
> see?
>
> On Mon, Aug 11, 2014 at 10:03 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> > On 11 August 2014 07:51, Brian Granger <ellisonbg at gmail.com> wrote:
> >>
> >> I don't remember that we made the decision to have this behavior. It
> >> is possibly I just don't remember, but today I would consider that to
> >> be a bug. Mark, could you open a GitHub issue on this to trigger a
> >> discussion.
> >
> >
> > I'm pretty sure that what Mark describes is how we expect it to work -
> when
> > you upload a file, the server doesn't know where in your filesystem that
> > data is coming from, so it can't switch to that directory and use the
> > original notebook there, which I think is what Mark wants. The
> disconnect is
> > that the UI implicitly encourages uploading, although that doesn't really
> > make sense when you're running the server as a local application.
> >
> > Thomas
> >
> -
> Brian E. Granger
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/91f1acd4/attachment.html>

From markbak at gmail.com  Mon Aug 11 16:14:01 2014
From: markbak at gmail.com (Mark Bakker)
Date: Mon, 11 Aug 2014 22:14:01 +0200
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAEX=yaZjLsszPKyCZHEMHYLiJcZ_vL5C_hmZgs-tG3vcTGK2hQ@mail.gmail.com>
References: <CAEX=yaZjLsszPKyCZHEMHYLiJcZ_vL5C_hmZgs-tG3vcTGK2hQ@mail.gmail.com>
Message-ID: <CAEX=yaZhmxDLZ+-9NYZifhVbZF6uAY6H3NoGo6Z1jzW_akNcWg@mail.gmail.com>

I forgot to add the expected behavior:
I expect that once I select the desired notebook in the dashboard, that it
is opened in the directory where it originates. So it doesn't get copied
and all the associated files are there in the same directory.



On Mon, Aug 11, 2014 at 10:11 PM, Mark Bakker <markbak at gmail.com> wrote:

> Thanks for considering this issue.
>
> The behavior now:
> I type: ipython notebook to start a notebook.
> At the dashboard,  I click on 'To import a notebook, drag the file onto
> the listing below or *click here*. When I click there, a window pops up, I
> go to the directory with my notebook file, click on it, and then it shows
> up in the list of files, with next to it a blue Upload button. When I click
> on the Upload button the notebook is copied to the directory where I
> started the IPython Notebook.
>
> The problem: The notebook gets copied to the directory where I started
> ipython notebook. That is not very useful, as all associated data files are
> still in the directory where the notebook originated.
>
> Does that make it clear? Should I file a bug report?
>
> Mark
>
> ps. It seems that the Anaconda guys are also expecting the behavior of the
> Notebook to change. They now have a useful 'start notebook' icon, but that
> starts in 'Documents/IPython Notebooks. And that faces the same problem as
> I described above.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/dff2c5bb/attachment.html>

From ellisonbg at gmail.com  Mon Aug 11 16:20:47 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 11 Aug 2014 13:20:47 -0700
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAEX=yaZjLsszPKyCZHEMHYLiJcZ_vL5C_hmZgs-tG3vcTGK2hQ@mail.gmail.com>
References: <CAEX=yaZjLsszPKyCZHEMHYLiJcZ_vL5C_hmZgs-tG3vcTGK2hQ@mail.gmail.com>
Message-ID: <CAH4pYpTmjmdTpjbv3UKe=UdrTukpheV2hqcJeebH-zzuPdQ4wQ@mail.gmail.com>

Mark,

OK I think I understand, and I don't consider this a bug. If the
notebook you are starting out with is on the same machine, there is no
reason to "upload" that notebook. Just navigate the notebook dashboard
to the directory where that notebook already is and open it. Is there
some reason you can't do that and are instead trying to upload the
notebook.

Cheers,

Brian

On Mon, Aug 11, 2014 at 1:11 PM, Mark Bakker <markbak at gmail.com> wrote:
> Thanks for considering this issue.
>
> The behavior now:
> I type: ipython notebook to start a notebook.
> At the dashboard,  I click on 'To import a notebook, drag the file onto the
> listing below or *click here*. When I click there, a window pops up, I go to
> the directory with my notebook file, click on it, and then it shows up in
> the list of files, with next to it a blue Upload button. When I click on the
> Upload button the notebook is copied to the directory where I started the
> IPython Notebook.
>
> The problem: The notebook gets copied to the directory where I started
> ipython notebook. That is not very useful, as all associated data files are
> still in the directory where the notebook originated.
>
> Does that make it clear? Should I file a bug report?
>
> Mark
>
> ps. It seems that the Anaconda guys are also expecting the behavior of the
> Notebook to change. They now have a useful 'start notebook' icon, but that
> starts in 'Documents/IPython Notebooks. And that faces the same problem as I
> described above.
>
>
>> From: Brian Granger <ellisonbg at gmail.com>
>> Subject: Re: [IPython-dev] changing working directory from IPython
>>         server
>> To: IPython developers list <ipython-dev at scipy.org>
>> Message-ID:
>>
>> <CAH4pYpTsm2Vykb27Zaa4kJibmvaN14nSgAg6uq0NBASoQCSXgA at mail.gmail.com>
>> Content-Type: text/plain; charset=UTF-8
>>
>> Oh, I might have understood.
>>
>> Mark can you clarify what exactly you are seeing and what you expect to
>> see?
>>
>> On Mon, Aug 11, 2014 at 10:03 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>> > On 11 August 2014 07:51, Brian Granger <ellisonbg at gmail.com> wrote:
>> >>
>> >> I don't remember that we made the decision to have this behavior. It
>> >> is possibly I just don't remember, but today I would consider that to
>> >> be a bug. Mark, could you open a GitHub issue on this to trigger a
>> >> discussion.
>> >
>> >
>> > I'm pretty sure that what Mark describes is how we expect it to work -
>> > when
>> > you upload a file, the server doesn't know where in your filesystem that
>> > data is coming from, so it can't switch to that directory and use the
>> > original notebook there, which I think is what Mark wants. The
>> > disconnect is
>> > that the UI implicitly encourages uploading, although that doesn't
>> > really
>> > make sense when you're running the server as a local application.
>> >
>> > Thomas
>> >
>> -
>> Brian E. Granger
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From ksmith at enthought.com  Mon Aug 11 16:38:12 2014
From: ksmith at enthought.com (Kurt Smith)
Date: Mon, 11 Aug 2014 15:38:12 -0500
Subject: [IPython-dev] ANN: DistArray 0.5 release
Message-ID: <CA+qbRt4e5QFXfgk=F+hYF6Qjo9-GgOmm0E-W-VcO=EZ58N39zA@mail.gmail.com>

======================================
DistArray 0.5 release
======================================

**Mailing list:** distarray at googlegroups.com

**Documentation:** http://distarray.readthedocs.org

**License:** Three-clause BSD

**Python versions:** 2.7, 3.3, and 3.4

**OS support:** \*nix and Mac OS X

What is DistArray?
------------------

DistArray aims to bring the ease-of-use of NumPy to data-parallel
high-performance computing.  It provides distributed multi-dimensional NumPy
arrays, distributed ufuncs, and distributed IO capabilities.  It can
efficiently interoperate with external distributed libraries like Trilinos.
DistArray works with NumPy and builds on top of it in a flexible and natural
way.

0.5 Release
-----------

Noteworthy improvements in this release include:

* closer alignment with NumPy's API,
* support for Python 3.4 (existing support for Python 2.7 and 3.3),
* a performance-oriented MPI-only mode for deployment on clusters and
  supercomputers,
* a way to register user-defined functions to be callable locally on worker
  processes,
* more consistent naming of sub-packages,
* testing with MPICH2 (already tested against OpenMPI),
* improved and expanded examples,
* installed version testable via ``distarray.test()``, and
* performance and scaling improvements.

With this release, DistArray ready for real-world testing and deployment.
 The
project is still evolving rapidly and we appreciate the continued input from
the larger scientific-Python community.

Existing features
-----------------

DistArray:

* supports NumPy-like slicing, reductions, and ufuncs on distributed
  multidimensional arrays;
* has a client-engine process design -- data resides on the worker
processes,
  commands are initiated from master;
* allows full control over what is executed on the worker processes and
  integrates transparently with the master process;
* allows direct communication between workers, bypassing the master process
  for scalability;
* integrates with IPython.parallel for interactive creation and exploration
of
  distributed data;
* supports distributed ufuncs (currently without broadcasting);
* builds on and leverages MPI via MPI4Py in a transparent and user-friendly
  way;
* has basic support for unstructured arrays;
* supports user-controllable array distributions across workers (block,
  cyclic, block-cyclic, and unstructured) on a per-axis basis;
* has a straightforward API to control how an array is distributed;
* has basic plotting support for visualization of array distributions;
* separates the array?s distribution from the array?s data -- useful for
  slicing, reductions, redistribution, broadcasting, and other operations;
* implements distributed random arrays;
* supports ``.npy``-like flat-file IO and hdf5 parallel IO (via ``h5py``);
  leverages MPI-based IO parallelism in an easy-to-use and transparent way;
  and
* supports the distributed array protocol [protocol]_, which allows
  independently developed parallel libraries to share distributed arrays
  without copying, analogous to the PEP-3118 new buffer protocol.

Planned features and roadmap
----------------------------

Near-term features and improvements include:

* array re-distribution capabilities;
* lazy evaluation and deferred computation for latency hiding;
* interoperation with Trilinos [Trilinos]_; and
* distributed broadcasting support.

The longer-term roadmap includes:

* Integration with other packages [petsc]_ that subscribe to the distributed
  array protocol [protocol]_;
* Distributed fancy indexing;
* Out-of-core computations;
* Support for distributed sorting and other non-trivial distributed
  algorithms; and
* End-user control over communication and temporary array creation, and
other
  performance aspects of distributed computations.

History and funding
-------------------

Brian Granger started DistArray as a NASA-funded SBIR project in 2008.
Enthought picked it up as part of a DOE Phase II SBIR [SBIR]_ to provide a
generally useful distributed array package.  It builds on NumPy, MPI,
MPI4Py,
IPython, IPython.parallel, and interfaces with the Trilinos suite of
distributed HPC solvers (via PyTrilinos [Trilinos]_).

This material is based upon work supported by the Department of Energy under
Award Number DE-SC0007699.

This report was prepared as an account of work sponsored by an agency of the
United States Government.  Neither the United States Government nor any
agency
thereof, nor any of their employees, makes any warranty, express or implied,
or assumes any legal liability or responsibility for the accuracy,
completeness, or usefulness of any information, apparatus, product, or
process
disclosed, or represents that its use would not infringe privately owned
rights.  Reference herein to any specific commercial product, process, or
service by trade name, trademark, manufacturer, or otherwise does not
necessarily constitute or imply its endorsement, recommendation, or favoring
by the United States Government or any agency thereof.  The views and
opinions
of authors expressed herein do not necessarily state or reflect those of the
United States Government or any agency thereof.


.. [protocol]
http://distributed-array-protocol.readthedocs.org/en/rel-0.10.0/
.. [Trilinos] http://trilinos.org/
.. [petsc] http://www.mcs.anl.gov/petsc/
.. [SBIR] http://www.sbir.gov/sbirsearch/detail/410257

-- 
Kurt W. Smith, Ph.D. <ksmith at enthought.com>
Enthought, Inc. <http://www.enthought.com>  | 512.536.1057
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/491719f7/attachment.html>

From ellisonbg at gmail.com  Mon Aug 11 16:47:15 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 11 Aug 2014 13:47:15 -0700
Subject: [IPython-dev] ANN: DistArray 0.5 release
In-Reply-To: <CA+qbRt4e5QFXfgk=F+hYF6Qjo9-GgOmm0E-W-VcO=EZ58N39zA@mail.gmail.com>
References: <CA+qbRt4e5QFXfgk=F+hYF6Qjo9-GgOmm0E-W-VcO=EZ58N39zA@mail.gmail.com>
Message-ID: <CAH4pYpTtBmhUFDKcYct99_6i8_1M+e7ZCmZ+Bcirha=QY462kg@mail.gmail.com>

Kurt,

Thanks for sharing, this looks great!

Cheers,

Brian

On Mon, Aug 11, 2014 at 1:38 PM, Kurt Smith <ksmith at enthought.com> wrote:
> ======================================
> DistArray 0.5 release
> ======================================
>
> **Mailing list:** distarray at googlegroups.com
>
> **Documentation:** http://distarray.readthedocs.org
>
> **License:** Three-clause BSD
>
> **Python versions:** 2.7, 3.3, and 3.4
>
> **OS support:** \*nix and Mac OS X
>
> What is DistArray?
> ------------------
>
> DistArray aims to bring the ease-of-use of NumPy to data-parallel
> high-performance computing.  It provides distributed multi-dimensional NumPy
> arrays, distributed ufuncs, and distributed IO capabilities.  It can
> efficiently interoperate with external distributed libraries like Trilinos.
> DistArray works with NumPy and builds on top of it in a flexible and natural
> way.
>
> 0.5 Release
> -----------
>
> Noteworthy improvements in this release include:
>
> * closer alignment with NumPy's API,
> * support for Python 3.4 (existing support for Python 2.7 and 3.3),
> * a performance-oriented MPI-only mode for deployment on clusters and
>   supercomputers,
> * a way to register user-defined functions to be callable locally on worker
>   processes,
> * more consistent naming of sub-packages,
> * testing with MPICH2 (already tested against OpenMPI),
> * improved and expanded examples,
> * installed version testable via ``distarray.test()``, and
> * performance and scaling improvements.
>
> With this release, DistArray ready for real-world testing and deployment.
> The
> project is still evolving rapidly and we appreciate the continued input from
> the larger scientific-Python community.
>
> Existing features
> -----------------
>
> DistArray:
>
> * supports NumPy-like slicing, reductions, and ufuncs on distributed
>   multidimensional arrays;
> * has a client-engine process design -- data resides on the worker
> processes,
>   commands are initiated from master;
> * allows full control over what is executed on the worker processes and
>   integrates transparently with the master process;
> * allows direct communication between workers, bypassing the master process
>   for scalability;
> * integrates with IPython.parallel for interactive creation and exploration
> of
>   distributed data;
> * supports distributed ufuncs (currently without broadcasting);
> * builds on and leverages MPI via MPI4Py in a transparent and user-friendly
>   way;
> * has basic support for unstructured arrays;
> * supports user-controllable array distributions across workers (block,
>   cyclic, block-cyclic, and unstructured) on a per-axis basis;
> * has a straightforward API to control how an array is distributed;
> * has basic plotting support for visualization of array distributions;
> * separates the array?s distribution from the array?s data -- useful for
>   slicing, reductions, redistribution, broadcasting, and other operations;
> * implements distributed random arrays;
> * supports ``.npy``-like flat-file IO and hdf5 parallel IO (via ``h5py``);
>   leverages MPI-based IO parallelism in an easy-to-use and transparent way;
>   and
> * supports the distributed array protocol [protocol]_, which allows
>   independently developed parallel libraries to share distributed arrays
>   without copying, analogous to the PEP-3118 new buffer protocol.
>
> Planned features and roadmap
> ----------------------------
>
> Near-term features and improvements include:
>
> * array re-distribution capabilities;
> * lazy evaluation and deferred computation for latency hiding;
> * interoperation with Trilinos [Trilinos]_; and
> * distributed broadcasting support.
>
> The longer-term roadmap includes:
>
> * Integration with other packages [petsc]_ that subscribe to the distributed
>   array protocol [protocol]_;
> * Distributed fancy indexing;
> * Out-of-core computations;
> * Support for distributed sorting and other non-trivial distributed
>   algorithms; and
> * End-user control over communication and temporary array creation, and
> other
>   performance aspects of distributed computations.
>
> History and funding
> -------------------
>
> Brian Granger started DistArray as a NASA-funded SBIR project in 2008.
> Enthought picked it up as part of a DOE Phase II SBIR [SBIR]_ to provide a
> generally useful distributed array package.  It builds on NumPy, MPI,
> MPI4Py,
> IPython, IPython.parallel, and interfaces with the Trilinos suite of
> distributed HPC solvers (via PyTrilinos [Trilinos]_).
>
> This material is based upon work supported by the Department of Energy under
> Award Number DE-SC0007699.
>
> This report was prepared as an account of work sponsored by an agency of the
> United States Government.  Neither the United States Government nor any
> agency
> thereof, nor any of their employees, makes any warranty, express or implied,
> or assumes any legal liability or responsibility for the accuracy,
> completeness, or usefulness of any information, apparatus, product, or
> process
> disclosed, or represents that its use would not infringe privately owned
> rights.  Reference herein to any specific commercial product, process, or
> service by trade name, trademark, manufacturer, or otherwise does not
> necessarily constitute or imply its endorsement, recommendation, or favoring
> by the United States Government or any agency thereof.  The views and
> opinions
> of authors expressed herein do not necessarily state or reflect those of the
> United States Government or any agency thereof.
>
>
> .. [protocol]
> http://distributed-array-protocol.readthedocs.org/en/rel-0.10.0/
> .. [Trilinos] http://trilinos.org/
> .. [petsc] http://www.mcs.anl.gov/petsc/
> .. [SBIR] http://www.sbir.gov/sbirsearch/detail/410257
>
> --
> Kurt W. Smith, Ph.D.
> Enthought, Inc.  | 512.536.1057
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From markbak at gmail.com  Mon Aug 11 16:56:19 2014
From: markbak at gmail.com (Mark Bakker)
Date: Mon, 11 Aug 2014 22:56:19 +0200
Subject: [IPython-dev] changing working directory from IPython server
Message-ID: <CAEX=yabmKenaQkcv7+4eFELpDD7WHwLZ9FZarNBfxp_t-5sRTw@mail.gmail.com>

Sorry to keep pestering you with this, Brian.

I explored it some more, and narrowed it down to the following.

Let's say my notebook is in /mark/notebooks

When I start ipython notebook in /mark, I can nicely click on the directory
notebooks on the dashboard and open the notebook I want.

If, on the other hand, I start ipython notebook in /mark/Documents. Then
there is no way to 'surf' to the correct /mark/notebooks, as there is no
cd.. place to click to go back to /mark and then to /mark/notebooks. When I
do the  'To import a notebook, drag the file onto the listing below or
*click here*' option, it only let's me 'upload' the selected notebook,
which means copying to the directory where I started ipython notebook.

The Anaconda guys run into exactly this problem, as their nice Notebook
icon starts in /user/Documents/IPython_Notebooks, and you can only surf to
a directory lower than that.

So what I think we need is either a way to surf to a higher directory, or
to have the option to 'click here' as it is now but then not being forced
to 'upload' the notebook.

Mark


From: Brian Granger <ellisonbg at gmail.com>
> Subject: Re: [IPython-dev] changing working directory from IPython
>         server
> To: IPython developers list <ipython-dev at scipy.org>
> Message-ID:
>         <CAH4pYpTmjmdTpjbv3UKe=
> UdrTukpheV2hqcJeebH-zzuPdQ4wQ at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Mark,
>
> OK I think I understand, and I don't consider this a bug. If the
> notebook you are starting out with is on the same machine, there is no
> reason to "upload" that notebook. Just navigate the notebook dashboard
> to the directory where that notebook already is and open it. Is there
> some reason you can't do that and are instead trying to upload the
> notebook.
>
> Cheers,
>
> Brian
>
> On Mon, Aug 11, 2014 at 1:11 PM, Mark Bakker <markbak at gmail.com> wrote:
> > Thanks for considering this issue.
> >
> > The behavior now:
> > I type: ipython notebook to start a notebook.
> > At the dashboard,  I click on 'To import a notebook, drag the file onto
> the
> > listing below or *click here*. When I click there, a window pops up, I
> go to
> > the directory with my notebook file, click on it, and then it shows up in
> > the list of files, with next to it a blue Upload button. When I click on
> the
> > Upload button the notebook is copied to the directory where I started the
> > IPython Notebook.
> >
> > The problem: The notebook gets copied to the directory where I started
> > ipython notebook. That is not very useful, as all associated data files
> are
> > still in the directory where the notebook originated.
> >
> > Does that make it clear? Should I file a bug report?
> >
> > Mark
> >
> > ps. It seems that the Anaconda guys are also expecting the behavior of
> the
> > Notebook to change. They now have a useful 'start notebook' icon, but
> that
> > starts in 'Documents/IPython Notebooks. And that faces the same problem
> as I
> > described above.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/7e4372a4/attachment.html>

From ellisonbg at gmail.com  Mon Aug 11 17:10:35 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 11 Aug 2014 17:10:35 -0400
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAEX=yabmKenaQkcv7+4eFELpDD7WHwLZ9FZarNBfxp_t-5sRTw@mail.gmail.com>
References: <CAEX=yabmKenaQkcv7+4eFELpDD7WHwLZ9FZarNBfxp_t-5sRTw@mail.gmail.com>
Message-ID: <88388EF9-C204-4E06-8003-6E4BE94708EA@gmail.com>

For security reasons we can't allow the notebook to browse to directories higher than it's starting directory. Thus, the only solution is to start the notebook from the highest directory you want to access notebooks from.

Given how the notebook works, I think that anaconda has made a less than ideal choice by not starting the notebook server in the users home directory.

Cheers,

Brian

Sent from my iPhone

> On Aug 11, 2014, at 4:56 PM, Mark Bakker <markbak at gmail.com> wrote:
> 
> Sorry to keep pestering you with this, Brian.
> 
> I explored it some more, and narrowed it down to the following.
> 
> Let's say my notebook is in /mark/notebooks
> 
> When I start ipython notebook in /mark, I can nicely click on the directory notebooks on the dashboard and open the notebook I want.
> 
> If, on the other hand, I start ipython notebook in /mark/Documents. Then there is no way to 'surf' to the correct /mark/notebooks, as there is no cd.. place to click to go back to /mark and then to /mark/notebooks. When I do the  'To import a notebook, drag the file onto the listing below or *click here*' option, it only let's me 'upload' the selected notebook, which means copying to the directory where I started ipython notebook.
> 
> The Anaconda guys run into exactly this problem, as their nice Notebook icon starts in /user/Documents/IPython_Notebooks, and you can only surf to a directory lower than that.
> 
> So what I think we need is either a way to surf to a higher directory, or to have the option to 'click here' as it is now but then not being forced to 'upload' the notebook.
> 
> Mark
> 
> 
>> From: Brian Granger <ellisonbg at gmail.com>
>> Subject: Re: [IPython-dev] changing working directory from IPython
>>         server
>> To: IPython developers list <ipython-dev at scipy.org>
>> Message-ID:
>>         <CAH4pYpTmjmdTpjbv3UKe=UdrTukpheV2hqcJeebH-zzuPdQ4wQ at mail.gmail.com>
>> Content-Type: text/plain; charset=UTF-8
>> 
>> Mark,
>> 
>> OK I think I understand, and I don't consider this a bug. If the
>> notebook you are starting out with is on the same machine, there is no
>> reason to "upload" that notebook. Just navigate the notebook dashboard
>> to the directory where that notebook already is and open it. Is there
>> some reason you can't do that and are instead trying to upload the
>> notebook.
>> 
>> Cheers,
>> 
>> Brian
>> 
>> On Mon, Aug 11, 2014 at 1:11 PM, Mark Bakker <markbak at gmail.com> wrote:
>> > Thanks for considering this issue.
>> >
>> > The behavior now:
>> > I type: ipython notebook to start a notebook.
>> > At the dashboard,  I click on 'To import a notebook, drag the file onto the
>> > listing below or *click here*. When I click there, a window pops up, I go to
>> > the directory with my notebook file, click on it, and then it shows up in
>> > the list of files, with next to it a blue Upload button. When I click on the
>> > Upload button the notebook is copied to the directory where I started the
>> > IPython Notebook.
>> >
>> > The problem: The notebook gets copied to the directory where I started
>> > ipython notebook. That is not very useful, as all associated data files are
>> > still in the directory where the notebook originated.
>> >
>> > Does that make it clear? Should I file a bug report?
>> >
>> > Mark
>> >
>> > ps. It seems that the Anaconda guys are also expecting the behavior of the
>> > Notebook to change. They now have a useful 'start notebook' icon, but that
>> > starts in 'Documents/IPython Notebooks. And that faces the same problem as I
>> > described above.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/14384d33/attachment.html>

From fperez.net at gmail.com  Mon Aug 11 18:34:49 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 11 Aug 2014 15:34:49 -0700
Subject: [IPython-dev] ANN: DistArray 0.5 release
In-Reply-To: <CAH4pYpTtBmhUFDKcYct99_6i8_1M+e7ZCmZ+Bcirha=QY462kg@mail.gmail.com>
References: <CA+qbRt4e5QFXfgk=F+hYF6Qjo9-GgOmm0E-W-VcO=EZ58N39zA@mail.gmail.com>
	<CAH4pYpTtBmhUFDKcYct99_6i8_1M+e7ZCmZ+Bcirha=QY462kg@mail.gmail.com>
Message-ID: <CAHAreOoUFpbX5OKpHpxxcVrGxc+D4cML3Y=C12H-VJTZGLC8cQ@mail.gmail.com>

On Mon, Aug 11, 2014 at 1:47 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Thanks for sharing, this looks great!


Indeed, and I'm very happy to see that IPython has made a contribution to
distarray :)  Great job, Kurt & team!

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/43df5186/attachment.html>

From fperez.net at gmail.com  Mon Aug 11 18:38:09 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 11 Aug 2014 15:38:09 -0700
Subject: [IPython-dev] IPython.parallel slow push
In-Reply-To: <CACfEFw-ZDFQZjQ_Voabq=EpDZNB8cdFPjz=X69Jq5ZvFa0LiSg@mail.gmail.com>
References: <CAFOFTpQJK+04XSx18FAQg5FnUQMFRdrTVGBhoTNiVU0RA8vXmA@mail.gmail.com>
	<CACfEFw_rKcAj2L_RSMmaCNKpeu4+SjBMZSKpjPexV8u0ZYa45A@mail.gmail.com>
	<CAFOFTpRJu7acU64GPpCv8FU_3Nw1Dt9_28E-VgdQ0xcH_uGrgQ@mail.gmail.com>
	<CACfEFw-ZDFQZjQ_Voabq=EpDZNB8cdFPjz=X69Jq5ZvFa0LiSg@mail.gmail.com>
Message-ID: <CAHAreOq9ehh3wBuEvx+kNLnio_XJ0U3Pr_QHEKw+yG_gEraxuA@mail.gmail.com>

On Mon, Aug 11, 2014 at 6:56 AM, Wes Turner <wes.turner at gmail.com> wrote:

> This [2] seems to suggest that anything that isn't a buffer,
> str/bytes, or numpy array is pickled and copied.
>

That is indeed correct.


>  Would it be faster to ETL into something like HDF5 (e.g. w/
> Pandas/PyTables) and just synchronize the dataset URI?
>

Absolutely.

IPython.parallel is NOT the right tool to use to move large amounts of data
around between machines. It's an important problem in parallel/distributed
computing, but also a very challenging one that is beyond our scope and
resources.

When using IPython.parallel, you should think of it as a good way to

- coordinate computation
- move code around
- move *small* data around
- have interactive control in parallel settings

But you should have a non-IPython strategy for moving big chunks of data
around. The right answer to that question will vary from one context to
another. In some cases a simple NFS mount may be enough, elsewhere
something like Hadoop FS or Disco FS may work, or a well-sharded database,
or whatever.

But it's simply a problem that we consider orthogonal to what
IPython.parallel can do well.

Hope this helps,

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/390234bc/attachment.html>

From takowl at gmail.com  Mon Aug 11 18:47:01 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 11 Aug 2014 15:47:01 -0700
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAEX=yabmKenaQkcv7+4eFELpDD7WHwLZ9FZarNBfxp_t-5sRTw@mail.gmail.com>
References: <CAEX=yabmKenaQkcv7+4eFELpDD7WHwLZ9FZarNBfxp_t-5sRTw@mail.gmail.com>
Message-ID: <CAOvn4qiihxEUH2Vjy9m-4bS4F9cXpasbgiCP0E0f5g5pN0DHuA@mail.gmail.com>

On 11 August 2014 13:56, Mark Bakker <markbak at gmail.com> wrote:

>  So what I think we need is either a way to surf to a higher directory, or
> to have the option to 'click here' as it is now but then not being forced
> to 'upload' the notebook.
>

Allow me to plug nbopen again: it's a tool to open notebooks. If there's a
notebook server running in any directory above where that notebook is, it
will use that. If not, it will start a new notebook server which can open
that. So there's no need to think about where you're launching the
notebook. It works from the command line on all platforms, and from GUI
file managers on open source platforms (pull requests for other platforms
welcome).

https://github.com/takluyver/nbopen

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/e8334333/attachment.html>

From cyrille.rossant at gmail.com  Tue Aug 12 00:16:00 2014
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Tue, 12 Aug 2014 09:16:00 +0500
Subject: [IPython-dev] IPython event loop integration
In-Reply-To: <CA+-1RQSXLFLk_e0PF35ozscoAyzHYYsCu4TehWTUzU1+F-SiJw@mail.gmail.com>
References: <CA+-1RQSXLFLk_e0PF35ozscoAyzHYYsCu4TehWTUzU1+F-SiJw@mail.gmail.com>
Message-ID: <CA+-1RQRoUwzStSN-scN+UCXUE3WjcGgZ=RL-ZwFStb9Wm6XYyg@mail.gmail.com>

Hi all,

We're currently implementing an IPython notebook backend for Vispy.
We're having a hard time figuring out how to integrate timer-based
animations in the notebook [1]. We'd really appreciate if one of you
could have a look at our problem. This is the last step of our GSoC
student's project, which terminates in a few days.

Many thanks,
Cyrille

[1] https://groups.google.com/forum/#!topic/vispy-dev/blh2odeMgvI
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/7afaede3/attachment.html>

From dave.hirschfeld at gmail.com  Tue Aug 12 03:51:21 2014
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Tue, 12 Aug 2014 07:51:21 +0000 (UTC)
Subject: [IPython-dev] changing working directory from IPython server
References: <CAEX=yabmKenaQkcv7+4eFELpDD7WHwLZ9FZarNBfxp_t-5sRTw@mail.gmail.com>
Message-ID: <loom.20140812T094821-434@post.gmane.org>

Mark Bakker <markbak <at> gmail.com> writes:

> 
> 
> Sorry to keep pestering you with this, Brian.
> 
> I explored it some more, and narrowed it down to the following.
> 
> Let's say my notebook is in /mark/notebooks
> 
> 
> When I start ipython notebook in /mark, I can nicely click on the 
directory notebooks on the dashboard and open the notebook I want.
> 
> If, on the other hand, I start ipython notebook in /mark/Documents. Then 
there is no way to 'surf' to the correct /mark/notebooks, as there is no 
cd.. place to click to go back to /mark and then to /mark/notebooks. When I 
do the ?'To import a notebook, drag the file onto the listing below or 
*click here*' option, it only let's me 'upload' the selected notebook, which 
means copying to the directory where I started ipython notebook.
> 
> 
> So what I think we need is either a way to surf to a higher directory, or 
to have the option to 'click here' as it is now but then not being forced to 
'upload' the notebook.
> 
> Mark
>

One solution if you have folders in different locations would be to simlink 
them (mklink /J in windows) into the notebook server directory

HTH,
Dave





From markbak at gmail.com  Tue Aug 12 04:16:23 2014
From: markbak at gmail.com (Mark Bakker)
Date: Tue, 12 Aug 2014 10:16:23 +0200
Subject: [IPython-dev] changing working directory from IPython server
Message-ID: <CAEX=yaabdoTxP0bV1hOKc--+ahZ6+GaGa-WCY3E_Kjsjjt2Q9Q@mail.gmail.com>

Brian,

All right, so browsing to a higher directory is out.
What about the 'uploading' feature. Would it not be possible to use the  'To
import a notebook, drag the file onto the listing below or *click here*'
option and then not be forced to upload the notebook? Or are there security
issues with that as well?

Mark

ps. I hope the Anaconda guys are reading this and will consider changing
their default starting location, as my students really like their clickable
icon to start ipython notebooks.


From: Brian Granger <ellisonbg at gmail.com>
> Subject: Re: [IPython-dev] changing working directory from IPython
>         server
> To: IPython developers list <ipython-dev at scipy.org>
> Message-ID: <88388EF9-C204-4E06-8003-6E4BE94708EA at gmail.com>
> Content-Type: text/plain; charset="us-ascii"
>
> For security reasons we can't allow the notebook to browse to directories
> higher than it's starting directory. Thus, the only solution is to start
> the notebook from the highest directory you want to access notebooks from.
>
> Given how the notebook works, I think that anaconda has made a less than
> ideal choice by not starting the notebook server in the users home
> directory.
>
> Cheers,
>
> Brian
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/2ec846b4/attachment.html>

From moritz.beber at gmail.com  Tue Aug 12 06:31:15 2014
From: moritz.beber at gmail.com (Moritz Beber)
Date: Tue, 12 Aug 2014 12:31:15 +0200
Subject: [IPython-dev] IPython.parallel slow push
In-Reply-To: <CAHAreOq9ehh3wBuEvx+kNLnio_XJ0U3Pr_QHEKw+yG_gEraxuA@mail.gmail.com>
References: <CAFOFTpQJK+04XSx18FAQg5FnUQMFRdrTVGBhoTNiVU0RA8vXmA@mail.gmail.com>
	<CACfEFw_rKcAj2L_RSMmaCNKpeu4+SjBMZSKpjPexV8u0ZYa45A@mail.gmail.com>
	<CAFOFTpRJu7acU64GPpCv8FU_3Nw1Dt9_28E-VgdQ0xcH_uGrgQ@mail.gmail.com>
	<CACfEFw-ZDFQZjQ_Voabq=EpDZNB8cdFPjz=X69Jq5ZvFa0LiSg@mail.gmail.com>
	<CAHAreOq9ehh3wBuEvx+kNLnio_XJ0U3Pr_QHEKw+yG_gEraxuA@mail.gmail.com>
Message-ID: <CAFOFTpQKQyePBFY0RG_pf=6agFRiH738yoNndN1UhdbFY99Vcg@mail.gmail.com>

On Tue, Aug 12, 2014 at 12:38 AM, Fernando Perez <fperez.net at gmail.com>
wrote:

> On Mon, Aug 11, 2014 at 6:56 AM, Wes Turner <wes.turner at gmail.com> wrote:
>
>> This [2] seems to suggest that anything that isn't a buffer,
>> str/bytes, or numpy array is pickled and copied.
>>
>
> That is indeed correct.
>
>
>>  Would it be faster to ETL into something like HDF5 (e.g. w/
>> Pandas/PyTables) and just synchronize the dataset URI?
>>
>
> Absolutely.
>
> IPython.parallel is NOT the right tool to use to move large amounts of
> data around between machines. It's an important problem in
> parallel/distributed computing, but also a very challenging one that is
> beyond our scope and resources.
>

As I said, I didn't move anything between machines, just locally. Still it
uses ZMQ and I get that IPython is not meant to handle this situation.
Simply using a shelve (relying on pickle here) and loading the contents in
each kernel already improved the time needed a lot.


>
> When using IPython.parallel, you should think of it as a good way to
>
> - coordinate computation
> - move code around
> - move *small* data around
> - have interactive control in parallel settings
>
> But you should have a non-IPython strategy for moving big chunks of data
> around. The right answer to that question will vary from one context to
> another. In some cases a simple NFS mount may be enough, elsewhere
> something like Hadoop FS or Disco FS may work, or a well-sharded database,
> or whatever.
>
> But it's simply a problem that we consider orthogonal to what
> IPython.parallel can do well.
>
> Hope this helps,
>
> f
>
> Thank you for your input.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/9349fc38/attachment.html>

From takowl at gmail.com  Tue Aug 12 12:36:25 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 12 Aug 2014 09:36:25 -0700
Subject: [IPython-dev] IPython event loop integration
In-Reply-To: <CA+-1RQRoUwzStSN-scN+UCXUE3WjcGgZ=RL-ZwFStb9Wm6XYyg@mail.gmail.com>
References: <CA+-1RQSXLFLk_e0PF35ozscoAyzHYYsCu4TehWTUzU1+F-SiJw@mail.gmail.com>
	<CA+-1RQRoUwzStSN-scN+UCXUE3WjcGgZ=RL-ZwFStb9Wm6XYyg@mail.gmail.com>
Message-ID: <CAOvn4qgB=sysq_njD3dodEV49ryA2+_PQG4M77HqQV2kwkaaUA@mail.gmail.com>

Hi Cyrille,

Having a quick scan through the thread you linked to, I wonder if it's
possible to use the kernel event loop to schedule update events. pyzmq
subclasses tornado's event loop, and provides a DelayedCallback class [1],
and tornado itself has a PeriodicCallback class [2].

[1]
http://zeromq.github.io/pyzmq/api/generated/zmq.eventloop.ioloop.html#delayedcallback
[2]
http://tornado.readthedocs.org/en/latest/ioloop.html#tornado.ioloop.PeriodicCallback

Thomas


On 11 August 2014 21:16, Cyrille Rossant <cyrille.rossant at gmail.com> wrote:

> Hi all,
>
> We're currently implementing an IPython notebook backend for Vispy.
> We're having a hard time figuring out how to integrate timer-based
> animations in the notebook [1]. We'd really appreciate if one of you
> could have a look at our problem. This is the last step of our GSoC
> student's project, which terminates in a few days.
>
> Many thanks,
> Cyrille
>
> [1] https://groups.google.com/forum/#!topic/vispy-dev/blh2odeMgvI
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/e99a5bc4/attachment.html>

From takowl at gmail.com  Tue Aug 12 12:38:54 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 12 Aug 2014 09:38:54 -0700
Subject: [IPython-dev] changing working directory from IPython server
In-Reply-To: <CAEX=yaabdoTxP0bV1hOKc--+ahZ6+GaGa-WCY3E_Kjsjjt2Q9Q@mail.gmail.com>
References: <CAEX=yaabdoTxP0bV1hOKc--+ahZ6+GaGa-WCY3E_Kjsjjt2Q9Q@mail.gmail.com>
Message-ID: <CAOvn4qjSSA=1w8=SNmCB-j2MXQtfBQRCmrPqf6bLFDrKc3D_Lg@mail.gmail.com>

On 12 August 2014 01:16, Mark Bakker <markbak at gmail.com> wrote:

> 'To import a notebook, drag the file onto the listing below or *click
> here*' option and then not be forced to upload the notebook? Or are there
> security issues with that as well?


As far as I know, the upload interface sees only the contents of the file,
not the location, so it's impossible to work with it on disk from its
original location based on pulling it into the browser.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/008ef037/attachment.html>

From shipman.william at gmail.com  Tue Aug 12 12:43:11 2014
From: shipman.william at gmail.com (William Shipman)
Date: Tue, 12 Aug 2014 18:43:11 +0200
Subject: [IPython-dev] Fwd: Using DirectView.execute to import modules on
	worker processes
In-Reply-To: <CAOUkfAzV-E8xCwM57+1SbmMBaUumW6pzE7eBAQqzXpEiRhuvQw@mail.gmail.com>
References: <CAOUkfAzV-E8xCwM57+1SbmMBaUumW6pzE7eBAQqzXpEiRhuvQw@mail.gmail.com>
Message-ID: <CAOUkfAz86rBJu_vce+xHGGKWhvKmfnMGAQWYqYrt98LSOXW9mw@mail.gmail.com>

I have looked through the documentation about importing modules on IPython
Parallel worker processes, and it seems imports of the form "import foo as
bar" are not supported.  However, I found that using DirectView.execute
allows me to get around this.  Is this safe, if not what can go wrong?

Here's the code I'm using to import part of the dateutil library and :

    from IPython.parallel import Client
    ipython_client = Client()
    dview = ipython_client.direct_view()
    dview.execute('from dateutil import parser as dtparser')

I am only using on my local PC, not a cluster.  After executing the above I
can happily run code that uses dtparser using dview.map_sync.  I can
rewrite my code to avoid doing this but I'd like to know why I shouldn't
use this method.

Thanks in advance.

P.S. I posted this to IPython-User but that mailing list seems not to be
active.  I anyone is receiving this for a second time, I apologize.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/dcbecf00/attachment.html>

From takowl at gmail.com  Tue Aug 12 12:47:20 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 12 Aug 2014 09:47:20 -0700
Subject: [IPython-dev] Fwd: Using DirectView.execute to import modules
 on worker processes
In-Reply-To: <CAOUkfAz86rBJu_vce+xHGGKWhvKmfnMGAQWYqYrt98LSOXW9mw@mail.gmail.com>
References: <CAOUkfAzV-E8xCwM57+1SbmMBaUumW6pzE7eBAQqzXpEiRhuvQw@mail.gmail.com>
	<CAOUkfAz86rBJu_vce+xHGGKWhvKmfnMGAQWYqYrt98LSOXW9mw@mail.gmail.com>
Message-ID: <CAOvn4qh=ptC_A+QacNp7HnUOBgd_Ap0CP14AoS6QYwqN5jD4QA@mail.gmail.com>

Hi William,

I don't think there should be any problem with using DirectView.execute()
for this. Min, who's the expert on IPython.parallel, is away at the moment,
but he should be able to give you a proper answer when he gets back in a
couple of days.

Thanks,
Thomas


On 12 August 2014 09:43, William Shipman <shipman.william at gmail.com> wrote:

> I have looked through the documentation about importing modules on IPython
> Parallel worker processes, and it seems imports of the form "import foo as
> bar" are not supported.  However, I found that using DirectView.execute
> allows me to get around this.  Is this safe, if not what can go wrong?
>
> Here's the code I'm using to import part of the dateutil library and :
>
>     from IPython.parallel import Client
>     ipython_client = Client()
>     dview = ipython_client.direct_view()
>     dview.execute('from dateutil import parser as dtparser')
>
> I am only using on my local PC, not a cluster.  After executing the above
> I can happily run code that uses dtparser using dview.map_sync.  I can
> rewrite my code to avoid doing this but I'd like to know why I shouldn't
> use this method.
>
> Thanks in advance.
>
> P.S. I posted this to IPython-User but that mailing list seems not to be
> active.  I anyone is receiving this for a second time, I apologize.
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/ed34af03/attachment.html>

From shipman.william at gmail.com  Tue Aug 12 13:03:50 2014
From: shipman.william at gmail.com (William Shipman)
Date: Tue, 12 Aug 2014 19:03:50 +0200
Subject: [IPython-dev] Fwd: Using DirectView.execute to import modules
 on worker processes
In-Reply-To: <CAOvn4qh=ptC_A+QacNp7HnUOBgd_Ap0CP14AoS6QYwqN5jD4QA@mail.gmail.com>
References: <CAOUkfAzV-E8xCwM57+1SbmMBaUumW6pzE7eBAQqzXpEiRhuvQw@mail.gmail.com>
	<CAOUkfAz86rBJu_vce+xHGGKWhvKmfnMGAQWYqYrt98LSOXW9mw@mail.gmail.com>
	<CAOvn4qh=ptC_A+QacNp7HnUOBgd_Ap0CP14AoS6QYwqN5jD4QA@mail.gmail.com>
Message-ID: <CAOUkfAyfQJZNHD1BgZHACq+6tjdETbid_L-JdPfy+mHNnzgV+Q@mail.gmail.com>

Thanks for the quick reply.  Should I post a message again next week once
Min is available again?

I guess IPython-User should not be used anymore?

Thanks,
William.



On 12 August 2014 18:47, Thomas Kluyver <takowl at gmail.com> wrote:

> Hi William,
>
> I don't think there should be any problem with using DirectView.execute()
> for this. Min, who's the expert on IPython.parallel, is away at the moment,
> but he should be able to give you a proper answer when he gets back in a
> couple of days.
>
> Thanks,
> Thomas
>
>
> On 12 August 2014 09:43, William Shipman <shipman.william at gmail.com>
> wrote:
>
>> I have looked through the documentation about importing modules on
>> IPython Parallel worker processes, and it seems imports of the form "import
>> foo as bar" are not supported.  However, I found that using
>> DirectView.execute allows me to get around this.  Is this safe, if not what
>> can go wrong?
>>
>> Here's the code I'm using to import part of the dateutil library and :
>>
>>     from IPython.parallel import Client
>>     ipython_client = Client()
>>     dview = ipython_client.direct_view()
>>     dview.execute('from dateutil import parser as dtparser')
>>
>> I am only using on my local PC, not a cluster.  After executing the above
>> I can happily run code that uses dtparser using dview.map_sync.  I can
>> rewrite my code to avoid doing this but I'd like to know why I shouldn't
>> use this method.
>>
>> Thanks in advance.
>>
>> P.S. I posted this to IPython-User but that mailing list seems not to be
>> active.  I anyone is receiving this for a second time, I apologize.
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/df0e2172/attachment.html>

From doug.blank at gmail.com  Tue Aug 12 13:09:58 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Tue, 12 Aug 2014 13:09:58 -0400
Subject: [IPython-dev] Kernel-independent magics?
Message-ID: <CAAusYCj__zUTtD9vrXMksJcrrP8pMN8c2epSnqTLOLEtStRu+Q@mail.gmail.com>

We are beginning to take our Pure-Python languages out from Calico and
implement them as stand-alone kernels (based on KernelBase) for Jupyter
(IPython 3). This includes Scheme, Basic, Logo, Little Computer (LC3,
assembly language), etc.

Wondering about how magics might be handled in Jupyter (In the Calico
Kernel, we handle magics in C# before passing the rest of the code to the
language). Currently, it looks like magics are handled in IPython by the
Interactive Python Shell. There are many magics that are language
independent and could be handled more abstractly (via JavaScript or server):

* %%latex
* %%html
* %connect_info
* %%file - create a file, with cell contents
* %%time - give execution time (not timeit, which figures out how many
times to run)
* %logstart
* many others

In addition, there is the need to allow others to write magics for specific
kernels, or for general kernels. That would imply some kind of %load_ext.

Also, there is generic ! shell execution.

Are there any plans on how to handle these functions in a
kernel-independent manner? If not, it would be good for kernel authors to
adopt some standards for magics.

Thanks for any pointers or comments!

-Doug
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/d37877ec/attachment.html>

From takowl at gmail.com  Tue Aug 12 13:20:48 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 12 Aug 2014 10:20:48 -0700
Subject: [IPython-dev] Fwd: Using DirectView.execute to import modules
 on worker processes
In-Reply-To: <CAOUkfAyfQJZNHD1BgZHACq+6tjdETbid_L-JdPfy+mHNnzgV+Q@mail.gmail.com>
References: <CAOUkfAzV-E8xCwM57+1SbmMBaUumW6pzE7eBAQqzXpEiRhuvQw@mail.gmail.com>
	<CAOUkfAz86rBJu_vce+xHGGKWhvKmfnMGAQWYqYrt98LSOXW9mw@mail.gmail.com>
	<CAOvn4qh=ptC_A+QacNp7HnUOBgd_Ap0CP14AoS6QYwqN5jD4QA@mail.gmail.com>
	<CAOUkfAyfQJZNHD1BgZHACq+6tjdETbid_L-JdPfy+mHNnzgV+Q@mail.gmail.com>
Message-ID: <CAOvn4qjpQPiLNi=HjjnoNz8ODX6tACyEkF_9hnoUw0U0_3zv2g@mail.gmail.com>

On 12 August 2014 10:03, William Shipman <shipman.william at gmail.com> wrote:

> Thanks for the quick reply.  Should I post a message again next week once
> Min is available again?
>

I'll try to point him to this thread, but if you don't hear from him by
next week, then ping the list again, or drop into the help chat room at
http://www.hipchat.com/ghSp7E1uY


> I guess IPython-User should not be used anymore?
>

We're gently phasing it out so there's one list for conversations - we
don't completely ignore -user, but yes, posting to -dev is preferred.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/e8f58a2f/attachment.html>

From takowl at gmail.com  Tue Aug 12 13:32:41 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 12 Aug 2014 10:32:41 -0700
Subject: [IPython-dev] Kernel-independent magics?
In-Reply-To: <CAAusYCj__zUTtD9vrXMksJcrrP8pMN8c2epSnqTLOLEtStRu+Q@mail.gmail.com>
References: <CAAusYCj__zUTtD9vrXMksJcrrP8pMN8c2epSnqTLOLEtStRu+Q@mail.gmail.com>
Message-ID: <CAOvn4qh-FetjV5tRrYFsY-G95SgrqzYOp9TboE7JjMPtuxwkmQ@mail.gmail.com>

Our take on this is that IPython's magic functions are a feature of the
IPython kernel. This kernel runs code written in 'IPython syntax', which is
a loosely specified superset of Python code, including some features
designed for convenient interactive use (%magics, !shell commands, help?,
etc.). If other kernels want to implement extra syntax for their own
languages to aid interactive use, that's up to them, but we don't expect
them to, and we're not trying to standardise those features.

The same goes for IPython (kernel side) extensions: if other kernels need
an extension system, they should implement that using the tools and
conventions that make sense for that language. We do, however, intend to
build tools to manage pure client side notebook extensions in a
kernel-independent way.

Thomas


On 12 August 2014 10:09, Doug Blank <doug.blank at gmail.com> wrote:

> We are beginning to take our Pure-Python languages out from Calico and
> implement them as stand-alone kernels (based on KernelBase) for Jupyter
> (IPython 3). This includes Scheme, Basic, Logo, Little Computer (LC3,
> assembly language), etc.
>
> Wondering about how magics might be handled in Jupyter (In the Calico
> Kernel, we handle magics in C# before passing the rest of the code to the
> language). Currently, it looks like magics are handled in IPython by the
> Interactive Python Shell. There are many magics that are language
> independent and could be handled more abstractly (via JavaScript or server):
>
> * %%latex
> * %%html
> * %connect_info
> * %%file - create a file, with cell contents
> * %%time - give execution time (not timeit, which figures out how many
> times to run)
> * %logstart
> * many others
>
> In addition, there is the need to allow others to write magics for
> specific kernels, or for general kernels. That would imply some kind of
> %load_ext.
>
> Also, there is generic ! shell execution.
>
> Are there any plans on how to handle these functions in a
> kernel-independent manner? If not, it would be good for kernel authors to
> adopt some standards for magics.
>
> Thanks for any pointers or comments!
>
> -Doug
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/15bd43ed/attachment.html>

From doug.blank at gmail.com  Tue Aug 12 14:04:35 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Tue, 12 Aug 2014 14:04:35 -0400
Subject: [IPython-dev] Kernel-independent magics?
In-Reply-To: <CAOvn4qh-FetjV5tRrYFsY-G95SgrqzYOp9TboE7JjMPtuxwkmQ@mail.gmail.com>
References: <CAAusYCj__zUTtD9vrXMksJcrrP8pMN8c2epSnqTLOLEtStRu+Q@mail.gmail.com>
	<CAOvn4qh-FetjV5tRrYFsY-G95SgrqzYOp9TboE7JjMPtuxwkmQ@mail.gmail.com>
Message-ID: <CAAusYCi-=XcXAsqpFVzK+S+EFNcB+CL_EOGGoo+da+Dfs5QOCg@mail.gmail.com>

On Tue, Aug 12, 2014 at 1:32 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> Our take on this is that IPython's magic functions are a feature of the
> IPython kernel. This kernel runs code written in 'IPython syntax', which is
> a loosely specified superset of Python code, including some features
> designed for convenient interactive use (%magics, !shell commands, help?,
> etc.). If other kernels want to implement extra syntax for their own
> languages to aid interactive use, that's up to them, but we don't expect
> them to, and we're not trying to standardise those features.
>

Ok, thanks for the quick response. That makes sense as many IPython magics
are intricately interwoven with Python.

I think that there is a role for a simplified, non-interwoven, standard set
of magics. Ok, we'll port our C# magic handler to Python, and share that
for use with those extending the KernelBase. Then a Python-based kernel can
easily get some nice/simple meta commands.


> The same goes for IPython (kernel side) extensions: if other kernels need
> an extension system, they should implement that using the tools and
> conventions that make sense for that language. We do, however, intend to
> build tools to manage pure client side notebook extensions in a
> kernel-independent way.
>

Sounds good! That should help on the front end.

-Doug


>
> Thomas
>
>
> On 12 August 2014 10:09, Doug Blank <doug.blank at gmail.com> wrote:
>
>> We are beginning to take our Pure-Python languages out from Calico and
>> implement them as stand-alone kernels (based on KernelBase) for Jupyter
>> (IPython 3). This includes Scheme, Basic, Logo, Little Computer (LC3,
>> assembly language), etc.
>>
>> Wondering about how magics might be handled in Jupyter (In the Calico
>> Kernel, we handle magics in C# before passing the rest of the code to the
>> language). Currently, it looks like magics are handled in IPython by the
>> Interactive Python Shell. There are many magics that are language
>> independent and could be handled more abstractly (via JavaScript or server):
>>
>> * %%latex
>> * %%html
>> * %connect_info
>> * %%file - create a file, with cell contents
>> * %%time - give execution time (not timeit, which figures out how many
>> times to run)
>> * %logstart
>> * many others
>>
>> In addition, there is the need to allow others to write magics for
>> specific kernels, or for general kernels. That would imply some kind of
>> %load_ext.
>>
>> Also, there is generic ! shell execution.
>>
>> Are there any plans on how to handle these functions in a
>> kernel-independent manner? If not, it would be good for kernel authors to
>> adopt some standards for magics.
>>
>> Thanks for any pointers or comments!
>>
>> -Doug
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/e84bc3fb/attachment.html>

From fperez.net at gmail.com  Wed Aug 13 02:47:08 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 12 Aug 2014 23:47:08 -0700
Subject: [IPython-dev] IPython.parallel slow push
In-Reply-To: <CAFOFTpQKQyePBFY0RG_pf=6agFRiH738yoNndN1UhdbFY99Vcg@mail.gmail.com>
References: <CAFOFTpQJK+04XSx18FAQg5FnUQMFRdrTVGBhoTNiVU0RA8vXmA@mail.gmail.com>
	<CACfEFw_rKcAj2L_RSMmaCNKpeu4+SjBMZSKpjPexV8u0ZYa45A@mail.gmail.com>
	<CAFOFTpRJu7acU64GPpCv8FU_3Nw1Dt9_28E-VgdQ0xcH_uGrgQ@mail.gmail.com>
	<CACfEFw-ZDFQZjQ_Voabq=EpDZNB8cdFPjz=X69Jq5ZvFa0LiSg@mail.gmail.com>
	<CAHAreOq9ehh3wBuEvx+kNLnio_XJ0U3Pr_QHEKw+yG_gEraxuA@mail.gmail.com>
	<CAFOFTpQKQyePBFY0RG_pf=6agFRiH738yoNndN1UhdbFY99Vcg@mail.gmail.com>
Message-ID: <CAHAreOpP427tvS_gm8R-L=JeD_mMhJWVasiQ=Uo5G+MQOzJmkA@mail.gmail.com>

On Tue, Aug 12, 2014 at 3:31 AM, Moritz Beber <moritz.beber at gmail.com>
wrote:

> As I said, I didn't move anything between machines, just locally.


Sorry, when I said machines I should have said 'engines'. It doesn't matter
that the engines are on the same physical node or not, the data is still
transferred between processes, and that's what was killing you.  Sorry for
the confusion.

Cheers,

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140812/dcd16aa1/attachment.html>

From benjaminrk at gmail.com  Wed Aug 13 12:02:26 2014
From: benjaminrk at gmail.com (Min RK)
Date: Wed, 13 Aug 2014 09:02:26 -0700
Subject: [IPython-dev] IPython event loop integration
In-Reply-To: <CAOvn4qgB=sysq_njD3dodEV49ryA2+_PQG4M77HqQV2kwkaaUA@mail.gmail.com>
References: <CA+-1RQSXLFLk_e0PF35ozscoAyzHYYsCu4TehWTUzU1+F-SiJw@mail.gmail.com>
	<CA+-1RQRoUwzStSN-scN+UCXUE3WjcGgZ=RL-ZwFStb9Wm6XYyg@mail.gmail.com>
	<CAOvn4qgB=sysq_njD3dodEV49ryA2+_PQG4M77HqQV2kwkaaUA@mail.gmail.com>
Message-ID: <19D5DC10-6002-40BB-99B0-49364E4FD822@gmail.com>

Tornado's PeriodicCallback ought to be the way to go for timers in an IPython kernel. There may be some issues if matplotlib is involved, setting up its own eventloop. In that case, you may want to use matplotlib's own Timer objects. An advantage of mpl timer objects is that they will work in all sorts of cases, not just tornado-based IPython.

-MinRK

> On Aug 12, 2014, at 9:36, Thomas Kluyver <takowl at gmail.com> wrote:
> 
> Hi Cyrille,
> 
> Having a quick scan through the thread you linked to, I wonder if it's possible to use the kernel event loop to schedule update events. pyzmq subclasses tornado's event loop, and provides a DelayedCallback class [1], and tornado itself has a PeriodicCallback class [2].
> 
> [1] http://zeromq.github.io/pyzmq/api/generated/zmq.eventloop.ioloop.html#delayedcallback
> [2] http://tornado.readthedocs.org/en/latest/ioloop.html#tornado.ioloop.PeriodicCallback
> 
> Thomas
> 
> 
>> On 11 August 2014 21:16, Cyrille Rossant <cyrille.rossant at gmail.com> wrote:
>> Hi all,
>> 
>> We're currently implementing an IPython notebook backend for Vispy.
>> We're having a hard time figuring out how to integrate timer-based
>> animations in the notebook [1]. We'd really appreciate if one of you
>> could have a look at our problem. This is the last step of our GSoC
>> student's project, which terminates in a few days.
>> 
>> Many thanks,
>> Cyrille
>> 
>> [1] https://groups.google.com/forum/#!topic/vispy-dev/blh2odeMgvI
>> 
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140813/916e8c32/attachment.html>

From benjaminrk at gmail.com  Wed Aug 13 13:17:29 2014
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 13 Aug 2014 10:17:29 -0700
Subject: [IPython-dev] mathjax equation number is incremented on
	cntr-enter
In-Reply-To: <53E45619.4040302@gmail.com>
References: <53E45619.4040302@gmail.com>
Message-ID: <CAHNn8BWykBV2a0-kN1W6=RG=TvzXxuej0dkS+XXSiFnqomHhnQ@mail.gmail.com>

This is a known issue <https://github.com/ipython/ipython/issues/4113>,
where mathjax generates an equation number for each equation it renders, so
they keep increasing on each render. In order to get proper equation
numbering, all equations (markdown cells) on the page would need to be
re-rendered every time any equation is. There isn?t a good shortcut for
this at the moment, but reloading the page should work.

-MinRK
?


On Thu, Aug 7, 2014 at 9:46 PM, Zolt?n V?r?s <zvoros at gmail.com> wrote:

> Hi all,
>
> I don't know, whether this is a known issue (a quick search did not turn
> up anything), in fact, I don't even know, whether this is an issue at all,
> but I believe, the markdown cell did behave differently in the past. In
> short, it seems to me that mathjax increments the equation number, whenever
> I press Cntr-Enter on a markdown cell. Here is my notebook, and here is the
> code that produces the problem:
>
> \begin{equation}
> x = y
> \end{equation}
>
> I haven't removed anything, yet, the equation is labelled (4). Can someone
> comment on this? In particular, whether this was intended? Is this,
> perhaps, related to the security problem that was floating around recently?
> I am running the latest code from master.
>
> Cheers,
>
> Zolt?n
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140813/5617218b/attachment.html>

From benjaminrk at gmail.com  Wed Aug 13 13:26:22 2014
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 13 Aug 2014 10:26:22 -0700
Subject: [IPython-dev] Separating the notebook from the file
	manager+text editor+terminal
In-Reply-To: <CAOvn4qh=EMRByNJyFiQ=RrGrO7Xx2EQzg9VNWKh16Rz3tsQSLA@mail.gmail.com>
References: <CAOvn4qh=EMRByNJyFiQ=RrGrO7Xx2EQzg9VNWKh16Rz3tsQSLA@mail.gmail.com>
Message-ID: <CAHNn8BWDLpi=8S6FBRZNm=hqS22auHrbWLbWkHq7c7m1Ha3NYg@mail.gmail.com>

I think this sounds like a good plan. I?m not 100% sure how to go about
implementing it, though. It would require that our webapp code be a bit
more modular. We took some steps in that direction with 2.0 - Brian and
Zach moved each component into discrete subpackages and URL areas, and are
loaded with a single call to load_handlers. We could extend this a bit
further to something like load_component, and figure out a way to configure
/ register available components.

-MinRK
?


On Sun, Aug 10, 2014 at 5:22 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> We've often said that, for people accessing the notebook on a remote
> server, we need to provide certain basic utilities - like the dashboard,
> which is now under pressure to evolve into a file manager, and an
> in-browser terminal and text editor, both of which we intend to add to the
> notebook.
>
> I propose that we separate development and packaging of the notebook from
> the project to assemble this trio. The new project would live under the
> Jupyter org, and be called something like 'utility-apps'. I see several
> advantages of this:
>
> - A different crowd of developers may be attracted to work on that,
> without having to know anything about the internals of IPython's machinery
> for handling and executing code. For instance, I'm pushing back on turning
> the dashboard into a full blown file manager, because I think that adds a
> lot of extra complexity in a codebase that's already complex, but if it was
> a separate project, it could grow into a file manager without being a
> problem.
> - People running the notebook locally may want to install just the
> notebook component, and not poor web-based clones of applications which
> they have well developed equivalents for already installed.
> - There are lots of potential users of the file manager/text
> editor/terminal trio who probably wouldn't use the notebook: for instance,
> it could be a basic admin interface to any server.
> - It could become a base on which people could write other HTML
> applications to meet their own needs. Following our usual MO, I wouldn't
> try to design it like this straight away, but keep it in mind, and evolve
> the APIs as things mature.
>
> There are still plenty of questions to be answered. How would this new
> component interface with the notebook server? What would the notebook look
> like without the file manager installed? But these could certainly be
> worked out, and I think they're worth working out in order to keep the
> scope of the notebook project in check.
>
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140813/6a6ee872/attachment.html>

From ellisonbg at gmail.com  Wed Aug 13 14:22:13 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 13 Aug 2014 14:22:13 -0400
Subject: [IPython-dev] Separating the notebook from the file
	manager+text editor+terminal
In-Reply-To: <CAHNn8BWDLpi=8S6FBRZNm=hqS22auHrbWLbWkHq7c7m1Ha3NYg@mail.gmail.com>
References: <CAOvn4qh=EMRByNJyFiQ=RrGrO7Xx2EQzg9VNWKh16Rz3tsQSLA@mail.gmail.com>
	<CAHNn8BWDLpi=8S6FBRZNm=hqS22auHrbWLbWkHq7c7m1Ha3NYg@mail.gmail.com>
Message-ID: <CAH4pYpRqxTAvCiqoHWuNf4pYZOMV64vKPQmyE7p_AzJ4ckn7hg@mail.gmail.com>

I am simultaneously +1 and -1 :)

+1: anything that helps us to write better, more modular code is a
huge improvement. As we add the terminal and text editor to the
notebook, we will have a great chance to improve these abstractions. I
think this will also help us to develop abstractions that enable our
different components to be reused in different contexts. I like that
and think it is important. Having separate repos for these components
*may* even make sense, although I wouldn't start there.

-1: I am have always been, and remain, opposed to turning our main
notebook app into a general web server project that can load and run
arbitrary combination of components. I think our notebook server
should run a fixed and consistent set of components for all users.
Anything different should be a "new app", not merely the same app
reconfigured.

Timing: I think the best way to proceed is to start developing the
terminal and text editor and see what abstractions start to develop
for these N=3 and N=4 cases

Dashboard: I too want to push back on the dashboard becoming a full
blown file manager. I am also opposed to there being multiple
dashboard that we maintain and which our users can choose between. I
think this is not a situation where we want to "settle disagreement by
making no decision and providing configuration instead." At the same
time, we are going to have to support more operations than we
currently do. The best model for balancing these concerns that I have
seen is the github UI. It basically has our UI, with a few extra
things for moving files around. I think we should aim for something
like that.


On Wed, Aug 13, 2014 at 1:26 PM, MinRK <benjaminrk at gmail.com> wrote:
> I think this sounds like a good plan. I?m not 100% sure how to go about
> implementing it, though. It would require that our webapp code be a bit more
> modular. We took some steps in that direction with 2.0 - Brian and Zach
> moved each component into discrete subpackages and URL areas, and are loaded
> with a single call to load_handlers. We could extend this a bit further to
> something like load_component, and figure out a way to configure / register
> available components.
>
> -MinRK
>
>
>
> On Sun, Aug 10, 2014 at 5:22 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>
>> We've often said that, for people accessing the notebook on a remote
>> server, we need to provide certain basic utilities - like the dashboard,
>> which is now under pressure to evolve into a file manager, and an in-browser
>> terminal and text editor, both of which we intend to add to the notebook.
>>
>> I propose that we separate development and packaging of the notebook from
>> the project to assemble this trio. The new project would live under the
>> Jupyter org, and be called something like 'utility-apps'. I see several
>> advantages of this:
>>
>> - A different crowd of developers may be attracted to work on that,
>> without having to know anything about the internals of IPython's machinery
>> for handling and executing code. For instance, I'm pushing back on turning
>> the dashboard into a full blown file manager, because I think that adds a
>> lot of extra complexity in a codebase that's already complex, but if it was
>> a separate project, it could grow into a file manager without being a
>> problem.
>> - People running the notebook locally may want to install just the
>> notebook component, and not poor web-based clones of applications which they
>> have well developed equivalents for already installed.
>> - There are lots of potential users of the file manager/text
>> editor/terminal trio who probably wouldn't use the notebook: for instance,
>> it could be a basic admin interface to any server.
>> - It could become a base on which people could write other HTML
>> applications to meet their own needs. Following our usual MO, I wouldn't try
>> to design it like this straight away, but keep it in mind, and evolve the
>> APIs as things mature.
>>
>> There are still plenty of questions to be answered. How would this new
>> component interface with the notebook server? What would the notebook look
>> like without the file manager installed? But these could certainly be worked
>> out, and I think they're worth working out in order to keep the scope of the
>> notebook project in check.
>>
>> Thomas
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From takowl at gmail.com  Wed Aug 13 14:31:24 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 13 Aug 2014 11:31:24 -0700
Subject: [IPython-dev] Separating the notebook from the file
	manager+text editor+terminal
In-Reply-To: <CAH4pYpRqxTAvCiqoHWuNf4pYZOMV64vKPQmyE7p_AzJ4ckn7hg@mail.gmail.com>
References: <CAOvn4qh=EMRByNJyFiQ=RrGrO7Xx2EQzg9VNWKh16Rz3tsQSLA@mail.gmail.com>
	<CAHNn8BWDLpi=8S6FBRZNm=hqS22auHrbWLbWkHq7c7m1Ha3NYg@mail.gmail.com>
	<CAH4pYpRqxTAvCiqoHWuNf4pYZOMV64vKPQmyE7p_AzJ4ckn7hg@mail.gmail.com>
Message-ID: <CAOvn4qg_VaNue7N7xiMGcMF7eCsoBOoHEv3cSuZ4Yj=yQW4TZw@mail.gmail.com>

On 13 August 2014 11:22, Brian Granger <ellisonbg at gmail.com> wrote:

> ...the github UI. It basically has our UI, with a few extra
> things for moving files around.
>

You can move files in the Github UI? I've never come across that.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140813/78854f32/attachment.html>

From damianavila at gmail.com  Wed Aug 13 14:43:34 2014
From: damianavila at gmail.com (=?UTF-8?Q?Dami=C3=A1n_Avila?=)
Date: Wed, 13 Aug 2014 15:43:34 -0300
Subject: [IPython-dev] Separating the notebook from the file
	manager+text editor+terminal
In-Reply-To: <CAOvn4qg_VaNue7N7xiMGcMF7eCsoBOoHEv3cSuZ4Yj=yQW4TZw@mail.gmail.com>
References: <CAOvn4qh=EMRByNJyFiQ=RrGrO7Xx2EQzg9VNWKh16Rz3tsQSLA@mail.gmail.com>
	<CAHNn8BWDLpi=8S6FBRZNm=hqS22auHrbWLbWkHq7c7m1Ha3NYg@mail.gmail.com>
	<CAH4pYpRqxTAvCiqoHWuNf4pYZOMV64vKPQmyE7p_AzJ4ckn7hg@mail.gmail.com>
	<CAOvn4qg_VaNue7N7xiMGcMF7eCsoBOoHEv3cSuZ4Yj=yQW4TZw@mail.gmail.com>
Message-ID: <CAH+mRR2Y=AZpLPPGNRAoTzvx4HuRpye16WZxozs+euV+YR7fng@mail.gmail.com>

I also agree on the modularity... with the code base getting bigger and
bigger it would be very useful to have the pieces split, and I think that
having separate repos is a necessary but not sufficient step towards a real
split. I think that having separate repos makes easier to make devs/users
to work in specific parts as a team as Thomas pointed out before...

Regarding the second Bryan's point, I also thing that our notebook
server should
run a fixed and consistent set of components for all users. Maybe my vision
is a little biased here, but I think the set of notebook, terminal,
text-editor and dashboard is the minimal and well balance set of tool to
completely achieve the majority of our user's uses cases, despite the
variability on the use cases community...

About the dashboard, we can (and have) to make it better, and people will
continue try to push toward a file manager, because it is the natural
evolution... and because probably everyone would like to have something
like that in their browsers. In any case, if enough people want to work on
that, as a separate repo, we can probably embrace them and replace the
dashboard, but it is clear, at least for me and I think fo all, that a
full-featured file manager is beyond our scope...


2014-08-13 15:31 GMT-03:00 Thomas Kluyver <takowl at gmail.com>:

> On 13 August 2014 11:22, Brian Granger <ellisonbg at gmail.com> wrote:
>
>> ...the github UI. It basically has our UI, with a few extra
>>
>> things for moving files around.
>>
>
> You can move files in the Github UI? I've never come across that.
>
> Thomas
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
*Dami?n*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140813/2045b043/attachment.html>

From ellisonbg at gmail.com  Wed Aug 13 14:53:28 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 13 Aug 2014 14:53:28 -0400
Subject: [IPython-dev] Separating the notebook from the file
	manager+text editor+terminal
In-Reply-To: <CAH+mRR2Y=AZpLPPGNRAoTzvx4HuRpye16WZxozs+euV+YR7fng@mail.gmail.com>
References: <CAOvn4qh=EMRByNJyFiQ=RrGrO7Xx2EQzg9VNWKh16Rz3tsQSLA@mail.gmail.com>
	<CAHNn8BWDLpi=8S6FBRZNm=hqS22auHrbWLbWkHq7c7m1Ha3NYg@mail.gmail.com>
	<CAH4pYpRqxTAvCiqoHWuNf4pYZOMV64vKPQmyE7p_AzJ4ckn7hg@mail.gmail.com>
	<CAOvn4qg_VaNue7N7xiMGcMF7eCsoBOoHEv3cSuZ4Yj=yQW4TZw@mail.gmail.com>
	<CAH+mRR2Y=AZpLPPGNRAoTzvx4HuRpye16WZxozs+euV+YR7fng@mail.gmail.com>
Message-ID: <CAH4pYpTgZ3yPWWP-xnMJXXsCaAHRuq7AV=orJBk9yFwFm-HgmA@mail.gmail.com>

Yes, it is really cool (moving files in GitHub UI). There are two ways
you can change a files directory and both use the same UI: 1) when
creating a new file in the UI and 2) when editing a file in the UI.

In both cases, the UI shows the name of the file, preceeded by its
path. It looks like only the name is editable, but if you type a "/"
in the name it turns that into a new subdir in the path and if you
press "delete" when in the L-most position of the name, you delete the
previous path segment. It is wicked cool and lightweight.


On Wed, Aug 13, 2014 at 2:43 PM, Dami?n Avila <damianavila at gmail.com> wrote:
> I also agree on the modularity... with the code base getting bigger and
> bigger it would be very useful to have the pieces split, and I think that
> having separate repos is a necessary but not sufficient step towards a real
> split. I think that having separate repos makes easier to make devs/users to
> work in specific parts as a team as Thomas pointed out before...
>
> Regarding the second Bryan's point, I also thing that our notebook server
> should run a fixed and consistent set of components for all users. Maybe my
> vision is a little biased here, but I think the set of notebook, terminal,
> text-editor and dashboard is the minimal and well balance set of tool to
> completely achieve the majority of our user's uses cases, despite the
> variability on the use cases community...
>
> About the dashboard, we can (and have) to make it better, and people will
> continue try to push toward a file manager, because it is the natural
> evolution... and because probably everyone would like to have something like
> that in their browsers. In any case, if enough people want to work on that,
> as a separate repo, we can probably embrace them and replace the dashboard,
> but it is clear, at least for me and I think fo all, that a full-featured
> file manager is beyond our scope...
>
>
> 2014-08-13 15:31 GMT-03:00 Thomas Kluyver <takowl at gmail.com>:
>>
>> On 13 August 2014 11:22, Brian Granger <ellisonbg at gmail.com> wrote:
>>>
>>> ...the github UI. It basically has our UI, with a few extra
>>>
>>> things for moving files around.
>>
>>
>> You can move files in the Github UI? I've never come across that.
>>
>> Thomas
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Dami?n
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From tritemio at gmail.com  Wed Aug 13 21:56:18 2014
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Wed, 13 Aug 2014 18:56:18 -0700
Subject: [IPython-dev] IPython Notebook in read-only mode
Message-ID: <CANn2QUyi71Eqg0U_NZ9+UQRS-OZ9YauZn7StsjRfcqXaFzXJJQ@mail.gmail.com>

Hi,

I need to share some notebooks with some colleagues from my desktop
(windows).

Would be possible to configure an ipython notebook server that accepts
connection from different machines but serves notebooks in read-only mode?

I remember that at some point in the past (maybe around 0.13) I had a
configuration like this, but I don't remember if it was just some side
effect of the firewall.

I could try to install nbviewer but I would also need to setup a second
server to host the ipynb files IIUC. Any guidance on how to setup nbviewer
for local files on Windows would also help :).

Antonio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140813/8a225588/attachment.html>

From ellisonbg at gmail.com  Wed Aug 13 22:03:51 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 13 Aug 2014 22:03:51 -0400
Subject: [IPython-dev] IPython Notebook in read-only mode
In-Reply-To: <CANn2QUyi71Eqg0U_NZ9+UQRS-OZ9YauZn7StsjRfcqXaFzXJJQ@mail.gmail.com>
References: <CANn2QUyi71Eqg0U_NZ9+UQRS-OZ9YauZn7StsjRfcqXaFzXJJQ@mail.gmail.com>
Message-ID: <CAH4pYpTCpt4=m07fKdkNri_EmfbVaA7x87YkYVWTVxbFS7SDYw@mail.gmail.com>

The multiuser notebook server we are working on will eventually have
something like this. But the single user server won't. In the meantime
I would use nbviewer, or just nbconvert and a static website.

Cheers,

Brian

On Wed, Aug 13, 2014 at 9:56 PM, Antonino Ingargiola <tritemio at gmail.com> wrote:
> Hi,
>
> I need to share some notebooks with some colleagues from my desktop
> (windows).
>
> Would be possible to configure an ipython notebook server that accepts
> connection from different machines but serves notebooks in read-only mode?
>
> I remember that at some point in the past (maybe around 0.13) I had a
> configuration like this, but I don't remember if it was just some side
> effect of the firewall.
>
> I could try to install nbviewer but I would also need to setup a second
> server to host the ipynb files IIUC. Any guidance on how to setup nbviewer
> for local files on Windows would also help :).
>
> Antonio
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From zvoros at gmail.com  Thu Aug 14 03:04:41 2014
From: zvoros at gmail.com (=?UTF-8?B?Wm9sdMOhbiBWw7Zyw7Zz?=)
Date: Thu, 14 Aug 2014 09:04:41 +0200
Subject: [IPython-dev] mathjax equation number is incremented on
	cntr-enter
In-Reply-To: <CAHNn8BWykBV2a0-kN1W6=RG=TvzXxuej0dkS+XXSiFnqomHhnQ@mail.gmail.com>
References: <53E45619.4040302@gmail.com>
	<CAHNn8BWykBV2a0-kN1W6=RG=TvzXxuej0dkS+XXSiFnqomHhnQ@mail.gmail.com>
Message-ID: <53EC5F89.1060107@gmail.com>

Hi Min,

Thanks for the pointer! And now I understand why I didn't find this 
issue: it has been known for too long a time...

Cheers,
Zolt?n

On 08/13/2014 07:17 PM, MinRK wrote:
>
> This is a known issue 
> <https://github.com/ipython/ipython/issues/4113>, where mathjax 
> generates an equation number for each equation it renders, so they 
> keep increasing on each render. In order to get proper equation 
> numbering, all equations (markdown cells) on the page would need to be 
> re-rendered every time any equation is. There isn?t a good shortcut 
> for this at the moment, but reloading the page should work.
>
> -MinRK
>
> ?
>
>
> On Thu, Aug 7, 2014 at 9:46 PM, Zolt?n V?r?s <zvoros at gmail.com 
> <mailto:zvoros at gmail.com>> wrote:
>
>     Hi all,
>
>     I don't know, whether this is a known issue (a quick search did
>     not turn up anything), in fact, I don't even know, whether this is
>     an issue at all, but I believe, the markdown cell did behave
>     differently in the past. In short, it seems to me that mathjax
>     increments the equation number, whenever I press Cntr-Enter on a
>     markdown cell. Here is my notebook, and here is the code that
>     produces the problem:
>
>     \begin{equation}
>     x = y
>     \end{equation}
>
>     I haven't removed anything, yet, the equation is labelled (4). Can
>     someone comment on this? In particular, whether this was intended?
>     Is this, perhaps, related to the security problem that was
>     floating around recently? I am running the latest code from master.
>
>     Cheers,
>
>     Zolt?n
>
>     _______________________________________________
>     IPython-dev mailing list
>     IPython-dev at scipy.org <mailto:IPython-dev at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From takowl at gmail.com  Thu Aug 14 14:20:59 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 14 Aug 2014 11:20:59 -0700
Subject: [IPython-dev] Mime type for IPython notebooks
Message-ID: <CAOvn4qiZUr7Mn=h++TngzETrWB6AW-7f6MBegBZNqpZ3_vyqaA@mail.gmail.com>

In the dev meeting, we discussed what the mime type should be for IPython
notebook files. The people in the meeting preferred
*application/x-ipynb+json* (we also considered
application/x-jupyter-notebook+json, and both of those without the +json
suffix). Looking at the official mime type registry [1], +json and +xml are
common suffixes to indicate container formats.

Does anyone want to argue for a different mime type before we declare
application/x-ipynb+json the victor?

1 http://www.iana.org/form/media-types

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140814/47a6e3dd/attachment.html>

From doug.blank at gmail.com  Thu Aug 14 14:55:00 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Thu, 14 Aug 2014 14:55:00 -0400
Subject: [IPython-dev] Mime type for IPython notebooks
In-Reply-To: <CAOvn4qiZUr7Mn=h++TngzETrWB6AW-7f6MBegBZNqpZ3_vyqaA@mail.gmail.com>
References: <CAOvn4qiZUr7Mn=h++TngzETrWB6AW-7f6MBegBZNqpZ3_vyqaA@mail.gmail.com>
Message-ID: <CAAusYCiCR59KD3JfF9MPzJX6KAvzuRgJvqd6wK7P0=2zq3iZjw@mail.gmail.com>

On Thu, Aug 14, 2014 at 2:20 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> In the dev meeting, we discussed what the mime type should be for IPython
> notebook files. The people in the meeting preferred
> *application/x-ipynb+json* (we also considered
> application/x-jupyter-notebook+json, and both of those without the +json
> suffix). Looking at the official mime type registry [1], +json and +xml are
> common suffixes to indicate container formats.
>
> Does anyone want to argue for a different mime type before we declare
> application/x-ipynb+json the victor?
>

Before you  wrote this, I would have guessed that the filename extensions
on notebooks would probably be changed to something more generic. Thinking
that you are going to also attach those 5 letters to the mime-type makes me
wonder. So, let me make a couple of points, and try to sway y'all in a more
generic naming scheme.

An aside, I think you could leave the +json off. The container indication
is useful on svg+xml to remind people that it is an image, and also xml.
Thus, many browsers can handle it directly. I don't think it makes sense on
a general format, unless you are considering adding application/...+xml (or
some other container format) at some point, too. Knowing that a format is
json doesn't get you much.

Jupyter will soon be the name that represents the big picture. Why wouldn't
the mime-type also represent the big picture? Also, there isn't any other
Jupyter file format, so why not just make it application/x-jupyter ? As for
file extension, consider the big picture, too. Maybe: .jnb, .jup, or even
.jupyter.

Anyway, thanks for asking for arguments; always willing to oblige :)

-Doug


>
> 1 http://www.iana.org/form/media-types
>
> Thanks,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140814/d1fc681a/attachment.html>

From benjaminrk at gmail.com  Thu Aug 14 15:48:56 2014
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 14 Aug 2014 12:48:56 -0700
Subject: [IPython-dev] Mime type for IPython notebooks
In-Reply-To: <CAAusYCiCR59KD3JfF9MPzJX6KAvzuRgJvqd6wK7P0=2zq3iZjw@mail.gmail.com>
References: <CAOvn4qiZUr7Mn=h++TngzETrWB6AW-7f6MBegBZNqpZ3_vyqaA@mail.gmail.com>
	<CAAusYCiCR59KD3JfF9MPzJX6KAvzuRgJvqd6wK7P0=2zq3iZjw@mail.gmail.com>
Message-ID: <CAHNn8BWoAbq5xEZuUJnLdU5hzifi37sZiLQFhBF=Y_F=bpA2Yw@mail.gmail.com>

On Thu, Aug 14, 2014 at 11:55 AM, Doug Blank <doug.blank at gmail.com> wrote:

On Thu, Aug 14, 2014 at 2:20 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
>> In the dev meeting, we discussed what the mime type should be for IPython
>> notebook files. The people in the meeting preferred
>> *application/x-ipynb+json* (we also considered
>> application/x-jupyter-notebook+json, and both of those without the +json
>> suffix). Looking at the official mime type registry [1], +json and +xml are
>> common suffixes to indicate container formats.
>>
>> Does anyone want to argue for a different mime type before we declare
>> application/x-ipynb+json the victor?
>>
>
> Before you  wrote this, I would have guessed that the filename extensions
> on notebooks would probably be changed to something more generic. Thinking
> that you are going to also attach those 5 letters to the mime-type makes me
> wonder. So, let me make a couple of points, and try to sway y'all in a more
> generic naming scheme.
>
We wouldn?t have picked .ipynb if we were choosing now, but the cost of
changing one letter seems to significantly outweigh the benefits. For a
slightly facetious argument: In Latin, there is no distinction between I
and J, so technically the name would be IUPYTER, so a .ipynb extension
still makes sense for project iupyter :)


> An aside, I think you could leave the +json off. The container indication
> is useful on svg+xml to remind people that it is an image, and also xml.
> Thus, many browsers can handle it directly. I don't think it makes sense on
> a general format, unless you are considering adding application/...+xml (or
> some other container format) at some point, too. Knowing that a format is
> json doesn't get you much.
>
It doesn?t get you much, but we found a number of examples following the
convention on the IANA registry
<http://www.iana.org/assignments/media-types/media-types.xhtml>, and it
seems helpful to note that it is indeed plain JSON, and the notebook just
describes the *structure* of that JSON.


> Jupyter will soon be the name that represents the big picture. Why
> wouldn't the mime-type also represent the big picture? Also, there isn't
> any other Jupyter file format, so why not just make it
> application/x-jupyter ? As for file extension, consider the big picture,
> too. Maybe: .jnb, .jup, or even .jupyter.
>
I think we would keep notebook as part of it, so honestly the most likely
candidate is .jpynb, which is almost identical to the existing .ipynb.
There could easily be other non-notebook Jupyter-related files in the
future.


> Anyway, thanks for asking for arguments; always willing to oblige :)
>
Thanks!

-MinRK


> -Doug
>
>
>>
>> 1 http://www.iana.org/form/media-types
>>
>> Thanks,
>> Thomas
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140814/3293e993/attachment.html>

From mkpaustin at gmail.com  Thu Aug 14 19:55:37 2014
From: mkpaustin at gmail.com (Phil Austin)
Date: Thu, 14 Aug 2014 23:55:37 +0000 (UTC)
Subject: [IPython-dev] making emacs keybindings the ipython default
Message-ID: <loom.20140815T014516-99@post.gmane.org>

I need a hint on how to turn on codemirror's emacs keybindings in custom.js for 
IPython 2.1.    I've found 

https://gist.github.com/minrk/5940801

and

https://nbviewer.jupyter.org/github/ivanov/scipy2014/blob/master/
v%20in%20IPython.ipynb#ipython-vimception

but when I insert

%%javascript
// change the mode of all current and future CodeMirror instances
function to(mode) {
    var mode = mode || 'vim'
    // first let's apply vim mode to all current cells
    function to_mode(c) { return c.code_mirror.setOption('keyMap', mode);};
    IPython.notebook.get_cells().map(to_mode);
    // apply the mode to future cells created
    IPython.Cell.options_default.cm_config.keyMap = mode;
}

require(["/static/components/codemirror/keymap/emacs"],
     function (emacs) { 
         to('emacs'); 
         console.log('emacs.js loaded'); 
     });

into ~/.ipython/profile_default/static/custom/custom.js  

the console.log (OSX Mavericks, Chrome) reports a 404:

2014-08-14 16:43:08.685 [NotebookApp] Use Control-C to stop this server and
shut down all kernels (twice to skip confirmation).
WARNING:tornado.access:404 GET /static/components/codemirror/keymap/emacs
(::1) 0.88ms referer=http://localhost:8889/notebooks/timetable.ipynbn
2014-08-14 16:43:10.988 [NotebookApp] Kernel started: 16eaa587-e1c8-4f51-8951-
3d6e60ab3ad6

and how to actually call the callback from custom.js seems to have gotten
deleted from the bottom of the notebook?

thanks, Phil




From pi at berkeley.edu  Thu Aug 14 20:23:37 2014
From: pi at berkeley.edu (Paul Ivanov)
Date: Thu, 14 Aug 2014 17:23:37 -0700
Subject: [IPython-dev] making emacs keybindings the ipython default
In-Reply-To: <loom.20140815T014516-99@post.gmane.org>
References: <loom.20140815T014516-99@post.gmane.org>
Message-ID: <20140815002337.GG3924@HbI-OTOH.berkeley.edu>

Hi Phil,

Sorry about the goof, you need to either put ".js" after emacs
the way I had it before, or not using a leading forward slash,
and just do the example below (adjusted for how you should add it
to custom.js)


$([IPython.events]).on('app_initialized.NotebookApp', function(){
function to(mode) {
    var mode = mode || 'vim'
    // first let's apply vim mode to all current cells
    function to_mode(c) { return c.code_mirror.setOption('keyMap', mode);};
    IPython.notebook.get_cells().map(to_mode);
    // apply the mode to future cells created
    IPython.Cell.options_default.cm_config.keyMap = mode;
}

require(["components/codemirror/keymap/emacs"],
     function (emacs) { 
         to('emacs'); 
         console.log('emacs.js loaded'); 
     });
})

Thanks for the bug report, let me go fix that in the notebook you
referenced,
-- 
                   _
                  / \
                A*   \^   -
             ,./   _.`\\ / \
            / ,--.S    \/   \
           /  `"~,_     \    \
     __o           ?
   _ \<,_         /:\
--(_)/-(_)----.../ | \
--------------.......J
Paul Ivanov
ipython and matplotlib core developer
@ivanov on github and twitter
http://pirsquared.org


From mkpaustin at gmail.com  Thu Aug 14 21:55:02 2014
From: mkpaustin at gmail.com (taleb brost)
Date: Thu, 14 Aug 2014 18:55:02 -0700
Subject: [IPython-dev] making emacs keybindings the ipython default
In-Reply-To: <20140815002337.GG3924@HbI-OTOH.berkeley.edu>
References: <loom.20140815T014516-99@post.gmane.org>
	<20140815002337.GG3924@HbI-OTOH.berkeley.edu>
Message-ID: <CAHz=VszdgKgErMXUmGaMbvHMM4-ab2zpDJAPfUY0+GUbvaSMgA@mail.gmail.com>

On Thu, Aug 14, 2014 at 5:23 PM, Paul Ivanov <pi at berkeley.edu> wrote:

> Hi Phil,
>
> Sorry about the goof, you need to either put ".js" after emacs
> the way I had it before, or not using a leading forward slash,
> and just do the example below (adjusted for how you should add it
> to custom.js)
>


 thanks, for the quick response --  almost there, in that, the 404 error is
silenced, but
when I start a new notebook I'm not getting the console message 'emacs.js
loaded'
and ctrl-p etc. are no-ops.  Is there a final step to call the function
loaded by require aside
from initializing the notebook?
My javascript knowledge is just above zero.  best, Phil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140814/91dd1f05/attachment.html>

From claresloggett at gmail.com  Fri Aug 15 10:48:56 2014
From: claresloggett at gmail.com (Clare Sloggett)
Date: Sat, 16 Aug 2014 00:48:56 +1000
Subject: [IPython-dev] IPython Notebook in read-only mode
In-Reply-To: <CAH4pYpTCpt4=m07fKdkNri_EmfbVaA7x87YkYVWTVxbFS7SDYw@mail.gmail.com>
References: <CANn2QUyi71Eqg0U_NZ9+UQRS-OZ9YauZn7StsjRfcqXaFzXJJQ@mail.gmail.com>
	<CAH4pYpTCpt4=m07fKdkNri_EmfbVaA7x87YkYVWTVxbFS7SDYw@mail.gmail.com>
Message-ID: <CAETqNqFPtA+vt=twZUrPSX5vXe-M7mPN6GOJn7_YwbhYHa8iWw@mail.gmail.com>

Hi Antonio,

If you're in an office with people and you don't want to set up an actual
website, you could also just send them the html file, or put it in a file
sharing space in your office, or Dropbox. Even if your colleagues aren't
particularly techie, they only need to "open" the html file to display it a
browser.

But, a read-only mode for Notebook is something I've wanted too. The reason
is that IPython Notebook provides its own authentication mechanism, so
something can be at a public address, but still relatively private. And
it's trivial to get going, since Notebook runs a webserver for you.

Of course I can use another authentication system on the static html, like
Dropbox sharing, or set up a webserver with a password-protected area, but
it would be pretty nice if Notebook supported read-only access as fully as
it supports read/write access.

Cheers,
Clare



On 14 August 2014 12:03, Brian Granger <ellisonbg at gmail.com> wrote:

> The multiuser notebook server we are working on will eventually have
> something like this. But the single user server won't. In the meantime
> I would use nbviewer, or just nbconvert and a static website.
>
> Cheers,
>
> Brian
>
> On Wed, Aug 13, 2014 at 9:56 PM, Antonino Ingargiola <tritemio at gmail.com>
> wrote:
> > Hi,
> >
> > I need to share some notebooks with some colleagues from my desktop
> > (windows).
> >
> > Would be possible to configure an ipython notebook server that accepts
> > connection from different machines but serves notebooks in read-only
> mode?
> >
> > I remember that at some point in the past (maybe around 0.13) I had a
> > configuration like this, but I don't remember if it was just some side
> > effect of the firewall.
> >
> > I could try to install nbviewer but I would also need to setup a second
> > server to host the ipynb files IIUC. Any guidance on how to setup
> nbviewer
> > for local files on Windows would also help :).
> >
> > Antonio
> >
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140816/59755c53/attachment.html>

From bussonniermatthias at gmail.com  Fri Aug 15 11:01:08 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Fri, 15 Aug 2014 16:01:08 +0100
Subject: [IPython-dev] IPython Notebook in read-only mode
In-Reply-To: <CANn2QUyi71Eqg0U_NZ9+UQRS-OZ9YauZn7StsjRfcqXaFzXJJQ@mail.gmail.com>
References: <CANn2QUyi71Eqg0U_NZ9+UQRS-OZ9YauZn7StsjRfcqXaFzXJJQ@mail.gmail.com>
Message-ID: <409FED76-DF37-4375-895A-A10FFE5E2C4F@gmail.com>

Nbviewer has an (unsupported) option at startup to serve ipynb from cwd. I suppose that will be enough for you.

Envoy? de mon iPhone

> Le 14 ao?t 2014 ? 02:56, Antonino Ingargiola <tritemio at gmail.com> a ?crit :
> 
> Hi,
> 
> I need to share some notebooks with some colleagues from my desktop (windows).
> 
> Would be possible to configure an ipython notebook server that accepts connection from different machines but serves notebooks in read-only mode?
> 
> I remember that at some point in the past (maybe around 0.13) I had a configuration like this, but I don't remember if it was just some side effect of the firewall.
> 
> I could try to install nbviewer but I would also need to setup a second server to host the ipynb files IIUC. Any guidance on how to setup nbviewer for local files on Windows would also help :).
> 
> Antonio
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From mankoff at gmail.com  Fri Aug 15 13:49:03 2014
From: mankoff at gmail.com (Ken Mankoff)
Date: Fri, 15 Aug 2014 13:49:03 -0400
Subject: [IPython-dev] making emacs keybindings the ipython default
In-Reply-To: <CAHz=VszdgKgErMXUmGaMbvHMM4-ab2zpDJAPfUY0+GUbvaSMgA@mail.gmail.com>
References: <loom.20140815T014516-99@post.gmane.org>
	<20140815002337.GG3924@HbI-OTOH.berkeley.edu>
	<CAHz=VszdgKgErMXUmGaMbvHMM4-ab2zpDJAPfUY0+GUbvaSMgA@mail.gmail.com>
Message-ID: <CAFdBzErvQABmVy7irbudtcA+J3h_Wsc1_Ug++K0uKHpY8m3i2A@mail.gmail.com>

Just a note that if you want emacs + IPython notebook, there is a project
for working with notebook files in emacs. See the EIN
https://tkf.github.io/emacs-ipython-notebook/

  -k.


On Thu, Aug 14, 2014 at 9:55 PM, taleb brost <mkpaustin at gmail.com> wrote:

> On Thu, Aug 14, 2014 at 5:23 PM, Paul Ivanov <pi at berkeley.edu> wrote:
>
>> Hi Phil,
>>
>> Sorry about the goof, you need to either put ".js" after emacs
>> the way I had it before, or not using a leading forward slash,
>> and just do the example below (adjusted for how you should add it
>> to custom.js)
>>
>
>
>  thanks, for the quick response --  almost there, in that, the 404 error
> is silenced, but
> when I start a new notebook I'm not getting the console message 'emacs.js
> loaded'
> and ctrl-p etc. are no-ops.  Is there a final step to call the function
> loaded by require aside
> from initializing the notebook?
> My javascript knowledge is just above zero.  best, Phil
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140815/1f9791b1/attachment.html>

From efiring at hawaii.edu  Fri Aug 15 16:22:12 2014
From: efiring at hawaii.edu (Eric Firing)
Date: Fri, 15 Aug 2014 10:22:12 -1000
Subject: [IPython-dev] ipython slowdown with qt
In-Reply-To: <574A53C0-860B-4622-B23A-E14E8693404B@astro.physik.uni-goettingen.de>
References: <53ED08DF.90304@gmail.com>	<CALGmxEKOjBFcoNQvF8zoLEfqj+xaFyGLqrjNAky7mpD7Hxb0Ew@mail.gmail.com>
	<574A53C0-860B-4622-B23A-E14E8693404B@astro.physik.uni-goettingen.de>
Message-ID: <53EE6BF4.2050407@hawaii.edu>

On 2014/08/15, 9:37 AM, Derek Homeier wrote:
> When using MPL with ipython ?pylab and the Quartz version of PyQT4,
> the interpreter seems to be slow down extremely after running for a
> little while. Weirdly this is not connected to any graphics display
> and in fact happens even without any plotting window opened, i.e. the
> ipython shell just randomly becomes completely unresponsive and hangs
> for several seconds on simple tasks like typing or navigating through
> history. The plotting itself actually does not appear to perform any
> worse than it used to under Mountain Lion.

[I'm switching the subject because my comments below relate to ipython 
and matplotlib, and are no longer Anaconda-specific.]

Derek,

Thanks.  A few days ago, when I switched from testing on linux to 
testing on osx, exactly this ipython slowdown was happening to me--but I 
lost track of what combination of versions and invocations was causing 
it.  Therefore I have been concentrating on the severe problem which 
was, for me, 100% repeatable, and involved macosx backend, not Qt.  I 
expect the macosx-relatec problem will go away after Ilan uploads the 
revised Anaconda ipython for python 3.

Now I find I can repeat the ipython problem on Homebrew python 3 
(framework--with Quartz app) and Anaconda with the un-fixed ipython 
(which is running without starting a Quartz app):

ipython --pylab=qt

Leave it alone for a bit.  Try scrolling through history.  Long delay, 
even in responding to Ctrl-C.  Evidently key events are stacking up and 
not being processed.  Now try:

ipython
%pylab qt

I see the slowdown with this, also.  The response delay seems to get 
worse with time. It renders the session unusable after only a few minutes.

ipython
%gui qt

And I still see it, so this appears to be a problem in ipython's PyQt4 
gui handling, not directly related to matplotlib.  All on Mavericks, 
running ipython from Apple's terminal.

Eric







From jenshnielsen at gmail.com  Fri Aug 15 17:05:14 2014
From: jenshnielsen at gmail.com (Jens Nielsen)
Date: Fri, 15 Aug 2014 22:05:14 +0100
Subject: [IPython-dev] ipython slowdown with qt
In-Reply-To: <53EE6BF4.2050407@hawaii.edu>
References: <53ED08DF.90304@gmail.com>
	<CALGmxEKOjBFcoNQvF8zoLEfqj+xaFyGLqrjNAky7mpD7Hxb0Ew@mail.gmail.com>
	<574A53C0-860B-4622-B23A-E14E8693404B@astro.physik.uni-goettingen.de>
	<53EE6BF4.2050407@hawaii.edu>
Message-ID: <CAM-Pw02FndL5_DpoaAne8qHK+9WGDjx8Rx8KmVptQDSerT6xng@mail.gmail.com>

While I can reproduce the issue using %gui qt I can also reproduce it with
the WX backend (%qui wx) with more or less the same symptoms. However, I
don't see the issue with either of the 'tk' or the 'osx' backends. And yes
the issue is reproducible in a python installation without any mpl
installed.

/Jens


On Fri, Aug 15, 2014 at 9:22 PM, Eric Firing <efiring at hawaii.edu> wrote:

> On 2014/08/15, 9:37 AM, Derek Homeier wrote:
> > When using MPL with ipython ?pylab and the Quartz version of PyQT4,
> > the interpreter seems to be slow down extremely after running for a
> > little while. Weirdly this is not connected to any graphics display
> > and in fact happens even without any plotting window opened, i.e. the
> > ipython shell just randomly becomes completely unresponsive and hangs
> > for several seconds on simple tasks like typing or navigating through
> > history. The plotting itself actually does not appear to perform any
> > worse than it used to under Mountain Lion.
>
> [I'm switching the subject because my comments below relate to ipython
> and matplotlib, and are no longer Anaconda-specific.]
>
> Derek,
>
> Thanks.  A few days ago, when I switched from testing on linux to
> testing on osx, exactly this ipython slowdown was happening to me--but I
> lost track of what combination of versions and invocations was causing
> it.  Therefore I have been concentrating on the severe problem which
> was, for me, 100% repeatable, and involved macosx backend, not Qt.  I
> expect the macosx-relatec problem will go away after Ilan uploads the
> revised Anaconda ipython for python 3.
>
> Now I find I can repeat the ipython problem on Homebrew python 3
> (framework--with Quartz app) and Anaconda with the un-fixed ipython
> (which is running without starting a Quartz app):
>
> ipython --pylab=qt
>
> Leave it alone for a bit.  Try scrolling through history.  Long delay,
> even in responding to Ctrl-C.  Evidently key events are stacking up and
> not being processed.  Now try:
>
> ipython
> %pylab qt
>
> I see the slowdown with this, also.  The response delay seems to get
> worse with time. It renders the session unusable after only a few minutes.
>
> ipython
> %gui qt
>
> And I still see it, so this appears to be a problem in ipython's PyQt4
> gui handling, not directly related to matplotlib.  All on Mavericks,
> running ipython from Apple's terminal.
>
> Eric
>
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140815/06f8e71b/attachment.html>

From efiring at hawaii.edu  Fri Aug 15 17:55:33 2014
From: efiring at hawaii.edu (Eric Firing)
Date: Fri, 15 Aug 2014 11:55:33 -1000
Subject: [IPython-dev] ipython slowdown with qt
In-Reply-To: <CAM-Pw02FndL5_DpoaAne8qHK+9WGDjx8Rx8KmVptQDSerT6xng@mail.gmail.com>
References: <53ED08DF.90304@gmail.com>	<CALGmxEKOjBFcoNQvF8zoLEfqj+xaFyGLqrjNAky7mpD7Hxb0Ew@mail.gmail.com>	<574A53C0-860B-4622-B23A-E14E8693404B@astro.physik.uni-goettingen.de>	<53EE6BF4.2050407@hawaii.edu>
	<CAM-Pw02FndL5_DpoaAne8qHK+9WGDjx8Rx8KmVptQDSerT6xng@mail.gmail.com>
Message-ID: <53EE81D5.8010309@hawaii.edu>

On 2014/08/15, 11:05 AM, Jens Nielsen wrote:
> While I can reproduce the issue using %gui qt I can also reproduce it
> with the WX backend (%qui wx) with more or less the same symptoms.
> However, I don't see the issue with either of the 'tk' or the 'osx'
> backends. And yes the issue is reproducible in a python installation
> without any mpl installed.
>
> /Jens
>

This is https://github.com/ipython/ipython/issues/6189.

Eric


From hughesadam87 at gmail.com  Sun Aug 17 13:54:59 2014
From: hughesadam87 at gmail.com (hugadams)
Date: Sun, 17 Aug 2014 10:54:59 -0700 (PDT)
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
Message-ID: <1408298099961-5068197.post@n6.nabble.com>

(Please disregard if doublepost; having some subscription issues)

Hello,

We are developing a spectroscopy library and want to put in some interactive
plots/utilities to make the software a bit more approachable to novices. 
I'd really like to build the entire interactive plotting interface through
the notebook's new widget system.  My reasoning is as follows:
  
   - All static plots are already matplotlib-based, so we wouldn't have to
write a seperate plotting api
   - Notebooks are inherently browser-ready, so sharing plots would be great
         - Is it likely that static nbviewer will eventually support
widgets?  Since the data is required, I figured not...
   - Our tutorials are already in notebooks, so we'd like to be consistent.

I've watched all the videos I can find from the widgets, and think I'm fine
with the Python end; however, am finding it hard to come by lots of
information on controlling the layout of the widgets.  I know it's mostly
done in javascript, and I think the most complex view we will have will look
something like this:

https://sites.google.com/site/shigemorita/home/2dshige

How much time would you estimate a javascript novice ought to spend on
getting something like that in the notebook?  Can anyone share tutorials for
the layout portion of the widget API?  Any information is helpful.  Lastly,
does anyone have any hacky ideas of how to bury the source code so that the
widgets merely display, rather than the user copy/pasting the widget source
into a cell?  

Thanks



--
View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From illanos at cal.berkeley.edu  Tue Aug 19 15:02:45 2014
From: illanos at cal.berkeley.edu (Imelda Llanos de Luna)
Date: Tue, 19 Aug 2014 12:02:45 -0700
Subject: [IPython-dev] IPython-Stakeholder Analysis
Message-ID: <BLU184-W4769D1BF3B3759BB5F9C01C7D50@phx.gbl>

Thanks for the warm welcome last week, everyone.
If it's okay, while the DRAFT of the Business Need is under review, I'd like to proceed with performing a Stakeholder Analysis of IPython project to show the different stakeholder groups and layers. I haven't spotted one on the various links, although different pages provide a good idea of most of the stakeholder groups if not all.  If there is a list or visualization and I missed seeing it, let me know.  Otherwise, I can draft something.
Also, if you'd like an example of a Stakeholder Analysis, please contact me at illanos(at)cal.berkeley .edu , and I'll email you a link to one I completed at UC Berkeley.
As for the draft of the Business Need, it can be further elaborated on, e.g. more benchmarking of competitors if it doesn't exist and it would be helpful. I would go into more detail of who they are, their size, their product, their customers, their reach, their cost/fees, etc. Just let me know with any feedback when you have a moment to provide feedback.  Thanks a bunch.
Sincerely,Imelda Llanos de Luna 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140819/eb0b8f97/attachment.html>

From cyrille.rossant at gmail.com  Wed Aug 20 04:15:38 2014
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Wed, 20 Aug 2014 10:15:38 +0200
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <1408298099961-5068197.post@n6.nabble.com>
References: <1408298099961-5068197.post@n6.nabble.com>
Message-ID: <CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>

Hi Adam,

I'll do a tutorial about IPython widgets at EuroSciPy in a week [1].
I'm notably planning to demonstrate how to build, in the notebook, a
GUI like the one you showed. I'll put here the link to the GitHub repo
when everything's ready.

Best,
Cyrille

[1] https://www.euroscipy.org/2014/schedule/presentation/60/

2014-08-17 19:54 GMT+02:00 hugadams <hughesadam87 at gmail.com>:
> (Please disregard if doublepost; having some subscription issues)
>
> Hello,
>
> We are developing a spectroscopy library and want to put in some interactive
> plots/utilities to make the software a bit more approachable to novices.
> I'd really like to build the entire interactive plotting interface through
> the notebook's new widget system.  My reasoning is as follows:
>
>    - All static plots are already matplotlib-based, so we wouldn't have to
> write a seperate plotting api
>    - Notebooks are inherently browser-ready, so sharing plots would be great
>          - Is it likely that static nbviewer will eventually support
> widgets?  Since the data is required, I figured not...
>    - Our tutorials are already in notebooks, so we'd like to be consistent.
>
> I've watched all the videos I can find from the widgets, and think I'm fine
> with the Python end; however, am finding it hard to come by lots of
> information on controlling the layout of the widgets.  I know it's mostly
> done in javascript, and I think the most complex view we will have will look
> something like this:
>
> https://sites.google.com/site/shigemorita/home/2dshige
>
> How much time would you estimate a javascript novice ought to spend on
> getting something like that in the notebook?  Can anyone share tutorials for
> the layout portion of the widget API?  Any information is helpful.  Lastly,
> does anyone have any hacky ideas of how to bury the source code so that the
> widgets merely display, rather than the user copy/pasting the widget source
> into a cell?
>
> Thanks


From rgbkrk at gmail.com  Mon Aug 18 01:09:12 2014
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Mon, 18 Aug 2014 00:09:12 -0500
Subject: [IPython-dev] Url route to download a notebook and open it
In-Reply-To: <CAFpY3ueFDvDSTzBY7wJf0ppfZg8i5zRB+LR14x0pDkA=snK=ZA@mail.gmail.com>
References: <CAFpY3ueFDvDSTzBY7wJf0ppfZg8i5zRB+LR14x0pDkA=snK=ZA@mail.gmail.com>
Message-ID: <CA+tbMaWq5whXLcvLKDznJHLY=9wzRp41V8XzAnxg52Pu_KfuNA@mail.gmail.com>

Going after the security issue, the nice thing is that we're now signing
the notebooks. No javascript from a notebook would be run. iframe embedding
is now blocked by the X-Frame-Options header as well.

With the notebook prior to v2, as an attacker I would create a site that
opens an iframe to a download route (or provides a link) to a malicious
notebook (hosted on one of the whitelisted domains, like
githubusercontent.com).

That's not to say that there wouldn't be a hole elsewhere with a download
route. The thing about making an explicit action on the user's part is to
make this avenue for attack less easy.



On Thu, Aug 7, 2014 at 10:24 AM, Paddy Mullen <paddy at paddymullen.com> wrote:

> It would be useful to have an url route that downloaded a notebook and
> opened it.
>
> I could see a route like http://localhost:8888/download/
> https://raw.githubusercontent.com/ipython/ipython/master/examples/Notebook/SymPy.ipynb
>
> that would download the SymPy notebook and open it.
>
> This would make it easy to link to generated notebooks.  The alternative
> workflow now is to provide a download link, and have the user drag that
> file into the notebook filebrowser.
>
> I realize that this would be a security risk, so it would probably be a
> feature that was best disabled by default.  A configurable whitelist of
> allowable download domains could help a lot.
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140818/65bb6ab3/attachment.html>

From doug.blank at gmail.com  Wed Aug 20 07:22:00 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Wed, 20 Aug 2014 07:22:00 -0400
Subject: [IPython-dev] IPython3/Jupyter Magics for Python-based kernels
Message-ID: <CAAusYCj+L81zm=vLay_VCgBLr6SGeNjYwzWKQVrssQBUbB2AaA@mail.gmail.com>

IPython developers,

One of the nice things that the IPython team did for the next version was
to create a base class for creating new kernels for code written in Python,
called wrapper kernels [1].

Unfortunately, such kernels don't have ! shell commands, nor magics. This
is understandable, as those are tightly integrated with IPython's Python.
However, one could get some of the basic functionality of these features on
top of the wrapper kernel, in a generic way. For example, you could run
shell commands, or have some magics that operate independent of the kernel.

We have created a prototype of magics for wrapper kernels [2]. The code is
in magickernel.py, and the magics are in the folder magics. The current
prototype works like this:

```
In [1]: %magics
Line magics:
     %cd PATH - change current directory of session
     %connect_info - show connection information
     %download URL [FILENAME] - download file from URL
     %html CODE - display code as HTML
     %install_magic URL - download and install magic from URL
     %javascript CODE - send code as JavaScript
     %latex TEXT - display text as LaTex
     %magics - show installed magics
     %reload_magics - reload the magics from the installed files
     %shell COMMAND - run the line as a shell command
     %time COMMAND - show time to run line

Cell magics:
    %%file FILENAME - write contents of cell to file
    %%html - display contents of cell as HTML
    %%javascript - send contents of cell as JavaScript
    %%latex - display contents of cell as LaTeX
    %%shell - run the contents of the cell as shell commands
    %%time - show time to run cell

Shell shortcut:
    ! COMMAND ... - execute command in shell

Any cell magic can be made persistent for rest of session by using %%%
prefix.
```

The idea is to be able to easily add some standard-like magics to 3rd-party
kernels by using MagicKernel. See the sample wrapperkernel EchoKernel with
magics [3]. Also, the idea here is not to replicate too much what comes
with IPython's Python, but to provide a small, simple core set of magics
that can't be easily done in the kernel language. You can install
MagicKernel from source, or using:

$ pip install calico --upgrade

This is a prototype (with some bugs), and we very much hope for others to
provide feedback or code, and more work needs to be done. What magics would
you need in your 3rd-party kernel, that can be accomplished this way? Will
this design be sufficient?

We hope that this is useful!

-Doug

[1] - http://ipython.org/ipython-doc/dev/development/wrapperkernels.html
[2] - https://bitbucket.org/ipre/calico/src/master/src/calico/
[3] -
https://bitbucket.org/ipre/calico/src/master/src/echo_kernel/echo_kernel.py
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140820/a5bb1166/attachment.html>

From ellisonbg at gmail.com  Wed Aug 20 11:21:26 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 20 Aug 2014 11:21:26 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <1408298099961-5068197.post@n6.nabble.com>
References: <1408298099961-5068197.post@n6.nabble.com>
Message-ID: <CAH4pYpQQE_pyXNMXc57ChxxmAvDqn4O_f_niQuXC1ppXNn1jJA@mail.gmail.com>

* You should be able to do this without any JavaScript - just python.
* I would wait a few weeks - we have some new container logic that is
about to be merged into ipython master that will make this much
easier.
* After those changes are in I would estimate it will take about 1-2
days to create this UI once you learn the widget APIs. May take more
time if you need to wire up complex event handling for the UI
elements.

Cheers,

Brian

On Sun, Aug 17, 2014 at 1:54 PM, hugadams <hughesadam87 at gmail.com> wrote:
> (Please disregard if doublepost; having some subscription issues)
>
> Hello,
>
> We are developing a spectroscopy library and want to put in some interactive
> plots/utilities to make the software a bit more approachable to novices.
> I'd really like to build the entire interactive plotting interface through
> the notebook's new widget system.  My reasoning is as follows:
>
>    - All static plots are already matplotlib-based, so we wouldn't have to
> write a seperate plotting api
>    - Notebooks are inherently browser-ready, so sharing plots would be great
>          - Is it likely that static nbviewer will eventually support
> widgets?  Since the data is required, I figured not...
>    - Our tutorials are already in notebooks, so we'd like to be consistent.
>
> I've watched all the videos I can find from the widgets, and think I'm fine
> with the Python end; however, am finding it hard to come by lots of
> information on controlling the layout of the widgets.  I know it's mostly
> done in javascript, and I think the most complex view we will have will look
> something like this:
>
> https://sites.google.com/site/shigemorita/home/2dshige
>
> How much time would you estimate a javascript novice ought to spend on
> getting something like that in the notebook?  Can anyone share tutorials for
> the layout portion of the widget API?  Any information is helpful.  Lastly,
> does anyone have any hacky ideas of how to bury the source code so that the
> widgets merely display, rather than the user copy/pasting the widget source
> into a cell?
>
> Thanks
>
>
>
> --
> View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Wed Aug 20 11:22:50 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 20 Aug 2014 11:22:50 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CAH4pYpQQE_pyXNMXc57ChxxmAvDqn4O_f_niQuXC1ppXNn1jJA@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CAH4pYpQQE_pyXNMXc57ChxxmAvDqn4O_f_niQuXC1ppXNn1jJA@mail.gmail.com>
Message-ID: <CAH4pYpSiECKEpe4+VmzMdSgiymLmf52V_81ahp373Dw-7Do_uQ@mail.gmail.com>

Oh a few more things:

* Plotting is supported
* nbviewer support for widgets is currently being worked on
* To hide the code, I would write the code in a spearate .py file, and
import a top level function and call it.

On Wed, Aug 20, 2014 at 11:21 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> * You should be able to do this without any JavaScript - just python.
> * I would wait a few weeks - we have some new container logic that is
> about to be merged into ipython master that will make this much
> easier.
> * After those changes are in I would estimate it will take about 1-2
> days to create this UI once you learn the widget APIs. May take more
> time if you need to wire up complex event handling for the UI
> elements.
>
> Cheers,
>
> Brian
>
> On Sun, Aug 17, 2014 at 1:54 PM, hugadams <hughesadam87 at gmail.com> wrote:
>> (Please disregard if doublepost; having some subscription issues)
>>
>> Hello,
>>
>> We are developing a spectroscopy library and want to put in some interactive
>> plots/utilities to make the software a bit more approachable to novices.
>> I'd really like to build the entire interactive plotting interface through
>> the notebook's new widget system.  My reasoning is as follows:
>>
>>    - All static plots are already matplotlib-based, so we wouldn't have to
>> write a seperate plotting api
>>    - Notebooks are inherently browser-ready, so sharing plots would be great
>>          - Is it likely that static nbviewer will eventually support
>> widgets?  Since the data is required, I figured not...
>>    - Our tutorials are already in notebooks, so we'd like to be consistent.
>>
>> I've watched all the videos I can find from the widgets, and think I'm fine
>> with the Python end; however, am finding it hard to come by lots of
>> information on controlling the layout of the widgets.  I know it's mostly
>> done in javascript, and I think the most complex view we will have will look
>> something like this:
>>
>> https://sites.google.com/site/shigemorita/home/2dshige
>>
>> How much time would you estimate a javascript novice ought to spend on
>> getting something like that in the notebook?  Can anyone share tutorials for
>> the layout portion of the widget API?  Any information is helpful.  Lastly,
>> does anyone have any hacky ideas of how to bury the source code so that the
>> widgets merely display, rather than the user copy/pasting the widget source
>> into a cell?
>>
>> Thanks
>>
>>
>>
>> --
>> View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197.html
>> Sent from the IPython - Development mailing list archive at Nabble.com.
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From nick.bollweg at gmail.com  Wed Aug 20 18:15:27 2014
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Wed, 20 Aug 2014 18:15:27 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
Message-ID: <CACejjWwrTKVHoUdbi2TeN8p-_JjJdxGZV1F0CJ0fB5mSuF3t=A@mail.gmail.com>

Didn't see this when it was posted (and then didn't see cyrille's and
brian's posts until just now! curse you gmail!), but in the meantime,
hacked this together:


   - http://nbviewer.ipython.org/gist/anonymous/840b5cec9e19f3e39090

Screenshot:

   - http://i.imgur.com/wP3VmWK.png

Obviously lots of room for improvement, but I think you can see you can be
very successful without writing any javacript at all.

There are some tricks: add_class and remove_class only happen to something
that is actually on the page.

As for wrapping it all: you'll end up exposing:

   - a function that returns the dialog widget
   - a function something that handles the add_remove class cleanup
      - it might be fixed in the new stuff
      - a function that will do both of the above and actually call
   IPython.display.display
      - this should also return the widget, in case people want custom
      handlers, I guess?

As Brian says, though: really some great stuff coming up!


On Wed, Aug 20, 2014 at 4:15 AM, Cyrille Rossant <cyrille.rossant at gmail.com>
wrote:

> Hi Adam,
>
> I'll do a tutorial about IPython widgets at EuroSciPy in a week [1].
> I'm notably planning to demonstrate how to build, in the notebook, a
> GUI like the one you showed. I'll put here the link to the GitHub repo
> when everything's ready.
>
> Best,
> Cyrille
>
> [1] https://www.euroscipy.org/2014/schedule/presentation/60/
>
> 2014-08-17 19:54 GMT+02:00 hugadams <hughesadam87 at gmail.com>:
> > (Please disregard if doublepost; having some subscription issues)
> >
> > Hello,
> >
> > We are developing a spectroscopy library and want to put in some
> interactive
> > plots/utilities to make the software a bit more approachable to novices.
> > I'd really like to build the entire interactive plotting interface
> through
> > the notebook's new widget system.  My reasoning is as follows:
> >
> >    - All static plots are already matplotlib-based, so we wouldn't have
> to
> > write a seperate plotting api
> >    - Notebooks are inherently browser-ready, so sharing plots would be
> great
> >          - Is it likely that static nbviewer will eventually support
> > widgets?  Since the data is required, I figured not...
> >    - Our tutorials are already in notebooks, so we'd like to be
> consistent.
> >
> > I've watched all the videos I can find from the widgets, and think I'm
> fine
> > with the Python end; however, am finding it hard to come by lots of
> > information on controlling the layout of the widgets.  I know it's mostly
> > done in javascript, and I think the most complex view we will have will
> look
> > something like this:
> >
> > https://sites.google.com/site/shigemorita/home/2dshige
> >
> > How much time would you estimate a javascript novice ought to spend on
> > getting something like that in the notebook?  Can anyone share tutorials
> for
> > the layout portion of the widget API?  Any information is helpful.
> Lastly,
> > does anyone have any hacky ideas of how to bury the source code so that
> the
> > widgets merely display, rather than the user copy/pasting the widget
> source
> > into a cell?
> >
> > Thanks
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140820/a975a031/attachment.html>

From hughesadam87 at gmail.com  Wed Aug 20 18:29:26 2014
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Wed, 20 Aug 2014 18:29:26 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CACejjWwrTKVHoUdbi2TeN8p-_JjJdxGZV1F0CJ0fB5mSuF3t=A@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CACejjWwrTKVHoUdbi2TeN8p-_JjJdxGZV1F0CJ0fB5mSuF3t=A@mail.gmail.com>
Message-ID: <CAMHV+dATOubFG7MJLzAtB-01JuoUhvLXDFcvkqkVB4rJ57ukgA@mail.gmail.com>

Guys,

Thanks for the help!  Nicholas, that's amazing; thank you for hacking this
just for our sake!  That's really cool of you.

The mailing list was having issues the last few days, so I actually started
a github discussion on this thread.  I'd just like to share that, where
Doug graciously helped me a great deal, and I posted a few followup
concerns.  Obviously they may become moot with all of the cool stuff coming
up.  Here's the link though in case you'd like to check it out and add
anything further.

As I elaborate on that link, the holy grail for us would be able to load
widgets anytime the user wants a GUI.  These widgets would need to accept
input data, and return output data.  For example in the example Nicholas
shared, I'd like the user to be able to pass in a matrix, calculate the
synchronous spectra, and then use that for further analysis in subsequent
cells in the primary notebook.  Having that as a capability would be huge
for us, because we envision a mixed API/GUI workflow.

https://github.com/ipython/ipython/issues/6336


On Wed, Aug 20, 2014 at 6:15 PM, Nicholas Bollweg <nick.bollweg at gmail.com>
wrote:

> Didn't see this when it was posted (and then didn't see cyrille's and
> brian's posts until just now! curse you gmail!), but in the meantime,
> hacked this together:
>
>
>    - http://nbviewer.ipython.org/gist/anonymous/840b5cec9e19f3e39090
>
> Screenshot:
>
>    - http://i.imgur.com/wP3VmWK.png
>
> Obviously lots of room for improvement, but I think you can see you can be
> very successful without writing any javacript at all.
>
> There are some tricks: add_class and remove_class only happen to something
> that is actually on the page.
>
> As for wrapping it all: you'll end up exposing:
>
>    - a function that returns the dialog widget
>    - a function something that handles the add_remove class cleanup
>       - it might be fixed in the new stuff
>       - a function that will do both of the above and actually call
>    IPython.display.display
>       - this should also return the widget, in case people want custom
>       handlers, I guess?
>
> As Brian says, though: really some great stuff coming up!
>
>
> On Wed, Aug 20, 2014 at 4:15 AM, Cyrille Rossant <
> cyrille.rossant at gmail.com> wrote:
>
>> Hi Adam,
>>
>> I'll do a tutorial about IPython widgets at EuroSciPy in a week [1].
>> I'm notably planning to demonstrate how to build, in the notebook, a
>> GUI like the one you showed. I'll put here the link to the GitHub repo
>> when everything's ready.
>>
>> Best,
>> Cyrille
>>
>> [1] https://www.euroscipy.org/2014/schedule/presentation/60/
>>
>> 2014-08-17 19:54 GMT+02:00 hugadams <hughesadam87 at gmail.com>:
>> > (Please disregard if doublepost; having some subscription issues)
>> >
>> > Hello,
>> >
>> > We are developing a spectroscopy library and want to put in some
>> interactive
>> > plots/utilities to make the software a bit more approachable to novices.
>> > I'd really like to build the entire interactive plotting interface
>> through
>> > the notebook's new widget system.  My reasoning is as follows:
>> >
>> >    - All static plots are already matplotlib-based, so we wouldn't have
>> to
>> > write a seperate plotting api
>> >    - Notebooks are inherently browser-ready, so sharing plots would be
>> great
>> >          - Is it likely that static nbviewer will eventually support
>> > widgets?  Since the data is required, I figured not...
>> >    - Our tutorials are already in notebooks, so we'd like to be
>> consistent.
>> >
>> > I've watched all the videos I can find from the widgets, and think I'm
>> fine
>> > with the Python end; however, am finding it hard to come by lots of
>> > information on controlling the layout of the widgets.  I know it's
>> mostly
>> > done in javascript, and I think the most complex view we will have will
>> look
>> > something like this:
>> >
>> > https://sites.google.com/site/shigemorita/home/2dshige
>> >
>> > How much time would you estimate a javascript novice ought to spend on
>> > getting something like that in the notebook?  Can anyone share
>> tutorials for
>> > the layout portion of the widget API?  Any information is helpful.
>> Lastly,
>> > does anyone have any hacky ideas of how to bury the source code so that
>> the
>> > widgets merely display, rather than the user copy/pasting the widget
>> source
>> > into a cell?
>> >
>> > Thanks
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140820/ed95aafa/attachment.html>

From fperez.net at gmail.com  Wed Aug 20 19:07:05 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 20 Aug 2014 16:07:05 -0700
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CAMHV+dATOubFG7MJLzAtB-01JuoUhvLXDFcvkqkVB4rJ57ukgA@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CACejjWwrTKVHoUdbi2TeN8p-_JjJdxGZV1F0CJ0fB5mSuF3t=A@mail.gmail.com>
	<CAMHV+dATOubFG7MJLzAtB-01JuoUhvLXDFcvkqkVB4rJ57ukgA@mail.gmail.com>
Message-ID: <CAHAreOojhJDxgB2Os7REus2_iVJuUJfj867x6-1gX_Tamsw+qw@mail.gmail.com>

On Wed, Aug 20, 2014 at 3:29 PM, Adam Hughes <hughesadam87 at gmail.com> wrote:

> These widgets would need to accept input data, and return output data.
>  For example in the example Nicholas shared, I'd like the user to be able
> to pass in a matrix, calculate the synchronous spectra, and then use that
> for further analysis in subsequent cells in the primary notebook.  Having
> that as a capability would be huge for us, because we envision a mixed
> API/GUI workflow.


One option here would be to provide a text field for the user to name the
variables they want returned for any given output (or you can fix those if
you want), and then have your kernel-side code store them in the user's
namespace directly.

Here's how you can immediately write variables in ipython:

In [7]: ip = get_ipython()

In [8]: a = 'hello'

In [9]: ip.user_ns['a']
Out[9]: 'hello'

In [10]: ip.user_ns['a'] = 100

In [11]: a
Out[11]: 100


We can think further about whether other paradigms might be cleaner, but
I'm not sure that making variables be the return value of the widget itself
will work all that well. But in the meantime, and until we decide what the
best solution is in the long term, this will already work and it uses a
100% stable IPython api. It may be in the end this *is* a long-term
solution we all like, but even if not, I think it's pretty serviceable.

Cheers

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140820/0416ce9f/attachment.html>

From ellisonbg at gmail.com  Wed Aug 20 19:25:48 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 20 Aug 2014 19:25:48 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CACejjWwrTKVHoUdbi2TeN8p-_JjJdxGZV1F0CJ0fB5mSuF3t=A@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CACejjWwrTKVHoUdbi2TeN8p-_JjJdxGZV1F0CJ0fB5mSuF3t=A@mail.gmail.com>
Message-ID: <CAH4pYpTLJ-6ckLrLo36swMtUc2Gc9ET0WeO8xmEp31M6MTHN5A@mail.gmail.com>

The need to do the messy add_class stuff after they are on the page
will go away. We have new HBox and VBox classes that automatially do
the layouts. That is the "great new stuff".

Glad you got this far!

On Wed, Aug 20, 2014 at 6:15 PM, Nicholas Bollweg
<nick.bollweg at gmail.com> wrote:
> Didn't see this when it was posted (and then didn't see cyrille's and
> brian's posts until just now! curse you gmail!), but in the meantime, hacked
> this together:
>
> http://nbviewer.ipython.org/gist/anonymous/840b5cec9e19f3e39090
>
> Screenshot:
>
> http://i.imgur.com/wP3VmWK.png
>
> Obviously lots of room for improvement, but I think you can see you can be
> very successful without writing any javacript at all.
>
> There are some tricks: add_class and remove_class only happen to something
> that is actually on the page.
>
> As for wrapping it all: you'll end up exposing:
>
> a function that returns the dialog widget
> a function something that handles the add_remove class cleanup
>
> it might be fixed in the new stuff
>
> a function that will do both of the above and actually call
> IPython.display.display
>
> this should also return the widget, in case people want custom handlers, I
> guess?
>
> As Brian says, though: really some great stuff coming up!
>
>
> On Wed, Aug 20, 2014 at 4:15 AM, Cyrille Rossant <cyrille.rossant at gmail.com>
> wrote:
>>
>> Hi Adam,
>>
>> I'll do a tutorial about IPython widgets at EuroSciPy in a week [1].
>> I'm notably planning to demonstrate how to build, in the notebook, a
>> GUI like the one you showed. I'll put here the link to the GitHub repo
>> when everything's ready.
>>
>> Best,
>> Cyrille
>>
>> [1] https://www.euroscipy.org/2014/schedule/presentation/60/
>>
>> 2014-08-17 19:54 GMT+02:00 hugadams <hughesadam87 at gmail.com>:
>> > (Please disregard if doublepost; having some subscription issues)
>> >
>> > Hello,
>> >
>> > We are developing a spectroscopy library and want to put in some
>> > interactive
>> > plots/utilities to make the software a bit more approachable to novices.
>> > I'd really like to build the entire interactive plotting interface
>> > through
>> > the notebook's new widget system.  My reasoning is as follows:
>> >
>> >    - All static plots are already matplotlib-based, so we wouldn't have
>> > to
>> > write a seperate plotting api
>> >    - Notebooks are inherently browser-ready, so sharing plots would be
>> > great
>> >          - Is it likely that static nbviewer will eventually support
>> > widgets?  Since the data is required, I figured not...
>> >    - Our tutorials are already in notebooks, so we'd like to be
>> > consistent.
>> >
>> > I've watched all the videos I can find from the widgets, and think I'm
>> > fine
>> > with the Python end; however, am finding it hard to come by lots of
>> > information on controlling the layout of the widgets.  I know it's
>> > mostly
>> > done in javascript, and I think the most complex view we will have will
>> > look
>> > something like this:
>> >
>> > https://sites.google.com/site/shigemorita/home/2dshige
>> >
>> > How much time would you estimate a javascript novice ought to spend on
>> > getting something like that in the notebook?  Can anyone share tutorials
>> > for
>> > the layout portion of the widget API?  Any information is helpful.
>> > Lastly,
>> > does anyone have any hacky ideas of how to bury the source code so that
>> > the
>> > widgets merely display, rather than the user copy/pasting the widget
>> > source
>> > into a cell?
>> >
>> > Thanks
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From tritemio at gmail.com  Mon Aug 18 18:21:32 2014
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Mon, 18 Aug 2014 15:21:32 -0700
Subject: [IPython-dev] IPython Notebook in read-only mode
In-Reply-To: <409FED76-DF37-4375-895A-A10FFE5E2C4F@gmail.com>
References: <CANn2QUyi71Eqg0U_NZ9+UQRS-OZ9YauZn7StsjRfcqXaFzXJJQ@mail.gmail.com>
	<409FED76-DF37-4375-895A-A10FFE5E2C4F@gmail.com>
Message-ID: <CANn2QUzXriAaOfzDQj7=vYaz=xQQ+43754P5pG_JVqsLm5gVGw@mail.gmail.com>

Thanks to all. It turns out that, for me, the quickest solution is just
make a copy of the notebooks folder and have a seconds server that launches
in that dir and accepts remote connections. Even if somebody saves some
modification I don't care and I can periodically overwrite the notebooks
copy with my working version.

Looking forward for ipython 3 with multi-user and read-only support :).

Antonio


On Fri, Aug 15, 2014 at 8:01 AM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

> Nbviewer has an (unsupported) option at startup to serve ipynb from cwd. I
> suppose that will be enough for you.
>
> Envoy? de mon iPhone
>
> > Le 14 ao?t 2014 ? 02:56, Antonino Ingargiola <tritemio at gmail.com> a
> ?crit :
> >
> > Hi,
> >
> > I need to share some notebooks with some colleagues from my desktop
> (windows).
> >
> > Would be possible to configure an ipython notebook server that accepts
> connection from different machines but serves notebooks in read-only mode?
> >
> > I remember that at some point in the past (maybe around 0.13) I had a
> configuration like this, but I don't remember if it was just some side
> effect of the firewall.
> >
> > I could try to install nbviewer but I would also need to setup a second
> server to host the ipynb files IIUC. Any guidance on how to setup nbviewer
> for local files on Windows would also help :).
> >
> > Antonio
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140818/4b44598d/attachment.html>

From hughesadam87 at gmail.com  Wed Aug 20 19:32:35 2014
From: hughesadam87 at gmail.com (hugadams)
Date: Wed, 20 Aug 2014 16:32:35 -0700 (PDT)
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CAHAreOojhJDxgB2Os7REus2_iVJuUJfj867x6-1gX_Tamsw+qw@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CACejjWwrTKVHoUdbi2TeN8p-_JjJdxGZV1F0CJ0fB5mSuF3t=A@mail.gmail.com>
	<CAMHV+dATOubFG7MJLzAtB-01JuoUhvLXDFcvkqkVB4rJ57ukgA@mail.gmail.com>
	<CAHAreOojhJDxgB2Os7REus2_iVJuUJfj867x6-1gX_Tamsw+qw@mail.gmail.com>
Message-ID: <CAMHV+dCaeFyo7QC4534V=F=OQoCvsy3tefJJKyy=3yQxzJr63A@mail.gmail.com>

GREAT!  Thank you


On Wed, Aug 20, 2014 at 7:03 PM, Fernando Perez [via Python] <
ml-node+s6n5068685h97 at n6.nabble.com> wrote:

>
> On Wed, Aug 20, 2014 at 3:29 PM, Adam Hughes <[hidden email]
> <http://user/SendEmail.jtp?type=node&node=5068685&i=0>> wrote:
>
>> These widgets would need to accept input data, and return output data.
>>  For example in the example Nicholas shared, I'd like the user to be able
>> to pass in a matrix, calculate the synchronous spectra, and then use that
>> for further analysis in subsequent cells in the primary notebook.  Having
>> that as a capability would be huge for us, because we envision a mixed
>> API/GUI workflow.
>
>
> One option here would be to provide a text field for the user to name the
> variables they want returned for any given output (or you can fix those if
> you want), and then have your kernel-side code store them in the user's
> namespace directly.
>
> Here's how you can immediately write variables in ipython:
>
> In [7]: ip = get_ipython()
>
> In [8]: a = 'hello'
>
> In [9]: ip.user_ns['a']
> Out[9]: 'hello'
>
> In [10]: ip.user_ns['a'] = 100
>
> In [11]: a
> Out[11]: 100
>
>
> We can think further about whether other paradigms might be cleaner, but
> I'm not sure that making variables be the return value of the widget itself
> will work all that well. But in the meantime, and until we decide what the
> best solution is in the long term, this will already work and it uses a
> 100% stable IPython api. It may be in the end this *is* a long-term
> solution we all like, but even if not, I think it's pretty serviceable.
>
> Cheers
>
> f
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> [hidden email] <http://user/SendEmail.jtp?type=node&node=5068685&i=1>
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5068685.html
>  To unsubscribe from Feasilibity of widgets/layout for mimicking this, click
> here
> <http://python.6.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5068197&code=aHVnaGVzYWRhbTg3QGdtYWlsLmNvbXw1MDY4MTk3fC03NjYyMTgyMzk=>
> .
> NAML
> <http://python.6.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5068690.html
Sent from the IPython - Development mailing list archive at Nabble.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140820/c43404b0/attachment.html>

From hughesadam87 at gmail.com  Wed Aug 20 20:22:18 2014
From: hughesadam87 at gmail.com (hugadams)
Date: Wed, 20 Aug 2014 17:22:18 -0700 (PDT)
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CAH4pYpTLJ-6ckLrLo36swMtUc2Gc9ET0WeO8xmEp31M6MTHN5A@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CACejjWwrTKVHoUdbi2TeN8p-_JjJdxGZV1F0CJ0fB5mSuF3t=A@mail.gmail.com>
	<CAH4pYpTLJ-6ckLrLo36swMtUc2Gc9ET0WeO8xmEp31M6MTHN5A@mail.gmail.com>
Message-ID: <1408580538091-5068692.post@n6.nabble.com>

Brian,

Will you guys make an announcement when this new stuff is merged?  If not,
if you think of it, can you drop a line back to this thread so we know it's
available to try out?



--
View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5068692.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From ellisonbg at gmail.com  Wed Aug 20 20:36:41 2014
From: ellisonbg at gmail.com (Brian)
Date: Wed, 20 Aug 2014 20:36:41 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <1408580538091-5068692.post@n6.nabble.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CACejjWwrTKVHoUdbi2TeN8p-_JjJdxGZV1F0CJ0fB5mSuF3t=A@mail.gmail.com>
	<CAH4pYpTLJ-6ckLrLo36swMtUc2Gc9ET0WeO8xmEp31M6MTHN5A@mail.gmail.com>
	<1408580538091-5068692.post@n6.nabble.com>
Message-ID: <75723EA7-161B-4B9D-A998-8901D6BE51B5@gmail.com>

Ok

Sent from my iPad

> On Aug 20, 2014, at 8:22 PM, hugadams <hughesadam87 at gmail.com> wrote:
> 
> Brian,
> 
> Will you guys make an announcement when this new stuff is merged?  If not,
> if you think of it, can you drop a line back to this thread so we know it's
> available to try out?
> 
> 
> 
> --
> View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5068692.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From ellisonbg at gmail.com  Wed Aug 20 21:12:53 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 20 Aug 2014 21:12:53 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <75723EA7-161B-4B9D-A998-8901D6BE51B5@gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CACejjWwrTKVHoUdbi2TeN8p-_JjJdxGZV1F0CJ0fB5mSuF3t=A@mail.gmail.com>
	<CAH4pYpTLJ-6ckLrLo36swMtUc2Gc9ET0WeO8xmEp31M6MTHN5A@mail.gmail.com>
	<1408580538091-5068692.post@n6.nabble.com>
	<75723EA7-161B-4B9D-A998-8901D6BE51B5@gmail.com>
Message-ID: <CAH4pYpQiRE+g0harfNcQ-3y6hddir9j4=vPotS4ytiOFjFxqng@mail.gmail.com>

Oh, actually, I just remembered that we merged this PR today into
master. We will probably do more work on this stuff before our
release, but master should be way better than latest stable.


On Wed, Aug 20, 2014 at 8:36 PM, Brian <ellisonbg at gmail.com> wrote:
> Ok
>
> Sent from my iPad
>
>> On Aug 20, 2014, at 8:22 PM, hugadams <hughesadam87 at gmail.com> wrote:
>>
>> Brian,
>>
>> Will you guys make an announcement when this new stuff is merged?  If not,
>> if you think of it, can you drop a line back to this thread so we know it's
>> available to try out?
>>
>>
>>
>> --
>> View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5068692.html
>> Sent from the IPython - Development mailing list archive at Nabble.com.
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From graffitici at yahoo.com  Thu Aug 21 04:15:27 2014
From: graffitici at yahoo.com (graffitici at yahoo.com)
Date: Thu, 21 Aug 2014 01:15:27 -0700
Subject: [IPython-dev] Using gevent for IPython CLI and notebook
Message-ID: <1408608927.3778.YahooMailNeo@web164806.mail.gq1.yahoo.com>

Hi,

I'm developing an interactive app that uses gevent Greenlets. I want to have some tasks running in the background while users are interacting with the IPython command-line or the notebook. I realize gevent support will probably never be in the trunk, but I don't mind distributing a modified version of IPython. I've been trying to get this to work in this thread:
https://github.com/ipython/ipython/issues/2785#issuecomment-52588108

adgaudio has been kind enough to help me there. But I still can't replicate what he observes. I tried it with both my system Python from homebrew, and the one in the Anaconda environment he suggested.


I created the following repo:
https://github.com/bbirand/gipython

Can somebody quickly give it a try, and see that they get the "Expected output", as opposed to the "Observed output" that I see?

I've been trying to get this for almost a month now, and I have no clue how to proceed..

Thanks!


From dave.hirschfeld at gmail.com  Thu Aug 21 07:21:36 2014
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Thu, 21 Aug 2014 11:21:36 +0000 (UTC)
Subject: [IPython-dev] ZMQ Segfault - IPython cluster
Message-ID: <loom.20140821T130457-544@post.gmane.org>

I'm running an IPython cluster on Windows HPC x64, Python 2.7.

When I start over ~80-90 engines I notice that the the controller segfaults 
with the only error message printed in the terminal being:

Assertion failed: fds.size () <= FD_SETSIZE 
(bundled\zeromq\src\select.cpp:68)


Looking on the net there's a fair few posts about this. My take on this is 
that it's a bug in libzmq - it should never just kill the host process which 
is exactly what the assert does (at least on windows).

The below script should segfault python when N > 1023.

```
import zmq, sys
N = 1024
print 'ZMQ v',zmq.zmq_version(),' N =',N
ctx = zmq.Context()
try:
    for _ in range(N):
        sock = ctx.socket(zmq.PAIR)
        sock.close()
finally:
    del ctx
```

The problem arises with the number of connections required by the IPython 
cluster which it seems can easily exceed this number.

I hadn't noticed this problem previously but that could well be because I 
wasn't using so many engines. Has anyone else come across this problem 
before? 

With 90 engines running the Task Manager reports the controller process as 
using 1353 handles and netstat says it has 1132 open sockets:

In [44]: controller_pid = '3744'

In [45]: socks = !netstat -a -o -n

In [46]: len(socks)
Out[46]: 2861

In [47]: len(socks.grep(controller_pid, field=-1))
Out[47]: 1132


Is this expected? If so, the default (compile time) value for FD_SETSIZE 
should really be increased IMHO - especially in light of the severity 
(segfault) of the problem.

I've recompiled pyzmq/libzmq manually with FD_SETSIZE=8192 and haven't had 
any further problems.

Regards,
Dave


{'commit_hash': '681fd77',
 'commit_source': 'installation',
 'default_encoding': 'cp1252',
 'ipython_path': 
'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\lib\\site-
packages\\IPython',
 'ipython_version': '2.1.0',
 'os_name': 'nt',
 'platform': 'Windows-2008ServerR2-6.1.7601-SP1',
 'sys_executable': 
'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\python.exe',
 'sys_platform': 'win32',
 'sys_version': '2.7.8 |Continuum Analytics, Inc.| (default, Jul  2 2014, 
15:12:11) [MSC v.1500 64 bit (AMD64)]'}





From Sebastien.Hinderer at inria.fr  Thu Aug 21 10:55:24 2014
From: Sebastien.Hinderer at inria.fr (=?utf-8?Q?S=C3=A9bastien?= Hinderer)
Date: Thu, 21 Aug 2014 16:55:24 +0200
Subject: [IPython-dev] Using IPython notebooks to teach Python
Message-ID: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr>

Dear all,

My team currently studies the possibility to use IPython notebooks in a
Python Mooc, more precisely for doing exercises. Teachers would prepare
the questions in a notebook and students could answer the questions
directly in the notebook itself. Actually, we would provide one copy of
the original teacher's notebook for each student, so that all of them have
their own notebook on which they can work independently.

What I'm trying to study here is the server's security. How to prevent
students from both doing malicious things on the server and accessing /
modifying the notebooks of other students.

Regarding the access to the server, we believe that we can secure things
with AppArmor. Has such a thing aleady been done? Does anybody have
experience in this area? Any help here would be greatly appreciated.

What we really don't know how to do is protecting students from
each other. Would it be possible to somehow run one kernel per student
so that each kernel could have a different AppArmor profile associated
with it? Can you think of any other approach we could use, here?

Many thanks in advance for any comment, remark or suggestion.

S?bastien.




From benjaminrk at gmail.com  Thu Aug 21 12:14:08 2014
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 21 Aug 2014 09:14:08 -0700
Subject: [IPython-dev] ZMQ Segfault - IPython cluster
In-Reply-To: <loom.20140821T130457-544@post.gmane.org>
References: <loom.20140821T130457-544@post.gmane.org>
Message-ID: <CAHNn8BVeudmYV--h4=mFpYA6wz3iFrCpM0HNnNm6Ye3C2rAYYw@mail.gmail.com>

On Thu, Aug 21, 2014 at 4:21 AM, Dave Hirschfeld <dave.hirschfeld at gmail.com>
wrote:

> I'm running an IPython cluster on Windows HPC x64, Python 2.7.
>
> When I start over ~80-90 engines I notice that the the controller segfaults
> with the only error message printed in the terminal being:
>
> Assertion failed: fds.size () <= FD_SETSIZE
> (bundled\zeromq\src\select.cpp:68)
>
>
> Looking on the net there's a fair few posts about this. My take on this is
> that it's a bug in libzmq - it should never just kill the host process
> which
> is exactly what the assert does (at least on windows).
>
> The below script should segfault python when N > 1023.
>
> ```
> import zmq, sys
> N = 1024
> print 'ZMQ v',zmq.zmq_version(),' N =',N
> ctx = zmq.Context()
> try:
>     for _ in range(N):
>         sock = ctx.socket(zmq.PAIR)
>         sock.close()
> finally:
>     del ctx
> ```
>
> The problem arises with the number of connections required by the IPython
> cluster which it seems can easily exceed this number.
>
> I hadn't noticed this problem previously but that could well be because I
> wasn't using so many engines. Has anyone else come across this problem
> before?
>
> With 90 engines running the Task Manager reports the controller process as
> using 1353 handles and netstat says it has 1132 open sockets:
>
> In [44]: controller_pid = '3744'
>
> In [45]: socks = !netstat -a -o -n
>
> In [46]: len(socks)
> Out[46]: 2861
>
> In [47]: len(socks.grep(controller_pid, field=-1))
> Out[47]: 1132
>
>
> Is this expected?

If so, the default (compile time) value for FD_SETSIZE
> should really be increased IMHO - especially in light of the severity
> (segfault) of the problem.
>
> I've recompiled pyzmq/libzmq manually with FD_SETSIZE=8192 and haven't had
> any further problems.
>


Yes, this is expected. The default for FD_SETSIZE is 1024 in libzmq (up
from the default of 64 on Windows), and can be increased at compile time.
As maintainer of pyzmq, I am reluctant for the defaults of pyzmq's bundled
libzmq to be different from the defaults for a traditionally built libzmq,
but zeromq-dev may be open to pushing up the default.

-MinRK


<pedant> a segfault is bad memory access; assert != segfault </pedant>


> Regards,
> Dave
>
>
> {'commit_hash': '681fd77',
>  'commit_source': 'installation',
>  'default_encoding': 'cp1252',
>  'ipython_path':
> 'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\lib\\site-
> packages\\IPython',
>  'ipython_version': '2.1.0',
>  'os_name': 'nt',
>  'platform': 'Windows-2008ServerR2-6.1.7601-SP1',
>  'sys_executable':
> 'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\python.exe',
>  'sys_platform': 'win32',
>  'sys_version': '2.7.8 |Continuum Analytics, Inc.| (default, Jul  2 2014,
> 15:12:11) [MSC v.1500 64 bit (AMD64)]'}
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140821/c4c5e78a/attachment.html>

From benjaminrk at gmail.com  Thu Aug 21 12:14:08 2014
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 21 Aug 2014 09:14:08 -0700
Subject: [IPython-dev] ZMQ Segfault - IPython cluster
In-Reply-To: <loom.20140821T130457-544@post.gmane.org>
References: <loom.20140821T130457-544@post.gmane.org>
Message-ID: <CAHNn8BVeudmYV--h4=mFpYA6wz3iFrCpM0HNnNm6Ye3C2rAYYw@mail.gmail.com>

On Thu, Aug 21, 2014 at 4:21 AM, Dave Hirschfeld <dave.hirschfeld at gmail.com>
wrote:

> I'm running an IPython cluster on Windows HPC x64, Python 2.7.
>
> When I start over ~80-90 engines I notice that the the controller segfaults
> with the only error message printed in the terminal being:
>
> Assertion failed: fds.size () <= FD_SETSIZE
> (bundled\zeromq\src\select.cpp:68)
>
>
> Looking on the net there's a fair few posts about this. My take on this is
> that it's a bug in libzmq - it should never just kill the host process
> which
> is exactly what the assert does (at least on windows).
>
> The below script should segfault python when N > 1023.
>
> ```
> import zmq, sys
> N = 1024
> print 'ZMQ v',zmq.zmq_version(),' N =',N
> ctx = zmq.Context()
> try:
>     for _ in range(N):
>         sock = ctx.socket(zmq.PAIR)
>         sock.close()
> finally:
>     del ctx
> ```
>
> The problem arises with the number of connections required by the IPython
> cluster which it seems can easily exceed this number.
>
> I hadn't noticed this problem previously but that could well be because I
> wasn't using so many engines. Has anyone else come across this problem
> before?
>
> With 90 engines running the Task Manager reports the controller process as
> using 1353 handles and netstat says it has 1132 open sockets:
>
> In [44]: controller_pid = '3744'
>
> In [45]: socks = !netstat -a -o -n
>
> In [46]: len(socks)
> Out[46]: 2861
>
> In [47]: len(socks.grep(controller_pid, field=-1))
> Out[47]: 1132
>
>
> Is this expected?

If so, the default (compile time) value for FD_SETSIZE
> should really be increased IMHO - especially in light of the severity
> (segfault) of the problem.
>
> I've recompiled pyzmq/libzmq manually with FD_SETSIZE=8192 and haven't had
> any further problems.
>


Yes, this is expected. The default for FD_SETSIZE is 1024 in libzmq (up
from the default of 64 on Windows), and can be increased at compile time.
As maintainer of pyzmq, I am reluctant for the defaults of pyzmq's bundled
libzmq to be different from the defaults for a traditionally built libzmq,
but zeromq-dev may be open to pushing up the default.

-MinRK


<pedant> a segfault is bad memory access; assert != segfault </pedant>


> Regards,
> Dave
>
>
> {'commit_hash': '681fd77',
>  'commit_source': 'installation',
>  'default_encoding': 'cp1252',
>  'ipython_path':
> 'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\lib\\site-
> packages\\IPython',
>  'ipython_version': '2.1.0',
>  'os_name': 'nt',
>  'platform': 'Windows-2008ServerR2-6.1.7601-SP1',
>  'sys_executable':
> 'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\python.exe',
>  'sys_platform': 'win32',
>  'sys_version': '2.7.8 |Continuum Analytics, Inc.| (default, Jul  2 2014,
> 15:12:11) [MSC v.1500 64 bit (AMD64)]'}
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140821/c4c5e78a/attachment-0001.html>

From moritz.beber at gmail.com  Thu Aug 21 12:51:28 2014
From: moritz.beber at gmail.com (Moritz Beber)
Date: Thu, 21 Aug 2014 18:51:28 +0200
Subject: [IPython-dev] Using IPython notebooks to teach Python
In-Reply-To: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr>
References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr>
Message-ID: <CAFOFTpSfG=1_LW9wLBH8sFOO2c8423GS9nGJr+q7RisyspoQ0A@mail.gmail.com>

Hey S?bastien,


On Thu, Aug 21, 2014 at 4:55 PM, S?bastien Hinderer <
Sebastien.Hinderer at inria.fr> wrote:

> Dear all,
>
> My team currently studies the possibility to use IPython notebooks in a
> Python Mooc, more precisely for doing exercises. Teachers would prepare
> the questions in a notebook and students could answer the questions
> directly in the notebook itself. Actually, we would provide one copy of
> the original teacher's notebook for each student, so that all of them have
> their own notebook on which they can work independently.
>

Teaching with the notebook has been discussed quite often. Search the
mailing archives for more information. Of course, some of what has been
said there is out-dated now. If you have the time, my recommendation would
be to wait for IPython 3.0 which will come with a multi-user notebook
server that should be secure to use. If you don't have the time then take a
look at my repository: https://github.com/Midnighter/Notebooks-Launcher.
I'm afraid it was written for Python 2 and I've not used it in a while but
it lets you create user accounts for each student and then runs a separate
password protected notebook server for each student in a specified
directory. You can also use it to push and pull material to/from students'
accounts. Naturally, you need su privileges to run it.

There's also https://github.com/UnataInc/ipydra which does a similar thing
but does not allow separate user accounts for each student. I talked to the
author about joining efforts at some point but never got around to it.

I hope these starting points help you out.

Best,
Moritz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140821/bf1d6ab7/attachment.html>

From fperez.net at gmail.com  Thu Aug 21 16:35:27 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 21 Aug 2014 13:35:27 -0700
Subject: [IPython-dev] Listing books on the home page,
	a policy proposal. Feedback?
Message-ID: <CAHAreOpJYoCFs-VEx7mdoOPryAt9im17=P_hrqDbbxzhxjK1zA@mail.gmail.com>

Hi folks,

we just went over this on the dev meeting, but I'd like to give it a day or
two here on-list for feedback in case anyone who wasn't at the meeting has
further comments...

For a while we've had on the homepage a link to Cyrille's IPython book, and
we've been contacted by a publisher of an upcoming title asking for a new
listing there. So I'd like to have uniform criteria that we use to make
that decision in a fair manner with *any* publisher.  Here's what we
discussed during the meeting, with some additional refinements I just
thought of to make things clearer.

One quick note: Brian noted that the website is way too busy already, which
is true. But I want to decouple this specific decision about books from an
entire website redesign conversation. I think that books are one of the
things that actually should belong on the main homepage, as they are a very
important resource related to the project and that benefits it in multiple
ways.  So let's assume that any redesigned home page will have *some* space
for a small image pointing to a book, and the discussion here is about what
to put there and how to provide a policy publishers can all know about.

Any feedback welcome.

Cheers

f

#### Proposed language

Our policy for listing books on the website is as follows. This will be
uniformly applied across publishers, so the project can maintain its
neutrality regarding IPython-related books (all language below would apply
in the future to Jupyter-oriented books).

1. Criteria for inclusion in the homepage

We will list books that are specifically and mostly about IPython itself,
rather than just mentioning IPython in a chapter or two (though see Note in
#2 below).

For example, the currently listed book by Cyrille Rossant amply qualifies,
but Wes McKinney's "Python for Data Analysis", while having an excellent
chapter devoted to IPython, would not get a link on the home page.


2. Our "Books" page:

We will create a main page listing in reverse chronological order the
various books that are mainly devoted to IPython. For each book, we simply
list its title, author and actual publisher page (or Amazon, whatever the
author prefers), along with a cover image.  No editorializing/reviews/etc.

There may be a section at the bottom listing books that include some useful
material about IPython (more than just a sentence but less than say 1/2 the
book).  These would be listed for reference but would have no front-page
link (e.g. Wes' book).

Note: the core team reserves the right to *not* include a book in this
listing. If someone out there publishes a book that we deem to be
absolutely terrible, we reserve the right to simply not list it.


3. What we show on the homepage

We will have a single image for books, and that will be a randomly rotating
screenshot of all currently available books.  That image will be a link to
that book's entry in the "Books" page.

Since now we'll only have at most two of these (the current one and
Cyrille's upcoming one), we can make the distribution give say 2/3 of views
to the new book and 1/3 to the old. If eventually there's multiple books,
we can figure out a *simple* distribution to highlight newer ones more
often while still showing the older ones (as long as they are in print).

-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140821/5c57ab00/attachment.html>

From moritz.beber at gmail.com  Thu Aug 21 17:11:32 2014
From: moritz.beber at gmail.com (Moritz Beber)
Date: Thu, 21 Aug 2014 23:11:32 +0200
Subject: [IPython-dev] Listing books on the home page,
	a policy proposal. Feedback?
In-Reply-To: <CAHAreOpJYoCFs-VEx7mdoOPryAt9im17=P_hrqDbbxzhxjK1zA@mail.gmail.com>
References: <CAHAreOpJYoCFs-VEx7mdoOPryAt9im17=P_hrqDbbxzhxjK1zA@mail.gmail.com>
Message-ID: <CAFOFTpR7BJXmvozRyyzU_6Fed4+_6wTn2NFKmn3TJza4GA51Ng@mail.gmail.com>

Hey,

I don't have strong opinions on this but wanted to give you my user
impressions.


> One quick note: Brian noted that the website is way too busy already,
> which is true. But I want to decouple this specific decision about books
> from an entire website redesign conversation. I think that books are one of
> the things that actually should belong on the main homepage, as they are a
> very important resource related to the project and that benefits it in
> multiple ways.  So let's assume that any redesigned home page will have
> *some* space for a small image pointing to a book, and the discussion here
> is about what to put there and how to provide a policy publishers can all
> know about.
>
>
In terms of website design: You probably discussed this in the meeting but
I would always look for books in the documentation section. Some authors
show a link right on the front page but I think that's mostly the case
where the package and book author are one and the same, and it is a means
of financing the project. ggplot2 did that for a while, for example. Since
IPython seems nicely financed by now and the book(s) were written by third
parties, I feel they don't need a prominent position on the front page.
Python.org has a suggestion in the FAQ:
https://docs.python.org/3/faq/general.html?highlight=book#are-there-any-books-on-python
which seems less than visible to me but if I had that many books written
about my project I would probably chose a similar path.

So overall the docs section just seems to hit the spot for me.

Best,
Moritz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140821/043a9cf3/attachment.html>

From fperez.net at gmail.com  Thu Aug 21 17:48:01 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 21 Aug 2014 14:48:01 -0700
Subject: [IPython-dev] Listing books on the home page,
	a policy proposal. Feedback?
In-Reply-To: <CAFOFTpR7BJXmvozRyyzU_6Fed4+_6wTn2NFKmn3TJza4GA51Ng@mail.gmail.com>
References: <CAHAreOpJYoCFs-VEx7mdoOPryAt9im17=P_hrqDbbxzhxjK1zA@mail.gmail.com>
	<CAFOFTpR7BJXmvozRyyzU_6Fed4+_6wTn2NFKmn3TJza4GA51Ng@mail.gmail.com>
Message-ID: <CAHAreOqPhUpAN3pUrLdK0Kiuf1iB4zPdBNrF-+C_zeHbQPJDTg@mail.gmail.com>

Hi Moritz,

thanks for your feedback!

On Thu, Aug 21, 2014 at 2:11 PM, Moritz Beber <moritz.beber at gmail.com>
wrote:

> In terms of website design: You probably discussed this in the meeting but
> I would always look for books in the documentation section. Some authors
> show a link right on the front page but I think that's mostly the case
> where the package and book author are one and the same, and it is a means
> of financing the project. ggplot2 did that for a while, for example. Since
> IPython seems nicely financed by now and the book(s) were written by third
> parties, I feel they don't need a prominent position on the front page.
> Python.org has a suggestion in the FAQ:
> https://docs.python.org/3/faq/general.html?highlight=book#are-there-any-books-on-python
> which seems less than visible to me but if I had that many books written
> about my project I would probably chose a similar path.
>
> So overall the docs section just seems to hit the spot for me.
>


A couple thoughts:

- good point, the "books page" could just be part of the docs page, or at
the very least linked there. I'd lean towards just making it a section of
the docs page for simplicity.

- I think our situation regarding books is a little different from Python's:

  * we do get contributions from the publisher (at least so far with
Packt). They have a policy of donating a part of the book's proceeds back
to the project. These funds come to our Numfocus account and are
unrestricted, so they are actually very useful and complementary to our
more directed grants.

  * By now, there's probably dozens, if not over a hundred, books about
Python.  There's *one* IPython book, with a second in the pipeline. So I
think at least at this stage, it makes sense to highlight them more
prominently. If we ever find ourselves with too many, we can obviously
revisit these decisions.

  * In our case, the books really complement our docs in a very useful way.
 The python docs, while not perfect, are way better than ours. Cyrille has
done a great job of covering the project in a way that is very useful to
newcomers, so I think it really helps our users.

  * Finally, I think listing official books published about the project on
the site is another signal of the value of IPython to potential visitors.
 That shouldn't be disciounted.


Based on your feedback, I'd adjust the proposal to merge 'books' into the
docs page, but would leave the rest as-is, with one caveat: should we
consider the policy of only listing on the home page books that contribute
back to the project (like Packt's)? This would be obviously still assuming
they pass our quality threshold.

I can see arguments both ways, but I'd like to hear people's thoughts.

Cheers,

f

-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140821/bff240e0/attachment.html>

From jbarratt at serialized.net  Fri Aug 22 02:48:00 2014
From: jbarratt at serialized.net (Joshua Barratt)
Date: Thu, 21 Aug 2014 23:48:00 -0700
Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython
 Notebook on your own machine
In-Reply-To: <CAOvn4qhN8DBS+ADwxjbsik=UqdubPdorxcvULZsAKs5MWnr-hw@mail.gmail.com>
References: <CAOvn4qjaZq5w-Y1hV5tJchWTNURcmzBf=V458-meR-fMSyAmkw@mail.gmail.com>
	<CAN4eD9AwsD3nW+PWCVSqZ65BsqcQ+Hc6H+_ZeoSvXx8+JGx8zw@mail.gmail.com>
	<CAOvn4qhN8DBS+ADwxjbsik=UqdubPdorxcvULZsAKs5MWnr-hw@mail.gmail.com>
Message-ID: <CAN4eD9DR0bJFihqA=nZADtNScXFjMejUha647+4K8LqmJ=V0nw@mail.gmail.com>

It *almost* works on IPython 2.x, except the nbserver-* JSON blobs only
have the PID's in the name, not the actual file.

I took a pass at fixing this up for IPython 2.x, by borrowing some of the
code from the master branch, and it seems to be working.

https://gist.github.com/jbarratt/ae8026493fedc79f122b


On Sun, Aug 3, 2014 at 10:49 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> Neat, thanks for those. For nblist, instead of scanning processes and
> ports, it may be easier to use our API function list_running_servers(),
> especially for cross platform support:
>
>
> http://ipython.org/ipython-doc/dev/api/generated/IPython.html.notebookapp.html#IPython.html.notebookapp.list_running_servers
>
> nbmanager also includes a slightly higher level API around that, which is
> completely independent of all the GUI code (though this requires IPython 3):
>
> https://github.com/takluyver/nbmanager/blob/master/nbmanager/api.py
>
> Best wishes,
> Thomas
>
>
> On 3 August 2014 22:34, Joshua Barratt <jbarratt at serialized.net> wrote:
>
>> Thanks for sharing those, very cool. I had been thinking about doing
>> something like nbopen, I'm happy to see that's something I can delete off
>> of the infinite to-do list.
>>
>> I created something similar to nbmanager, but much more stripped down and
>> only allows viewing.
>>
>> I called mine nblist
>> <https://gist.github.com/jbarratt/ae8026493fedc79f122b>.
>>
>>         $ nblist
>>           http://127.0.0.1:8088 | /Users/me/work/notebooks
>>
>>           http://127.0.0.1:8089 | /Users/me/work/otherproject
>>
>>
>> I also recently wrote something called nbgrep
>> <https://gist.github.com/jbarratt/fa1d3473048e5f856aeb> (blog post here
>> <http://serialized.net/2014/07/mining-your-ipython-notebooks-with-nbgrep/>)
>> which allows you to search the input cells of all your notebooks,
>> regardless of directory.
>>
>> Unfortunately both of mine only work on OSX, though they could be
>> modified to be cross-platform.
>>
>> Josh
>>
>> On Fri, Aug 1, 2014 at 2:55 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>
>>> I've been working on a couple of tools to improve the experience of
>>> working with IPython notebooks on your own computer.
>>>
>>> First, nbopen, a tool to open a notebook in the nearest available
>>> server, or start a new server if there isn't one available. It now
>>> integrates with your file manager so you can double click on a notebook
>>> file to open it (open source desktops only for now, PRs for other systems
>>> welcome):
>>>
>>> https://github.com/takluyver/nbopen
>>>
>>> However, there's no easy way to shut down notebook servers launched by
>>> double clicking on notebooks, because there isn't a terminal where you can
>>> press ctrl-c. So I also made nbmanager, a GUI app which discovers running
>>> notebook servers and open notebooks, and lets you easily shut them down:
>>>
>>> https://github.com/takluyver/nbmanager
>>>
>>> nbopen will work on IPython 2, while nbmanager, which is more
>>> experimental, requires you to be running IPython from git at the moment.
>>>
>>> Thomas
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140821/78fbea3f/attachment.html>

From Sebastien.Hinderer at inria.fr  Fri Aug 22 02:48:02 2014
From: Sebastien.Hinderer at inria.fr (=?utf-8?Q?S=C3=A9bastien?= Hinderer)
Date: Fri, 22 Aug 2014 08:48:02 +0200
Subject: [IPython-dev] Using IPython notebooks to teach Python
In-Reply-To: <CAFOFTpSfG=1_LW9wLBH8sFOO2c8423GS9nGJr+q7RisyspoQ0A@mail.gmail.com>
References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr>
	<CAFOFTpSfG=1_LW9wLBH8sFOO2c8423GS9nGJr+q7RisyspoQ0A@mail.gmail.com>
Message-ID: <20140822064802.GA17770@pl-59055.rocqadm.inria.fr>

Dear Moritz,

Manyh thanks for your prompt and helpfll response!

> Teaching with the notebook has been discussed quite often. Search the
> mailing archives for more information. Of course, some of what has been
> said there is out-dated now. If you have the time, my recommendation would
> be to wait for IPython 3.0 which will come with a multi-user notebook
> server that should be secure to use. If you don't have the time then take a
> look at my repository:
> https://github.com/Midnighter/Notebooks-Launcher.

Well unless iPython 3.0 is released very, very soon, I think we don't
have the time to wait for it so I'll definitely have a look to your
repository, thanks! Is there any planned date for the release of iPython
3.0? 

> I'm afraid it was written for Python 2 and I've not used it in a while but
> it lets you create user accounts for each student and then runs a separate
> password protected notebook server for each student in a specified
> directory.

This seems a very interesting starting point for expeimenting!

> You can also use it to push and pull material to/from students'
> accounts. Naturally, you need su privileges to run it.

Sure. I presume it will be the sme with iPython 3.0? Will the server
have to be run as root and then, once a user logs in, the serverwill
continue running as that user? So does it mean that users will need to
have a full Unix account on the server?
 
> There's also https://github.com/UnataInc/ipydra which does a similar thing
> but does not allow separate user accounts for each student. I talked to the
> author about joining efforts at some point but never got around to it.

Will have a look to that, too! Thanks!

> I hope these starting points help you out.

They definitely do, thanks.

Any comment on the AppArmor aspect would also be helpful.

Best wishes and thanks again,
S?bastien.


From moritz.beber at gmail.com  Fri Aug 22 04:41:31 2014
From: moritz.beber at gmail.com (Moritz Beber)
Date: Fri, 22 Aug 2014 10:41:31 +0200
Subject: [IPython-dev] Using IPython notebooks to teach Python
In-Reply-To: <20140822064802.GA17770@pl-59055.rocqadm.inria.fr>
References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr>
	<CAFOFTpSfG=1_LW9wLBH8sFOO2c8423GS9nGJr+q7RisyspoQ0A@mail.gmail.com>
	<20140822064802.GA17770@pl-59055.rocqadm.inria.fr>
Message-ID: <CAFOFTpRWjPLUp_Qmd2EVObFEJpk+Wp64V2xLRWAWn79SOHT9wA@mail.gmail.com>

Hi,


> Well unless iPython 3.0 is released very, very soon, I think we don't
> have the time to wait for it so I'll definitely have a look to your
> repository, thanks! Is there any planned date for the release of iPython
> 3.0?
>

According to this: https://github.com/ipython/ipython/wiki/Roadmap:-IPython
fairly soon but any dev will be able to provide a better answer.


>
> > I'm afraid it was written for Python 2 and I've not used it in a while
> but
> > it lets you create user accounts for each student and then runs a
> separate
> > password protected notebook server for each student in a specified
> > directory.
>
> This seems a very interesting starting point for expeimenting!
>
> > You can also use it to push and pull material to/from students'
> > accounts. Naturally, you need su privileges to run it.
>
> Sure. I presume it will be the sme with iPython 3.0? Will the server
> have to be run as root and then, once a user logs in, the serverwill
> continue running as that user? So does it mean that users will need to
> have a full Unix account on the server?
>

I'm not familiar with the plans for the multi-user server in 3.0.  It's
possible that the users will simply be distinguished by login and thus
restricted in their rights but that's just guessing on my part now. What my
script does is create a full Unix account for each student and then start a
separate password protected notebook server for each student. It then
spawns a simple website with links for each student to the appropriate
address/port of their notebook server.


> Any comment on the AppArmor aspect would also be helpful.
>
>
I have zero experience with AppArmor. All I can say is that the users are
running their notebook under their unix user account. So as long as the
unix system is set up properly they should not be able to do any harm.
Furthermore the notebooks are password protected so unless a student
chooses to grant the necessary rights or hands out his password, others
cannot affect it. I have also always clearly announced at the beginning of
a course that a student who tries to mess with the unix system or other
people's stuff will (a) fail the course and (b) be brought before a
committee for academic conduct.  This was obviously at a university so I
don't know if this is relevant to you.

Good luck and feel free to mail me off list or open issues on github if you
have problems with my script.

Moritz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/39f7d6eb/attachment.html>

From bussonniermatthias at gmail.com  Fri Aug 22 05:57:13 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Fri, 22 Aug 2014 11:57:13 +0200
Subject: [IPython-dev] Listing books on the home page,
	a policy proposal. Feedback?
In-Reply-To: <CAHAreOqPhUpAN3pUrLdK0Kiuf1iB4zPdBNrF-+C_zeHbQPJDTg@mail.gmail.com>
References: <CAHAreOpJYoCFs-VEx7mdoOPryAt9im17=P_hrqDbbxzhxjK1zA@mail.gmail.com>
	<CAFOFTpR7BJXmvozRyyzU_6Fed4+_6wTn2NFKmn3TJza4GA51Ng@mail.gmail.com>
	<CAHAreOqPhUpAN3pUrLdK0Kiuf1iB4zPdBNrF-+C_zeHbQPJDTg@mail.gmail.com>
Message-ID: <FF4100D8-3979-46EB-9477-24877992EA8D@gmail.com>


>   * By now, there's probably dozens, if not over a hundred, books about Python.  There's *one* IPython book, with a second in the pipeline. So I think at least at this stage, it makes sense to highlight them more prominently. If we ever find ourselves with too many, we can obviously revisit these decisions.

Just to be clear and have it written somewhere : 
What about book written as notebook that do not speak about IPython in particular ? 
(assuming the notebook are available to be re-executed of course) 
-- 
M



From Sebastien.Hinderer at inria.fr  Fri Aug 22 05:55:58 2014
From: Sebastien.Hinderer at inria.fr (=?utf-8?Q?S=C3=A9bastien?= Hinderer)
Date: Fri, 22 Aug 2014 11:55:58 +0200
Subject: [IPython-dev] Using IPython notebooks to teach Python
In-Reply-To: <CAFOFTpRWjPLUp_Qmd2EVObFEJpk+Wp64V2xLRWAWn79SOHT9wA@mail.gmail.com>
References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr>
	<CAFOFTpSfG=1_LW9wLBH8sFOO2c8423GS9nGJr+q7RisyspoQ0A@mail.gmail.com>
	<20140822064802.GA17770@pl-59055.rocqadm.inria.fr>
	<CAFOFTpRWjPLUp_Qmd2EVObFEJpk+Wp64V2xLRWAWn79SOHT9wA@mail.gmail.com>
Message-ID: <20140822095558.GA5722@pl-59055.rocqadm.inria.fr>

Dear Moritz,

I have just discovered this project:

https://github.com/cni/ipython-hydra

Which seems even closer to yours than ipydra.

Are you able to say what differs between your project and this one?
Is it just a matter of one having a web interface and the other not, or
can you see other differences?

Many thanks!
S?bastien.


From Sebastien.Hinderer at inria.fr  Fri Aug 22 06:06:21 2014
From: Sebastien.Hinderer at inria.fr (=?utf-8?Q?S=C3=A9bastien?= Hinderer)
Date: Fri, 22 Aug 2014 12:06:21 +0200
Subject: [IPython-dev] Adding support for more cell types?
Message-ID: <20140822100621.GA32228@pl-59055.rocqadm.inria.fr>

Dear all,

Another question we have, at a longer term, though, would be to figure
out whether iPython could be used to deal with notebooks containing code
blocks for other programming languages. How easy would it be to add,
say, suport for Haskell, OCaml or Lisp cells?

Many thanks in advance for any pointer!
Best wishes,
S?bastien.


From bussonniermatthias at gmail.com  Fri Aug 22 06:16:52 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Fri, 22 Aug 2014 12:16:52 +0200
Subject: [IPython-dev] Adding support for more cell types?
In-Reply-To: <20140822100621.GA32228@pl-59055.rocqadm.inria.fr>
References: <20140822100621.GA32228@pl-59055.rocqadm.inria.fr>
Message-ID: <2554C7B4-A304-4B22-A69D-714BD7718F66@gmail.com>


Le 22 ao?t 2014 ? 12:06, S?bastien Hinderer a ?crit :

> Dear all,
> 
> Another question we have, at a longer term, though, would be to figure
> out whether iPython could be used to deal with notebooks containing code
> blocks for other programming languages. How easy would it be to add,
> say, suport for Haskell, OCaml or Lisp cells?

Haskell and OCamel already ave full kernel implementation :

https://github.com/ipython/ipython/wiki/Projects-using-IPython

You are welcome to create a Lisp on.

If you want to mix/match language you could create a custom magic:

(see cython one for ex [1] )

Or look at calico (above link)
-- 
M

[1] : http://nbviewer.ipython.org/github/ipython/ipython/blob/3607712653c66d63e0d7f13f073bde8c0f209ba8/docs/examples/notebooks/cython_extension.ipynb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/3770c96c/attachment.html>

From gvwilson at third-bit.com  Fri Aug 22 06:51:44 2014
From: gvwilson at third-bit.com (Greg Wilson)
Date: Fri, 22 Aug 2014 06:51:44 -0400
Subject: [IPython-dev] Using IPython notebooks to teach Python
In-Reply-To: <20140822095558.GA5722@pl-59055.rocqadm.inria.fr>
References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr>	<CAFOFTpSfG=1_LW9wLBH8sFOO2c8423GS9nGJr+q7RisyspoQ0A@mail.gmail.com>	<20140822064802.GA17770@pl-59055.rocqadm.inria.fr>	<CAFOFTpRWjPLUp_Qmd2EVObFEJpk+Wp64V2xLRWAWn79SOHT9wA@mail.gmail.com>
	<20140822095558.GA5722@pl-59055.rocqadm.inria.fr>
Message-ID: <53F720C0.5010903@third-bit.com>

Hi all,

We have been using the IPython Notebook to teach Software Carpentry 
classes for a year now, and while we like it a lot, we've been bumping 
into two issues:

1. For most students, the notebook is an entirely new kind of interface 
unlike any they've seen before, which means they're trying to learn two 
things at once: Python, and the notebook itself.  I've had good success 
introducing Python first via the standard command-line interpreter, 
letting them get comfortable with a few basic ideas (say, from "assign a 
value to a variable" to "the code block belonging to a loop must be 
indented"), and *then* introducing the notebook.  This fall, I'm going 
to experiment with showing them Python via Wing101 (a simple IDE that's 
widely used in undergrad computer science teaching), then switching to 
the notebook.

2. Notebooks and version control don't play nicely together.  This may 
not be an issue for you if you're not trying to teach students to use 
version control as part of the class, but diffing and merging the source 
of notebooks is out of reach for novice programmers.

Cheers,
Greg

On 2014-08-22 5:55 AM, S?bastien Hinderer wrote:
> Dear Moritz,
>
> I have just discovered this project:
>
> https://github.com/cni/ipython-hydra
>
> Which seems even closer to yours than ipydra.
>
> Are you able to say what differs between your project and this one?
> Is it just a matter of one having a web interface and the other not, or
> can you see other differences?
>
> Many thanks!
> S?bastien.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From moritz.beber at gmail.com  Fri Aug 22 08:12:54 2014
From: moritz.beber at gmail.com (Moritz Beber)
Date: Fri, 22 Aug 2014 14:12:54 +0200
Subject: [IPython-dev] Using IPython notebooks to teach Python
In-Reply-To: <20140822095558.GA5722@pl-59055.rocqadm.inria.fr>
References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr>
	<CAFOFTpSfG=1_LW9wLBH8sFOO2c8423GS9nGJr+q7RisyspoQ0A@mail.gmail.com>
	<20140822064802.GA17770@pl-59055.rocqadm.inria.fr>
	<CAFOFTpRWjPLUp_Qmd2EVObFEJpk+Wp64V2xLRWAWn79SOHT9wA@mail.gmail.com>
	<20140822095558.GA5722@pl-59055.rocqadm.inria.fr>
Message-ID: <CAFOFTpSDi1wZ-rMdzxonvp-1WWoNvcrCcB75wvOs+XHfm5+jXQ@mail.gmail.com>

>
> I have just discovered this project:
>
> https://github.com/cni/ipython-hydra
>
> Which seems even closer to yours than ipydra.
>
> Are you able to say what differs between your project and this one?
> Is it just a matter of one having a web interface and the other not, or
> can you see other differences?
>
>
The main difference is explained in one of their paragraphs:

"addldapuser will add a new user to the system, if they don't already
exist. It must be run as root. The password is disabled because we use
Stanford's kerberos authentication, so if users log in via ssh they can
authenticate with their SUNet ID and password. The new user is also added
to the ipython group. If the user exists in Stanford LDAP, then their UID
will be assigned by the LDAP result. Otherwise, a local UID is used."

They integrated it with an LDAP server and that is something I shied away
from. The integration with that LDAP server is hard coded into their bash
scripts so it would have taken me some time to decouple it. On top of that
I needed to run the server both on Mac and Linux, so it was easier for me
to use Python for that cross-platform compatibility.

Another difference is that they spawn notebooks and then check the port
assigned whereas my script assigns the port deterministically (but does not
actually check whether that port was already in use before - so that's
desperately needed). So assigning ports beforehand (with the proper checks)
might be nicer when you need to open firewall ports only in a certain range.

I wouldn't mind improving the whole script to update it to the new IPython
2.0 and make it Python 3.0 compatible, and use a simple sqlite database. I
have time for that after September 8. If that's not too late for you, we
could then also talk about specific problems and feature requests.

Cheers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/668849e8/attachment.html>

From doug.blank at gmail.com  Fri Aug 22 08:23:53 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Fri, 22 Aug 2014 08:23:53 -0400
Subject: [IPython-dev] Using IPython notebooks to teach Python
In-Reply-To: <CAFOFTpSDi1wZ-rMdzxonvp-1WWoNvcrCcB75wvOs+XHfm5+jXQ@mail.gmail.com>
References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr>
	<CAFOFTpSfG=1_LW9wLBH8sFOO2c8423GS9nGJr+q7RisyspoQ0A@mail.gmail.com>
	<20140822064802.GA17770@pl-59055.rocqadm.inria.fr>
	<CAFOFTpRWjPLUp_Qmd2EVObFEJpk+Wp64V2xLRWAWn79SOHT9wA@mail.gmail.com>
	<20140822095558.GA5722@pl-59055.rocqadm.inria.fr>
	<CAFOFTpSDi1wZ-rMdzxonvp-1WWoNvcrCcB75wvOs+XHfm5+jXQ@mail.gmail.com>
Message-ID: <CAAusYChjJaEVJ=fjywLVLk3uNaEOXUWdT+V7RNF87bTUZD33ow@mail.gmail.com>

You may also want to follow the development of the Jupyter Hub for IPython3:

https://github.com/jupyter/jupyterhub

Although I suspect that IPython3/Jupyter won't be released for months, I
was able to login (via my Linux id and password) with the code as of
yesterday. In addition, I was able to use our new IPython3 kernels without
any obvious issues. It looks like it will be very easy to substitute the os
authentication for another.

I'm making plans to use this multi-user hub in the classroom as soon as
possible this semester. (I'm teaching Programming Languages, and so easily
switching between languages should make an interesting basis for the
course.)

-Doug


On Fri, Aug 22, 2014 at 8:12 AM, Moritz Beber <moritz.beber at gmail.com>
wrote:

>
>
>> I have just discovered this project:
>>
>> https://github.com/cni/ipython-hydra
>>
>> Which seems even closer to yours than ipydra.
>>
>> Are you able to say what differs between your project and this one?
>> Is it just a matter of one having a web interface and the other not, or
>> can you see other differences?
>>
>>
> The main difference is explained in one of their paragraphs:
>
> "addldapuser will add a new user to the system, if they don't already
> exist. It must be run as root. The password is disabled because we use
> Stanford's kerberos authentication, so if users log in via ssh they can
> authenticate with their SUNet ID and password. The new user is also added
> to the ipython group. If the user exists in Stanford LDAP, then their UID
> will be assigned by the LDAP result. Otherwise, a local UID is used."
>
> They integrated it with an LDAP server and that is something I shied away
> from. The integration with that LDAP server is hard coded into their bash
> scripts so it would have taken me some time to decouple it. On top of that
> I needed to run the server both on Mac and Linux, so it was easier for me
> to use Python for that cross-platform compatibility.
>
> Another difference is that they spawn notebooks and then check the port
> assigned whereas my script assigns the port deterministically (but does not
> actually check whether that port was already in use before - so that's
> desperately needed). So assigning ports beforehand (with the proper checks)
> might be nicer when you need to open firewall ports only in a certain range.
>
> I wouldn't mind improving the whole script to update it to the new IPython
> 2.0 and make it Python 3.0 compatible, and use a simple sqlite database. I
> have time for that after September 8. If that's not too late for you, we
> could then also talk about specific problems and feature requests.
>
> Cheers
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/4694c5d4/attachment.html>

From dave.hirschfeld at gmail.com  Fri Aug 22 10:26:11 2014
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Fri, 22 Aug 2014 14:26:11 +0000 (UTC)
Subject: [IPython-dev] ZMQ Segfault - IPython cluster
References: <loom.20140821T130457-544@post.gmane.org>
	<CAHNn8BVeudmYV--h4=mFpYA6wz3iFrCpM0HNnNm6Ye3C2rAYYw@mail.gmail.com>
Message-ID: <loom.20140822T151124-771@post.gmane.org>

MinRK <benjaminrk <at> gmail.com> writes:

> 
> 
> Yes, this is expected. The default for FD_SETSIZE is 1024 in libzmq (up 
from the default of 64 on Windows), and can be increased at compile time. As 
maintainer of pyzmq, I am reluctant for the defaults of pyzmq's bundled 
libzmq to be different from the defaults for a traditionally built libzmq, 
but zeromq-dev may be open to pushing up the default.
>  
> 
> -MinRK
> ?
> 
> <pedant> a segfault is bad memory access; assert != segfault </pedant>
> 


It was certainly unexpected for me! :P

There's also no mention of FD_SETSIZE on either the IPython or PyZMQ 
websites AFAICS.

It was a fairly nasty bug to track down because the remote process failed 
with no indication of why it did so. This is because Windows HPC 2008 only 
caches the first 4000 characters of stdout/stderr so the error message never 
got displayed (this has changed to a much more useful last 4k chars in 
2012). To actually see the message I had to remote on the node and start the 
ipcontroller process manually in a terminal. 

After actually managing to get an error message I then had to search the 
internet for what it meant and then recompile libzmq itself with VS2008 
which isn't exactly a cakewalk for the uninitiated.

I agree it should be fixed in the upstream library so I've opened an issue
there - we can see what they say there...

https://github.com/zeromq/libzmq/issues/1165


Thanks,
Dave








From doug.blank at gmail.com  Fri Aug 22 11:46:47 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Fri, 22 Aug 2014 11:46:47 -0400
Subject: [IPython-dev] Just for fun: chatbot in Jupyter/IPython3
Message-ID: <CAAusYCg0Oe6Z_-zaVDttU0etzued+63K4oY1jT=zYH4vEsuuTA@mail.gmail.com>

Just testing out the new ipython-master and jupyterhub repositories. If you
are trying those, you can easily add a chatbot kernel:

$ pip install chatbot_kernel

Now you can use a console with chatbot_kernel, or select "Chatbot" from the
notebook. Perhaps this type of kernel could actually be made into a
help-bot, or tutor. Or just to have some fun :)

$ ipython console --kernel chatbot_kernel
(loads after a few seconds; loading databases...)
(hit enter to display messages; need to flush)

In [1]: Hello
Hi there!

In [2]: Who are you?
Nameless

In [3]: What time is it?
The time here is Fri Aug 22 11:35:00 2014

In [4]: Do you like movies?
Yes, I love film, especially science-fiction and comedy.

In [5]: What is your favorite movie?
My favorite movie is . Have you seen     ?

In [6]: You have some bugs
What makes you think I have bugs?

In [7]: Do you have a mother?
No I only have a father.

In [8]: What is your father's name?
My  is .

In [9]: I have a fear of spiders
Interesting. I have a lot of answers to silly questions.

In [10]: What do you want to do?
I would like to study psychology and maybe go into law or politics.

In [11]: I like computer science
What do you think of the way the press covers your field?

In [12]: It is mostly hype
Really.  mostly hype is?

In [13]: Have you heard of Jupyter?
No, tell me about him, her or it.

In [14]: It is a system for computing
mostly hype is a system for computing. I suppose that makes sense.

In [15]: I see you take my words and re use them
Yes I take his words and re use them.

In [16]: Ok, goodbye
"Yes I take his words and re use them."? Ok, goodbye?. Sayonara.

-Doug

PS - source is at:
https://bitbucket.org/ipre/calico/src/master/src/chatbot_kernel/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/dd76a0b6/attachment.html>

From damianavila at gmail.com  Fri Aug 22 11:49:03 2014
From: damianavila at gmail.com (=?UTF-8?Q?Dami=C3=A1n_Avila?=)
Date: Fri, 22 Aug 2014 12:49:03 -0300
Subject: [IPython-dev] Just for fun: chatbot in Jupyter/IPython3
In-Reply-To: <CAAusYCg0Oe6Z_-zaVDttU0etzued+63K4oY1jT=zYH4vEsuuTA@mail.gmail.com>
References: <CAAusYCg0Oe6Z_-zaVDttU0etzued+63K4oY1jT=zYH4vEsuuTA@mail.gmail.com>
Message-ID: <CAH+mRR064iRupER=JCcT-JGvihZ-5LL3V7=JQ8_RMuxeFyCmmQ@mail.gmail.com>

Ja ja... nice!!!


2014-08-22 12:46 GMT-03:00 Doug Blank <doug.blank at gmail.com>:

> Just testing out the new ipython-master and jupyterhub repositories. If
> you are trying those, you can easily add a chatbot kernel:
>
> $ pip install chatbot_kernel
>
> Now you can use a console with chatbot_kernel, or select "Chatbot" from
> the notebook. Perhaps this type of kernel could actually be made into a
> help-bot, or tutor. Or just to have some fun :)
>
> $ ipython console --kernel chatbot_kernel
> (loads after a few seconds; loading databases...)
> (hit enter to display messages; need to flush)
>
> In [1]: Hello
> Hi there!
>
> In [2]: Who are you?
> Nameless
>
> In [3]: What time is it?
> The time here is Fri Aug 22 11:35:00 2014
>
> In [4]: Do you like movies?
> Yes, I love film, especially science-fiction and comedy.
>
> In [5]: What is your favorite movie?
> My favorite movie is . Have you seen     ?
>
> In [6]: You have some bugs
> What makes you think I have bugs?
>
> In [7]: Do you have a mother?
> No I only have a father.
>
> In [8]: What is your father's name?
> My  is .
>
> In [9]: I have a fear of spiders
> Interesting. I have a lot of answers to silly questions.
>
> In [10]: What do you want to do?
> I would like to study psychology and maybe go into law or politics.
>
> In [11]: I like computer science
> What do you think of the way the press covers your field?
>
> In [12]: It is mostly hype
> Really.  mostly hype is?
>
> In [13]: Have you heard of Jupyter?
> No, tell me about him, her or it.
>
> In [14]: It is a system for computing
> mostly hype is a system for computing. I suppose that makes sense.
>
> In [15]: I see you take my words and re use them
> Yes I take his words and re use them.
>
> In [16]: Ok, goodbye
> "Yes I take his words and re use them."? Ok, goodbye?. Sayonara.
>
> -Doug
>
> PS - source is at:
> https://bitbucket.org/ipre/calico/src/master/src/chatbot_kernel/
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
*Dami?n*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/62ac7e63/attachment.html>

From ocefpaf at gmail.com  Fri Aug 22 11:55:48 2014
From: ocefpaf at gmail.com (Filipe Pires Alvarenga Fernandes)
Date: Fri, 22 Aug 2014 12:55:48 -0300
Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter
Message-ID: <CAH2VmmA0Tv-1TPjdPN7Dkjw4q+4ZMde_ZFS-BhZx6J+tPWp2YQ@mail.gmail.com>

Have anyone tried to use venvs as kernels?  Making the installed vens show
up in the drop-down menu in the new Jupyter Notebook?

I was toying with the idea but got nowhere near something useful.

I think it would be a nice way to switch between virtual environments.

-Filipe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/8c591ea4/attachment.html>

From fperez.net at gmail.com  Fri Aug 22 13:34:04 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 22 Aug 2014 10:34:04 -0700
Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter
In-Reply-To: <CAH2VmmA0Tv-1TPjdPN7Dkjw4q+4ZMde_ZFS-BhZx6J+tPWp2YQ@mail.gmail.com>
References: <CAH2VmmA0Tv-1TPjdPN7Dkjw4q+4ZMde_ZFS-BhZx6J+tPWp2YQ@mail.gmail.com>
Message-ID: <CAHAreOro4BTe_GjLKafKA8yT5XcNEHPnROi2UXVZO=GqZS_VVQ@mail.gmail.com>

We looked into it and as far as we can tell, all the pieces we're providing
for the kernel spec should make it possible to have your kernel entry point
do the switching.

But we will NOT bake into the kernel spec direct knowledge of venv, conda
envs, or any other 'enviornment manager'. We want to make it *possible* do
declare kernels that are connected to environments, but as of yet, there
are just too many ways of 'managing environments' across languages and
toolchains for us to enshrine any particular approach into the spec itself.

As I say above, we looked into it, but we didn't actually implement
anything yet. So we'd encourage you to try things out, and report both with
what works (I'm sure others are interested) and what doesn't (so we can
help out or fix things if needed)

Cheers

f


On Fri, Aug 22, 2014 at 8:55 AM, Filipe Pires Alvarenga Fernandes <
ocefpaf at gmail.com> wrote:

> Have anyone tried to use venvs as kernels?  Making the installed vens show
> up in the drop-down menu in the new Jupyter Notebook?
>
> I was toying with the idea but got nowhere near something useful.
>
> I think it would be a nice way to switch between virtual environments.
>
> -Filipe
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/245445f0/attachment.html>

From fperez.net at gmail.com  Fri Aug 22 13:36:01 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 22 Aug 2014 10:36:01 -0700
Subject: [IPython-dev] Using IPython notebooks to teach Python
In-Reply-To: <CAAusYChjJaEVJ=fjywLVLk3uNaEOXUWdT+V7RNF87bTUZD33ow@mail.gmail.com>
References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr>
	<CAFOFTpSfG=1_LW9wLBH8sFOO2c8423GS9nGJr+q7RisyspoQ0A@mail.gmail.com>
	<20140822064802.GA17770@pl-59055.rocqadm.inria.fr>
	<CAFOFTpRWjPLUp_Qmd2EVObFEJpk+Wp64V2xLRWAWn79SOHT9wA@mail.gmail.com>
	<20140822095558.GA5722@pl-59055.rocqadm.inria.fr>
	<CAFOFTpSDi1wZ-rMdzxonvp-1WWoNvcrCcB75wvOs+XHfm5+jXQ@mail.gmail.com>
	<CAAusYChjJaEVJ=fjywLVLk3uNaEOXUWdT+V7RNF87bTUZD33ow@mail.gmail.com>
Message-ID: <CAHAreOqD9WF4wxCeJRpQNg9EZjRQavpAL2+=ZC_dQA2iKmOK_Q@mail.gmail.com>

On Fri, Aug 22, 2014 at 5:23 AM, Doug Blank <doug.blank at gmail.com> wrote:

> You may also want to follow the development of the Jupyter Hub for
> IPython3:
>
> https://github.com/jupyter/jupyterhub
>

BTW, that's what we mean when we say "the multiuser IPython server". Since
that machinery is fully language agnostic, we're putting it as part of
Jupyter itself from the get-go.  I know Doug knows this, I just want to
make sure it's clear for everyone else...


> Although I suspect that IPython3/Jupyter won't be released for months, I
> was able to login (via my Linux id and password) with the code as of
> yesterday. In addition, I was able to use our new IPython3 kernels without
> any obvious issues. It looks like it will be very easy to substitute the os
> authentication for another.
>

We're hoping to have a release by October/November.  I actually think
that's pretty realistic. It may not have too many bells and whistles, but
given that Min's current prototype already does work, we should be OK
pushing out a basic MVP in that timeframe.

Cheers

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/bf3c0c15/attachment.html>

From fperez.net at gmail.com  Fri Aug 22 13:36:49 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 22 Aug 2014 10:36:49 -0700
Subject: [IPython-dev] Just for fun: chatbot in Jupyter/IPython3
In-Reply-To: <CAAusYCg0Oe6Z_-zaVDttU0etzued+63K4oY1jT=zYH4vEsuuTA@mail.gmail.com>
References: <CAAusYCg0Oe6Z_-zaVDttU0etzued+63K4oY1jT=zYH4vEsuuTA@mail.gmail.com>
Message-ID: <CAHAreOp8hP5heJJGzNfcT12dnO8AUUbrV=010Q1nbFd_q=JuOg@mail.gmail.com>

On Fri, Aug 22, 2014 at 8:46 AM, Doug Blank <doug.blank at gmail.com> wrote:

> Just testing out the new ipython-master and jupyterhub repositories. If
> you are trying those, you can easily add a chatbot kernel:
>
> $ pip install chatbot_kernel
>

This is just awesome! Thanks :)

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/01f1100c/attachment.html>

From ellisonbg at gmail.com  Fri Aug 22 14:35:42 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 22 Aug 2014 14:35:42 -0400
Subject: [IPython-dev] Just for fun: chatbot in Jupyter/IPython3
In-Reply-To: <CAAusYCg0Oe6Z_-zaVDttU0etzued+63K4oY1jT=zYH4vEsuuTA@mail.gmail.com>
References: <CAAusYCg0Oe6Z_-zaVDttU0etzued+63K4oY1jT=zYH4vEsuuTA@mail.gmail.com>
Message-ID: <8489708C-68D9-4C9E-AB75-060C7CBD0CC8@gmail.com>

:-)

Sent from my iPhone

> On Aug 22, 2014, at 11:46 AM, Doug Blank <doug.blank at gmail.com> wrote:
> 
> Just testing out the new ipython-master and jupyterhub repositories. If you are trying those, you can easily add a chatbot kernel:
> 
> $ pip install chatbot_kernel
> 
> Now you can use a console with chatbot_kernel, or select "Chatbot" from the notebook. Perhaps this type of kernel could actually be made into a help-bot, or tutor. Or just to have some fun :)
> 
> $ ipython console --kernel chatbot_kernel
> (loads after a few seconds; loading databases...)
> (hit enter to display messages; need to flush)
> 
> In [1]: Hello
> Hi there!
> 
> In [2]: Who are you?
> Nameless
> 
> In [3]: What time is it?
> The time here is Fri Aug 22 11:35:00 2014
> 
> In [4]: Do you like movies?
> Yes, I love film, especially science-fiction and comedy.
> 
> In [5]: What is your favorite movie?
> My favorite movie is . Have you seen     ?
> 
> In [6]: You have some bugs
> What makes you think I have bugs?
> 
> In [7]: Do you have a mother?
> No I only have a father.
> 
> In [8]: What is your father's name?
> My  is .
> 
> In [9]: I have a fear of spiders
> Interesting. I have a lot of answers to silly questions.
> 
> In [10]: What do you want to do?
> I would like to study psychology and maybe go into law or politics.
> 
> In [11]: I like computer science
> What do you think of the way the press covers your field?
> 
> In [12]: It is mostly hype
> Really.  mostly hype is?
> 
> In [13]: Have you heard of Jupyter?
> No, tell me about him, her or it.
> 
> In [14]: It is a system for computing
> mostly hype is a system for computing. I suppose that makes sense.
> 
> In [15]: I see you take my words and re use them
> Yes I take his words and re use them.
> 
> In [16]: Ok, goodbye
> "Yes I take his words and re use them."? Ok, goodbye?. Sayonara.
> 
> -Doug
> 
> PS - source is at:
> https://bitbucket.org/ipre/calico/src/master/src/chatbot_kernel/
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/2217b9fe/attachment.html>

From ocefpaf at gmail.com  Fri Aug 22 14:40:39 2014
From: ocefpaf at gmail.com (Filipe Pires Alvarenga Fernandes)
Date: Fri, 22 Aug 2014 15:40:39 -0300
Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter
In-Reply-To: <CAHAreOro4BTe_GjLKafKA8yT5XcNEHPnROi2UXVZO=GqZS_VVQ@mail.gmail.com>
References: <CAH2VmmA0Tv-1TPjdPN7Dkjw4q+4ZMde_ZFS-BhZx6J+tPWp2YQ@mail.gmail.com>
	<CAHAreOro4BTe_GjLKafKA8yT5XcNEHPnROi2UXVZO=GqZS_VVQ@mail.gmail.com>
Message-ID: <CAH2VmmA+Z--q3HS7QdJWPN8As-pjq2PHLQ+NQ5-z9D9tzw8GRA@mail.gmail.com>

On Fri, Aug 22, 2014 at 2:34 PM, Fernando Perez <fperez.net at gmail.com>
wrote:

> We looked into it and as far as we can tell, all the pieces we're
> providing for the kernel spec should make it possible to have your kernel
> entry point do the switching.
>

OK, I will keep trying to have something more robust.  Right now I am doing
a very hackish workaround.


> But we will NOT bake into the kernel spec direct knowledge of venv, conda
> envs, or any other 'enviornment manager'. We want to make it *possible* do
> declare kernels that are connected to environments, but as of yet, there
> are just too many ways of 'managing environments' across languages and
> toolchains for us to enshrine any particular approach into the spec itself.
>

I understand (and I agree) with that.


> As I say above, we looked into it, but we didn't actually implement
> anything yet. So we'd encourage you to try things out, and report both with
> what works (I'm sure others are interested) and what doesn't (so we can
> help out or fix things if needed)
>

As soon as I have something better I will post it here.


> Cheers
>
> f
>

Thanks,

-Filipe


>
>
> On Fri, Aug 22, 2014 at 8:55 AM, Filipe Pires Alvarenga Fernandes <
> ocefpaf at gmail.com> wrote:
>
>> Have anyone tried to use venvs as kernels?  Making the installed vens
>> show up in the drop-down menu in the new Jupyter Notebook?
>>
>> I was toying with the idea but got nowhere near something useful.
>>
>> I think it would be a nice way to switch between virtual environments.
>>
>> -Filipe
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/c8822c14/attachment.html>

From rsarava at gmail.com  Fri Aug 22 20:24:33 2014
From: rsarava at gmail.com (Ramalingam Saravanan)
Date: Fri, 22 Aug 2014 19:24:33 -0500
Subject: [IPython-dev]  Using IPython notebooks to teach Python
Message-ID: <CAPoFz=cy1eJYmT0H-JSYfN+LK_Y-xtPUjcGZtt8DK1uYuK8DQA@mail.gmail.com>

Hi S?bastien,

The multi-user IPython server would be the best option for your needs, but
here are some alternatives:

    https://github.com/ptone/jiffylab

    https://cloud.sagemath.com

    https://github.com/mitotic/graphterm (my own project; somewhat
experimental)

Here's a paper from SciPy14 proceedings that discusses these alternatives:

    https://dl.dropboxusercontent.com/u/72208800/docs/Saravanan-SciPy14.pdf

Best,
Saravanan


On Thu, Aug 21, 2014 at 4:55 PM, S?bastien Hinderer <
Sebastien.Hinderer at inria.fr> wrote:

> Dear all,
>
> My team currently studies the possibility to use IPython notebooks in a
> Python Mooc, more precisely for doing exercises. Teachers would prepare
> the questions in a notebook and students could answer the questions
> directly in the notebook itself. Actually, we would provide one copy of
> the original teacher's notebook for each student, so that all of them have
> their own notebook on which they can work independently.
>
> What I'm trying to study here is the server's security. How to prevent
> students from both doing malicious things on the server and accessing /
> modifying the notebooks of other students.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/096fb836/attachment.html>

From benjaminrk at gmail.com  Sat Aug 23 01:12:59 2014
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 22 Aug 2014 22:12:59 -0700
Subject: [IPython-dev] IPython3/Jupyter Magics for Python-based kernels
In-Reply-To: <CAAusYCj+L81zm=vLay_VCgBLr6SGeNjYwzWKQVrssQBUbB2AaA@mail.gmail.com>
References: <CAAusYCj+L81zm=vLay_VCgBLr6SGeNjYwzWKQVrssQBUbB2AaA@mail.gmail.com>
Message-ID: <CAHNn8BVg69vG9CxwiPHPduUqZY3Zzwwarx7c5eLwyO8=YyWBVw@mail.gmail.com>

I think if you want to add something like magics for Python-wrapped kernels
based on your magickernel.py code, that would be interesting. Standardizing
magics across kernel languages is something that is attractive on its face,
but we deemed outside the scope of the message / kernel specification,
because it is probably not feasible to do in a language agnostic way (For
example, IPython magics can contain Python string-formatting code with
Python expressions that are evaluated in the interactive namespace).
Clarification of discussions like this are part of the reasoning behind the
Jupyter/IPython split:

   - Jupyter is language-agnostic computing tools, such as the message
   protocol, notebook document format, and notebook UI.
   - IPython is a better interactive Python environment, with some
   extensions to the Python language for interactive use.
   - IPython provides a Jupyter Kernel.
   - Magics are 100% an IPython feature, and not in any way related to
   Jupyter.
   - Other kernels are welcome to implement extensions to their language
   (including Python!) as they see fit,
   as similar or different to IPython?s magics system as makes sense.

Implementing magics for basic Python-wrapped kernels (i.e. not R, Julia,
Ruby, Haskell, etc.) would be simple, and potentially useful, depending on
how many of these kernels are developed. I do not expect most first-class
kernels to go this route, in which case they would not share any magic
code. Further, I don?t think the magic code developed for this use case
would even be shared by IPython.

But it would still be an interesting exercise, and might encourage other
kernels to take the multi-interpreter approach if it is successful.

Sounds like a fun experiment!

-MinRK
?


On Wed, Aug 20, 2014 at 4:22 AM, Doug Blank <doug.blank at gmail.com> wrote:

> IPython developers,
>
> One of the nice things that the IPython team did for the next version was
> to create a base class for creating new kernels for code written in Python,
> called wrapper kernels [1].
>
> Unfortunately, such kernels don't have ! shell commands, nor magics. This
> is understandable, as those are tightly integrated with IPython's Python.
> However, one could get some of the basic functionality of these features on
> top of the wrapper kernel, in a generic way. For example, you could run
> shell commands, or have some magics that operate independent of the kernel.
>
> We have created a prototype of magics for wrapper kernels [2]. The code is
> in magickernel.py, and the magics are in the folder magics. The current
> prototype works like this:
>
> ```
> In [1]: %magics
> Line magics:
>      %cd PATH - change current directory of session
>      %connect_info - show connection information
>      %download URL [FILENAME] - download file from URL
>      %html CODE - display code as HTML
>      %install_magic URL - download and install magic from URL
>      %javascript CODE - send code as JavaScript
>      %latex TEXT - display text as LaTex
>      %magics - show installed magics
>      %reload_magics - reload the magics from the installed files
>      %shell COMMAND - run the line as a shell command
>      %time COMMAND - show time to run line
>
> Cell magics:
>     %%file FILENAME - write contents of cell to file
>     %%html - display contents of cell as HTML
>     %%javascript - send contents of cell as JavaScript
>     %%latex - display contents of cell as LaTeX
>     %%shell - run the contents of the cell as shell commands
>     %%time - show time to run cell
>
> Shell shortcut:
>     ! COMMAND ... - execute command in shell
>
> Any cell magic can be made persistent for rest of session by using %%%
> prefix.
> ```
>
> The idea is to be able to easily add some standard-like magics to
> 3rd-party kernels by using MagicKernel. See the sample wrapperkernel
> EchoKernel with magics [3]. Also, the idea here is not to replicate too
> much what comes with IPython's Python, but to provide a small, simple core
> set of magics that can't be easily done in the kernel language. You can
> install MagicKernel from source, or using:
>
> $ pip install calico --upgrade
>
> This is a prototype (with some bugs), and we very much hope for others to
> provide feedback or code, and more work needs to be done. What magics would
> you need in your 3rd-party kernel, that can be accomplished this way? Will
> this design be sufficient?
>
> We hope that this is useful!
>
> -Doug
>
> [1] - http://ipython.org/ipython-doc/dev/development/wrapperkernels.html
> [2] - https://bitbucket.org/ipre/calico/src/master/src/calico/
> [3] -
> https://bitbucket.org/ipre/calico/src/master/src/echo_kernel/echo_kernel.py
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140822/412538aa/attachment.html>

From claresloggett at gmail.com  Sat Aug 23 07:30:55 2014
From: claresloggett at gmail.com (Clare Sloggett)
Date: Sat, 23 Aug 2014 21:30:55 +1000
Subject: [IPython-dev] Using IPython notebooks to teach Python
In-Reply-To: <CAPoFz=cy1eJYmT0H-JSYfN+LK_Y-xtPUjcGZtt8DK1uYuK8DQA@mail.gmail.com>
References: <CAPoFz=cy1eJYmT0H-JSYfN+LK_Y-xtPUjcGZtt8DK1uYuK8DQA@mail.gmail.com>
Message-ID: <CAETqNqG8xDOimx_HWzk18z1RE6R+5+kTyT1-2k9zY3tpL9CDSw@mail.gmail.com>

Wakari (wakari.io) is another free service with fairly quick sign-up. I've
used Wakari and SageMathCloud as backup options for when a student had an
issue with the uni infrastructure.

My understanding is that these services manage security is to provide each
user with their own virtualised environment so they can't really do any
harm, and that this can be something lightweight, like docker.

If you're going to use a service outside your institution you might want to
watch out for websocket proxying issues.. I've run into these problems in
some places!


On 23 August 2014 10:24, Ramalingam Saravanan <rsarava at gmail.com> wrote:

> Hi S?bastien,
>
> The multi-user IPython server would be the best option for your needs, but
> here are some alternatives:
>
>     https://github.com/ptone/jiffylab
>
>     https://cloud.sagemath.com
>
>     https://github.com/mitotic/graphterm (my own project; somewhat
> experimental)
>
> Here's a paper from SciPy14 proceedings that discusses these alternatives:
>
>
> https://dl.dropboxusercontent.com/u/72208800/docs/Saravanan-SciPy14.pdf
>
> Best,
> Saravanan
>
>
> On Thu, Aug 21, 2014 at 4:55 PM, S?bastien Hinderer <
> Sebastien.Hinderer at inria.fr> wrote:
>
> > Dear all,
> >
> > My team currently studies the possibility to use IPython notebooks in a
> > Python Mooc, more precisely for doing exercises. Teachers would prepare
> > the questions in a notebook and students could answer the questions
> > directly in the notebook itself. Actually, we would provide one copy of
> > the original teacher's notebook for each student, so that all of them
> have
> > their own notebook on which they can work independently.
> >
> > What I'm trying to study here is the server's security. How to prevent
> > students from both doing malicious things on the server and accessing /
> > modifying the notebooks of other students.
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140823/1ce4092f/attachment.html>

From cyrille.rossant at gmail.com  Sat Aug 23 12:50:20 2014
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Sat, 23 Aug 2014 18:50:20 +0200
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
Message-ID: <CA+-1RQRoZrO+r1BF3amKisJE23PAjtcY1C0giYq1g1+W4E4+YA@mail.gmail.com>

Adam,

FYI, you'll find my EuroSciPy tutorial on the widgets here
<https://github.com/rossant/euroscipy2014>

Best,
Cyrille

2014-08-20 10:15 GMT+02:00 Cyrille Rossant <cyrille.rossant at gmail.com>:
> Hi Adam,
>
> I'll do a tutorial about IPython widgets at EuroSciPy in a week [1].
> I'm notably planning to demonstrate how to build, in the notebook, a
> GUI like the one you showed. I'll put here the link to the GitHub repo
> when everything's ready.
>
> Best,
> Cyrille
>
> [1] https://www.euroscipy.org/2014/schedule/presentation/60/
>
> 2014-08-17 19:54 GMT+02:00 hugadams <hughesadam87 at gmail.com>:
>> (Please disregard if doublepost; having some subscription issues)
>>
>> Hello,
>>
>> We are developing a spectroscopy library and want to put in some interactive
>> plots/utilities to make the software a bit more approachable to novices.
>> I'd really like to build the entire interactive plotting interface through
>> the notebook's new widget system.  My reasoning is as follows:
>>
>>    - All static plots are already matplotlib-based, so we wouldn't have to
>> write a seperate plotting api
>>    - Notebooks are inherently browser-ready, so sharing plots would be great
>>          - Is it likely that static nbviewer will eventually support
>> widgets?  Since the data is required, I figured not...
>>    - Our tutorials are already in notebooks, so we'd like to be consistent.
>>
>> I've watched all the videos I can find from the widgets, and think I'm fine
>> with the Python end; however, am finding it hard to come by lots of
>> information on controlling the layout of the widgets.  I know it's mostly
>> done in javascript, and I think the most complex view we will have will look
>> something like this:
>>
>> https://sites.google.com/site/shigemorita/home/2dshige
>>
>> How much time would you estimate a javascript novice ought to spend on
>> getting something like that in the notebook?  Can anyone share tutorials for
>> the layout portion of the widget API?  Any information is helpful.  Lastly,
>> does anyone have any hacky ideas of how to bury the source code so that the
>> widgets merely display, rather than the user copy/pasting the widget source
>> into a cell?
>>
>> Thanks


From doug.blank at gmail.com  Sat Aug 23 13:02:01 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Sat, 23 Aug 2014 13:02:01 -0400
Subject: [IPython-dev] Drag image onto notebook
Message-ID: <CAAusYCiBufRQfFgFWwbcns1YPPtFmz-+JQeEvbdYZko4ufh_QA@mail.gmail.com>

Devs,

It used to be the case (early IPython 2, I believe) that the attached code
would allow an image to be embedded into a notebook by dragging and
dropping it onto the notebook.

However, now there is a message in the browser console:

HTML Sanitizer img.src removed -> Object

Is there a way to allow this to work?

Also, is there another option to allow local images to be embedded in a
notebook (and won't vanish if one re-executes a cell)?

Thanks for advice,

-Doug
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140823/3b8f2eea/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: drag-and-drop.js
Type: application/x-javascript
Size: 5833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140823/3b8f2eea/attachment.bin>

From doug.blank at gmail.com  Sat Aug 23 13:26:01 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Sat, 23 Aug 2014 13:26:01 -0400
Subject: [IPython-dev] IPython3/Jupyter Magics for Python-based kernels
In-Reply-To: <CAHNn8BVg69vG9CxwiPHPduUqZY3Zzwwarx7c5eLwyO8=YyWBVw@mail.gmail.com>
References: <CAAusYCj+L81zm=vLay_VCgBLr6SGeNjYwzWKQVrssQBUbB2AaA@mail.gmail.com>
	<CAHNn8BVg69vG9CxwiPHPduUqZY3Zzwwarx7c5eLwyO8=YyWBVw@mail.gmail.com>
Message-ID: <CAAusYCg7ODU9e=OqXxeKje68SFcB5GvqvjW-8x4=60BZMF8RUA@mail.gmail.com>

On Sat, Aug 23, 2014 at 1:12 AM, MinRK <benjaminrk at gmail.com> wrote:

> I think if you want to add something like magics for Python-wrapped
> kernels based on your magickernel.py code, that would be interesting.
> Standardizing magics across kernel languages is something that is
> attractive on its face, but we deemed outside the scope of the message /
> kernel specification, because it is probably not feasible to do in a
> language agnostic way (For example, IPython magics can contain Python
> string-formatting code with Python expressions that are evaluated in the
> interactive namespace). Clarification of discussions like this are part of
> the reasoning behind the Jupyter/IPython split:
>
>    - Jupyter is language-agnostic computing tools, such as the message
>    protocol, notebook document format, and notebook UI.
>    - IPython is a better interactive Python environment, with some
>    extensions to the Python language for interactive use.
>    - IPython provides a Jupyter Kernel.
>    - Magics are 100% an IPython feature, and not in any way related to
>    Jupyter.
>    - Other kernels are welcome to implement extensions to their language
>    (including Python!) as they see fit,
>    as similar or different to IPython?s magics system as makes sense.
>
> Implementing magics for basic Python-wrapped kernels (i.e. not R, Julia,
> Ruby, Haskell, etc.) would be simple, and potentially useful, depending on
> how many of these kernels are developed. I do not expect most first-class
> kernels to go this route, in which case they would not share any magic
> code. Further, I don?t think the magic code developed for this use case
> would even be shared by IPython.
>
> But it would still be an interesting exercise, and might encourage other
> kernels to take the multi-interpreter approach if it is successful.
>
I see your point, but from a general Jupyter user's perspective, IF a
kernel supports a magic-like metacommand, THEN it would be nice to be
uniform in syntax, and behavior.

Imagine that 5 different kernels implement a magic like "%%time" with 5
different syntaxes, and 5 slightly different results.

I am not advocating that the IPython team need to do anything. I am just
suggesting that where possible, it would be great from the user's
perspective if there was some standardization across these metacommands.

I am finding that there are many "magics" (I know that I'm defining that
word differently than IPython uses it) that can be implemented in a manner
that doesn't affect (nor interact with) the underlying language. In
MagicKernel, these commands act as pre and post processors, but are not
"in" the language as they are in IPython's Python.

Sounds like a fun experiment!
>
Well, it is very useful to have the set that are independent of the
language, and we are implementing them. If they are useful for other
Python-based kernels, then great! All of the following kernels in pip now
have these standard magics: eval_kernel (simple Python evaluator),
echo_kernel (from docs), chatbot_kernel, and scheme_kernel.

-Doug


> -MinRK
> ?
>
>
> On Wed, Aug 20, 2014 at 4:22 AM, Doug Blank <doug.blank at gmail.com> wrote:
>
>> IPython developers,
>>
>> One of the nice things that the IPython team did for the next version was
>> to create a base class for creating new kernels for code written in Python,
>> called wrapper kernels [1].
>>
>> Unfortunately, such kernels don't have ! shell commands, nor magics. This
>> is understandable, as those are tightly integrated with IPython's Python.
>> However, one could get some of the basic functionality of these features on
>> top of the wrapper kernel, in a generic way. For example, you could run
>> shell commands, or have some magics that operate independent of the kernel.
>>
>> We have created a prototype of magics for wrapper kernels [2]. The code
>> is in magickernel.py, and the magics are in the folder magics. The current
>> prototype works like this:
>>
>> ```
>> In [1]: %magics
>> Line magics:
>>      %cd PATH - change current directory of session
>>      %connect_info - show connection information
>>      %download URL [FILENAME] - download file from URL
>>      %html CODE - display code as HTML
>>      %install_magic URL - download and install magic from URL
>>      %javascript CODE - send code as JavaScript
>>      %latex TEXT - display text as LaTex
>>      %magics - show installed magics
>>      %reload_magics - reload the magics from the installed files
>>      %shell COMMAND - run the line as a shell command
>>      %time COMMAND - show time to run line
>>
>> Cell magics:
>>     %%file FILENAME - write contents of cell to file
>>     %%html - display contents of cell as HTML
>>     %%javascript - send contents of cell as JavaScript
>>     %%latex - display contents of cell as LaTeX
>>     %%shell - run the contents of the cell as shell commands
>>     %%time - show time to run cell
>>
>> Shell shortcut:
>>     ! COMMAND ... - execute command in shell
>>
>> Any cell magic can be made persistent for rest of session by using %%%
>> prefix.
>> ```
>>
>> The idea is to be able to easily add some standard-like magics to
>> 3rd-party kernels by using MagicKernel. See the sample wrapperkernel
>> EchoKernel with magics [3]. Also, the idea here is not to replicate too
>> much what comes with IPython's Python, but to provide a small, simple core
>> set of magics that can't be easily done in the kernel language. You can
>> install MagicKernel from source, or using:
>>
>> $ pip install calico --upgrade
>>
>> This is a prototype (with some bugs), and we very much hope for others to
>> provide feedback or code, and more work needs to be done. What magics would
>> you need in your 3rd-party kernel, that can be accomplished this way? Will
>> this design be sufficient?
>>
>> We hope that this is useful!
>>
>> -Doug
>>
>> [1] - http://ipython.org/ipython-doc/dev/development/wrapperkernels.html
>> [2] - https://bitbucket.org/ipre/calico/src/master/src/calico/
>> [3] -
>> https://bitbucket.org/ipre/calico/src/master/src/echo_kernel/echo_kernel.py
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140823/03ce2834/attachment.html>

From nick.bollweg at gmail.com  Sat Aug 23 13:52:55 2014
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Sat, 23 Aug 2014 13:52:55 -0400
Subject: [IPython-dev] eco: a composed language editor
Message-ID: <CACejjWzk9z64EidqFUmbeK40Fq_j4aGMe-XLdwLjuMq70Wu2yA@mail.gmail.com>

More of a thought exercise than a discussion of actual features: i recently
read about this editor for composed languages, eco:

   - site: http://soft-dev.org/src/eco/
   - paper:
   http://tratt.net/laurie/blog/entries/an_editor_for_composed_programs
   - repo: https://bitbucket.org/softdevteam/eco/

Just from their examples, I now have this picture in my mind of a
codemirror-based, eco-like editor in a cell that lets me prototype an
interactive GUI driven by a couple SPARQL queries with python interpolated
inside the querie, without any templating stuff in sight. I can mix in
Widgets wherever I like. Hell, throw in some coffeescript and LESS.

At first glance, it's a 90's PHP nightmare, but then I remember that I can
immediately tell if all of the layers are syntactically correct, since each
layer was interactively built against a grammar, augmented by what is known
about the current running environment.

Some things I thought about:

   - what is a composed kernel(s)?
      - they don't really cover how one might actually *execute *one of
      these files. for their example with HTML, Python, SQL, what's really
      happening? in my mind, this is really just a python kernel that
wraps HTML
      and SQL as strings, but a more involved example?
   - how would parser-driven editing in context work?
   - the completion available in a running kernel is already far beyond
      what a  grammar-only completion tool can do. what would a more
aggressively
      parser-driven editing experience be like?
      - how would one store a tree of tokens?
   - presumably, storing serialized JSON in cells[].source, while ugly,
      would work. or metadata.
      - what are visual tokens
   <http://soft-dev.org/pubs/html/diekmann_tratt__eco_a_language_composition_editor/#x1-220009.2>
   like?
   - actually, in line with the jupyter iconography (i.e. galileo's moon),
      the inlining of visual tokens would be incredible. codemirror doesn't
      appear to support flowed widgets (yet?). their suggested "real" use, of a
      formula editor, is interesting. And, of course, matrices.
      - eco native format is gzipped JSON
      - i think this has been discussed before (ipynbz). i don't know what
      advantage this provides over just having the json out there, but i have
      created some pretty big notebooks before. also, notebook files are not
      really hand editable (though I have done so).

Interested in what other folks think...

Cheers!

Nick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140823/1efce827/attachment.html>

From hughesadam87 at gmail.com  Sat Aug 23 13:49:59 2014
From: hughesadam87 at gmail.com (hugadams)
Date: Sat, 23 Aug 2014 10:49:59 -0700 (PDT)
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CA+-1RQRoZrO+r1BF3amKisJE23PAjtcY1C0giYq1g1+W4E4+YA@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CA+-1RQRoZrO+r1BF3amKisJE23PAjtcY1C0giYq1g1+W4E4+YA@mail.gmail.com>
Message-ID: <CAMHV+dDZ9Q-Ljd3NujK77CfO7+Nk54SDsERhj5LghX5_5hrhzA@mail.gmail.com>

Wow, thanks for sharing.  This is going to be an excellent resource!

Since this thread probably has the most widget experts I'm ever going to
find, may I ask a followup that I was chewing over on a plane ride
yesterday?

For our package, I think it would be beneficial to make a series of useful
containers of various widgets for a particular task.  For example, a
"filtering" container would have a series of widgets related to filtering
data.  Maybe a dropdown widget with "fourier, hamming, wavelet" and other
related things like a "transform" and "back transform button".  Likewise,
maybe a 1-d plotting widget would store a plot, have a save button,
publicize some common plotting options like "color" and so forth.  I'd like
to build these, and then assemble them into more complex GUI's, where the
main gui would define all of the events to link up all of these sub-UI's,
or conatiners as I'm calling them for lack of a better word.

What's unclear to me is if a complex event system like this is feasible in
the current design of the widget API.  Let's say I have two sub-UI's for
Plotting and Filtering.  Most likely, they'll be in some sort of class, so
pretend I could instantiate as:

plot_ui = Plotting()
filtering_ui = Filtering()

Imagine a simple event where the user changes an attribute through a widget
in the filtering UI and this is to trigger a redraw of the plot.  In my
main program, I'd like to link these up.  Is this type of functionality
supported?  Something like:

@on_trait_change(filtering_ui.windowtype):
    if filtering_ui.windowtype == 'hamming':
        plot_ui.redraw(scipy.filter.hamming(data)
   etc...




On Sat, Aug 23, 2014 at 12:45 PM, rossant [via Python] <
ml-node+s6n5069009h41 at n6.nabble.com> wrote:

> Adam,
>
> FYI, you'll find my EuroSciPy tutorial on the widgets here
> <https://github.com/rossant/euroscipy2014>
>
> Best,
> Cyrille
>
> 2014-08-20 10:15 GMT+02:00 Cyrille Rossant <[hidden email]
> <http://user/SendEmail.jtp?type=node&node=5069009&i=0>>:
>
> > Hi Adam,
> >
> > I'll do a tutorial about IPython widgets at EuroSciPy in a week [1].
> > I'm notably planning to demonstrate how to build, in the notebook, a
> > GUI like the one you showed. I'll put here the link to the GitHub repo
> > when everything's ready.
> >
> > Best,
> > Cyrille
> >
> > [1] https://www.euroscipy.org/2014/schedule/presentation/60/
> >
> > 2014-08-17 19:54 GMT+02:00 hugadams <[hidden email]
> <http://user/SendEmail.jtp?type=node&node=5069009&i=1>>:
> >> (Please disregard if doublepost; having some subscription issues)
> >>
> >> Hello,
> >>
> >> We are developing a spectroscopy library and want to put in some
> interactive
> >> plots/utilities to make the software a bit more approachable to
> novices.
> >> I'd really like to build the entire interactive plotting interface
> through
> >> the notebook's new widget system.  My reasoning is as follows:
> >>
> >>    - All static plots are already matplotlib-based, so we wouldn't have
> to
> >> write a seperate plotting api
> >>    - Notebooks are inherently browser-ready, so sharing plots would be
> great
> >>          - Is it likely that static nbviewer will eventually support
> >> widgets?  Since the data is required, I figured not...
> >>    - Our tutorials are already in notebooks, so we'd like to be
> consistent.
> >>
> >> I've watched all the videos I can find from the widgets, and think I'm
> fine
> >> with the Python end; however, am finding it hard to come by lots of
> >> information on controlling the layout of the widgets.  I know it's
> mostly
> >> done in javascript, and I think the most complex view we will have will
> look
> >> something like this:
> >>
> >> https://sites.google.com/site/shigemorita/home/2dshige
> >>
> >> How much time would you estimate a javascript novice ought to spend on
> >> getting something like that in the notebook?  Can anyone share
> tutorials for
> >> the layout portion of the widget API?  Any information is helpful.
>  Lastly,
> >> does anyone have any hacky ideas of how to bury the source code so that
> the
> >> widgets merely display, rather than the user copy/pasting the widget
> source
> >> into a cell?
> >>
> >> Thanks
> _______________________________________________
> IPython-dev mailing list
> [hidden email] <http://user/SendEmail.jtp?type=node&node=5069009&i=2>
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5069009.html
>  To unsubscribe from Feasilibity of widgets/layout for mimicking this, click
> here
> <http://python.6.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5068197&code=aHVnaGVzYWRhbTg3QGdtYWlsLmNvbXw1MDY4MTk3fC03NjYyMTgyMzk=>
> .
> NAML
> <http://python.6.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5069013.html
Sent from the IPython - Development mailing list archive at Nabble.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140823/bf0300ac/attachment.html>

From python at elbonia.de  Sat Aug 23 14:06:42 2014
From: python at elbonia.de (Juergen Hasch)
Date: Sat, 23 Aug 2014 20:06:42 +0200
Subject: [IPython-dev] Drag image onto notebook
In-Reply-To: <CAAusYCiBufRQfFgFWwbcns1YPPtFmz-+JQeEvbdYZko4ufh_QA@mail.gmail.com>
References: <CAAusYCiBufRQfFgFWwbcns1YPPtFmz-+JQeEvbdYZko4ufh_QA@mail.gmail.com>
Message-ID: <53F8D832.6090906@elbonia.de>

This is not possible due to the HTML sanitizer, as you noticed. Google-caja sanitizes base64 uris.
There is an updated extension on https://github.com/ipython-contrib/IPython-notebook-extensions using the new 3.x
IPython contents webservice. Dragging an image into a notebook now uploads it to the server.


Am 23.08.2014 19:02, schrieb Doug Blank:
> Devs,
> 
> It used to be the case (early IPython 2, I believe) that the attached code would allow an image to be embedded into a
> notebook by dragging and dropping it onto the notebook.
> 
> However, now there is a message in the browser console:
> 
> HTML Sanitizer img.src removed -> Object
> 
> Is there a way to allow this to work?
> 
> Also, is there another option to allow local images to be embedded in a notebook (and won't vanish if one re-executes a
> cell)?
> 
> Thanks for advice,
> 
> -Doug
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 



From jason-sage at creativetrax.com  Sat Aug 23 15:33:46 2014
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 23 Aug 2014 15:33:46 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CAMHV+dDZ9Q-Ljd3NujK77CfO7+Nk54SDsERhj5LghX5_5hrhzA@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>	<CA+-1RQRoZrO+r1BF3amKisJE23PAjtcY1C0giYq1g1+W4E4+YA@mail.gmail.com>
	<CAMHV+dDZ9Q-Ljd3NujK77CfO7+Nk54SDsERhj5LghX5_5hrhzA@mail.gmail.com>
Message-ID: <53F8EC9A.7090003@creativetrax.com>

On 8/23/14, 13:49, hugadams wrote:

> What's unclear to me is if a complex event system like this is feasible
> in the current design of the widget API.

Yes.  Resoundingly yes. Someone (you?) should go for it---it would be 
pretty straightforward with the current design, and would be a great 
thing to see.

Thanks,

Jason



From nick.bollweg at gmail.com  Sat Aug 23 15:50:25 2014
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Sat, 23 Aug 2014 15:50:25 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CAMHV+dDZ9Q-Ljd3NujK77CfO7+Nk54SDsERhj5LghX5_5hrhzA@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CA+-1RQRoZrO+r1BF3amKisJE23PAjtcY1C0giYq1g1+W4E4+YA@mail.gmail.com>
	<CAMHV+dDZ9Q-Ljd3NujK77CfO7+Nk54SDsERhj5LghX5_5hrhzA@mail.gmail.com>
Message-ID: <CACejjWw8E4diExgrww1S1-FmJh-bqMtQ=U60knA-LLrR08O8rw@mail.gmail.com>

One way to do this would be to instead of requiring the notebook user to
understand everything, expose important bits as traitlets, and then
change/traitlet.link all of them together. In your example, if:

   - Plotting knows to redraw itself when its data changes
   - Filtering , updates its windowtype, to be, instead of "hamming",
   scipy.filter.hamming

then suddenly your code is now:

> def change_windowtype(name, old_value, new_value):
>     plot_ui.data = filtering_ui.windowtype(data)
> filtering_ui.on_trait_change("windowtype", change_windowtype)
>
(I don't think your usage of on_trait_changed as a decorator is legit, but
that would be awesome!)

That moves a lot of the event stuff into the internal structure of each of
your GUI components. This pattern would also be much easier to wrap up
later into a "higher order" GUI. Then, if you want to "hoist" a property,
you would just use traitlet.link:

> class GUI(widgets.ContainerWidget)
>     windowtype = traitlets.Any(sync=True)
>     plot_data = traitlets.Any(sync=True)
>     data = traitlets.Any(sync=True)
>
>     def __init__(self, **kwargs)
>         super(GUI, self).__init__(**kwargs)
>
>         self.filtering_ui = Filtering()
>         self.plot_ui = Plotting()
>
>         self.children = [self.filtering_ui, self.plot_ui]
>
>         traitlets.link(self, "plot_data"), (self.plot_ui, "data"),
>         traitlets.link(self, "windowtype"), (self.filtering_ui,
> "windowtype"),
>
>     def _windowtype_changed(self, name, old_value, new_value):
>         self.plot_data = self.windowtype(self.data)
>
And now the notebook user would still be able to inspect and change the
important parts of even deeply nested settings with simple assignment.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140823/1ebcdc1f/attachment.html>

From paddy at paddymullen.com  Mon Aug 25 08:45:42 2014
From: paddy at paddymullen.com (Paddy Mullen)
Date: Mon, 25 Aug 2014 08:45:42 -0400
Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter
In-Reply-To: <CAH2VmmA+Z--q3HS7QdJWPN8As-pjq2PHLQ+NQ5-z9D9tzw8GRA@mail.gmail.com>
References: <CAH2VmmA0Tv-1TPjdPN7Dkjw4q+4ZMde_ZFS-BhZx6J+tPWp2YQ@mail.gmail.com>
	<CAHAreOro4BTe_GjLKafKA8yT5XcNEHPnROi2UXVZO=GqZS_VVQ@mail.gmail.com>
	<CAH2VmmA+Z--q3HS7QdJWPN8As-pjq2PHLQ+NQ5-z9D9tzw8GRA@mail.gmail.com>
Message-ID: <CAFpY3ufd+cvMTs5k67HamRrW+_xO-niE7yT0Ua+jRZozc1kqNg@mail.gmail.com>

Would it make sense to give notebooks a preamble cell where the notebook
could check that all of its required dependencies are installed.  I guess
this could be done by convention.  Notebooks could also include a
dependency list in requirements.txt or conda.yaml or a docker file, but the
installation of those venvs is left as an implementation detail.


On Fri, Aug 22, 2014 at 2:40 PM, Filipe Pires Alvarenga Fernandes <
ocefpaf at gmail.com> wrote:

> On Fri, Aug 22, 2014 at 2:34 PM, Fernando Perez <fperez.net at gmail.com>
> wrote:
>
>> We looked into it and as far as we can tell, all the pieces we're
>> providing for the kernel spec should make it possible to have your kernel
>> entry point do the switching.
>>
>
> OK, I will keep trying to have something more robust.  Right now I am
> doing a very hackish workaround.
>
>
>> But we will NOT bake into the kernel spec direct knowledge of venv, conda
>> envs, or any other 'enviornment manager'. We want to make it *possible* do
>> declare kernels that are connected to environments, but as of yet, there
>> are just too many ways of 'managing environments' across languages and
>> toolchains for us to enshrine any particular approach into the spec itself.
>>
>
> I understand (and I agree) with that.
>
>
>> As I say above, we looked into it, but we didn't actually implement
>> anything yet. So we'd encourage you to try things out, and report both with
>> what works (I'm sure others are interested) and what doesn't (so we can
>> help out or fix things if needed)
>>
>
> As soon as I have something better I will post it here.
>
>
>> Cheers
>>
>> f
>>
>
> Thanks,
>
> -Filipe
>
>
>>
>>
>> On Fri, Aug 22, 2014 at 8:55 AM, Filipe Pires Alvarenga Fernandes <
>> ocefpaf at gmail.com> wrote:
>>
>>> Have anyone tried to use venvs as kernels?  Making the installed vens
>>> show up in the drop-down menu in the new Jupyter Notebook?
>>>
>>> I was toying with the idea but got nowhere near something useful.
>>>
>>> I think it would be a nice way to switch between virtual environments.
>>>
>>> -Filipe
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>>
>> --
>> Fernando Perez (@fperez_org; http://fperez.org)
>> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>> fernando.perez-at-berkeley: contact me here for any direct mail
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140825/9dc6228d/attachment.html>

From bussonniermatthias at gmail.com  Mon Aug 25 08:55:59 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Mon, 25 Aug 2014 14:55:59 +0200
Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter
In-Reply-To: <CAFpY3ufd+cvMTs5k67HamRrW+_xO-niE7yT0Ua+jRZozc1kqNg@mail.gmail.com>
References: <CAH2VmmA0Tv-1TPjdPN7Dkjw4q+4ZMde_ZFS-BhZx6J+tPWp2YQ@mail.gmail.com>
	<CAHAreOro4BTe_GjLKafKA8yT5XcNEHPnROi2UXVZO=GqZS_VVQ@mail.gmail.com>
	<CAH2VmmA+Z--q3HS7QdJWPN8As-pjq2PHLQ+NQ5-z9D9tzw8GRA@mail.gmail.com>
	<CAFpY3ufd+cvMTs5k67HamRrW+_xO-niE7yT0Ua+jRZozc1kqNg@mail.gmail.com>
Message-ID: <8B05D1E8-DB46-423A-A486-911C2AF105F7@gmail.com>


Le 25 ao?t 2014 ? 14:45, Paddy Mullen a ?crit :

> Would it make sense to give notebooks a preamble cell where the notebook could check that all of its required dependencies are installed.  I guess this could be done by convention.  Notebooks could also include a dependency list in requirements.txt or conda.yaml or a docker file, but the installation of those venvs is left as an implementation detail.  

This as already been discusses here and there on the mailing list,
and we had decided that this is outside of IPython scope. 

Konrad Hinsen have done Active Paper which is meant to improve things like that. 

Everything is an implementation detail !
-- 
M




From benjaminrk at gmail.com  Mon Aug 25 12:40:36 2014
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 25 Aug 2014 09:40:36 -0700
Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter
In-Reply-To: <8B05D1E8-DB46-423A-A486-911C2AF105F7@gmail.com>
References: <CAH2VmmA0Tv-1TPjdPN7Dkjw4q+4ZMde_ZFS-BhZx6J+tPWp2YQ@mail.gmail.com>
	<CAHAreOro4BTe_GjLKafKA8yT5XcNEHPnROi2UXVZO=GqZS_VVQ@mail.gmail.com>
	<CAH2VmmA+Z--q3HS7QdJWPN8As-pjq2PHLQ+NQ5-z9D9tzw8GRA@mail.gmail.com>
	<CAFpY3ufd+cvMTs5k67HamRrW+_xO-niE7yT0Ua+jRZozc1kqNg@mail.gmail.com>
	<8B05D1E8-DB46-423A-A486-911C2AF105F7@gmail.com>
Message-ID: <CAHNn8BU934Q0cLpQxHRjvH=4BCDKuv-r1AKg021OcCmmTMqYdw@mail.gmail.com>

Aron Ahmadia has an extension (thisworks
<https://github.com/ahmadia/thisworks>) that ensures dependencies are
installed, and if not, creates a new hashdist and relaunches a notebook
server in the environment with all of the dependencies installed.
?


On Mon, Aug 25, 2014 at 5:55 AM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

>
> Le 25 ao?t 2014 ? 14:45, Paddy Mullen a ?crit :
>
> > Would it make sense to give notebooks a preamble cell where the notebook
> could check that all of its required dependencies are installed.  I guess
> this could be done by convention.  Notebooks could also include a
> dependency list in requirements.txt or conda.yaml or a docker file, but the
> installation of those venvs is left as an implementation detail.
>
> This as already been discusses here and there on the mailing list,
> and we had decided that this is outside of IPython scope.
>
> Konrad Hinsen have done Active Paper which is meant to improve things like
> that.
>
> Everything is an implementation detail !
> --
> M
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140825/6515c554/attachment.html>

From takowl at gmail.com  Tue Aug 26 04:53:45 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 26 Aug 2014 09:53:45 +0100
Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython
 Notebook on your own machine
In-Reply-To: <CAN4eD9DR0bJFihqA=nZADtNScXFjMejUha647+4K8LqmJ=V0nw@mail.gmail.com>
References: <CAOvn4qjaZq5w-Y1hV5tJchWTNURcmzBf=V458-meR-fMSyAmkw@mail.gmail.com>
	<CAN4eD9AwsD3nW+PWCVSqZ65BsqcQ+Hc6H+_ZeoSvXx8+JGx8zw@mail.gmail.com>
	<CAOvn4qhN8DBS+ADwxjbsik=UqdubPdorxcvULZsAKs5MWnr-hw@mail.gmail.com>
	<CAN4eD9DR0bJFihqA=nZADtNScXFjMejUha647+4K8LqmJ=V0nw@mail.gmail.com>
Message-ID: <CAOvn4qhxAYQfm6StBcS7p1gNKL4JEfOTeK-3vYRtbJ80VFvUtg@mail.gmail.com>

On 22 August 2014 07:48, Joshua Barratt <jbarratt at serialized.net> wrote:

> It *almost* works on IPython 2.x, except the nbserver-* JSON blobs only
> have the PID's in the name, not the actual file.
>
> I took a pass at fixing this up for IPython 2.x, by borrowing some of the
> code from the master branch, and it seems to be working.


Great! I was just being lazy with my own implementation, because I live on
the development version anyway.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140826/b8a2a784/attachment.html>

From ellisonbg at gmail.com  Tue Aug 26 10:49:05 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 26 Aug 2014 10:49:05 -0400
Subject: [IPython-dev] Listing books on the home page,
	a policy proposal. Feedback?
In-Reply-To: <CAHAreOpJYoCFs-VEx7mdoOPryAt9im17=P_hrqDbbxzhxjK1zA@mail.gmail.com>
References: <CAHAreOpJYoCFs-VEx7mdoOPryAt9im17=P_hrqDbbxzhxjK1zA@mail.gmail.com>
Message-ID: <CAH4pYpTfi+fu2JtB_nn13-HDj=AS5OVNejo=UAefQNDaa49OFg@mail.gmail.com>

+1

On Thu, Aug 21, 2014 at 4:35 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi folks,
>
> we just went over this on the dev meeting, but I'd like to give it a day or
> two here on-list for feedback in case anyone who wasn't at the meeting has
> further comments...
>
> For a while we've had on the homepage a link to Cyrille's IPython book, and
> we've been contacted by a publisher of an upcoming title asking for a new
> listing there. So I'd like to have uniform criteria that we use to make that
> decision in a fair manner with *any* publisher.  Here's what we discussed
> during the meeting, with some additional refinements I just thought of to
> make things clearer.
>
> One quick note: Brian noted that the website is way too busy already, which
> is true. But I want to decouple this specific decision about books from an
> entire website redesign conversation. I think that books are one of the
> things that actually should belong on the main homepage, as they are a very
> important resource related to the project and that benefits it in multiple
> ways.  So let's assume that any redesigned home page will have *some* space
> for a small image pointing to a book, and the discussion here is about what
> to put there and how to provide a policy publishers can all know about.
>
> Any feedback welcome.
>
> Cheers
>
> f
>
> #### Proposed language
>
> Our policy for listing books on the website is as follows. This will be
> uniformly applied across publishers, so the project can maintain its
> neutrality regarding IPython-related books (all language below would apply
> in the future to Jupyter-oriented books).
>
> 1. Criteria for inclusion in the homepage
>
> We will list books that are specifically and mostly about IPython itself,
> rather than just mentioning IPython in a chapter or two (though see Note in
> #2 below).
>
> For example, the currently listed book by Cyrille Rossant amply qualifies,
> but Wes McKinney's "Python for Data Analysis", while having an excellent
> chapter devoted to IPython, would not get a link on the home page.
>
>
> 2. Our "Books" page:
>
> We will create a main page listing in reverse chronological order the
> various books that are mainly devoted to IPython. For each book, we simply
> list its title, author and actual publisher page (or Amazon, whatever the
> author prefers), along with a cover image.  No editorializing/reviews/etc.
>
> There may be a section at the bottom listing books that include some useful
> material about IPython (more than just a sentence but less than say 1/2 the
> book).  These would be listed for reference but would have no front-page
> link (e.g. Wes' book).
>
> Note: the core team reserves the right to *not* include a book in this
> listing. If someone out there publishes a book that we deem to be absolutely
> terrible, we reserve the right to simply not list it.
>
>
> 3. What we show on the homepage
>
> We will have a single image for books, and that will be a randomly rotating
> screenshot of all currently available books.  That image will be a link to
> that book's entry in the "Books" page.
>
> Since now we'll only have at most two of these (the current one and
> Cyrille's upcoming one), we can make the distribution give say 2/3 of views
> to the new book and 1/3 to the old. If eventually there's multiple books, we
> can figure out a *simple* distribution to highlight newer ones more often
> while still showing the older ones (as long as they are in print).
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com


From david.trem at gmail.com  Tue Aug 26 15:15:23 2014
From: david.trem at gmail.com (=?ISO-8859-1?Q?Tr=E9mouilles_David?=)
Date: Tue, 26 Aug 2014 21:15:23 +0200
Subject: [IPython-dev] ipython notebook "current cell changed signal"
Message-ID: <53FCDCCB.3090208@gmail.com>

Hello,

In the IPython notebook, I would like to trigger a javascript
function when the notebook currently selected cell changes.

I have found this kind of solution:

%%javascript
$([IPython.events]).on('select.Cell', function (event, data) {
            console.log("changed...")
         });

However it only works when the cell is changed by clicking on
another one but not when navigating with the up and down arrow key...
Is there any simple solution to trigger a javascript function
whenever the selected cell changes ?

(I'm using the current ipython github repo head.)

Thanks in advance,

David


From tritemio at gmail.com  Tue Aug 26 19:29:22 2014
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Tue, 26 Aug 2014 16:29:22 -0700
Subject: [IPython-dev] ipython notebook widgets: combobox?
Message-ID: <CANn2QUw+chUAE+VzEuz6KLeNfPCbwxaMgJ7c8QrV48Nam8B2fQ@mail.gmail.com>

Hi to the list,

I'm playing with the widgets and I can't find any example of a combo-box
widget. I need a numeric text entry (like BoundedIntTextWidget) with two
small buttons to increment or decrement the value. All the GUI toolkits
have something similar.

If this widget does not yet exist, would it be possible to bind two buttons
(Up, Down) to a BoundedIntTextWidget in order to achieve roughly the same
result?

Thanks,
Antonio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140826/b1d5aae9/attachment.html>

From cyrille.rossant at gmail.com  Tue Aug 26 19:46:04 2014
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Wed, 27 Aug 2014 00:46:04 +0100
Subject: [IPython-dev] ipython notebook widgets: combobox?
In-Reply-To: <CANn2QUw+chUAE+VzEuz6KLeNfPCbwxaMgJ7c8QrV48Nam8B2fQ@mail.gmail.com>
References: <CANn2QUw+chUAE+VzEuz6KLeNfPCbwxaMgJ7c8QrV48Nam8B2fQ@mail.gmail.com>
Message-ID: <CA+-1RQS7BnTKikqN9g2giWH1FkkY=JemiR6M8sded38dLzJKOg@mail.gmail.com>

Hi
See example 4 here: https://github.com/rossant/euroscipy2014
Cyrille

Le mercredi 27 ao?t 2014, Antonino Ingargiola <tritemio at gmail.com> a ?crit :

> Hi to the list,
>
> I'm playing with the widgets and I can't find any example of a combo-box
> widget. I need a numeric text entry (like BoundedIntTextWidget) with two
> small buttons to increment or decrement the value. All the GUI toolkits
> have something similar.
>
> If this widget does not yet exist, would it be possible to bind two
> buttons (Up, Down) to a BoundedIntTextWidget in order to achieve roughly
> the same result?
>
> Thanks,
> Antonio
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/d680461f/attachment.html>

From david.trem at gmail.com  Wed Aug 27 05:17:37 2014
From: david.trem at gmail.com (=?ISO-8859-1?Q?Tr=E9mouilles_David?=)
Date: Wed, 27 Aug 2014 11:17:37 +0200
Subject: [IPython-dev] ipython notebook "current cell changed signal"
In-Reply-To: <53FCDCCB.3090208@gmail.com>
References: <53FCDCCB.3090208@gmail.com>
Message-ID: <53FDA231.5080607@gmail.com>

OK. I have finally figured this out by myself :-)

'selected_cell_type_changed.Notebook' is
actually what I was looking for.

I was misled by the naming. My first thought was
that the 'selected_cell_type_changed.Notebook' signal
was /only/ triggered when the selected "cell type" was
changed, actually it is triggered either when the
"selected cell" or the "selected cell type" are changed...

Best regards,

David

Le 26/08/14 21:15, Tr?mouilles David a ?crit :
> Hello,
>
> In the IPython notebook, I would like to trigger a javascript
> function when the notebook currently selected cell changes.
>
> I have found this kind of solution:
>
> %%javascript
> $([IPython.events]).on('select.Cell', function (event, data) {
>            console.log("changed...")
>         });
>
> However it only works when the cell is changed by clicking on
> another one but not when navigating with the up and down arrow key...
> Is there any simple solution to trigger a javascript function
> whenever the selected cell changes ?
>
> (I'm using the current ipython github repo head.)
>
> Thanks in advance,
>
> David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/cda405a1/attachment.html>

From takowl at gmail.com  Wed Aug 27 05:34:04 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 27 Aug 2014 10:34:04 +0100
Subject: [IPython-dev] ipython notebook "current cell changed signal"
In-Reply-To: <53FDA231.5080607@gmail.com>
References: <53FCDCCB.3090208@gmail.com> <53FDA231.5080607@gmail.com>
Message-ID: <CAOvn4qi+u6tQPQ_xZfoeSwBJ+C-HpcoZkB4s4KAM2dULF7p5UQ@mail.gmail.com>

That could be a bug - if the events don't make sense at present, be aware
that we may fix them in a future release.


On 27 August 2014 10:17, Tr?mouilles David <david.trem at gmail.com> wrote:

>  OK. I have finally figured this out by myself :-)
>
> 'selected_cell_type_changed.Notebook' is
> actually what I was looking for.
>
> I was misled by the naming. My first thought was
> that the 'selected_cell_type_changed.Notebook' signal
> was *only* triggered when the selected "cell type" was
> changed, actually it is triggered either when the
> "selected cell" or the "selected cell type" are changed...
>
> Best regards,
>
> David
>
> Le 26/08/14 21:15, Tr?mouilles David a ?crit :
>
> Hello,
>
> In the IPython notebook, I would like to trigger a javascript
> function when the notebook currently selected cell changes.
>
> I have found this kind of solution:
>
> %%javascript
> $([IPython.events]).on('select.Cell', function (event, data) {
>            console.log("changed...")
>         });
>
> However it only works when the cell is changed by clicking on
> another one but not when navigating with the up and down arrow key...
> Is there any simple solution to trigger a javascript function
> whenever the selected cell changes ?
>
> (I'm using the current ipython github repo head.)
>
> Thanks in advance,
>
> David
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/3877225d/attachment.html>

From david.trem at gmail.com  Wed Aug 27 07:04:26 2014
From: david.trem at gmail.com (=?ISO-8859-1?Q?Tr=E9mouilles_David?=)
Date: Wed, 27 Aug 2014 13:04:26 +0200
Subject: [IPython-dev] ipython notebook "current cell changed signal"
In-Reply-To: <CAOvn4qi+u6tQPQ_xZfoeSwBJ+C-HpcoZkB4s4KAM2dULF7p5UQ@mail.gmail.com>
References: <53FCDCCB.3090208@gmail.com> <53FDA231.5080607@gmail.com>
	<CAOvn4qi+u6tQPQ_xZfoeSwBJ+C-HpcoZkB4s4KAM2dULF7p5UQ@mail.gmail.com>
Message-ID: <53FDBB3A.4080005@gmail.com>

Yes indeed, this might be considered as a bug.
I'll let you (core developers) decide whether or not...

Is there any plan to propose and document
a kind of notebook javascript API ? (I did not
find such document in the IPython docs but
it could be that I was not enough persistent (?))

Anyway, thanks to all of you for IPython. It rocks!

David

Le 27/08/14 11:34, Thomas Kluyver a ?crit :
> That could be a bug - if the events don't make sense at present, be 
> aware that we may fix them in a future release.
>
>
> On 27 August 2014 10:17, Tr?mouilles David <david.trem at gmail.com 
> <mailto:david.trem at gmail.com>> wrote:
>
>     OK. I have finally figured this out by myself :-)
>
>     'selected_cell_type_changed.Notebook' is
>     actually what I was looking for.
>
>     I was misled by the naming. My first thought was
>     that the 'selected_cell_type_changed.Notebook' signal
>     was /only/ triggered when the selected "cell type" was
>     changed, actually it is triggered either when the
>     "selected cell" or the "selected cell type" are changed...
>
>     Best regards,
>
>     David
>
>     Le 26/08/14 21:15, Tr?mouilles David a ?crit :
>>     Hello,
>>
>>     In the IPython notebook, I would like to trigger a javascript
>>     function when the notebook currently selected cell changes.
>>
>>     I have found this kind of solution:
>>
>>     %%javascript
>>     $([IPython.events]).on('select.Cell', function (event, data) {
>>                console.log("changed...")
>>             });
>>
>>     However it only works when the cell is changed by clicking on
>>     another one but not when navigating with the up and down arrow
>>     key...
>>     Is there any simple solution to trigger a javascript function
>>     whenever the selected cell changes ?
>>
>>     (I'm using the current ipython github repo head.)
>>
>>     Thanks in advance,
>>
>>     David
>
>
>     _______________________________________________
>     IPython-dev mailing list
>     IPython-dev at scipy.org <mailto:IPython-dev at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/06845971/attachment.html>

From takowl at gmail.com  Wed Aug 27 07:11:57 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 27 Aug 2014 12:11:57 +0100
Subject: [IPython-dev] ipython notebook "current cell changed signal"
In-Reply-To: <53FDBB3A.4080005@gmail.com>
References: <53FCDCCB.3090208@gmail.com> <53FDA231.5080607@gmail.com>
	<CAOvn4qi+u6tQPQ_xZfoeSwBJ+C-HpcoZkB4s4KAM2dULF7p5UQ@mail.gmail.com>
	<53FDBB3A.4080005@gmail.com>
Message-ID: <CAOvn4qizEvuESC2wt5D7qUwXzV=LEyLDyo-6GQ6mQs7YgJ7f6A@mail.gmail.com>

On 27 August 2014 12:04, Tr?mouilles David <david.trem at gmail.com> wrote:

> Is there any plan to propose and document
> a kind of notebook javascript API ? (I did not
> find such document in the IPython docs but
> it could be that I was not enough persistent (?))
>

That is the plan, but there's nothing to find yet. The JS API has been
changing quite quickly, but when it's settled down enough, we do want to
define and document what people should rely on.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/0a3b3c31/attachment.html>

From bussonniermatthias at gmail.com  Wed Aug 27 07:15:37 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Wed, 27 Aug 2014 12:15:37 +0100
Subject: [IPython-dev] ipython notebook "current cell changed signal"
In-Reply-To: <CAOvn4qizEvuESC2wt5D7qUwXzV=LEyLDyo-6GQ6mQs7YgJ7f6A@mail.gmail.com>
References: <53FCDCCB.3090208@gmail.com> <53FDA231.5080607@gmail.com>
	<CAOvn4qi+u6tQPQ_xZfoeSwBJ+C-HpcoZkB4s4KAM2dULF7p5UQ@mail.gmail.com>
	<53FDBB3A.4080005@gmail.com>
	<CAOvn4qizEvuESC2wt5D7qUwXzV=LEyLDyo-6GQ6mQs7YgJ7f6A@mail.gmail.com>
Message-ID: <47404C39-C66F-4E9B-B981-A77B73D69801@gmail.com>


Le 27 ao?t 2014 ? 12:11, Thomas Kluyver <takowl at gmail.com> a ?crit :

> On 27 August 2014 12:04, Tr?mouilles David <david.trem at gmail.com> wrote:
> Is there any plan to propose and document
> a kind of notebook javascript API ? (I did not
> find such document in the IPython docs but
> it could be that I was not enough persistent (?))
> 
> That is the plan, but there's nothing to find yet. The JS API has been changing quite quickly, but when it's settled down enough, we do want to define and document what people should rely on.

And the more people we have to **maintain**[1] javascript/HTML and write doc, the sooner this will happen. 

? 
M

PS.: did that some time ago: http://ipjsdoc.herokuapp.com/
[1]: that does not mean add new features :-) 

> 
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/ec98b572/attachment.html>

From doug.blank at gmail.com  Wed Aug 27 12:24:11 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Wed, 27 Aug 2014 12:24:11 -0400
Subject: [IPython-dev] What's the easiest way to render an .ipynb file as
	html?
Message-ID: <CAAusYChKXpDajdqUxPHgxchctC=hn-v5wj2_aXPewgLHHHnaeQ@mail.gmail.com>

Short of calling nbviewer, what is the easiest way to render a local .ipynb
file as html, just like nbviewer does? Would prefer to use the IPython
libraries directly rather than externally calling ipython, but that would
work too.

-Doug
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/e0727bf2/attachment.html>

From jakevdp at cs.washington.edu  Wed Aug 27 12:54:11 2014
From: jakevdp at cs.washington.edu (Jacob Vanderplas)
Date: Wed, 27 Aug 2014 09:54:11 -0700
Subject: [IPython-dev] What's the easiest way to render an .ipynb file
	as html?
In-Reply-To: <CAAusYChKXpDajdqUxPHgxchctC=hn-v5wj2_aXPewgLHHHnaeQ@mail.gmail.com>
References: <CAAusYChKXpDajdqUxPHgxchctC=hn-v5wj2_aXPewgLHHHnaeQ@mail.gmail.com>
Message-ID: <CACpqBg0NkgwwQedFOnwXeaNy7Q-3nhtcmxqNgqkVvGK9Y+sCZA@mail.gmail.com>

Here's an example of a possible approach:

from IPython.nbformat import current as nbformat
from IPython.nbconvert.exporters import HTMLExporter

input_file = 'notebook.ipynb'
output_file = 'notebook.html'

with open(input_file) as f:
    notebook_content = f.read()

exporter = HTMLExporter(template_file='full')
nb_json = nbformat.reads_json(notebook_content)
(body, resources) = exporter.from_notebook_node(nb_json)

with open(output_file, 'w') as f:
    f.write(body)

Hope that helps!
   Jake

 Jake VanderPlas
 Director of Research ? Physical Sciences
 eScience Institute, University of Washington
 http://www.vanderplas.com


On Wed, Aug 27, 2014 at 9:24 AM, Doug Blank <doug.blank at gmail.com> wrote:

> Short of calling nbviewer, what is the easiest way to render a local
> .ipynb file as html, just like nbviewer does? Would prefer to use the
> IPython libraries directly rather than externally calling ipython, but that
> would work too.
>
> -Doug
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/967d4539/attachment.html>

From doug.blank at gmail.com  Wed Aug 27 13:25:45 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Wed, 27 Aug 2014 13:25:45 -0400
Subject: [IPython-dev] What's the easiest way to render an .ipynb file
	as html?
In-Reply-To: <CACpqBg0NkgwwQedFOnwXeaNy7Q-3nhtcmxqNgqkVvGK9Y+sCZA@mail.gmail.com>
References: <CAAusYChKXpDajdqUxPHgxchctC=hn-v5wj2_aXPewgLHHHnaeQ@mail.gmail.com>
	<CACpqBg0NkgwwQedFOnwXeaNy7Q-3nhtcmxqNgqkVvGK9Y+sCZA@mail.gmail.com>
Message-ID: <CAAusYCgZ+bBzGfoOZB1JWDAs4Xr9oGmM9-3ez0QbjhEc9eRQng@mail.gmail.com>

Great, thanks! That gets me very close to being complete... need to make
sure I get the css to load. (I'm adding a method to dynamically create a
static version of a live, shared notebook in the new multiuser server [1]).

-Doug

[1] - https://github.com/jupyter/jupyterhub


On Wed, Aug 27, 2014 at 12:54 PM, Jacob Vanderplas <
jakevdp at cs.washington.edu> wrote:

> Here's an example of a possible approach:
>
> from IPython.nbformat import current as nbformat
> from IPython.nbconvert.exporters import HTMLExporter
>
> input_file = 'notebook.ipynb'
> output_file = 'notebook.html'
>
> with open(input_file) as f:
>     notebook_content = f.read()
>
> exporter = HTMLExporter(template_file='full')
> nb_json = nbformat.reads_json(notebook_content)
> (body, resources) = exporter.from_notebook_node(nb_json)
>
> with open(output_file, 'w') as f:
>     f.write(body)
>
> Hope that helps!
>    Jake
>
>  Jake VanderPlas
>  Director of Research ? Physical Sciences
>  eScience Institute, University of Washington
>  http://www.vanderplas.com
>
>
> On Wed, Aug 27, 2014 at 9:24 AM, Doug Blank <doug.blank at gmail.com> wrote:
>
>> Short of calling nbviewer, what is the easiest way to render a local
>> .ipynb file as html, just like nbviewer does? Would prefer to use the
>> IPython libraries directly rather than externally calling ipython, but that
>> would work too.
>>
>> -Doug
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/45684f81/attachment.html>

From hughesadam87 at gmail.com  Wed Aug 27 13:33:24 2014
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Wed, 27 Aug 2014 13:33:24 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CACejjWw8E4diExgrww1S1-FmJh-bqMtQ=U60knA-LLrR08O8rw@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CA+-1RQRoZrO+r1BF3amKisJE23PAjtcY1C0giYq1g1+W4E4+YA@mail.gmail.com>
	<CAMHV+dDZ9Q-Ljd3NujK77CfO7+Nk54SDsERhj5LghX5_5hrhzA@mail.gmail.com>
	<CACejjWw8E4diExgrww1S1-FmJh-bqMtQ=U60knA-LLrR08O8rw@mail.gmail.com>
Message-ID: <CAMHV+dDyVeoxPOo_Pg8Qu1utVrjePTXs78WNche3VAtN0PFNjw@mail.gmail.com>

Thanks for the help Nick and Jason, sorry for the late reply.  WIll try out
your suggestions.

PS, the trait notification decorator is standard to Traits but probably not
Traitlets.


On Sat, Aug 23, 2014 at 3:50 PM, Nicholas Bollweg <nick.bollweg at gmail.com>
wrote:

> One way to do this would be to instead of requiring the notebook user to
> understand everything, expose important bits as traitlets, and then
> change/traitlet.link all of them together. In your example, if:
>
>    - Plotting knows to redraw itself when its data changes
>    - Filtering , updates its windowtype, to be, instead of "hamming",
>    scipy.filter.hamming
>
> then suddenly your code is now:
>
>> def change_windowtype(name, old_value, new_value):
>>     plot_ui.data = filtering_ui.windowtype(data)
>> filtering_ui.on_trait_change("windowtype", change_windowtype)
>>
> (I don't think your usage of on_trait_changed as a decorator is legit, but
> that would be awesome!)
>
> That moves a lot of the event stuff into the internal structure of each of
> your GUI components. This pattern would also be much easier to wrap up
> later into a "higher order" GUI. Then, if you want to "hoist" a property,
> you would just use traitlet.link:
>
>> class GUI(widgets.ContainerWidget)
>>     windowtype = traitlets.Any(sync=True)
>>     plot_data = traitlets.Any(sync=True)
>>     data = traitlets.Any(sync=True)
>>
>>     def __init__(self, **kwargs)
>>         super(GUI, self).__init__(**kwargs)
>>
>>         self.filtering_ui = Filtering()
>>         self.plot_ui = Plotting()
>>
>>         self.children = [self.filtering_ui, self.plot_ui]
>>
>>         traitlets.link(self, "plot_data"), (self.plot_ui, "data"),
>>         traitlets.link(self, "windowtype"), (self.filtering_ui,
>> "windowtype"),
>>
>>     def _windowtype_changed(self, name, old_value, new_value):
>>         self.plot_data = self.windowtype(self.data)
>>
> And now the notebook user would still be able to inspect and change the
> important parts of even deeply nested settings with simple assignment.
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/af15baa6/attachment.html>

From tritemio at gmail.com  Wed Aug 27 14:41:19 2014
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Wed, 27 Aug 2014 11:41:19 -0700
Subject: [IPython-dev] ipython notebook widgets: combobox?
In-Reply-To: <CA+-1RQS7BnTKikqN9g2giWH1FkkY=JemiR6M8sded38dLzJKOg@mail.gmail.com>
References: <CANn2QUw+chUAE+VzEuz6KLeNfPCbwxaMgJ7c8QrV48Nam8B2fQ@mail.gmail.com>
	<CA+-1RQS7BnTKikqN9g2giWH1FkkY=JemiR6M8sded38dLzJKOg@mail.gmail.com>
Message-ID: <CANn2QUycDTuKE4mYyU9hL+UPAA-b14MhfhQfRDq=ueesRVX67A@mail.gmail.com>

Thanks Cyrille, your collection of notebooks for euroscipy2014 is amazing!

I let you know if I encounter problems in implementing the combo-box.

Antonio


On Tue, Aug 26, 2014 at 4:46 PM, Cyrille Rossant <cyrille.rossant at gmail.com>
wrote:

> Hi
> See example 4 here: https://github.com/rossant/euroscipy2014
> Cyrille
>
> Le mercredi 27 ao?t 2014, Antonino Ingargiola <tritemio at gmail.com> a
> ?crit :
>
> Hi to the list,
>>
>> I'm playing with the widgets and I can't find any example of a combo-box
>> widget. I need a numeric text entry (like BoundedIntTextWidget) with two
>> small buttons to increment or decrement the value. All the GUI toolkits
>> have something similar.
>>
>> If this widget does not yet exist, would it be possible to bind two
>> buttons (Up, Down) to a BoundedIntTextWidget in order to achieve roughly
>> the same result?
>>
>> Thanks,
>> Antonio
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/dc556497/attachment.html>

From matthew.brett at gmail.com  Wed Aug 27 15:52:56 2014
From: matthew.brett at gmail.com (Matthew Brett)
Date: Wed, 27 Aug 2014 15:52:56 -0400
Subject: [IPython-dev] javascript / Python communication for image viewer
Message-ID: <CAH6Pt5q1WAP9bipPBm5YMDfBNRU5uu8PqY5ZJbayC_58wU37WQ@mail.gmail.com>

Guys / gals,

I want to ask for advice about writing a brain image display widget for
IPython.

I would like to make an IPython widget that can take an in-memory numpy array
and do an interactive display of orthogonal slices from the array.  The
display will look something like Papaya:

http://rii.uthscsa.edu/mango/papaya

where clicking or moving the mouse causes matching slices to be displayed
through the three axes of the numpy array (here a brain image).

Papaya is pure javascript, so I am assuming that it loads the whole array
(brain image) into a javascript variable and takes slices from that.

What I would like to do, is to be able to keep the whole 3D array only in
Python, and pass the slices as needed to a javascript viewer.

In my ignorance, I am not sure which approach to go for first.

Should I use the comm / widget interface for this?  In that case I guess the
procedure would be:

* mouse movement generates a 'need slice' message from javascript
* python kernel accepts 'need slice' message, takes slice from array, base64
  encodes into JSON, sends 'here is your slice' message back to javascript
  with the data
* javascript accepts message, unpacks base64'ed JSONed slice into variable and
  displays slice variable

Is that right?  I guess that involves Is there any chance that this
will be fast enough for
satisfying interactive movement through the image, which would likely require
something like 20 slices per second, each of say 64 x 64 floating point?

If not - is there something else I should look at instead?

Another question for more thanks - should I use a Canvas or SVG element to
display the images?  The fabric.js README [1] seems to imply the
Canvas element is
faster for some interactive stuff, does anyone have relevant experience to
share?

Thanks a lot,

Matthew

[1] https://github.com/kangax/fabric.js/#history


From cyrille.rossant at gmail.com  Wed Aug 27 16:53:28 2014
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Wed, 27 Aug 2014 21:53:28 +0100
Subject: [IPython-dev] javascript / Python communication for image viewer
In-Reply-To: <CAH6Pt5q1WAP9bipPBm5YMDfBNRU5uu8PqY5ZJbayC_58wU37WQ@mail.gmail.com>
References: <CAH6Pt5q1WAP9bipPBm5YMDfBNRU5uu8PqY5ZJbayC_58wU37WQ@mail.gmail.com>
Message-ID: <CA+-1RQST1BiYF+0yPQfvttSZyy4wfTH2zokQjAiyU58MAqqEhg@mail.gmail.com>

We have pretty similar requirements for Vispy, as we target fast
visualization of big datasets using OpenGL in the IPython notebook. In
particular, we can't use Canvas or SVG because it's just too slow for
big data, so we have to use WebGL. This lets us leverage the GPU for
visualization.

I would be very interested in seeing the answers to your questions.
Notably, is it possible to use binary sockets for transferring large
amounts of binary data between Python and JavaScript?

Cyrille

2014-08-27 20:52 GMT+01:00 Matthew Brett <matthew.brett at gmail.com>:
> Guys / gals,
>
> I want to ask for advice about writing a brain image display widget for
> IPython.
>
> I would like to make an IPython widget that can take an in-memory numpy array
> and do an interactive display of orthogonal slices from the array.  The
> display will look something like Papaya:
>
> http://rii.uthscsa.edu/mango/papaya
>
> where clicking or moving the mouse causes matching slices to be displayed
> through the three axes of the numpy array (here a brain image).
>
> Papaya is pure javascript, so I am assuming that it loads the whole array
> (brain image) into a javascript variable and takes slices from that.
>
> What I would like to do, is to be able to keep the whole 3D array only in
> Python, and pass the slices as needed to a javascript viewer.
>
> In my ignorance, I am not sure which approach to go for first.
>
> Should I use the comm / widget interface for this?  In that case I guess the
> procedure would be:
>
> * mouse movement generates a 'need slice' message from javascript
> * python kernel accepts 'need slice' message, takes slice from array, base64
>   encodes into JSON, sends 'here is your slice' message back to javascript
>   with the data
> * javascript accepts message, unpacks base64'ed JSONed slice into variable and
>   displays slice variable
>
> Is that right?  I guess that involves Is there any chance that this
> will be fast enough for
> satisfying interactive movement through the image, which would likely require
> something like 20 slices per second, each of say 64 x 64 floating point?
>
> If not - is there something else I should look at instead?
>
> Another question for more thanks - should I use a Canvas or SVG element to
> display the images?  The fabric.js README [1] seems to imply the
> Canvas element is
> faster for some interactive stuff, does anyone have relevant experience to
> share?
>
> Thanks a lot,
>
> Matthew
>
> [1] https://github.com/kangax/fabric.js/#history
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From nick.bollweg at gmail.com  Wed Aug 27 18:29:47 2014
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Wed, 27 Aug 2014 18:29:47 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CAMHV+dDyVeoxPOo_Pg8Qu1utVrjePTXs78WNche3VAtN0PFNjw@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CA+-1RQRoZrO+r1BF3amKisJE23PAjtcY1C0giYq1g1+W4E4+YA@mail.gmail.com>
	<CAMHV+dDZ9Q-Ljd3NujK77CfO7+Nk54SDsERhj5LghX5_5hrhzA@mail.gmail.com>
	<CACejjWw8E4diExgrww1S1-FmJh-bqMtQ=U60knA-LLrR08O8rw@mail.gmail.com>
	<CAMHV+dDyVeoxPOo_Pg8Qu1utVrjePTXs78WNche3VAtN0PFNjw@mail.gmail.com>
Message-ID: <CACejjWwDroWHm403OkwXEK+HVZMiV0Km56HaWJft0wxoLjasWw@mail.gmail.com>

Adam:
It's really no problem... this is exactly the way we want to be building
engineering tools at work, so it's been beneficial to have a more-defined
problem than "make interactive dashboards better/faster/stronger".

I really dug into the 3.0 notebook over the weekend, and do find it a bit
nicer.

Also, your decorator made me want some similar stuff, so I hacked together
these two classes:

http://nbviewer.ipython.org/gist/bollwyvl/8001ed62c6868301c26e
>

again, no widget actually show up, but I think one gets the idea.

it does provide a decorator (I'm not happy with it yet) as well as
overloaded assignment and transformed values. I hope this might be useful!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/df238a78/attachment.html>

From hughesadam87 at gmail.com  Wed Aug 27 19:47:10 2014
From: hughesadam87 at gmail.com (hugadams)
Date: Wed, 27 Aug 2014 16:47:10 -0700 (PDT)
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CACejjWwDroWHm403OkwXEK+HVZMiV0Km56HaWJft0wxoLjasWw@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CA+-1RQRoZrO+r1BF3amKisJE23PAjtcY1C0giYq1g1+W4E4+YA@mail.gmail.com>
	<CAMHV+dDZ9Q-Ljd3NujK77CfO7+Nk54SDsERhj5LghX5_5hrhzA@mail.gmail.com>
	<CACejjWw8E4diExgrww1S1-FmJh-bqMtQ=U60knA-LLrR08O8rw@mail.gmail.com>
	<CAMHV+dDyVeoxPOo_Pg8Qu1utVrjePTXs78WNche3VAtN0PFNjw@mail.gmail.com>
	<CACejjWwDroWHm403OkwXEK+HVZMiV0Km56HaWJft0wxoLjasWw@mail.gmail.com>
Message-ID: <CAMHV+dAwJ-YqqJR3-Q-=r58SoznKJeJuk2DmAr5fRmYx8ofEaQ@mail.gmail.com>

These are def cool.  Can you keep me updated if you add more to this?  I'd
like to steal it (with credit) for our widgets.

One question about the decorator:

@wire(wire.e.value, wire.a.value)def thrice(a):
    return a * 3


Does this mean that anytime the "e" widget value is changed, the "a" widget
value will be multiplied by 3?  Or am I not understanding what is happening
here?

On Wed, Aug 27, 2014 at 6:24 PM, Nicholas Bollweg [via Python] <
ml-node+s6n5069419h51 at n6.nabble.com> wrote:

> Adam:
> It's really no problem... this is exactly the way we want to be building
> engineering tools at work, so it's been beneficial to have a more-defined
> problem than "make interactive dashboards better/faster/stronger".
>
> I really dug into the 3.0 notebook over the weekend, and do find it a bit
> nicer.
>
> Also, your decorator made me want some similar stuff, so I hacked together
> these two classes:
>
> http://nbviewer.ipython.org/gist/bollwyvl/8001ed62c6868301c26e
>>
>
> again, no widget actually show up, but I think one gets the idea.
>
> it does provide a decorator (I'm not happy with it yet) as well as
> overloaded assignment and transformed values. I hope this might be useful!
>
> _______________________________________________
> IPython-dev mailing list
> [hidden email] <http://user/SendEmail.jtp?type=node&node=5069419&i=0>
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5069419.html
>  To unsubscribe from Feasilibity of widgets/layout for mimicking this, click
> here
> <http://python.6.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5068197&code=aHVnaGVzYWRhbTg3QGdtYWlsLmNvbXw1MDY4MTk3fC03NjYyMTgyMzk=>
> .
> NAML
> <http://python.6.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5069424.html
Sent from the IPython - Development mailing list archive at Nabble.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/e34c9e47/attachment.html>

From nick.bollweg at gmail.com  Wed Aug 27 22:57:07 2014
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Wed, 27 Aug 2014 22:57:07 -0400
Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this
In-Reply-To: <CAMHV+dAwJ-YqqJR3-Q-=r58SoznKJeJuk2DmAr5fRmYx8ofEaQ@mail.gmail.com>
References: <1408298099961-5068197.post@n6.nabble.com>
	<CA+-1RQT6RL9bKp0D7yKQ718-H_UDvs-P0QTO0AmNZO2-YdBSBA@mail.gmail.com>
	<CA+-1RQRoZrO+r1BF3amKisJE23PAjtcY1C0giYq1g1+W4E4+YA@mail.gmail.com>
	<CAMHV+dDZ9Q-Ljd3NujK77CfO7+Nk54SDsERhj5LghX5_5hrhzA@mail.gmail.com>
	<CACejjWw8E4diExgrww1S1-FmJh-bqMtQ=U60knA-LLrR08O8rw@mail.gmail.com>
	<CAMHV+dDyVeoxPOo_Pg8Qu1utVrjePTXs78WNche3VAtN0PFNjw@mail.gmail.com>
	<CACejjWwDroWHm403OkwXEK+HVZMiV0Km56HaWJft0wxoLjasWw@mail.gmail.com>
	<CAMHV+dAwJ-YqqJR3-Q-=r58SoznKJeJuk2DmAr5fRmYx8ofEaQ@mail.gmail.com>
Message-ID: <CACejjWwkY85LF55MwoZrG2DmMn2NbqCSS+PLS6ebokhV0qZORg@mail.gmail.com>

I was suggesting (with the naming) that e would be updated by a changing,
but I agree, it's confusing.

I tried something more complicated with kwargs, but it ended up being a
whole second implementation of the thing that Wire does, which seemed like
too much, but would look like this (which I think is somewhat more
readable):

@wire(wire.e.value, a=wire.a.value)def thrice(a):
    return a * 3


I've updated to use that, as well as an example of the dictionary-based
assignment it is using. I welcome any other suggestions: the decorator
syntax is somewhat limiting, which is probably good as this is already
really evil :)

On Wed, Aug 27, 2014 at 7:47 PM, hugadams <hughesadam87 at gmail.com> wrote:

> These are def cool.  Can you keep me updated if you add more to this?  I'd
> like to steal it (with credit) for our widgets.
>
> One question about the decorator:
>
> @wire(wire.e.value, wire.a.value)def thrice(a):
>     return a * 3
>
>
> Does this mean that anytime the "e" widget value is changed, the "a"
> widget value will be multiplied by 3?  Or am I not understanding what is
> happening here?
>
> On Wed, Aug 27, 2014 at 6:24 PM, Nicholas Bollweg [via Python] <[hidden
> email] <http://user/SendEmail.jtp?type=node&node=5069424&i=0>> wrote:
>
>> Adam:
>> It's really no problem... this is exactly the way we want to be building
>> engineering tools at work, so it's been beneficial to have a more-defined
>> problem than "make interactive dashboards better/faster/stronger".
>>
>> I really dug into the 3.0 notebook over the weekend, and do find it a bit
>> nicer.
>>
>> Also, your decorator made me want some similar stuff, so I hacked
>> together these two classes:
>>
>> http://nbviewer.ipython.org/gist/bollwyvl/8001ed62c6868301c26e
>>>
>>
>> again, no widget actually show up, but I think one gets the idea.
>>
>> it does provide a decorator (I'm not happy with it yet) as well as
>> overloaded assignment and transformed values. I hope this might be useful!
>>
>> _______________________________________________
>> IPython-dev mailing list
>> [hidden email] <http://user/SendEmail.jtp?type=node&node=5069419&i=0>
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>> ------------------------------
>>  If you reply to this email, your message will be added to the
>> discussion below:
>>
>> http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5069419.html
>>  To unsubscribe from Feasilibity of widgets/layout for mimicking this, click
>> here.
>> NAML
>> <http://python.6.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
> ------------------------------
> View this message in context: Re: Feasilibity of widgets/layout for
> mimicking this
> <http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5069424.html>
> Sent from the IPython - Development mailing list archive
> <http://python.6.x6.nabble.com/IPython-Development-f1646922.html> at
> Nabble.com.
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140827/b174681f/attachment.html>

From jabooth at gmail.com  Thu Aug 28 03:45:15 2014
From: jabooth at gmail.com (James Booth)
Date: Thu, 28 Aug 2014 00:45:15 -0700 (PDT)
Subject: [IPython-dev] javascript / Python communication for image viewer
In-Reply-To: <CA+-1RQST1BiYF+0yPQfvttSZyy4wfTH2zokQjAiyU58MAqqEhg@mail.gmail.com>
References: <CA+-1RQST1BiYF+0yPQfvttSZyy4wfTH2zokQjAiyU58MAqqEhg@mail.gmail.com>
Message-ID: <1409211914670.efeba11e@Nodemailer>

Hey Matthew, Cyrille,




I?m developing a three.js-based web app for image/object annotation called landmarker.io (I work in computer vision/machine learning, getting high quality annotations of 3D and 2D data is often critical to training models). It?s located here




www.landmarker.io (tool, launches in demo by default)

https://github.com/menpo/landmarker.io (code)





I?m also one of the developer?s of Menpo (www.menpo.io), which is a Python package for building and testing deformable models. I?m going to be developing an IPython widget-version of the landmarker.io tool, so we can correct annotations as we sport problems in the notebook.




I mention all this as:




1. I?m also interested in understanding how we can transfer large arrays efficiently to IPython widgets

2. I have a little experience in how best to handle this in a traditional client/server model, but I don?t know how well this will translate to the comm interface.




Basically, landmarker.io expects to be able to talk to a RESTful interface that is implemented currently by:




https://github.com/menpo/landmarkerio-server




Originally, I sent meshes as large JSON objects, but this was pretty slow as:




1. It?s more work in JS to parse the JSON into an array

2. It?s more work to turn this into the most efficient JS type for numerical arrays (which WebGL loves) which is an ArrayBuffer.





https://developer.mozilla.org/en-US/docs/Web/API/ArrayBuffer

https://developer.mozilla.org/en-US/docs/Web/API/Float32Array






This meant that even with a server on localhost, skipping through faces to annotate felt a little sluggish.




Instead now I just directly build a pure ArrayBuffer in Python on the server and ship it to the client. That happens here in the server:





https://github.com/menpo/landmarkerio-server/blob/master/landmarkerio/cache.py#L165




(I?m a little lazy here and literally save the file to disk, then gzip it. It could be done in memory with StringIO through, but I do this as a caching process so it?s going to disk anyway).


On the client side, I make an XMLHttpRequest?and demand an array buffer from the server:


https://github.com/menpo/landmarker.io/blob/master/src/js/app/lib/get.js#L9




For completion, the actual parsing of the array buffer is done here:


https://github.com/menpo/landmarker.io/blob/master/src/js/app/model/mesh.js#L104






Because three.js supports ArrayBuffers, this is really fast. I just point three to the array and we are away. With this implementation, browsing through subjects to annotate with a server on localhost (akin to moving between slices in your brain scan I imagine) is very fast.




I?m afraid I don?t have much knowledge of the comm interface - is there a document I could be pointed out that lays out the protocol? Does it sound possible to send a pure array to JS in the way I?m doing in the landmarkerio-server?




Best,

James






?
Sent from Mailbox

On Wed, Aug 27, 2014 at 9:53 PM, Cyrille Rossant
<cyrille.rossant at gmail.com> wrote:

> We have pretty similar requirements for Vispy, as we target fast
> visualization of big datasets using OpenGL in the IPython notebook. In
> particular, we can't use Canvas or SVG because it's just too slow for
> big data, so we have to use WebGL. This lets us leverage the GPU for
> visualization.
> I would be very interested in seeing the answers to your questions.
> Notably, is it possible to use binary sockets for transferring large
> amounts of binary data between Python and JavaScript?
> Cyrille
> 2014-08-27 20:52 GMT+01:00 Matthew Brett <matthew.brett at gmail.com>:
>> Guys / gals,
>>
>> I want to ask for advice about writing a brain image display widget for
>> IPython.
>>
>> I would like to make an IPython widget that can take an in-memory numpy array
>> and do an interactive display of orthogonal slices from the array.  The
>> display will look something like Papaya:
>>
>> http://rii.uthscsa.edu/mango/papaya
>>
>> where clicking or moving the mouse causes matching slices to be displayed
>> through the three axes of the numpy array (here a brain image).
>>
>> Papaya is pure javascript, so I am assuming that it loads the whole array
>> (brain image) into a javascript variable and takes slices from that.
>>
>> What I would like to do, is to be able to keep the whole 3D array only in
>> Python, and pass the slices as needed to a javascript viewer.
>>
>> In my ignorance, I am not sure which approach to go for first.
>>
>> Should I use the comm / widget interface for this?  In that case I guess the
>> procedure would be:
>>
>> * mouse movement generates a 'need slice' message from javascript
>> * python kernel accepts 'need slice' message, takes slice from array, base64
>>   encodes into JSON, sends 'here is your slice' message back to javascript
>>   with the data
>> * javascript accepts message, unpacks base64'ed JSONed slice into variable and
>>   displays slice variable
>>
>> Is that right?  I guess that involves Is there any chance that this
>> will be fast enough for
>> satisfying interactive movement through the image, which would likely require
>> something like 20 slices per second, each of say 64 x 64 floating point?
>>
>> If not - is there something else I should look at instead?
>>
>> Another question for more thanks - should I use a Canvas or SVG element to
>> display the images?  The fabric.js README [1] seems to imply the
>> Canvas element is
>> faster for some interactive stuff, does anyone have relevant experience to
>> share?
>>
>> Thanks a lot,
>>
>> Matthew
>>
>> [1] https://github.com/kangax/fabric.js/#history
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140828/3e80f057/attachment.html>

From Sebastien.Hinderer at inria.fr  Thu Aug 28 04:05:15 2014
From: Sebastien.Hinderer at inria.fr (=?utf-8?Q?S=C3=A9bastien?= Hinderer)
Date: Thu, 28 Aug 2014 10:05:15 +0200
Subject: [IPython-dev] Using IPython notebooks to teach Python
In-Reply-To: <CAETqNqG8xDOimx_HWzk18z1RE6R+5+kTyT1-2k9zY3tpL9CDSw@mail.gmail.com>
References: <CAPoFz=cy1eJYmT0H-JSYfN+LK_Y-xtPUjcGZtt8DK1uYuK8DQA@mail.gmail.com>
	<CAETqNqG8xDOimx_HWzk18z1RE6R+5+kTyT1-2k9zY3tpL9CDSw@mail.gmail.com>
Message-ID: <20140828080515.GB2951@pl-59055.rocqadm.inria.fr>

Dear all,

First of all, let me thank all those who have provided so useful
responses to the question about setting up a multi-user notebook server.

Basically, we will do it as all of you seem to have done it: creating a
Unix account for each user so that security relies on Unix permissions.
Technically, we will write our own scripts but they will definitely
devrive from existing work, so thanks again for having provided
valuable resources.

One issue I'm curious about and that did not seem to be addressed in the
resources I explored is the termination of webserver and
kernel processes. Since we develop the solution in the context of a
MOOC, there is the possibility of having plenty of users. So, letting
all the processes run forever does not seem an option. There is
indeed a limit in how many processes can run simultaneously on a system
and, perhaps even more importantly, the number of available TCP ports is
even smaller than that. Of course, the transpor beween web servers and
kernels can be set-up to use Unix local sockets, but the webservs will
all need a different port so if they are kept alive that means a limit
on the number of users.

Perhaps one possibility here would be to have a web server running in
front and redirecting connections to the notebook severs, based on a URL
prefix or a similar solution. Not sure whether/ how this can be
implemented, though.

Another suggestion, regarding IPython 3.0, would be to provide
configuration parameters to let a kernel and a web server stop after
some given idle time, or something like that.

Any comment on this aspect will of course be welcome!

Bes wishes,
S?bastien.




From bussonniermatthias at gmail.com  Thu Aug 28 04:46:36 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Thu, 28 Aug 2014 09:46:36 +0100
Subject: [IPython-dev] javascript / Python communication for image viewer
In-Reply-To: <1409211914670.efeba11e@Nodemailer>
References: <CA+-1RQST1BiYF+0yPQfvttSZyy4wfTH2zokQjAiyU58MAqqEhg@mail.gmail.com>
	<1409211914670.efeba11e@Nodemailer>
Message-ID: <AD8B94DB-2541-4E06-9D3B-8FE87AF85567@gmail.com>

Hi all, 


You probably don?t want to use widget framework directly but more comms object. 
Binary data exchange will be possible with https://github.com/ipython/ipython/pull/6110

? 
M
Le 28 ao?t 2014 ? 08:45, James Booth <jabooth at gmail.com> a ?crit :

> Hey Matthew, Cyrille,
> 
> I?m developing a three.js-based web app for image/object annotation called landmarker.io (I work in computer vision/machine learning, getting high quality annotations of 3D and 2D data is often critical to training models). It?s located here
> 
> www.landmarker.io (tool, launches in demo by default)
> https://github.com/menpo/landmarker.io (code)
> 
> I?m also one of the developer?s of Menpo (www.menpo.io), which is a Python package for building and testing deformable models. I?m going to be developing an IPython widget-version of the landmarker.io tool, so we can correct annotations as we sport problems in the notebook.
> 
> I mention all this as:
> 
> 1. I?m also interested in understanding how we can transfer large arrays efficiently to IPython widgets
> 2. I have a little experience in how best to handle this in a traditional client/server model, but I don?t know how well this will translate to the comm interface.
> 
> Basically, landmarker.io expects to be able to talk to a RESTful interface that is implemented currently by:
> 
> https://github.com/menpo/landmarkerio-server
> 
> Originally, I sent meshes as large JSON objects, but this was pretty slow as:
> 
> 1. It?s more work in JS to parse the JSON into an array
> 2. It?s more work to turn this into the most efficient JS type for numerical arrays (which WebGL loves) which is an ArrayBuffer.
> 
> https://developer.mozilla.org/en-US/docs/Web/API/ArrayBuffer
> https://developer.mozilla.org/en-US/docs/Web/API/Float32Array
> 
> This meant that even with a server on localhost, skipping through faces to annotate felt a little sluggish.
> 
> Instead now I just directly build a pure ArrayBuffer in Python on the server and ship it to the client. That happens here in the server:
> 
> https://github.com/menpo/landmarkerio-server/blob/master/landmarkerio/cache.py#L165
> 
> (I?m a little lazy here and literally save the file to disk, then gzip it. It could be done in memory with StringIO through, but I do this as a caching process so it?s going to disk anyway).
> 
> On the client side, I make an XMLHttpRequest and demand an array buffer from the server:
> https://github.com/menpo/landmarker.io/blob/master/src/js/app/lib/get.js#L9
> 
> For completion, the actual parsing of the array buffer is done here:
> https://github.com/menpo/landmarker.io/blob/master/src/js/app/model/mesh.js#L104
> 
> Because three.js supports ArrayBuffers, this is really fast. I just point three to the array and we are away. With this implementation, browsing through subjects to annotate with a server on localhost (akin to moving between slices in your brain scan I imagine) is very fast.
> 
> I?m afraid I don?t have much knowledge of the comm interface - is there a document I could be pointed out that lays out the protocol? Does it sound possible to send a pure array to JS in the way I?m doing in the landmarkerio-server?
> 
> Best,
> James
> 
> 
> ?
> Sent from Mailbox
> 
> 
> On Wed, Aug 27, 2014 at 9:53 PM, Cyrille Rossant <cyrille.rossant at gmail.com> wrote:
> 
> We have pretty similar requirements for Vispy, as we target fast 
> visualization of big datasets using OpenGL in the IPython notebook. In 
> particular, we can't use Canvas or SVG because it's just too slow for 
> big data, so we have to use WebGL. This lets us leverage the GPU for 
> visualization. 
> 
> I would be very interested in seeing the answers to your questions. 
> Notably, is it possible to use binary sockets for transferring large 
> amounts of binary data between Python and JavaScript? 
> 
> Cyrille 
> 
> 2014-08-27 20:52 GMT+01:00 Matthew Brett <matthew.brett at gmail.com>: 
> > Guys / gals, 
> > 
> > I want to ask for advice about writing a brain image display widget for 
> > IPython. 
> > 
> > I would like to make an IPython widget that can take an in-memory numpy array 
> > and do an interactive display of orthogonal slices from the array. The 
> > display will look something like Papaya: 
> > 
> > http://rii.uthscsa.edu/mango/papaya 
> > 
> > where clicking or moving the mouse causes matching slices to be displayed 
> > through the three axes of the numpy array (here a brain image). 
> > 
> > Papaya is pure javascript, so I am assuming that it loads the whole array 
> > (brain image) into a javascript variable and takes slices from that. 
> > 
> > What I would like to do, is to be able to keep the whole 3D array only in 
> > Python, and pass the slices as needed to a javascript viewer. 
> > 
> > In my ignorance, I am not sure which approach to go for first. 
> > 
> > Should I use the comm / widget interface for this? In that case I guess the 
> > procedure would be: 
> > 
> > * mouse movement generates a 'need slice' message from javascript 
> > * python kernel accepts 'need slice' message, takes slice from array, base64 
> > encodes into JSON, sends 'here is your slice' message back to javascript 
> > with the data 
> > * javascript accepts message, unpacks base64'ed JSONed slice into variable and 
> > displays slice variable 
> > 
> > Is that right? I guess that involves Is there any chance that this 
> > will be fast enough for 
> > satisfying interactive movement through the image, which would likely require 
> > something like 20 slices per second, each of say 64 x 64 floating point? 
> > 
> > If not - is there something else I should look at instead? 
> > 
> > Another question for more thanks - should I use a Canvas or SVG element to 
> > display the images? The fabric.js README [1] seems to imply the 
> > Canvas element is 
> > faster for some interactive stuff, does anyone have relevant experience to 
> > share? 
> > 
> > Thanks a lot, 
> > 
> > Matthew 
> > 
> > [1] https://github.com/kangax/fabric.js/#history 
> > _______________________________________________ 
> > IPython-dev mailing list 
> > IPython-dev at scipy.org 
> > http://mail.scipy.org/mailman/listinfo/ipython-dev 
> _______________________________________________ 
> IPython-dev mailing list 
> IPython-dev at scipy.org 
> http://mail.scipy.org/mailman/listinfo/ipython-dev 
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140828/8496a9ee/attachment.html>

From bussonniermatthias at gmail.com  Thu Aug 28 05:07:37 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Thu, 28 Aug 2014 10:07:37 +0100
Subject: [IPython-dev] Using IPython notebooks to teach Python
In-Reply-To: <20140828080515.GB2951@pl-59055.rocqadm.inria.fr>
References: <CAPoFz=cy1eJYmT0H-JSYfN+LK_Y-xtPUjcGZtt8DK1uYuK8DQA@mail.gmail.com>
	<CAETqNqG8xDOimx_HWzk18z1RE6R+5+kTyT1-2k9zY3tpL9CDSw@mail.gmail.com>
	<20140828080515.GB2951@pl-59055.rocqadm.inria.fr>
Message-ID: <89D82AF9-4DCF-42F5-85DC-869FD98D5F72@gmail.com>

Hi, 

Please have a look at 
    https://github.com/jupyter/jupyterhub
and
    https://github.com/jupyter/configurable-http-proxy


This is what will be used for IPython multi-user.

Le 28 ao?t 2014 ? 09:05, S?bastien Hinderer <Sebastien.Hinderer at inria.fr> a ?crit :

> Dear all,
> 
> First of all, let me thank all those who have provided so useful
> responses to the question about setting up a multi-user notebook server.
> 
> Basically, we will do it as all of you seem to have done it: creating a
> Unix account for each user so that security relies on Unix permissions.
> Technically, we will write our own scripts but they will definitely
> devrive from existing work, so thanks again for having provided
> valuable resources.
> 
> One issue I'm curious about and that did not seem to be addressed in the
> resources I explored is the termination of webserver and
> kernel processes. Since we develop the solution in the context of a
> MOOC, there is the possibility of having plenty of users. So, letting
> all the processes run forever does not seem an option. There is
> indeed a limit in how many processes can run simultaneously on a system
> and, perhaps even more importantly, the number of available TCP ports is
> even smaller than that. Of course, the transpor beween web servers and
> kernels can be set-up to use Unix local sockets, but the webservs will
> all need a different port so if they are kept alive that means a limit
> on the number of users.
> 
> Perhaps one possibility here would be to have a web server running in
> front and redirecting connections to the notebook severs, based on a URL
> prefix or a similar solution. Not sure whether/ how this can be
> implemented, though.

This is what above link do. 


> Another suggestion, regarding IPython 3.0, would be to provide
> configuration parameters to let a kernel and a web server stop after
> some given idle time, or something like that.


Probably not. 
as you are in control of the server/proxy, you can have a 
deamon that monitor the connexion and send the rest command
 to shutdown with the logic you like. 

It will be much more reliable and flexible than baking in all the imaginable logic in IPython.

> Any comment on this aspect will of course be welcome!


We will be happy to get help on the developpement of this, 
The current funding of IPython is (as far as I know) coming to 
an end at the end of this year, so  we cannot promise that things 
will continue to move fast in the future. 

If your MOOC as a lot of audience and you use IPython/Python a lot, 
please consider proposing your hierarchy to give something to Python community. 

Be aware that isolating user on the server-side using container or alike is 
not the hard part of the security issue in using IPython.
The hard part is on the browser side. 

To anyone that say that container are used on server-side for security, 
be aware that what you say is equivalent to say :

	?  The server of your bank is in a vault with 5 ton door, of course you can 
	 send plain text email with your credit card number to any people that claim 
         to be the server, no one can access this server ?

The risk is js injection in the browser, if it is a MOOC, steeling credential, replaying cookie auth,
CORS on web socket, and so on and so forth. 
? 
M


> 
> Bes wishes,
> S?bastien.
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140828/cdf84048/attachment.html>

From dave.hirschfeld at gmail.com  Thu Aug 28 06:42:41 2014
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Thu, 28 Aug 2014 10:42:41 +0000 (UTC)
Subject: [IPython-dev] =?utf-8?q?AttributeError=3A_=27Kernel=27_object_has?=
	=?utf-8?q?_no_attribute_=27abort=5Fqueues=27?=
Message-ID: <loom.20140828T114459-236@post.gmane.org>

I got the aforementioned error in an ipengine process.
I'm not sure of the sequence of events which led up to it, but from what I can 
see it doesn't exist in 2.1.

[IPEngineApp] ERROR | Exception in control handler:
Traceback (most recent call last):
  File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site-
packages\IPython\kernel\zmq\ipkernel.py", line 200, in dispatch_control
    handler(self.control_stream, idents, msg)
  File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site-
packages\IPython\kernel\zmq\ipkernel.py", line 666, in abort_request
    self.abort_queues()
AttributeError: 'Kernel' object has no attribute 'abort_queues'

https://github.com/ipython/ipython/blob/rel-
2.1.0/IPython/kernel/zmq/ipkernel.py

I've confirmed that this bug is also present in master:

https://github.com/ipython/ipython/blob/4dcc51a102619862141ee68742117adb4cd1e5
1d/IPython/kernel/zmq/kernelbase.py


PR available @ https://github.com/ipython/ipython/pull/6374


HTH,
Dave



From benjaminrk at gmail.com  Thu Aug 28 14:03:12 2014
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 28 Aug 2014 11:03:12 -0700
Subject: [IPython-dev] AttributeError: 'Kernel' object has no attribute
	'abort_queues'
In-Reply-To: <loom.20140828T114459-236@post.gmane.org>
References: <loom.20140828T114459-236@post.gmane.org>
Message-ID: <CAHNn8BWBsADLm23zuQnrDmmYhhGxeADTHkRDBbf9rcD1Rbjh+w@mail.gmail.com>

Thanks for catching it


On Thu, Aug 28, 2014 at 3:42 AM, Dave Hirschfeld <dave.hirschfeld at gmail.com>
wrote:

> I got the aforementioned error in an ipengine process.
> I'm not sure of the sequence of events which led up to it, but from what I
> can
> see it doesn't exist in 2.1.
>
> [IPEngineApp] ERROR | Exception in control handler:
> Traceback (most recent call last):
>   File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site-
> packages\IPython\kernel\zmq\ipkernel.py", line 200, in dispatch_control
>     handler(self.control_stream, idents, msg)
>   File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site-
> packages\IPython\kernel\zmq\ipkernel.py", line 666, in abort_request
>     self.abort_queues()
> AttributeError: 'Kernel' object has no attribute 'abort_queues'
>
> https://github.com/ipython/ipython/blob/rel-
> 2.1.0/IPython/kernel/zmq/ipkernel.py
>
> I've confirmed that this bug is also present in master:
>
>
> https://github.com/ipython/ipython/blob/4dcc51a102619862141ee68742117adb4cd1e5
> 1d/IPython/kernel/zmq/kernelbase.py
>
>
> PR available @ https://github.com/ipython/ipython/pull/6374
>
>
> HTH,
> Dave
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140828/d023eb74/attachment.html>

From benjaminrk at gmail.com  Thu Aug 28 14:03:12 2014
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 28 Aug 2014 11:03:12 -0700
Subject: [IPython-dev] AttributeError: 'Kernel' object has no attribute
	'abort_queues'
In-Reply-To: <loom.20140828T114459-236@post.gmane.org>
References: <loom.20140828T114459-236@post.gmane.org>
Message-ID: <CAHNn8BWBsADLm23zuQnrDmmYhhGxeADTHkRDBbf9rcD1Rbjh+w@mail.gmail.com>

Thanks for catching it


On Thu, Aug 28, 2014 at 3:42 AM, Dave Hirschfeld <dave.hirschfeld at gmail.com>
wrote:

> I got the aforementioned error in an ipengine process.
> I'm not sure of the sequence of events which led up to it, but from what I
> can
> see it doesn't exist in 2.1.
>
> [IPEngineApp] ERROR | Exception in control handler:
> Traceback (most recent call last):
>   File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site-
> packages\IPython\kernel\zmq\ipkernel.py", line 200, in dispatch_control
>     handler(self.control_stream, idents, msg)
>   File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site-
> packages\IPython\kernel\zmq\ipkernel.py", line 666, in abort_request
>     self.abort_queues()
> AttributeError: 'Kernel' object has no attribute 'abort_queues'
>
> https://github.com/ipython/ipython/blob/rel-
> 2.1.0/IPython/kernel/zmq/ipkernel.py
>
> I've confirmed that this bug is also present in master:
>
>
> https://github.com/ipython/ipython/blob/4dcc51a102619862141ee68742117adb4cd1e5
> 1d/IPython/kernel/zmq/kernelbase.py
>
>
> PR available @ https://github.com/ipython/ipython/pull/6374
>
>
> HTH,
> Dave
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140828/d023eb74/attachment-0001.html>

From benjaminrk at gmail.com  Thu Aug 28 18:44:22 2014
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 28 Aug 2014 15:44:22 -0700
Subject: [IPython-dev] nbformat updates
Message-ID: <CAHNn8BUZqqkAi7hefJNAmTvuZCvA6YF55tziEfKKgBQ4zsRtJA@mail.gmail.com>

Hello all,

IPython 3.0 will carry major updates to both the kernel message spec
<http://ipython.org/ipython-doc/dev/development/messaging.html> and
the notebook
document format <https://github.com/ipython/ipython/pull/6045>. The message
spec changes landed months ago, and include shims to continue supporting
kernels that talk msg spec v4 via adapters. We will backport the new
nbformat to 2.3 once it is finalized, so that 2.x users will still be able
to read IPython notebooks authored in 3.0 (they will be downgraded to the
old nbformat on open).

The question under discussion right now: Do we need shims like those in the
message specification in the notebook format? The main place where the
nbformat is a public API is in nbconvert, where custom templates or
preprocessors need to know the nbformat structure. Code such as Jake
Vanderplas? liquid_tags for pelican will need to be updated to work with
IPython 3.0. Shims for nbformat would be a lot more complicated than for
the message spec, so I want to gauge interest / need before investing too
much time in it.

-MinRK
?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140828/7ebe81b9/attachment.html>

From jklymak at gmail.com  Fri Aug 29 12:00:50 2014
From: jklymak at gmail.com (Klymak Jody)
Date: Fri, 29 Aug 2014 09:00:50 -0700
Subject: [IPython-dev] Latex highlighting for markdown cells?
Message-ID: <F89FB897-ADD9-4624-B3C1-A9C869605D1C@gmail.com>

Sorry if I've double posted this - I keep sending from the wrong account...

Hi all,

I am using a lot of <equation> tags in my markdown cells, and that works great for what I am writing.  However, is there a way to get latex-style highlighting in my cells rather than markdown (or some combination of both)?  As soon as you put an underscore in the cell, everything until the next underscore is rendered as oblique, which is not very easy to read.  

i.e. if I type:

Hello $a_b$ there.   

Everything after the "b" is oblique.  

Thanks for any pointers.

Cheers,  Jody



From nathan12343 at gmail.com  Fri Aug 29 12:04:48 2014
From: nathan12343 at gmail.com (Nathan Goldbaum)
Date: Fri, 29 Aug 2014 09:04:48 -0700
Subject: [IPython-dev] Latex highlighting for markdown cells?
In-Reply-To: <F89FB897-ADD9-4624-B3C1-A9C869605D1C@gmail.com>
References: <F89FB897-ADD9-4624-B3C1-A9C869605D1C@gmail.com>
Message-ID: <CAJXewOmhbBmfdQufiQA4HrF5mkfUvhnyZNVQAkufkXXSWU5Lng@mail.gmail.com>

This is a known issue I raised in January:

https://github.com/ipython/ipython/issues/4850

I get the impression from Mon's and Brian's
comments that this would be not very straightforward to fix.

On Friday, August 29, 2014, Klymak Jody <jklymak at gmail.com> wrote:

> Sorry if I've double posted this - I keep sending from the wrong account...
>
> Hi all,
>
> I am using a lot of <equation> tags in my markdown cells, and that works
> great for what I am writing.  However, is there a way to get latex-style
> highlighting in my cells rather than markdown (or some combination of
> both)?  As soon as you put an underscore in the cell, everything until the
> next underscore is rendered as oblique, which is not very easy to read.
>
> i.e. if I type:
>
> Hello $a_b$ there.
>
> Everything after the "b" is oblique.
>
> Thanks for any pointers.
>
> Cheers,  Jody
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org <javascript:;>
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140829/c7a43827/attachment.html>

From bussonniermatthias at gmail.com  Fri Aug 29 12:05:42 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Fri, 29 Aug 2014 17:05:42 +0100
Subject: [IPython-dev] Latex highlighting for markdown cells?
In-Reply-To: <F89FB897-ADD9-4624-B3C1-A9C869605D1C@gmail.com>
References: <F89FB897-ADD9-4624-B3C1-A9C869605D1C@gmail.com>
Message-ID: <4C684A7E-7FEC-4D25-8E5A-BA05EDFD46C5@gmail.com>


Le 29 ao?t 2014 ? 17:00, Klymak Jody <jklymak at gmail.com> a ?crit :

> Sorry if I've double posted this - I keep sending from the wrong account...
> 
> Hi all,
> 
> I am using a lot of <equation> tags in my markdown cells, and that works great for what I am writing.  However, is there a way to get latex-style highlighting in my cells rather than markdown (or some combination of both)?  As soon as you put an underscore in the cell, everything until the next underscore is rendered as oblique, which is not very easy to read.  
> 

Someone would have to write a markdown+latex mode in code mirror that detect $/$$ et switch to latex. 
It would have the advantage of also highlighting latex correctly. 
But it need to be done. 

is not hard, but not trivial either.
? 
M

> i.e. if I type:
> 
> Hello $a_b$ there.   
> 
> Everything after the "b" is oblique.  
> 
> Thanks for any pointers.



> 
> Cheers,  Jody
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From zvoros at gmail.com  Fri Aug 29 13:38:25 2014
From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Fri, 29 Aug 2014 19:38:25 +0200
Subject: [IPython-dev] Latex highlighting for markdown cells?
In-Reply-To: <4C684A7E-7FEC-4D25-8E5A-BA05EDFD46C5@gmail.com>
References: <F89FB897-ADD9-4624-B3C1-A9C869605D1C@gmail.com>
	<4C684A7E-7FEC-4D25-8E5A-BA05EDFD46C5@gmail.com>
Message-ID: <5400BA91.1090202@gmail.com>

Hi Matthias,

I have recently been wondering myself about this. What I wanted to do is 
insert the closing \end{environment} tags in the markdown cell, and 
insert e.g., \frac{}{} and move the cursor back to the first curly 
bracket, if the user types \fra, but syntax highlighting would also be 
useful. Could you point me to some sort of documentation as to how one 
starts out writing a plugin for this? I would be willing to work on this.

Cheers,
Zolt?n

On 08/29/2014 06:05 PM, Matthias Bussonnier wrote:
> Le 29 ao?t 2014 ? 17:00, Klymak Jody <jklymak at gmail.com> a ?crit :
>
>> Sorry if I've double posted this - I keep sending from the wrong account...
>>
>> Hi all,
>>
>> I am using a lot of <equation> tags in my markdown cells, and that works great for what I am writing.  However, is there a way to get latex-style highlighting in my cells rather than markdown (or some combination of both)?  As soon as you put an underscore in the cell, everything until the next underscore is rendered as oblique, which is not very easy to read.
>>
> Someone would have to write a markdown+latex mode in code mirror that detect $/$$ et switch to latex.
> It would have the advantage of also highlighting latex correctly.
> But it need to be done.
>
> is not hard, but not trivial either.
> ?
> M
>



From cyrille.rossant at gmail.com  Sat Aug 30 08:57:45 2014
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Sat, 30 Aug 2014 13:57:45 +0100
Subject: [IPython-dev] ANN: Vispy 0.3,
	high-performance visualization in Python
Message-ID: <CA+-1RQR+zo2K_1Y_dsyhC+ZVyoWHj7cKBFdh9k7_nm-YTkRkPA@mail.gmail.com>

Hi,

We are pleased to announce the release of Vispy version 0.3!

This release provides new graphics API (including a scene graph), a
basic OpenGL backend for matplotlib, and experimental integration in
the IPython notebook. Further, we have improved the existing modules
and added several application backends (e.g. wx and sdl2).

We are now focusing our efforts on:

* Consolidating the higher-level graphics APIs: visuals, transforms,
scene graph.
* Improving the performance of the scene graph, notably through batch
rendering (Collections system).
* Adding more features to the matplotlib backend.
* Improving the integration in the IPython notebook: better
performance, improved event loop integration, client-side backend with
WebGL.

Eventually, scientists with no knowledge of OpenGL will be able to
easily create fast and scalable interactive visualizations.

We will greatly appreciate any feedback, bug reports, and feature
requests. See http://vispy.org for downloads and more information.

The Vispy team


From tom at hackerschool.com  Sat Aug 30 12:32:16 2014
From: tom at hackerschool.com (Thomas Ballinger)
Date: Sat, 30 Aug 2014 09:32:16 -0700
Subject: [IPython-dev] __future__ statements in ipython -i test.py
Message-ID: <CABb6=DWh8VY2J3gOGLPs3tRV0bveuPAXEK3cWWLvaKdx0kYqiw@mail.gmail.com>

While addressing a bug in bpython, I found that (terminal) ipython and
python differ in behavior when a file test.py contains

    from __future__ import division

and I run (i)python -i test.py and then enter 1/2. The behavior is the same
for

    $ (i)python -i -c 'from __future__ import division'

and then doing some division.

Before I look too far into if fixing the problem in bpython is possible, I
wanted to see if I could crib off of your collective experience with this.
Is this behavior difficult to implement, not actually desirable, or just
not something someone has gotten around to? After Thomas's excellent
answers last time to my encoding questions I tend to think you all have
dealt with any problem I face a few times already and thought about it more
than I have. :)

Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140830/22f1cfd7/attachment.html>

From hughesadam87 at gmail.com  Sat Aug 30 12:58:40 2014
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Sat, 30 Aug 2014 12:58:40 -0400
Subject: [IPython-dev] ANN: Vispy 0.3,
	high-performance visualization in Python
In-Reply-To: <CA+-1RQR+zo2K_1Y_dsyhC+ZVyoWHj7cKBFdh9k7_nm-YTkRkPA@mail.gmail.com>
References: <CA+-1RQR+zo2K_1Y_dsyhC+ZVyoWHj7cKBFdh9k7_nm-YTkRkPA@mail.gmail.com>
Message-ID: <CAMHV+dBEeiGW=0yjfBkS8ND2NjftHs6-rHkKcfwutpENwXYr0Q@mail.gmail.com>

Looks really awesome, thanks for sharing!


On Sat, Aug 30, 2014 at 8:57 AM, Cyrille Rossant <cyrille.rossant at gmail.com>
wrote:

> Hi,
>
> We are pleased to announce the release of Vispy version 0.3!
>
> This release provides new graphics API (including a scene graph), a
> basic OpenGL backend for matplotlib, and experimental integration in
> the IPython notebook. Further, we have improved the existing modules
> and added several application backends (e.g. wx and sdl2).
>
> We are now focusing our efforts on:
>
> * Consolidating the higher-level graphics APIs: visuals, transforms,
> scene graph.
> * Improving the performance of the scene graph, notably through batch
> rendering (Collections system).
> * Adding more features to the matplotlib backend.
> * Improving the integration in the IPython notebook: better
> performance, improved event loop integration, client-side backend with
> WebGL.
>
> Eventually, scientists with no knowledge of OpenGL will be able to
> easily create fast and scalable interactive visualizations.
>
> We will greatly appreciate any feedback, bug reports, and feature
> requests. See http://vispy.org for downloads and more information.
>
> The Vispy team
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140830/62f3e980/attachment.html>

From hughesadam87 at gmail.com  Sat Aug 30 13:41:30 2014
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Sat, 30 Aug 2014 13:41:30 -0400
Subject: [IPython-dev] Widget example that uses classes instead of globals?
Message-ID: <CAMHV+dDXAJkWYztJAUN2jzNoekd4Cz6dnGjcSVuqh6b10myJHA@mail.gmail.com>

Hi,

This question follows directly from this recent thread on the feasiblity of
widgets (
http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-td5068197.html)
but I thought was specific enough to be a new thread.

In all of the notebook examples I've seen, a GUI is defined in its own cell
with global variables that are shared between functions.  I'd really like
to see a complete, simple example of a GUI build as a class, where most of
the events, updates and default values are set within the class.  And then
when the class is instantiated, the notebook somehow can still output the
GUI.

For example, would something of this design be compatible with the current
widget design pattern:

*class Foo(object):*

*   variable = defaultvalue*

*   def update_variable(self):*
*      ....*

*   def redraw(self):*
*      ...*

*# Instantiate the class*
*f = Foo()*
*f.variable_widget.on_trait_change(update_variable, 'value')*

Or is this type of design pattern out of the question?  We're trying to
move in this direction so that we can compartmentalize and template as much
behavior as possible, and make a simple framework for building up more
complex GUIs.  It's just now clear to me if such a pattern is possible.  If
anyone has any insights or examples, would you mind sharing?

THanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140830/b6b0d92c/attachment.html>

From hughesadam87 at gmail.com  Sat Aug 30 13:43:26 2014
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Sat, 30 Aug 2014 13:43:26 -0400
Subject: [IPython-dev] Widget example that uses classes instead of
	globals?
In-Reply-To: <CAMHV+dDXAJkWYztJAUN2jzNoekd4Cz6dnGjcSVuqh6b10myJHA@mail.gmail.com>
References: <CAMHV+dDXAJkWYztJAUN2jzNoekd4Cz6dnGjcSVuqh6b10myJHA@mail.gmail.com>
Message-ID: <CAMHV+dAffihnHg7g3yO1BLhxHOBbLHX8n3XxcProvCF7kmuHnQ@mail.gmail.com>

And just to clarify, I presume that the notebook is actively "looking" for
particular functions or something in the cell that let's the notebook know
"hey, this is widget/GUI code, I need to output it as so".  I had worried
that if this critical piece was buried in a class, the notebook wouldn't
recognize it, and the GUI wouldn't display.


On Sat, Aug 30, 2014 at 1:41 PM, Adam Hughes <hughesadam87 at gmail.com> wrote:

> Hi,
>
> This question follows directly from this recent thread on the feasiblity
> of widgets (
> http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-td5068197.html)
> but I thought was specific enough to be a new thread.
>
> In all of the notebook examples I've seen, a GUI is defined in its own
> cell with global variables that are shared between functions.  I'd really
> like to see a complete, simple example of a GUI build as a class, where
> most of the events, updates and default values are set within the class.
>  And then when the class is instantiated, the notebook somehow can still
> output the GUI.
>
> For example, would something of this design be compatible with the current
> widget design pattern:
>
> *class Foo(object):*
>
> *   variable = defaultvalue*
>
> *   def update_variable(self):*
> *      ....*
>
> *   def redraw(self):*
> *      ...*
>
> *# Instantiate the class*
> *f = Foo()*
> *f.variable_widget.on_trait_change(update_variable, 'value')*
>
> Or is this type of design pattern out of the question?  We're trying to
> move in this direction so that we can compartmentalize and template as much
> behavior as possible, and make a simple framework for building up more
> complex GUIs.  It's just now clear to me if such a pattern is possible.  If
> anyone has any insights or examples, would you mind sharing?
>
> THanks
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140830/72ca26a2/attachment.html>

From cyrille.rossant at gmail.com  Sat Aug 30 13:49:49 2014
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Sat, 30 Aug 2014 10:49:49 -0700
Subject: [IPython-dev] Widget example that uses classes instead of
	globals?
Message-ID: <-1189340719239998470@unknownmsgid>

I think you can use the "display" function to render a widget instance,
does that answer your question?
------------------------------
De : Adam Hughes <hughesadam87 at gmail.com>
Envoy? : ?30/?08/?2014 18:43
? : IPython developers list <ipython-dev at scipy.org>
Objet : Re: [IPython-dev] Widget example that uses classes instead of
globals?

And just to clarify, I presume that the notebook is actively "looking" for
particular functions or something in the cell that let's the notebook know
"hey, this is widget/GUI code, I need to output it as so".  I had worried
that if this critical piece was buried in a class, the notebook wouldn't
recognize it, and the GUI wouldn't display.


On Sat, Aug 30, 2014 at 1:41 PM, Adam Hughes <hughesadam87 at gmail.com> wrote:

> Hi,
>
> This question follows directly from this recent thread on the feasiblity
> of widgets (
> http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-td5068197.html)
> but I thought was specific enough to be a new thread.
>
> In all of the notebook examples I've seen, a GUI is defined in its own
> cell with global variables that are shared between functions.  I'd really
> like to see a complete, simple example of a GUI build as a class, where
> most of the events, updates and default values are set within the class.
>  And then when the class is instantiated, the notebook somehow can still
> output the GUI.
>
> For example, would something of this design be compatible with the current
> widget design pattern:
>
> *class Foo(object):*
>
> *   variable = defaultvalue*
>
> *   def update_variable(self):*
> *      ....*
>
> *   def redraw(self):*
> *      ...*
>
> *# Instantiate the class*
> *f = Foo()*
> *f.variable_widget.on_trait_change(update_variable, 'value')*
>
> Or is this type of design pattern out of the question?  We're trying to
> move in this direction so that we can compartmentalize and template as much
> behavior as possible, and make a simple framework for building up more
> complex GUIs.  It's just now clear to me if such a pattern is possible.  If
> anyone has any insights or examples, would you mind sharing?
>
> THanks
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140830/b3390938/attachment.html>

From tritemio at gmail.com  Sat Aug 30 14:21:33 2014
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Sat, 30 Aug 2014 11:21:33 -0700
Subject: [IPython-dev] Widget example that uses classes instead of
	globals?
In-Reply-To: <-1189340719239998470@unknownmsgid>
References: <-1189340719239998470@unknownmsgid>
Message-ID: <CANn2QUyLBTMi=6zOsXCNuUq5PYfokHHGrwUACSdYedBObaEgVQ@mail.gmail.com>

Hi,

Sorry for entering in the discussion on-fly...

My question is more specific (but possibly very similar). Would be possible
to call interact with a class object (i.e. a callable class) instead of a
function? This would allow to store all the GUI specific state in the class
(for example precomputing figures) without the need of using globals.

When I quickly tried (with ipython 2.2), interact returned an error
complaining about the number of parameters passed to the object. So I
resorted to using globals. Below I attached the code I currently use to
pre-compute a set of figures (a slider selects which figure to display). I
would appreciate any comment on possibly improving this pattern.

fig_cache = {}
def plot_mfit_i(ich=0, nodisplay=False):
    global fig_cache
    if ich not in fig_cache:
        fig = mfit.plot_mfit(fitter, ich=ich, residuals=True,
return_fig=True)
        fig_cache[ich] = fig
        plt.close(fig)
    if not nodisplay:
        display(fig_cache[ich])

for i in range(8): plot_mfit_i(i, nodisplay=True)  # Precompute all the
figures

interact(plot_mfit_i, ich=(0, 7), nodisplay=fixed(False));  # display the
figure selected through the slider

--
Antonio

On Sat, Aug 30, 2014 at 10:49 AM, Cyrille Rossant <cyrille.rossant at gmail.com
> wrote:

> I think you can use the "display" function to render a widget instance,
> does that answer your question?
> ------------------------------
> De : Adam Hughes <hughesadam87 at gmail.com>
> Envoy? : ?30/?08/?2014 18:43
> ? : IPython developers list <ipython-dev at scipy.org>
> Objet : Re: [IPython-dev] Widget example that uses classes instead of
> globals?
>
> And just to clarify, I presume that the notebook is actively "looking" for
> particular functions or something in the cell that let's the notebook know
> "hey, this is widget/GUI code, I need to output it as so".  I had worried
> that if this critical piece was buried in a class, the notebook wouldn't
> recognize it, and the GUI wouldn't display.
>
>
> On Sat, Aug 30, 2014 at 1:41 PM, Adam Hughes <hughesadam87 at gmail.com>
> wrote:
>
>> Hi,
>>
>> This question follows directly from this recent thread on the feasiblity
>> of widgets (
>> http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-td5068197.html)
>> but I thought was specific enough to be a new thread.
>>
>> In all of the notebook examples I've seen, a GUI is defined in its own
>> cell with global variables that are shared between functions.  I'd really
>> like to see a complete, simple example of a GUI build as a class, where
>> most of the events, updates and default values are set within the class.
>>  And then when the class is instantiated, the notebook somehow can still
>> output the GUI.
>>
>> For example, would something of this design be compatible with the
>> current widget design pattern:
>>
>> *class Foo(object):*
>>
>> *   variable = defaultvalue*
>>
>> *   def update_variable(self):*
>> *      ....*
>>
>> *   def redraw(self):*
>> *      ...*
>>
>> *# Instantiate the class*
>> *f = Foo()*
>> *f.variable_widget.on_trait_change(update_variable, 'value')*
>>
>> Or is this type of design pattern out of the question?  We're trying to
>> move in this direction so that we can compartmentalize and template as much
>> behavior as possible, and make a simple framework for building up more
>> complex GUIs.  It's just now clear to me if such a pattern is possible.  If
>> anyone has any insights or examples, would you mind sharing?
>>
>> THanks
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140830/da460a66/attachment.html>

From mirage007 at gmail.com  Sat Aug 30 21:36:28 2014
From: mirage007 at gmail.com (Ivan Zhang)
Date: Sat, 30 Aug 2014 21:36:28 -0400
Subject: [IPython-dev] Runtime Console Using IPython
Message-ID: <CAAmwedGh=WWpyeRr-1xm7iCpd00NMW85nZg+NPT8HPNzfK5EMQ@mail.gmail.com>

Hi All,

As a way to familiarize myself with IPython, I wrote a simple tool that
allows an IPython Kernel to be run as part of a thread, effectively
creating a way to remotely monitor and change the state of a running
application in real time.

There are a few items that I have yet to figure out, I am hoping for some
feedback from everyone on whether there is a better approach, or an easy
way to fix some of the limitations.

I wrote a post describing the specifics of the tool here:
http://mirage007.github.io/runtime/console/python/ipython/2014/08/28/rtconsole/

and the source code is located here:
https://github.com/mirage007/rtconsole

Any feedback would be greatly appreciated!

Thanks,

-- 
Ivan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140830/15afb92e/attachment.html>

From fperez.net at gmail.com  Sat Aug 30 22:50:30 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 30 Aug 2014 19:50:30 -0700
Subject: [IPython-dev] Runtime Console Using IPython
In-Reply-To: <CAAmwedGh=WWpyeRr-1xm7iCpd00NMW85nZg+NPT8HPNzfK5EMQ@mail.gmail.com>
References: <CAAmwedGh=WWpyeRr-1xm7iCpd00NMW85nZg+NPT8HPNzfK5EMQ@mail.gmail.com>
Message-ID: <CAHAreOoCYYnuWJQ2nuZyJhXW-6Do=dDC7GakMx46y6ia-+QUxg@mail.gmail.com>

Hi Ivan,

On Sat, Aug 30, 2014 at 6:36 PM, Ivan Zhang <mirage007 at gmail.com> wrote:

> Hi All,
>
> As a way to familiarize myself with IPython, I wrote a simple tool that
> allows an IPython Kernel to be run as part of a thread, effectively
> creating a way to remotely monitor and change the state of a running
> application in real time.
>

Great! Thanks for sharing that... I'm sorry that I haven't looked at your
code in detail, but along those lines, I thought at least it would be
useful to point you to these examples:

https://github.com/ipython/ipython/tree/master/examples/Embedding

They illustrate how to embed a kernel in a process that may have its own
event loop (Qt and Wx provided), and how to connect to it from an external
Qt console, with both the Qt console and the GUI controls manipulating the
same namespace.

It's a different take on the same kind of ideas, and you may find it useful.

Cheers

f
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140830/1a645eaf/attachment.html>

From jason-sage at creativetrax.com  Sat Aug 30 23:25:46 2014
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 30 Aug 2014 23:25:46 -0400
Subject: [IPython-dev] Runtime Console Using IPython
In-Reply-To: <CAHAreOoCYYnuWJQ2nuZyJhXW-6Do=dDC7GakMx46y6ia-+QUxg@mail.gmail.com>
References: <CAAmwedGh=WWpyeRr-1xm7iCpd00NMW85nZg+NPT8HPNzfK5EMQ@mail.gmail.com>
	<CAHAreOoCYYnuWJQ2nuZyJhXW-6Do=dDC7GakMx46y6ia-+QUxg@mail.gmail.com>
Message-ID: <540295BA.6000802@creativetrax.com>

On 8/30/14, 22:50, Fernando Perez wrote:

> They illustrate how to embed a kernel in a process that may have its own
> event loop (Qt and Wx provided), and how to connect to it from an
> external Qt console, with both the Qt console and the GUI controls
> manipulating the same namespace.
>
> It's a different take on the same kind of ideas, and you may find it useful.
>

Related to this, I was thinking the other day that it would be really 
useful if there was an easy standard way to make small, short widget 
control update messages and resulting event triggers do their work on a 
separate thread.  I had some usecases the other day where the 
interactivity was severely lagging because the kernel was tied up doing 
other unrelated work.  This probably wouldn't make sense to do by 
default, but maybe with some special decorator or something.  I 
experimented with this sort of thing several years ago with Sage 
interacts, but I'm thinking through it in the context of the IPython 
widgets now.  Just thought I'd bring it up as a related idea to your 
paragraph above.

Thanks,

Jason



From wes.turner at gmail.com  Sun Aug 31 08:46:16 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Sun, 31 Aug 2014 07:46:16 -0500
Subject: [IPython-dev] ANN: Vispy 0.3,
	high-performance visualization in Python
In-Reply-To: <CA+-1RQR+zo2K_1Y_dsyhC+ZVyoWHj7cKBFdh9k7_nm-YTkRkPA@mail.gmail.com>
References: <CA+-1RQR+zo2K_1Y_dsyhC+ZVyoWHj7cKBFdh9k7_nm-YTkRkPA@mail.gmail.com>
Message-ID: <CACfEFw_dENAJS-6u48wsNhrNRe=OfAqBNMYQ93Yid2b5EZLbqw@mail.gmail.com>

Looks cool from afar!

A few posts to reddit may also find your audience:

* http://www.reddit.com/r/python
* http://www.reddit.com/r/visualization
* http://www.reddit.com/r/IPython
On Aug 30, 2014 7:57 AM, "Cyrille Rossant" <cyrille.rossant at gmail.com>
wrote:

> Hi,
>
> We are pleased to announce the release of Vispy version 0.3!
>
> This release provides new graphics API (including a scene graph), a
> basic OpenGL backend for matplotlib, and experimental integration in
> the IPython notebook. Further, we have improved the existing modules
> and added several application backends (e.g. wx and sdl2).
>
> We are now focusing our efforts on:
>
> * Consolidating the higher-level graphics APIs: visuals, transforms,
> scene graph.
> * Improving the performance of the scene graph, notably through batch
> rendering (Collections system).
> * Adding more features to the matplotlib backend.
> * Improving the integration in the IPython notebook: better
> performance, improved event loop integration, client-side backend with
> WebGL.
>
> Eventually, scientists with no knowledge of OpenGL will be able to
> easily create fast and scalable interactive visualizations.
>
> We will greatly appreciate any feedback, bug reports, and feature
> requests. See http://vispy.org for downloads and more information.
>
> The Vispy team
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140831/8490d321/attachment.html>

From asmeurer at gmail.com  Sun Aug 31 12:51:15 2014
From: asmeurer at gmail.com (Aaron Meurer)
Date: Sun, 31 Aug 2014 11:51:15 -0500
Subject: [IPython-dev] __future__ statements in ipython -i test.py
In-Reply-To: <CABb6=DWh8VY2J3gOGLPs3tRV0bveuPAXEK3cWWLvaKdx0kYqiw@mail.gmail.com>
References: <CABb6=DWh8VY2J3gOGLPs3tRV0bveuPAXEK3cWWLvaKdx0kYqiw@mail.gmail.com>
Message-ID: <-7772341799178964587@unknownmsgid>

I believe this has been discussed before on this list at least in the
context of %run.

Aaron Meurer

> On Aug 30, 2014, at 11:32 AM, Thomas Ballinger <tom at hackerschool.com> wrote:
>
> While addressing a bug in bpython, I found that (terminal) ipython and python differ in behavior when a file test.py contains
>
>     from __future__ import division
>
> and I run (i)python -i test.py and then enter 1/2. The behavior is the same for
>
>     $ (i)python -i -c 'from __future__ import division'
>
> and then doing some division.
>
> Before I look too far into if fixing the problem in bpython is possible, I wanted to see if I could crib off of your collective experience with this. Is this behavior difficult to implement, not actually desirable, or just not something someone has gotten around to? After Thomas's excellent answers last time to my encoding questions I tend to think you all have dealt with any problem I face a few times already and thought about it more than I have. :)
>
> Tom
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From asmeurer at gmail.com  Sun Aug 31 14:07:59 2014
From: asmeurer at gmail.com (Aaron Meurer)
Date: Sun, 31 Aug 2014 13:07:59 -0500
Subject: [IPython-dev] __future__ statements in ipython -i test.py
In-Reply-To: <-7772341799178964587@unknownmsgid>
References: <CABb6=DWh8VY2J3gOGLPs3tRV0bveuPAXEK3cWWLvaKdx0kYqiw@mail.gmail.com>
	<-7772341799178964587@unknownmsgid>
Message-ID: <CAKgW=6JRoWLyvtNXMxWxgQBgTiF-g2Sn4i70G4rbUEkRY+jksw@mail.gmail.com>

http://mail.scipy.org/pipermail/ipython-dev/2012-April/008956.html


On Sun, Aug 31, 2014 at 11:51 AM, Aaron Meurer <asmeurer at gmail.com> wrote:

> I believe this has been discussed before on this list at least in the
> context of %run.
>
> Aaron Meurer
>
> > On Aug 30, 2014, at 11:32 AM, Thomas Ballinger <tom at hackerschool.com>
> wrote:
> >
> > While addressing a bug in bpython, I found that (terminal) ipython and
> python differ in behavior when a file test.py contains
> >
> >     from __future__ import division
> >
> > and I run (i)python -i test.py and then enter 1/2. The behavior is the
> same for
> >
> >     $ (i)python -i -c 'from __future__ import division'
> >
> > and then doing some division.
> >
> > Before I look too far into if fixing the problem in bpython is possible,
> I wanted to see if I could crib off of your collective experience with
> this. Is this behavior difficult to implement, not actually desirable, or
> just not something someone has gotten around to? After Thomas's excellent
> answers last time to my encoding questions I tend to think you all have
> dealt with any problem I face a few times already and thought about it more
> than I have. :)
> >
> > Tom
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140831/f39834ea/attachment.html>

From bussonniermatthias at gmail.com  Sun Aug 31 14:14:57 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Sun, 31 Aug 2014 19:14:57 +0100
Subject: [IPython-dev] Latex highlighting for markdown cells?
In-Reply-To: <5400BA91.1090202@gmail.com>
References: <F89FB897-ADD9-4624-B3C1-A9C869605D1C@gmail.com>
	<4C684A7E-7FEC-4D25-8E5A-BA05EDFD46C5@gmail.com>
	<5400BA91.1090202@gmail.com>
Message-ID: <55FBA463-ED7D-499E-9293-0A5D90245861@gmail.com>


Le 29 ao?t 2014 ? 18:38, Zolt?n V?r?s <zvoros at gmail.com> a ?crit :

> Hi Matthias,
> 
> I have recently been wondering myself about this. What I wanted to do is 
> insert the closing \end{environment} tags in the markdown cell, and 
> insert e.g., \frac{}{} and move the cursor back to the first curly 
> bracket, if the user types \fra, but syntax highlighting would also be 
> useful. Could you point me to some sort of documentation as to how one 
> starts out writing a plugin for this? I would be willing to work on this.

Sorry for the short mail : 

http://codemirror.net/doc/manual.html#modeapi

Have also a look at this part of code mirror doc : 

Sometimes, it is useful for modes to nest?to have one mode delegate work to another mode. An example of this kind of mode is the mixed-mode HTML mode. To implement such nesting, it is usually necessary to create mode objects and copy states yourself. To create a mode object, there are CodeMirror.getMode(options, parserConfig), where the first argument is a configuration object as passed to the mode constructor function, and the second argument is a mode specification as in the mode option. To copy a state object, call CodeMirror.copyState(mode, state), where mode is the mode that created the given state.

? 


> 
> Cheers,
> Zolt?n
> 
> On 08/29/2014 06:05 PM, Matthias Bussonnier wrote:
>> Le 29 ao?t 2014 ? 17:00, Klymak Jody <jklymak at gmail.com> a ?crit :
>> 
>>> Sorry if I've double posted this - I keep sending from the wrong account...
>>> 
>>> Hi all,
>>> 
>>> I am using a lot of <equation> tags in my markdown cells, and that works great for what I am writing.  However, is there a way to get latex-style highlighting in my cells rather than markdown (or some combination of both)?  As soon as you put an underscore in the cell, everything until the next underscore is rendered as oblique, which is not very easy to read.
>>> 
>> Someone would have to write a markdown+latex mode in code mirror that detect $/$$ et switch to latex.
>> It would have the advantage of also highlighting latex correctly.
>> But it need to be done.
>> 
>> is not hard, but not trivial either.
>> ?
>> M
>> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140831/70d9e4a1/attachment.html>

From doug.blank at gmail.com  Sun Aug 31 14:37:49 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Sun, 31 Aug 2014 14:37:49 -0400
Subject: [IPython-dev] Latex highlighting for markdown cells?
In-Reply-To: <55FBA463-ED7D-499E-9293-0A5D90245861@gmail.com>
References: <F89FB897-ADD9-4624-B3C1-A9C869605D1C@gmail.com>
	<4C684A7E-7FEC-4D25-8E5A-BA05EDFD46C5@gmail.com>
	<5400BA91.1090202@gmail.com>
	<55FBA463-ED7D-499E-9293-0A5D90245861@gmail.com>
Message-ID: <CAAusYCiriKZunGbsx=7zOja5N5W-EzCKL0M3LpcULmGySv-tyA@mail.gmail.com>

Here is an example of the codemirror overlay with IPython's mode:

https://bitbucket.org/ipre/calico/src/master/notebooks/nbextensions/calico-spell-check.js

That code is a little bit complicated as it has a function, makeOverlay,
that makes a function overlay. That is so you can make spell-check overlays
for as many modes as you want.

BTW, this appears that it will need some changes to work with
IPython3/Jupyter.

Hope that helps,

-Doug


On Sun, Aug 31, 2014 at 2:14 PM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

>
> Le 29 ao?t 2014 ? 18:38, Zolt?n V?r?s <zvoros at gmail.com> a ?crit :
>
> Hi Matthias,
>
> I have recently been wondering myself about this. What I wanted to do is
> insert the closing \end{environment} tags in the markdown cell, and
> insert e.g., \frac{}{} and move the cursor back to the first curly
> bracket, if the user types \fra, but syntax highlighting would also be
> useful. Could you point me to some sort of documentation as to how one
> starts out writing a plugin for this? I would be willing to work on this.
>
>
> Sorry for the short mail :
>
> http://codemirror.net/doc/manual.html#modeapi
>
> Have also a look at this part of code mirror doc :
>
> Sometimes, it is useful for modes to nest?to have one mode delegate work
> to another mode. An example of this kind of mode is the mixed-mode HTML
> mode. To implement such nesting, it is usually necessary to create mode
> objects and copy states yourself. To create a mode object, there
> are CodeMirror.getMode(options, parserConfig), where the first argument is
> a configuration object as passed to the mode constructor function, and the
> second argument is a mode specification as in the mode option. To copy a
> state object, call CodeMirror.copyState(mode, state), where mode is the
> mode that created the given state.
>
> ?
>
>
>
> Cheers,
> Zolt?n
>
> On 08/29/2014 06:05 PM, Matthias Bussonnier wrote:
>
> Le 29 ao?t 2014 ? 17:00, Klymak Jody <jklymak at gmail.com> a ?crit :
>
> Sorry if I've double posted this - I keep sending from the wrong account...
>
> Hi all,
>
> I am using a lot of <equation> tags in my markdown cells, and that works
> great for what I am writing.  However, is there a way to get latex-style
> highlighting in my cells rather than markdown (or some combination of
> both)?  As soon as you put an underscore in the cell, everything until the
> next underscore is rendered as oblique, which is not very easy to read.
>
> Someone would have to write a markdown+latex mode in code mirror that
> detect $/$$ et switch to latex.
> It would have the advantage of also highlighting latex correctly.
> But it need to be done.
>
> is not hard, but not trivial either.
> ?
> M
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140831/cbe1c524/attachment.html>