From vasco+python at tenner.nl  Mon Dec  1 07:47:31 2014
From: vasco+python at tenner.nl (Vasco)
Date: Mon, 01 Dec 2014 13:47:31 +0100
Subject: [IPython-dev] [IPython-User] Standard cells at the beginning of
 a new ipython notebook
In-Reply-To: <CAAipwu9_23hXRz_hKVSiYHyuM67hsgP6X1bvo1PfZY+6qTZyCA@mail.gmail.com>
References: <CAAipwu9_23hXRz_hKVSiYHyuM67hsgP6X1bvo1PfZY+6qTZyCA@mail.gmail.com>
Message-ID: <547C6363.9030104@tenner.nl>

Hi Kirstie,

On 01-12-14 12:48, Kirstie Whitaker wrote:
> Hi IPythoners,
>
> I have a few standard cells that I like to put at the beginning of each
> new notebook. Mostly they're standard imports and a few definitions that
> I use a lot. (For example the first cell is always: %load_ext
> autoreload, %autoreload 2, %matplotlib inline).
>
> Does anyone have any good suggestions for how to have this appear
> automatically in a new notebook? At the moment I just open an old one
> and copy over each cell individually....but it seems like maybe there's
> a better way....
One solution is to create a new template notebook, and copy that whole 
notebook around (In the filebrowser I use, I can select new 
file->Ipython notebook).

Vasco
Ps. The ipython-user list is not in use anymore. There is now only 1 
ipython userlist: ipython-dev at scipy.org


From jakevdp at cs.washington.edu  Mon Dec  1 10:04:10 2014
From: jakevdp at cs.washington.edu (Jacob Vanderplas)
Date: Mon, 1 Dec 2014 07:04:10 -0800
Subject: [IPython-dev] [IPython-User] Standard cells at the beginning of
 a new ipython notebook
In-Reply-To: <547C6363.9030104@tenner.nl>
References: <CAAipwu9_23hXRz_hKVSiYHyuM67hsgP6X1bvo1PfZY+6qTZyCA@mail.gmail.com>
	<547C6363.9030104@tenner.nl>
Message-ID: <CACpqBg2asgMihubuDcT3hd49L2NdKBGGNSKTN6c+BZoAg-CJnA@mail.gmail.com>

Hi,
Adrian Price-Whelan has a macrocell extension that does something like
this: https://github.com/adrn/macro-cell
  Jake

 Jake VanderPlas
 Director of Research ? Physical Sciences
 eScience Institute, University of Washington
 http://www.vanderplas.com

On Mon, Dec 1, 2014 at 4:47 AM, Vasco <vasco+python at tenner.nl> wrote:

> Hi Kirstie,
>
> On 01-12-14 12:48, Kirstie Whitaker wrote:
> > Hi IPythoners,
> >
> > I have a few standard cells that I like to put at the beginning of each
> > new notebook. Mostly they're standard imports and a few definitions that
> > I use a lot. (For example the first cell is always: %load_ext
> > autoreload, %autoreload 2, %matplotlib inline).
> >
> > Does anyone have any good suggestions for how to have this appear
> > automatically in a new notebook? At the moment I just open an old one
> > and copy over each cell individually....but it seems like maybe there's
> > a better way....
> One solution is to create a new template notebook, and copy that whole
> notebook around (In the filebrowser I use, I can select new
> file->Ipython notebook).
>
> Vasco
> Ps. The ipython-user list is not in use anymore. There is now only 1
> ipython userlist: ipython-dev at scipy.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141201/e8d75256/attachment.html>

From maplabs at light42.com  Tue Dec  2 20:19:09 2014
From: maplabs at light42.com (=?utf-8?b?QnJpYW4gTSBIYW1saW4=?=)
Date: Tue, 02 Dec 2014 17:19:09 -0800
Subject: [IPython-dev] misc debian packaging recomm.
Message-ID: <20141202171909.mjdsk9majo4k4ssk@webmail.light42.com>

    Hi All -

 ? we are ramping up our excellent Ubuntu-based Linux "distro" and reading the build log, I see this package list for the IPython install. 
 Any comments ? ? thanks in advance

 --
 LUbuntu 14.04 base build
 The following extra packages will be installed:?
 ? fonts-lyx libamd2.3.1 libarmadillo4 libarpack2 libblas3 libcamd2.3.1? libccolamd2.8.0 libcholmod2.1.2 libcolamd2.8.0 libdap11 libdapclient3? libepsilon1 libfreexl1 libgdal1h libgeos-3.4.2 libgeos-c1 libgfortran3? libhdf4-0-alt libhdf5-7 libkml0 liblapack3 libnetcdfc7 libodbc1 libogdi3.2? libpq5 libproj0 libspatialite5 libtcl8.6 libtk8.6 libumfpack5.6.2? liburiparser1 libxerces-c3.1 odbcinst odbcinst1debian2 proj-data? python-affine python-dateutil python-decorator python-gdal? python-matplotlib-data python-numpy python-pandas-lib python-pyparsing? python-scientific python-tz

 Suggested packages:? 
 ? libhdf4-doc libhdf4-alt-dev hdf4-tools libnetcdf4 libmyodbc odbc-postgresql? tdsodbc unixodbc-bin ogdi-bin tcl8.6 tk8.6 dvipng inkscape ipython? python-configobj python-excelerator python-matplotlib-doc python-nose? python-qt4 python-sip python-tornado python-traits python-wxgtk2.8? texlive-extra-utils texlive-latex-extra ttf-staypuft gfortran? python-numpy-dbg python-numpy-doc python-pandas-doc python-scientific-doc

 Recommended packages:? 
 ? proj-bin python-imaging python-tk python-tables python-numexpr python-xlrd? python-statsmodels python-openpyxl python-xlwt python-bs4 python-html5lib? pyro

 The following NEW packages will be installed:?
 ? fonts-lyx libamd2.3.1 libarmadillo4 libarpack2 libblas3 libcamd2.3.1? libccolamd2.8.0 libcholmod2.1.2 libcolamd2.8.0 libdap11 libdapclient3? libepsilon1 libfreexl1 libgdal1h libgeos-3.4.2 libgeos-c1 libgfortran3? libhdf4-0-alt libhdf5-7 libkml0 liblapack3 libnetcdfc7 libodbc1 libogdi3.2? libpq5 libproj0 libspatialite5 libtcl8.6 libtk8.6 libumfpack5.6.2? liburiparser1 libxerces-c3.1 odbcinst odbcinst1debian2 proj-data? python-affine python-dateutil python-decorator python-fiona python-gdal? python-matplotlib python-matplotlib-data python-netcdf python-numpy? python-pandas python-pandas-lib python-pyparsing python-rasterio? python-scientific python-scipy python-tz

 0 upgraded, 51 newly installed, 0 to remove and 3 not upgraded.<br><br>

 ps- I have a tool that reads all of this into a queryable database too, for all 50+ projects.. more info on request
 pps- I have taken the stance that pip install is "ok" or better, while others are not convinced..?

 --
Brian M Hamlin
OSGeo California Chapter
blog.light42.com

  


From takowl at gmail.com  Tue Dec  2 20:34:44 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 2 Dec 2014 17:34:44 -0800
Subject: [IPython-dev] misc debian packaging recomm.
In-Reply-To: <20141202171909.mjdsk9majo4k4ssk@webmail.light42.com>
References: <20141202171909.mjdsk9majo4k4ssk@webmail.light42.com>
Message-ID: <CAOvn4qhE211rr8aSNRvWnNqH7xAOPQFaDDWvsVRsEa=0q7K7AA@mail.gmail.com>

On 2 December 2014 at 17:19, Brian M Hamlin <maplabs at light42.com> wrote:

> we are ramping up our excellent Ubuntu-based Linux "distro" and reading
> the build log, I see this package list for the IPython install.
>  Any comments ?   thanks in advance
>
>  --
>  LUbuntu 14.04 base build
>  The following extra packages will be installed:
>    fonts-lyx libamd2.3.1 libarmadillo4 libarpack2 libblas3 libcamd2.3.1
> libccolamd2.8.0 libcholmod2.1.2 libcolamd2.8.0 libdap11 libdapclient3
> libepsilon1 libfreexl1 libgdal1h libgeos-3.4.2 libgeos-c1 libgfortran3
> libhdf4-0-alt libhdf5-7 libkml0 liblapack3 libnetcdfc7 libodbc1 libogdi3.2
> libpq5 libproj0 libspatialite5 libtcl8.6 libtk8.6 libumfpack5.6.2
> liburiparser1 libxerces-c3.1 odbcinst odbcinst1debian2 proj-data
> python-affine python-dateutil python-decorator python-gdal
> python-matplotlib-data python-numpy python-pandas-lib python-pyparsing
> python-scientific python-tz
>

How are you getting this? IPython shouldn't depend on all that stuff, and
Ubuntu does not appear to be doing anything strange with dependencies:
http://packages.ubuntu.com/trusty/ipython


>  pps- I have taken the stance that pip install is "ok" or better, while
> others are not convinced..
>

Apt is technically a much superior packaging system, but for anything
that's actively developed and released separately from the distro, it's
almost always out of date. Ubuntu 14.04 has IPython 1.2.1, while the
current release is 2.3.1, and we're looking to have 3.0 out some time early
in the new year. So I'd strongly recommend installing a newer version if
possible.

The distros' argument is that they want to give users a stable version, and
they'll backport bugfixes as necessary. But, especially if you're sticking
with Ubuntu LTS versions, that does mean the IPython packages get left way
behind the current state of development.

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141202/30ea85a5/attachment.html>

From vasco+python at tenner.nl  Wed Dec  3 06:37:48 2014
From: vasco+python at tenner.nl (Vasco)
Date: Wed, 03 Dec 2014 12:37:48 +0100
Subject: [IPython-dev] open IPython notebooks from file browser (Linux)
Message-ID: <547EF60C.9060607@tenner.nl>

Hi,
I wrote a little helper script to open ipynb files directly from my 
filebrowser (caja in my case). It assumes that there is already a 
notebookserver running, that can access the ipynb files.

It contains a .desktop file that calls a bash script. This script 
removes the "ipython notebook server root directory" from the file name, 
and opens a browser with the url pointing to the notebook.

Installation:
copy the desktop file to ~/.local/share/applications
copy the sh file to ~/bin/ and chmod +x

I realize this is a very crude approach, but it works for me.

Vasco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipython-notebook_here.sh
Type: application/x-shellscript
Size: 414 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/97acacf8/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipython-notebook_here.desktop
Type: application/x-desktop
Size: 268 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/97acacf8/attachment-0001.bin>

From vasco+python at tenner.nl  Wed Dec  3 09:38:52 2014
From: vasco+python at tenner.nl (Vasco)
Date: Wed, 03 Dec 2014 15:38:52 +0100
Subject: [IPython-dev] Interactive matplotlib figures in the IPython
	notebook
In-Reply-To: <CA+L60sCPGLPez5SX3OcFx2rv_0C=yqHUgVeYHmb9q=9mJ_Ti1A@mail.gmail.com>
References: <CA+L60sCPGLPez5SX3OcFx2rv_0C=yqHUgVeYHmb9q=9mJ_Ti1A@mail.gmail.com>
Message-ID: <547F207C.4070009@tenner.nl>

Dear Phil,

On 24-04-14 15:57, Phil Elson wrote:
> Cross posted to IPython-dev and mpl-dev.
>
> Over the Easter holidays I had a chance to take a look at implementing a
> new matplotlib backend which would allow interactive figures inline in
> the IPython notebook. It's something that has been on the radar for a
> couple of years now, with work needed from both projects to make the
> functionality possible, so I'm pleased to have been able to submit a PR
> (https://github.com/matplotlib/matplotlib/pull/3008) to matplotlib which
> finally adds the nbagg backend - the final piece of the jigsaw.

I just discovered that this nbagg backend is working! What a very nice 
tool! This was exactely what I was missing.

I didn't see comming along on the mailinglist that this was fixed. Thus 
by this, to draw everyones attention: try the nbagg plotting backend, 
simply by executing:
%matplotlib nbagg

Vasco


From darcamo at gmail.com  Wed Dec  3 09:47:53 2014
From: darcamo at gmail.com (darcamo at gmail.com)
Date: Wed, 03 Dec 2014 14:47:53 +0000
Subject: [IPython-dev] Interactive matplotlib figures in the IPython
	notebook
References: <CA+L60sCPGLPez5SX3OcFx2rv_0C=yqHUgVeYHmb9q=9mJ_Ti1A@mail.gmail.com>
	<547F207C.4070009@tenner.nl>
Message-ID: <CAGjSdXrwPjVwqXHHd7hbhZp91rBPu39XfUySZ=XfQgtTydgqAA@mail.gmail.com>

I just tried it with a plot I have that uses pick events. Basically when I
click on one of the points of a scatter plot the color of a few other
related points is changed. This works flawless in the IPython notebook with
the nbagg backend. It is indeed amazing (requires recent version of
matplotlib).

Em Wed Dec 03 2014 at 11:38:58, Vasco <vasco+python at tenner.nl> escreveu:

> Dear Phil,
>
> On 24-04-14 15:57, Phil Elson wrote:
> > Cross posted to IPython-dev and mpl-dev.
> >
> > Over the Easter holidays I had a chance to take a look at implementing a
> > new matplotlib backend which would allow interactive figures inline in
> > the IPython notebook. It's something that has been on the radar for a
> > couple of years now, with work needed from both projects to make the
> > functionality possible, so I'm pleased to have been able to submit a PR
> > (https://github.com/matplotlib/matplotlib/pull/3008) to matplotlib which
> > finally adds the nbagg backend - the final piece of the jigsaw.
>
> I just discovered that this nbagg backend is working! What a very nice
> tool! This was exactely what I was missing.
>
> I didn't see comming along on the mailinglist that this was fixed. Thus
> by this, to draw everyones attention: try the nbagg plotting backend,
> simply by executing:
> %matplotlib nbagg
>
> Vasco
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/9861dc60/attachment.html>

From j.wuttke at fz-juelich.de  Wed Dec  3 11:37:14 2014
From: j.wuttke at fz-juelich.de (Joachim Wuttke)
Date: Wed, 3 Dec 2014 17:37:14 +0100
Subject: [IPython-dev] How to parse a domain-specific language on top of
	Python
Message-ID: <547F3C3A.6050106@fz-juelich.de>

((cross-posting from http://stackoverflow.com/questions/27274607
   where this question gets close votes for being "too broad":))


I want to construct a domain-specific language as a superset of Python. 
Cryptic commands like

     f7:10y=x^2

designed to minimize typing shall be parsed into plain Python

     for k in range(7,10):
         f[k].set_y( expr='x^2' )

before being executed. Probably, the command-line interface shall be 
IPython.

What would be an appropriate architecture: Shall I implement the 
cryptic-to-plain-Python translation in the IPython command-line shell or 
in its kernel daemon? Are there helpful libraries / tutorials / examples?

Or more generically: Are there examples how to add complex syntactic 
sugar to Python?


Thanks in advance - Joachim


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5013 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/ef624d4f/attachment.bin>

From takowl at gmail.com  Wed Dec  3 11:37:56 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 3 Dec 2014 08:37:56 -0800
Subject: [IPython-dev] open IPython notebooks from file browser (Linux)
In-Reply-To: <547EF60C.9060607@tenner.nl>
References: <547EF60C.9060607@tenner.nl>
Message-ID: <CAOvn4qhSVAG5Qt2SNdoVUjp8S+FSXWqOu10TQzuzdnM83UhY+Q@mail.gmail.com>

Hi Vasco,

Have a look at my nbopen repo on Github, it's much the same idea, but it
doesn't need the server to already be running, and it works across
platforms.

Best wishes,
Thomas
On 3 Dec 2014 03:37, "Vasco" <vasco+python at tenner.nl> wrote:

> Hi,
> I wrote a little helper script to open ipynb files directly from my
> filebrowser (caja in my case). It assumes that there is already a
> notebookserver running, that can access the ipynb files.
>
> It contains a .desktop file that calls a bash script. This script removes
> the "ipython notebook server root directory" from the file name, and opens
> a browser with the url pointing to the notebook.
>
> Installation:
> copy the desktop file to ~/.local/share/applications
> copy the sh file to ~/bin/ and chmod +x
>
> I realize this is a very crude approach, but it works for me.
>
> Vasco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/495ccb72/attachment.html>

From doug.blank at gmail.com  Wed Dec  3 11:47:45 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Wed, 3 Dec 2014 11:47:45 -0500
Subject: [IPython-dev] How to parse a domain-specific language on top of
	Python
In-Reply-To: <547F3C3A.6050106@fz-juelich.de>
References: <547F3C3A.6050106@fz-juelich.de>
Message-ID: <CAAusYCjPvg+n2YW6HB4iszS2jUzdaEKrEoycdFn6qz8b-yymoA@mail.gmail.com>

On Wed, Dec 3, 2014 at 11:37 AM, Joachim Wuttke <j.wuttke at fz-juelich.de>
wrote:

> ((cross-posting from http://stackoverflow.com/questions/27274607
>   where this question gets close votes for being "too broad":))
>
>
> I want to construct a domain-specific language as a superset of Python.
> Cryptic commands like
>
>     f7:10y=x^2
>
> designed to minimize typing shall be parsed into plain Python
>
>     for k in range(7,10):
>         f[k].set_y( expr='x^2' )
>
> before being executed. Probably, the command-line interface shall be
> IPython.
>
> What would be an appropriate architecture: Shall I implement the
> cryptic-to-plain-Python translation in the IPython command-line shell or in
> its kernel daemon? Are there helpful libraries / tutorials / examples?
>
> Or more generically: Are there examples how to add complex syntactic sugar
> to Python?
>

You could handle this in a number of ways:

1) You could create your own language, parse it, and generate Python AST.
The Hy Lisp kernel (for IPython 3) does this, and turns it back into Python
so that all of the IPython stuff just works. Even shows the actual Python
code in the console history. See https://github.com/bollwyvl/hy_kernel

2) You could do this as a magic. You could use a simple parser to just
output regular Python. This is the least complex, and should be easier to
keep working over time.

3) You could implement your own parser written in Python as its own kernel
(google "wrapper kernel"). You can even have your own magics etc. (see
https://github.com/blink1073/metakernel)

4) I think that there are also syntactic transformers built into IPython...
that may be the way to go, but I don't have any info on that.

Good luck!

-Doug


>
>
> Thanks in advance - Joachim
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/6c537b6d/attachment.html>

From wes.turner at gmail.com  Wed Dec  3 11:48:32 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Wed, 3 Dec 2014 10:48:32 -0600
Subject: [IPython-dev] open IPython notebooks from file browser (Linux)
In-Reply-To: <547EF60C.9060607@tenner.nl>
References: <547EF60C.9060607@tenner.nl>
Message-ID: <CACfEFw8e3TGKVukZpB1drPrELsdZr4=EVg2adX0whvX86EhRPw@mail.gmail.com>

Great idea! This could be a helpful installation option.

https://github.com/westurner/dotfiles/blob/a384790eff/scripts/x-www-browser

This Python scripts should open one or more browser windows as:

    x-www-browser <url> [<url_n>]




On Wed, Dec 3, 2014 at 5:37 AM, Vasco <vasco+python at tenner.nl> wrote:

> Hi,
> I wrote a little helper script to open ipynb files directly from my
> filebrowser (caja in my case). It assumes that there is already a
> notebookserver running, that can access the ipynb files.
>
> It contains a .desktop file that calls a bash script. This script removes
> the "ipython notebook server root directory" from the file name, and opens
> a browser with the url pointing to the notebook.
>
> Installation:
> copy the desktop file to ~/.local/share/applications
> copy the sh file to ~/bin/ and chmod +x
>
> I realize this is a very crude approach, but it works for me.
>
> Vasco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/dbe616f0/attachment.html>

From j.wuttke at fz-juelich.de  Wed Dec  3 11:55:22 2014
From: j.wuttke at fz-juelich.de (Joachim Wuttke)
Date: Wed, 3 Dec 2014 17:55:22 +0100
Subject: [IPython-dev] How to parse a domain-specific language on top of
 Python
In-Reply-To: <CAAusYCjPvg+n2YW6HB4iszS2jUzdaEKrEoycdFn6qz8b-yymoA@mail.gmail.com>
References: <547F3C3A.6050106@fz-juelich.de>
	<CAAusYCjPvg+n2YW6HB4iszS2jUzdaEKrEoycdFn6qz8b-yymoA@mail.gmail.com>
Message-ID: <547F407A.9010708@fz-juelich.de>

Thank you, Doug!

Could you possibly give me a hint what you mean by "magic" under
(2) and (3) ?

- Joachim

> 2) You could do this as a magic. You could use a simple parser to just
> output regular Python. This is the least complex, and should be easier
> to keep working over time.
>
> 3) You could implement your own parser written in Python as its own
> kernel (google "wrapper kernel"). You can even have your own magics etc.
> (see https://github.com/blink1073/metakernel)


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5013 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/fa269cfb/attachment.bin>

From rjtskolasinski at gmail.com  Wed Dec  3 12:05:08 2014
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Wed, 3 Dec 2014 18:05:08 +0100
Subject: [IPython-dev] Using IPython Parallel inside a class
Message-ID: <CAAiurtdn3B1PLQkXWBnXCahTfDaW3HXG+e+LFRcMu+hA0wp8CA@mail.gmail.com>

Hi Guys,

I hope this is the right place to ask this kind of question.

I would like to use IPython parallel in my physical simulation.
In my mind I've got class Simulation, which contain as a field object
hamiltonian of class Hamiltonian.

Method get_energy should call  hamiltonian.get_matrix(momentum),
diagonalize it with scipy method and return energies.

I would like to execute it in parallel for a list of different momenta.


I tested it with a standard python 'map' function and it worked so I
thought that easiest way would be to use 'dview.map_sync' in the same way
but it doesn't looks like a proper way of doing it.

I am attaching my testing code. I would be very grateful for any help and
ideas how to do it in a nice and clean way.

Cheers,
Raphael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/21e7122d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: parallel_class.py
Type: text/x-python
Size: 1481 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/21e7122d/attachment.py>

From doug.blank at gmail.com  Wed Dec  3 12:08:51 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Wed, 3 Dec 2014 12:08:51 -0500
Subject: [IPython-dev] How to parse a domain-specific language on top of
	Python
In-Reply-To: <547F407A.9010708@fz-juelich.de>
References: <547F3C3A.6050106@fz-juelich.de>
	<CAAusYCjPvg+n2YW6HB4iszS2jUzdaEKrEoycdFn6qz8b-yymoA@mail.gmail.com>
	<547F407A.9010708@fz-juelich.de>
Message-ID: <CAAusYChakun_9q3_qVGz1EkXA5GThBvOD8MB1_-kFmhcDcsepA@mail.gmail.com>

On Wed, Dec 3, 2014 at 11:55 AM, Joachim Wuttke <j.wuttke at fz-juelich.de>
wrote:

> Thank you, Doug!
>
> Could you possibly give me a hint what you mean by "magic" under
> (2) and (3) ?


Sure! "Magics" are metacommands in IPython... things like "%lsmagic" will
list all of the magics. They can do lots of different things... almost
limitless. They can also be packaged up, and distributed independently. See
http://ipython.org/ipython-doc/dev/interactive/tutorial.html for use...
there may be developer docs for creating your own magic... don't know.

IPython is in the process of being split into two main parts: language
agnostic parts and IPython/Python specific parts. This allows other
languages to use the IPython infrastructure. It sounds like you want to
stay with Python, but even if you did, you could build your own version of
a kernel (for Python or another language) and then use a different magic
system (a language-agnostic version). That is what metakernel does.

-Doug


>
>
> - Joachim
>
>
>  2) You could do this as a magic. You could use a simple parser to just
>> output regular Python. This is the least complex, and should be easier
>> to keep working over time.
>>
>> 3) You could implement your own parser written in Python as its own
>> kernel (google "wrapper kernel"). You can even have your own magics etc.
>> (see https://github.com/blink1073/metakernel)
>>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/b96d2f7c/attachment.html>

From rjtskolasinski at gmail.com  Wed Dec  3 12:21:31 2014
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Wed, 3 Dec 2014 18:21:31 +0100
Subject: [IPython-dev] Using IPython Parallel inside a class
Message-ID: <CAAiurtf0B79pCxZBr=WoSof-zbHykUJ3+vF7fLfimFOk+ScuMA@mail.gmail.com>

Oh, in the file I attached there should ofc be line
spectrum = dview.map_sync(self.get_energy,momenta)
instead of
spectrum = dview.map_sync(self.get_energy_for_parallel,momenta)

Sorry for mistake.
Raphael

2014-12-03 18:05 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>:

> Hi Guys,
>
> I hope this is the right place to ask this kind of question.
>
> I would like to use IPython parallel in my physical simulation.
> In my mind I've got class Simulation, which contain as a field object
> hamiltonian of class Hamiltonian.
>
> Method get_energy should call  hamiltonian.get_matrix(momentum),
> diagonalize it with scipy method and return energies.
>
> I would like to execute it in parallel for a list of different momenta.
>
>
> I tested it with a standard python 'map' function and it worked so I
> thought that easiest way would be to use 'dview.map_sync' in the same way
> but it doesn't looks like a proper way of doing it.
>
> I am attaching my testing code. I would be very grateful for any help and
> ideas how to do it in a nice and clean way.
>
> Cheers,
> Raphael
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/0d42ab78/attachment.html>

From bussonniermatthias at gmail.com  Wed Dec  3 12:26:04 2014
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 3 Dec 2014 18:26:04 +0100
Subject: [IPython-dev] How to parse a domain-specific language on top of
	Python
In-Reply-To: <547F407A.9010708@fz-juelich.de>
References: <547F3C3A.6050106@fz-juelich.de>
	<CAAusYCjPvg+n2YW6HB4iszS2jUzdaEKrEoycdFn6qz8b-yymoA@mail.gmail.com>
	<547F407A.9010708@fz-juelich.de>
Message-ID: <6D600A7E-807E-4F7B-B639-30425275D533@gmail.com>


Le 3 d?c. 2014 ? 17:55, Joachim Wuttke a ?crit :

> Thank you, Doug!
> 
> Could you possibly give me a hint what you mean by "magic" under
> (2) and (3) ?

In both cases you need <your language>  to python converter. 
Magics  are a short syntactic sugar that allow you to transforme a string and do whatever. 
One of the use is to integrate with other language. 

Tipycally you can use 

%%ruby
puts 'hi' from ruby

(this one will call ruby in a subshell) 

or 

%%hy
(print "Hi from Hy")

And it will call hy for you (second one has integration with python)

Cf https://github.com/rossant/ipycache/blob/master/ipycache.py#L249-L289 for example as how to write a magic.

Nonetheless IPython itself does not provide anything to parse your language. 
So in the end what you want is write some (python) code that take a string as output and emit some python, 
after deciding in between a kernel and a maid is a detail.

Depending on your complexity you can try to produce either python code as string. 
Or do as Hy does and create an AST directly. I think there is an interesting talk as 
Pycon 2014 that I've some pointer on how they did this.

I'm sure this does not completely  help. 

Looking forward to et some news though, 
-- 
M

> 
> - Joachim
> 
>> 2) You could do this as a magic. You could use a simple parser to just
>> output regular Python. This is the least complex, and should be easier
>> to keep working over time.
>> 
>> 3) You could implement your own parser written in Python as its own
>> kernel (google "wrapper kernel"). You can even have your own magics etc.
>> (see https://github.com/blink1073/metakernel)
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/7d749faf/attachment.html>

From doug.blank at gmail.com  Wed Dec  3 12:28:57 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Wed, 3 Dec 2014 12:28:57 -0500
Subject: [IPython-dev] Using IPython Parallel inside a class
In-Reply-To: <CAAiurtf0B79pCxZBr=WoSof-zbHykUJ3+vF7fLfimFOk+ScuMA@mail.gmail.com>
References: <CAAiurtf0B79pCxZBr=WoSof-zbHykUJ3+vF7fLfimFOk+ScuMA@mail.gmail.com>
Message-ID: <CAAusYCjuw=434fmH3rD+Loc9ysKZYO7mAAW0iAUg5ZBsG7dadw@mail.gmail.com>

I haven't done much parallel IPython, but it looks like you just need to
define/import everything on the remote clients, something like:

def get_energy(momentum):
    import scipy.sparse.linalg as sla
    from scipy.sparse import coo_matrix
    from scipy.sparse import csc_matrix
    import numpy as np
    class Hamiltonian:
        def get_matrix(self, k):
            # sleep(3.0)
            return k**2 * np.diag(range(100))

    hamiltonian = Hamiltonian()
    mat = hamiltonian.get_matrix(momentum)
    mat = csc_matrix(mat)
    ev = sla.eigsh(mat,k=5, sigma = 0.5, which = 'LM')
    return ev[0]

def compute_parallel(momenta):
    rc = parallel.Client()
    dview = rc[:]
    spectrum = dview.map_sync(get_energy,momenta)
    return spectrum

-Doug

On Wed, Dec 3, 2014 at 12:21 PM, Rafa? Skolasi?ski <rjtskolasinski at gmail.com
> wrote:

> Oh, in the file I attached there should ofc be line
> spectrum = dview.map_sync(self.get_energy,momenta)
> instead of
> spectrum = dview.map_sync(self.get_energy_for_parallel,momenta)
>
> Sorry for mistake.
> Raphael
>
>
> 2014-12-03 18:05 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>:
>
>> Hi Guys,
>>
>> I hope this is the right place to ask this kind of question.
>>
>> I would like to use IPython parallel in my physical simulation.
>> In my mind I've got class Simulation, which contain as a field object
>> hamiltonian of class Hamiltonian.
>>
>> Method get_energy should call  hamiltonian.get_matrix(momentum),
>> diagonalize it with scipy method and return energies.
>>
>> I would like to execute it in parallel for a list of different momenta.
>>
>>
>> I tested it with a standard python 'map' function and it worked so I
>> thought that easiest way would be to use 'dview.map_sync' in the same way
>> but it doesn't looks like a proper way of doing it.
>>
>> I am attaching my testing code. I would be very grateful for any help and
>> ideas how to do it in a nice and clean way.
>>
>> Cheers,
>> Raphael
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/8eb85cd9/attachment.html>

From rjtskolasinski at gmail.com  Wed Dec  3 12:37:46 2014
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Wed, 3 Dec 2014 18:37:46 +0100
Subject: [IPython-dev] Using IPython Parallel inside a class
In-Reply-To: <CAAusYCjuw=434fmH3rD+Loc9ysKZYO7mAAW0iAUg5ZBsG7dadw@mail.gmail.com>
References: <CAAiurtf0B79pCxZBr=WoSof-zbHykUJ3+vF7fLfimFOk+ScuMA@mail.gmail.com>
	<CAAusYCjuw=434fmH3rD+Loc9ysKZYO7mAAW0iAUg5ZBsG7dadw@mail.gmail.com>
Message-ID: <CAAiurtc90doEi_nfrK-J7OT_wUPBry9-5YrEc_5PL=NVj3EEkQ@mail.gmail.com>

Hi,

It doesn't look like a solution. I still get bunch of errors. And I would
like to use self.hamiltonian instance, not create more instances of it. the
hamiltonian object in my real (not just a test) simulation code enters as
the argument to the Simulation class constructor.

And I would also like to execute hamiltonian.get_matrix method in parallel.
The version of that method I use in my real simulation code is a little
more complex and also take some time to be executed. Not maybe as much as
diagonalization but still.

Cheers,
Rafal

2014-12-03 18:28 GMT+01:00 Doug Blank <doug.blank at gmail.com>:

> I haven't done much parallel IPython, but it looks like you just need to
> define/import everything on the remote clients, something like:
>
> def get_energy(momentum):
>     import scipy.sparse.linalg as sla
>     from scipy.sparse import coo_matrix
>     from scipy.sparse import csc_matrix
>     import numpy as np
>     class Hamiltonian:
>         def get_matrix(self, k):
>             # sleep(3.0)
>             return k**2 * np.diag(range(100))
>
>     hamiltonian = Hamiltonian()
>     mat = hamiltonian.get_matrix(momentum)
>     mat = csc_matrix(mat)
>     ev = sla.eigsh(mat,k=5, sigma = 0.5, which = 'LM')
>     return ev[0]
>
> def compute_parallel(momenta):
>     rc = parallel.Client()
>     dview = rc[:]
>     spectrum = dview.map_sync(get_energy,momenta)
>     return spectrum
>
> -Doug
>
> On Wed, Dec 3, 2014 at 12:21 PM, Rafa? Skolasi?ski <
> rjtskolasinski at gmail.com> wrote:
>
>> Oh, in the file I attached there should ofc be line
>> spectrum = dview.map_sync(self.get_energy,momenta)
>> instead of
>> spectrum = dview.map_sync(self.get_energy_for_parallel,momenta)
>>
>> Sorry for mistake.
>> Raphael
>>
>>
>> 2014-12-03 18:05 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>:
>>
>>> Hi Guys,
>>>
>>> I hope this is the right place to ask this kind of question.
>>>
>>> I would like to use IPython parallel in my physical simulation.
>>> In my mind I've got class Simulation, which contain as a field object
>>> hamiltonian of class Hamiltonian.
>>>
>>> Method get_energy should call  hamiltonian.get_matrix(momentum),
>>> diagonalize it with scipy method and return energies.
>>>
>>> I would like to execute it in parallel for a list of different momenta.
>>>
>>>
>>> I tested it with a standard python 'map' function and it worked so I
>>> thought that easiest way would be to use 'dview.map_sync' in the same way
>>> but it doesn't looks like a proper way of doing it.
>>>
>>> I am attaching my testing code. I would be very grateful for any help
>>> and ideas how to do it in a nice and clean way.
>>>
>>> Cheers,
>>> Raphael
>>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/d3a7e383/attachment.html>

From rawlins at gmail.com  Wed Dec  3 12:44:04 2014
From: rawlins at gmail.com (Kyle Rawlins)
Date: Wed, 3 Dec 2014 12:44:04 -0500
Subject: [IPython-dev] IPython-dev Digest, Vol 131, Issue 2
In-Reply-To: <mailman.4323.1417624638.1087.ipython-dev@scipy.org>
References: <mailman.4323.1417624638.1087.ipython-dev@scipy.org>
Message-ID: <F4FBE2DC-9957-4185-8D4E-43A0737B54A8@gmail.com>


On Dec 3, 2014, at 11:37 AM, ipython-dev-request at scipy.org wrote:
> Date: Wed, 3 Dec 2014 17:37:14 +0100
> From: Joachim Wuttke <j.wuttke at fz-juelich.de>
> Subject: [IPython-dev] How to parse a domain-specific language on top
> 	of	Python
> To: <ipython-dev at scipy.org>
> Message-ID: <547F3C3A.6050106 at fz-juelich.de>
> Content-Type: text/plain; charset="utf-8"
> 
> ((cross-posting from http://stackoverflow.com/questions/27274607
>   where this question gets close votes for being "too broad":))
> 
> 
> I want to construct a domain-specific language as a superset of Python. 
> Cryptic commands like
> 
>     f7:10y=x^2
> 
> designed to minimize typing shall be parsed into plain Python
> 
>     for k in range(7,10):
>         f[k].set_y( expr='x^2' )
> 
> before being executed. Probably, the command-line interface shall be 
> IPython.
> 
> What would be an appropriate architecture: Shall I implement the 
> cryptic-to-plain-Python translation in the IPython command-line shell or 
> in its kernel daemon? Are there helpful libraries / tutorials / examples?

I think the most straightforward approach would be to do this kind of thing using IPython magics, probably in IPython notebook.  For an (alpha) example of this, you may find it helpful to look at my IPython Lambda Notebook project which uses cell magics to interface notebooks with a parser for a domain-specific mini-language (the domain is linguistics, in particular linguistic semantics, so the mini-language is a typed logic/lambda calculus).  The magics pass things to the parser, and variable assignments in the mini-language are injected back into the environment in a very straightforward way, so it?s easy to switch back and forth between the mini-language and python code.  I wouldn?t really recommend my approach to parsing and interpreting the mini-language, though, which is tailored to an extremely specific brand of scientific computing (and not for actually doing efficient parsing or computation).

https://github.com/rawlins/lambda-notebook

Less relevant to this mailing list but I?d also caution against literally targeting a superset of python, this will be really, really hard to get right.  At least start with a domain-specific language that is python-like but tractable.

Best,
-kyle

-- 
Kyle Rawlins
http://sites.krieger.jhu.edu/rawlins/
Department of Cognitive Science
Johns Hopkins University



From bussonniermatthias at gmail.com  Wed Dec  3 12:49:45 2014
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 3 Dec 2014 18:49:45 +0100
Subject: [IPython-dev] IPython-dev Digest, Vol 131, Issue 2
In-Reply-To: <F4FBE2DC-9957-4185-8D4E-43A0737B54A8@gmail.com>
References: <mailman.4323.1417624638.1087.ipython-dev@scipy.org>
	<F4FBE2DC-9957-4185-8D4E-43A0737B54A8@gmail.com>
Message-ID: <F756D44B-8989-45FB-ACFA-D591954115D4@gmail.com>


Le 3 d?c. 2014 ? 18:44, Kyle Rawlins a ?crit :

> I think the most straightforward approach would be to do this kind of thing using IPython magics, probably in IPython notebook.  For an (alpha) example of this, you may find it helpful to look at my IPython Lambda Notebook project which uses cell magics to interface notebooks with a parser for a domain-specific mini-language (the domain is linguistics, in particular linguistic semantics, so the mini-language is a typed logic/lambda calculus).  The magics pass things to the parser, and variable assignments in the mini-language are injected back into the environment in a very straightforward way, so it?s easy to switch back and forth between the mini-language and python code.  I wouldn?t really recommend my approach to parsing and interpreting the mini-language, though, which is tailored to an extremely specific brand of scientific computing (and not for actually doing efficient parsing or computation).
> 
> https://github.com/rawlins/lambda-notebook
> 
> Less relevant to this mailing list but I?d also caution against literally targeting a superset of python, this will be really, really hard to get right.  At least start with a domain-specific language that is python-like but tractable.

Looks cool !

Nbviewer link for the lazy:

http://nbviewer.ipython.org/github/rawlins/lambda-notebook/blob/master/notebooks/Lambda%20Notebook%20Intro%20%28start%20here%29.ipynb

-- 
M



> 
> Best,
> -kyle
> 
> -- 
> Kyle Rawlins
> http://sites.krieger.jhu.edu/rawlins/
> Department of Cognitive Science
> Johns Hopkins University
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/87924dcd/attachment.html>

From damontallen at gmail.com  Wed Dec  3 15:07:44 2014
From: damontallen at gmail.com (Damon Allen)
Date: Wed, 3 Dec 2014 15:07:44 -0500
Subject: [IPython-dev] Repr method hierarchy order
Message-ID: <CAMYKURYZHfidcBd2zSS4cHKdCyCTuYi8V9H10W8mAwQX8vEaog@mail.gmail.com>

Hi,

Firstly, thank you for all your hard work on IPython, nbconvert, Jupyter,
and everything else all of you have been working on.  I just have a quick
question, is the order of the repr method hierarchy being changed in the
development version of IPython?  The reason I ask is that I just wrote a
simple example to demonstrate some of IPython's repr options and when I'm
running the notebook locally, in IPython 2.3, _repr_latex_ is a higher
priority than _repr_svg_ and LaTex is displayed when both are available.
However, when I shared it on nbviewer the SVG repr is displayed rather than
the LaTex repr in the same location.

Notebook
<http://nbviewer.ipython.org/urls/gist.githubusercontent.com/damontallen/5e591948346f7ae8ee2f/raw/bfeae94552285fe1a3b72292afb3bddb623b1a2a/repr_options.ipynb>

Thank you,
Damon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/77bafc2a/attachment.html>

From sk.spiros at hotmail.com  Wed Dec  3 15:26:23 2014
From: sk.spiros at hotmail.com (Hack-The-Paradise)
Date: Wed, 3 Dec 2014 12:26:23 -0800 (PST)
Subject: [IPython-dev] How-To create persistent,
	interactive UI elements outside cells
In-Reply-To: <1417364813834-5079209.post@n6.nabble.com>
References: <1417183142119-5079080.post@n6.nabble.com>
	<3EB0767F-032E-4E0E-8E11-56586EB87A92@gmail.com>
	<1417208710660-5079115.post@n6.nabble.com>
	<13063C4B-1E4A-409F-8070-5B0012142FE3@gmail.com>
	<1417364813834-5079209.post@n6.nabble.com>
Message-ID: <1417638383572-5079562.post@n6.nabble.com>

bump!



--
View this message in context: http://python.6.x6.nabble.com/How-To-create-persistent-interactive-UI-elements-outside-cells-tp5079080p5079562.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From evelzw at gmail.com  Wed Dec  3 15:36:03 2014
From: evelzw at gmail.com (EvelynLiu)
Date: Wed, 3 Dec 2014 12:36:03 -0800 (PST)
Subject: [IPython-dev] How to select multiple items in widgets like
	RadioButtons
Message-ID: <1417638963059-5079566.post@n6.nabble.com>

Hi,

For IPython.html.widgets.RadioButtons, it says only one radio button can be
toggled at any point in time. However, I am trying to build a widget where
several items can be selected simultaneously, like: 

widgets.RadioButtons(
    description='Speed:',
    values=['fast', 'regular', 'slow'],
)

What should I do to toggle 'fast' and 'regular' at the same time? 

I noticed in matplotlib.widghets, there is one called CheckButtons. Any
widgets in ipython widgets working like this?

Thanks!



--
View this message in context: http://python.6.x6.nabble.com/How-to-select-multiple-items-in-widgets-like-RadioButtons-tp5079566.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From ccordoba12 at gmail.com  Wed Dec  3 16:07:26 2014
From: ccordoba12 at gmail.com (=?UTF-8?B?Q2FybG9zIEPDs3Jkb2Jh?=)
Date: Wed, 03 Dec 2014 16:07:26 -0500
Subject: [IPython-dev] How-To create persistent,
 interactive UI elements outside cells
In-Reply-To: <1417638383572-5079562.post@n6.nabble.com>
References: <1417183142119-5079080.post@n6.nabble.com>	<3EB0767F-032E-4E0E-8E11-56586EB87A92@gmail.com>	<1417208710660-5079115.post@n6.nabble.com>	<13063C4B-1E4A-409F-8070-5B0012142FE3@gmail.com>	<1417364813834-5079209.post@n6.nabble.com>
	<1417638383572-5079562.post@n6.nabble.com>
Message-ID: <547F7B8E.7060706@gmail.com>

Hi,

I found this repo yesterday. It could help you to solve your problem:

https://github.com/jdfreder/ipython-widgetmode

Cheers,
Carlos

El 03/12/14 a las 15:26, Hack-The-Paradise escribi?:
> bump!
>
>
>
> --
> View this message in context: http://python.6.x6.nabble.com/How-To-create-persistent-interactive-UI-elements-outside-cells-tp5079080p5079562.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/3806802b/attachment.html>

From doug.blank at gmail.com  Wed Dec  3 16:19:34 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Wed, 3 Dec 2014 16:19:34 -0500
Subject: [IPython-dev] Using IPython Parallel inside a class
In-Reply-To: <CAAiurtc90doEi_nfrK-J7OT_wUPBry9-5YrEc_5PL=NVj3EEkQ@mail.gmail.com>
References: <CAAiurtf0B79pCxZBr=WoSof-zbHykUJ3+vF7fLfimFOk+ScuMA@mail.gmail.com>
	<CAAusYCjuw=434fmH3rD+Loc9ysKZYO7mAAW0iAUg5ZBsG7dadw@mail.gmail.com>
	<CAAiurtc90doEi_nfrK-J7OT_wUPBry9-5YrEc_5PL=NVj3EEkQ@mail.gmail.com>
Message-ID: <CAAusYCgpBn6svjWA4QxgO05Lct7GhJPXDpaAAwviw3pMchyr_Q@mail.gmail.com>

On Wed, Dec 3, 2014 at 12:37 PM, Rafa? Skolasi?ski <rjtskolasinski at gmail.com
> wrote:

> Hi,
>
> It doesn't look like a solution. I still get bunch of errors. And I would
> like to use self.hamiltonian instance, not create more instances of it. the
> hamiltonian object in my real (not just a test) simulation code enters as
> the argument to the Simulation class constructor.
>

Your original code gave errors trying to serialize the Hamiltonian instance
in order to make copies for each machine in the cluster. Some objects can't
be serialized like that. In any event, each machine needs to have access to
the data that it will process. So, you need to somehow get that data (in
whole or in part) to each machine.

Hope that helps,

-Doug



>
> And I would also like to execute hamiltonian.get_matrix method in
> parallel. The version of that method I use in my real simulation code is a
> little more complex and also take some time to be executed. Not maybe as
> much as diagonalization but still.
>
> Cheers,
> Rafal
>
> 2014-12-03 18:28 GMT+01:00 Doug Blank <doug.blank at gmail.com>:
>
>> I haven't done much parallel IPython, but it looks like you just need to
>> define/import everything on the remote clients, something like:
>>
>> def get_energy(momentum):
>>     import scipy.sparse.linalg as sla
>>     from scipy.sparse import coo_matrix
>>     from scipy.sparse import csc_matrix
>>     import numpy as np
>>     class Hamiltonian:
>>         def get_matrix(self, k):
>>             # sleep(3.0)
>>             return k**2 * np.diag(range(100))
>>
>>     hamiltonian = Hamiltonian()
>>     mat = hamiltonian.get_matrix(momentum)
>>     mat = csc_matrix(mat)
>>     ev = sla.eigsh(mat,k=5, sigma = 0.5, which = 'LM')
>>     return ev[0]
>>
>> def compute_parallel(momenta):
>>     rc = parallel.Client()
>>     dview = rc[:]
>>     spectrum = dview.map_sync(get_energy,momenta)
>>     return spectrum
>>
>> -Doug
>>
>> On Wed, Dec 3, 2014 at 12:21 PM, Rafa? Skolasi?ski <
>> rjtskolasinski at gmail.com> wrote:
>>
>>> Oh, in the file I attached there should ofc be line
>>> spectrum = dview.map_sync(self.get_energy,momenta)
>>> instead of
>>> spectrum = dview.map_sync(self.get_energy_for_parallel,momenta)
>>>
>>> Sorry for mistake.
>>> Raphael
>>>
>>>
>>> 2014-12-03 18:05 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>:
>>>
>>>> Hi Guys,
>>>>
>>>> I hope this is the right place to ask this kind of question.
>>>>
>>>> I would like to use IPython parallel in my physical simulation.
>>>> In my mind I've got class Simulation, which contain as a field object
>>>> hamiltonian of class Hamiltonian.
>>>>
>>>> Method get_energy should call  hamiltonian.get_matrix(momentum),
>>>> diagonalize it with scipy method and return energies.
>>>>
>>>> I would like to execute it in parallel for a list of different momenta.
>>>>
>>>>
>>>> I tested it with a standard python 'map' function and it worked so I
>>>> thought that easiest way would be to use 'dview.map_sync' in the same way
>>>> but it doesn't looks like a proper way of doing it.
>>>>
>>>> I am attaching my testing code. I would be very grateful for any help
>>>> and ideas how to do it in a nice and clean way.
>>>>
>>>> Cheers,
>>>> Raphael
>>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141203/2967fc07/attachment.html>

From sk.spiros at hotmail.com  Wed Dec  3 18:50:32 2014
From: sk.spiros at hotmail.com (Hack-The-Paradise)
Date: Wed, 3 Dec 2014 15:50:32 -0800 (PST)
Subject: [IPython-dev] How-To create persistent,
	interactive UI elements outside cells
In-Reply-To: <547F7B8E.7060706@gmail.com>
References: <1417183142119-5079080.post@n6.nabble.com>
	<3EB0767F-032E-4E0E-8E11-56586EB87A92@gmail.com>
	<1417208710660-5079115.post@n6.nabble.com>
	<13063C4B-1E4A-409F-8070-5B0012142FE3@gmail.com>
	<1417364813834-5079209.post@n6.nabble.com>
	<1417638383572-5079562.post@n6.nabble.com>
	<547F7B8E.7060706@gmail.com>
Message-ID: <1417650632635-5079576.post@n6.nabble.com>

Thank you but this is not in the direction that my team and i are going...

I think it's in beta, probably abandoned and it's best case scenario is
this:
https://github.com/ipython-contrib/IPython-notebook-extensions/wiki/hide_input_all

As i explicitly described we aim to communicate with the kernel in order to
achieve what we want, maybe hiding specific cells would be an alternative
but it seems very primitive one...

I thought that this would be a lively topic but it also seems that the Devs
are not interested...

P.S.: I also found:
https://github.com/ipython-contrib/IPython-notebook-extensions/tree/master/testing/hierarchical_collapse
But i have not dig into to it yet...

Any thoughts would be really appreciated...



--
View this message in context: http://python.6.x6.nabble.com/How-To-create-persistent-interactive-UI-elements-outside-cells-tp5079080p5079576.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From mmckerns at caltech.edu  Wed Dec  3 20:58:02 2014
From: mmckerns at caltech.edu (Michael McKerns)
Date: Wed, 3 Dec 2014 20:58:02 -0500
Subject: [IPython-dev] Using IPython Parallel inside a class
In-Reply-To: <CAAusYCgpBn6svjWA4QxgO05Lct7GhJPXDpaAAwviw3pMchyr_Q@mail.gmail.com>
References: <CAAiurtf0B79pCxZBr=WoSof-zbHykUJ3+vF7fLfimFOk+ScuMA@mail.gmail.com>
	<CAAusYCjuw=434fmH3rD+Loc9ysKZYO7mAAW0iAUg5ZBsG7dadw@mail.gmail.com>
	<CAAiurtc90doEi_nfrK-J7OT_wUPBry9-5YrEc_5PL=NVj3EEkQ@mail.gmail.com>
	<CAAusYCgpBn6svjWA4QxgO05Lct7GhJPXDpaAAwviw3pMchyr_Q@mail.gmail.com>
Message-ID: <938dd195bbb412e9fbf137a27994f4ba.squirrel@webmail.caltech.edu>

Try replacing the serialization engine with `dill` or `cloudpickle`.
That's probably your best chance at serializing and shipping the
objects across the wire.  IPython.parallel lets you override the
serializer with either of these.


> On Wed, Dec 3, 2014 at 12:37 PM, Rafa? Skolasi?ski
> <rjtskolasinski at gmail.com
>> wrote:
>
>> Hi,
>>
>> It doesn't look like a solution. I still get bunch of errors. And I
>> would
>> like to use self.hamiltonian instance, not create more instances of it.
>> the
>> hamiltonian object in my real (not just a test) simulation code enters
>> as
>> the argument to the Simulation class constructor.
>>
>
> Your original code gave errors trying to serialize the Hamiltonian
> instance
> in order to make copies for each machine in the cluster. Some objects
> can't
> be serialized like that. In any event, each machine needs to have access
> to
> the data that it will process. So, you need to somehow get that data (in
> whole or in part) to each machine.
>
> Hope that helps,
>
> -Doug
>
>
>
>>
>> And I would also like to execute hamiltonian.get_matrix method in
>> parallel. The version of that method I use in my real simulation code is
>> a
>> little more complex and also take some time to be executed. Not maybe as
>> much as diagonalization but still.
>>
>> Cheers,
>> Rafal
>>
>> 2014-12-03 18:28 GMT+01:00 Doug Blank <doug.blank at gmail.com>:
>>
>>> I haven't done much parallel IPython, but it looks like you just need
>>> to
>>> define/import everything on the remote clients, something like:
>>>
>>> def get_energy(momentum):
>>>     import scipy.sparse.linalg as sla
>>>     from scipy.sparse import coo_matrix
>>>     from scipy.sparse import csc_matrix
>>>     import numpy as np
>>>     class Hamiltonian:
>>>         def get_matrix(self, k):
>>>             # sleep(3.0)
>>>             return k**2 * np.diag(range(100))
>>>
>>>     hamiltonian = Hamiltonian()
>>>     mat = hamiltonian.get_matrix(momentum)
>>>     mat = csc_matrix(mat)
>>>     ev = sla.eigsh(mat,k=5, sigma = 0.5, which = 'LM')
>>>     return ev[0]
>>>
>>> def compute_parallel(momenta):
>>>     rc = parallel.Client()
>>>     dview = rc[:]
>>>     spectrum = dview.map_sync(get_energy,momenta)
>>>     return spectrum
>>>
>>> -Doug
>>>
>>> On Wed, Dec 3, 2014 at 12:21 PM, Rafa? Skolasi?ski <
>>> rjtskolasinski at gmail.com> wrote:
>>>
>>>> Oh, in the file I attached there should ofc be line
>>>> spectrum = dview.map_sync(self.get_energy,momenta)
>>>> instead of
>>>> spectrum = dview.map_sync(self.get_energy_for_parallel,momenta)
>>>>
>>>> Sorry for mistake.
>>>> Raphael
>>>>
>>>>
>>>> 2014-12-03 18:05 GMT+01:00 Rafa? Skolasi?ski
>>>> <rjtskolasinski at gmail.com>:
>>>>
>>>>> Hi Guys,
>>>>>
>>>>> I hope this is the right place to ask this kind of question.
>>>>>
>>>>> I would like to use IPython parallel in my physical simulation.
>>>>> In my mind I've got class Simulation, which contain as a field object
>>>>> hamiltonian of class Hamiltonian.
>>>>>
>>>>> Method get_energy should call  hamiltonian.get_matrix(momentum),
>>>>> diagonalize it with scipy method and return energies.
>>>>>
>>>>> I would like to execute it in parallel for a list of different
>>>>> momenta.
>>>>>
>>>>>
>>>>> I tested it with a standard python 'map' function and it worked so I
>>>>> thought that easiest way would be to use 'dview.map_sync' in the same
>>>>> way
>>>>> but it doesn't looks like a proper way of doing it.
>>>>>
>>>>> I am attaching my testing code. I would be very grateful for any help
>>>>> and ideas how to do it in a nice and clean way.
>>>>>
>>>>> Cheers,
>>>>> Raphael
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


---

Mike McKerns
California Institute of Technology
TEL: (626)395-5773 or (626)590-8470
http://www.its.caltech.edu/~mmckerns
mmckerns at caltech.edu



From maplabs at light42.com  Wed Dec  3 21:09:37 2014
From: maplabs at light42.com (=?utf-8?b?QnJpYW4gTSBIYW1saW4=?=)
Date: Wed, 03 Dec 2014 18:09:37 -0800
Subject: [IPython-dev] misc debian packaging recomm.
Message-ID: <20141203180937.c8ervyx5ic088kss@webmail.light42.com>

    well, its a bit murky honestly.. I had a short discussion with one of our debian packagers and I was able to show three different sets of related packages using three different package system interogation methods (!)

 at any rate, a tool called ?apt-rdepends shows these:

 ? ct.light42 dot com / ?www_shared / ?ipy_deps / ?*pdf

 ? its probably not a huge topic for those in a pure-python world.. but it is actually how software lands in Ubuntu.. 

 I will look into it more and summarize .. thx

 --
Brian M Hamlin
OSGeo California Chapter
blog.light42.com

  


From vasco+python at tenner.nl  Thu Dec  4 04:30:12 2014
From: vasco+python at tenner.nl (Vasco)
Date: Thu, 04 Dec 2014 10:30:12 +0100
Subject: [IPython-dev] How-To create persistent,
 interactive UI elements outside cells
In-Reply-To: <1417650632635-5079576.post@n6.nabble.com>
References: <1417183142119-5079080.post@n6.nabble.com>	<3EB0767F-032E-4E0E-8E11-56586EB87A92@gmail.com>	<1417208710660-5079115.post@n6.nabble.com>	<13063C4B-1E4A-409F-8070-5B0012142FE3@gmail.com>	<1417364813834-5079209.post@n6.nabble.com>	<1417638383572-5079562.post@n6.nabble.com>	<547F7B8E.7060706@gmail.com>
	<1417650632635-5079576.post@n6.nabble.com>
Message-ID: <548029A4.7090606@tenner.nl>

Hi,

On 04-12-14 00:50, Hack-The-Paradise wrote:
> I thought that this would be a lively topic but it also seems that the Devs
> are not interested...

I have the feeling that Matthias gave a reasonable explanation of which 
things have to be used, in his email on nov 29. However, you probably 
have to do some work yourself...

Kind regards,
Vasco


From bussonniermatthias at gmail.com  Thu Dec  4 05:58:55 2014
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 4 Dec 2014 11:58:55 +0100
Subject: [IPython-dev] How-To create persistent,
	interactive UI elements outside cells
In-Reply-To: <1417650632635-5079576.post@n6.nabble.com>
References: <1417183142119-5079080.post@n6.nabble.com>
	<3EB0767F-032E-4E0E-8E11-56586EB87A92@gmail.com>
	<1417208710660-5079115.post@n6.nabble.com>
	<13063C4B-1E4A-409F-8070-5B0012142FE3@gmail.com>
	<1417364813834-5079209.post@n6.nabble.com>
	<1417638383572-5079562.post@n6.nabble.com>
	<547F7B8E.7060706@gmail.com>
	<1417650632635-5079576.post@n6.nabble.com>
Message-ID: <D541B7A9-D856-4830-B077-4550DE842AEA@gmail.com>

Hi, 

Le 4 d?c. 2014 ? 00:50, Hack-The-Paradise a ?crit :

> Thank you but this is not in the direction that my team and i are going...
> 
> I think it's in beta, probably abandoned and it's best case scenario is
> this:
> https://github.com/ipython-contrib/IPython-notebook-extensions/wiki/hide_input_all

This still forces you to ass code in cells input. 

> As i explicitly described we aim to communicate with the kernel in order to
> achieve what we want, maybe hiding specific cells would be an alternative
> but it seems very primitive one?

You might not have received my previous mail, so here is the archive version:

	

In it I gave you an example on how to load custom javascript in each notebook 
loaded from a specific server. As well as the ability to communicate with the kernel. 



> I thought that this would be a lively topic but it also seems that the Devs
> are not interested?

Did you investigate the user-expressions as I suggested you to ? It allows you to execute code just before/after
user code. 

Did you even achieved to get a side panel which is capable of executing 1+1 ? 
If you are not even there, the rest will not make much sense. 


As you now know how to execute your own string of code in the kernel, you are aware that you are actually sending a message
with registered callback.

You can also manually rester a callback for a specify message with `kernel.set_callback_for_message`.

In particular if you are not especially running from from a cell, manner.js:111-135 expect you to define the `iopub.get_cell` callback
(name might be unfortunate but is mostly returning cells-like object) that provide , in a nested propertyy a dom element.

You can get convinced of that by exec :

> IPython.notebook.get_selected_cell().widget_subarea

That return the Dom element of widgets.

"Just" define now the dom element in which you want your widget.


Here is another starting point that should move you closer to your goal. 


%%html
<div id='xxx'></div>
<div id='yyy'></div>

%%javascript
IPython.notebook.kernel.execute(
        'from IPython.html.widgets import interact\n'+
        '@interact\n'+
        'def fun(n=1):\n'+
        '    print(n)\n',     
          {
                shell : {
                  reply : function(data){console.log('reply',data.content)},
                  payload : {
                    set_next_input : function(data){console.log('sni',data)},
                    page : function(data){console.log('page',data)},
                  }
                },
                iopub : {
                    output : function(data){$("#yyy").append((data.content.data||{'text/plain':'text/plain'})['text/plain']); console.log('output',data.content)},
                    clear_output : function(data){console.log('cloup',data)},
                    get_cell: function(data){console.warn('got celllllll1111111!!!!'); return {widget_subarea:$('#xxx')}}
                    
                },
                input : function(data){console.log('rawin',data)}
          }
        , {silent: false, store_history: true})


It will at least I've you a slider in a custom div.
You will still have to hook up the custom update triggered by the changing value of the slider. 

Still I'm still not sure what you did as you don't safe code example beyond notebook.
and I'm not sure of what you want to achieve : 
	- display the same tools in a sidebar for all notebook. 
	- display tools that depends on which notebook are loaded in a sidebar.

-- 
M


> P.S.: I also found:
> https://github.com/ipython-contrib/IPython-notebook-extensions/tree/master/testing/hierarchical_collapse
> But i have not dig into to it yet...
> 
> Any thoughts would be really appreciated...
> 
> 
> 
> --
> View this message in context: http://python.6.x6.nabble.com/How-To-create-persistent-interactive-UI-elements-outside-cells-tp5079080p5079576.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From samuel.cozannet at canonical.com  Thu Dec  4 06:09:42 2014
From: samuel.cozannet at canonical.com (Samuel Cozannet)
Date: Thu, 4 Dec 2014 12:09:42 +0100
Subject: [IPython-dev] Deploying and using iPython Notebooks easily on the
	cloud
Message-ID: <CAHzBGdiwx1SoRtjNEDC2MJjMnvvuzz6mNtOAqfJg_vjkKCwsxg@mail.gmail.com>

Hi All,

Following the example of tmpnb that deploys notebooks in containers, I
wrote a charm for Juju
<https://demo.jujucharms.com/~samuel-cozannet/trusty/ipython-notebook-3/>
to easily consume iPython Notebooks on any cloud and/or locally (LXC on
Linux, Vagrant/Virtualbox for Mac)

You can now spin a private iPython Notebook VM on any public cloud or in a
container on your local machine using Juju. Install the client by following
https://jujucharms.com/docs/ (there are clients for MacOS, Ubuntu and
Windows), configure your Cloud API keys, and just do

*juju deploy cs:~samuel-cozannet/trusty/ipython-notebook*
*juju expose ipython-notebook*

Give it some time to deploy, then run

*juju status*

to find out the public IP address, and connect on https. The VM (hence the
notebook) will then be available for as long as you want. If you run it on
AWS you can use a free instance and run your notebook for free and for
ever.

If you want to get your notebooks files on your laptop from the command
line, you can do

*juju scp ipython-notebook/0:/home/ubuntu/*.pynb ./ *

which btw means you can implement a rsync strategy between the VM and your
laptop to keep in sync.

Feel free to contact me for further instructions or if you need assistance
to deploy. Any input for new features or feedback if I forgot libs or other
items more than welcome.
Please use either of

* https://github.com/SaMnCo/charm-ipython-notebook
*
https://code.launchpad.net/~samuel-cozannet/charms/trusty/ipython-notebook/trunk

to report those.

BTW, have a look at the other solutions we can deploy on
https://demo.jujucharms.com. I am interested in feedback, ideas for new
solutions... And obviously if you want to join the community, don't
hesitate to come back to me for training and more info.

Best,
Samuel

--
Samuel Cozannet
Cloud, Big Data and IoT Strategy Team
Strategic Program Manager
Changing the Future of Cloud
Ubuntu <http://ubuntu.com> / Canonical <http://canonical.com> UK LTD
samuel.cozannet at canonical.com
+33 616 702 389
Skype: samnco
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141204/50df2ced/attachment.html>

From maidos93 at laposte.net  Thu Dec  4 10:57:09 2014
From: maidos93 at laposte.net (thwiouz)
Date: Thu, 4 Dec 2014 07:57:09 -0800 (PST)
Subject: [IPython-dev] Parallel magic and exception
Message-ID: <1417708629805-5079639.post@n6.nabble.com>

Hi guys,

I'm currently testing the parallel mode of IPython for a project. My problem
is that I need to run a lot of 
simulations (for loop) for a Monte Carlo program with some code I wrote.
Because I had no idea about the 
computation cost of my program, I totally handled it into a notebook. Thus
everything is self contained 
(definition of new classes, functions and so on).
In my last cell I do my loop and then I get the result I want.

In order to parallelize my Notebook without modifying it a lot, I started to
read the doc and use the %autopx 
magic which works well. In order to do so, I first create a client, then a
view that I put into synchronous mode 
and then I execute my whole notebook.

At the end, I use the view dict to get my variables back to get the results
of the computations and everything 
should work well. Everything looks like okay, but I have got a f***ing error
in my code, well this is a 
RuntimeWarning (that I filtered to an 'error') for some of my engines.
Obviously, I do not handle the error 
right because the  engines that encountered the error stop without getting
any stack of what happened.

I saw something about CompositeError, but how do I manage it into a loop
into the engine side?

Thanks a lot,



--
View this message in context: http://python.6.x6.nabble.com/Parallel-magic-and-exception-tp5079639.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From jsw at fnal.gov  Thu Dec  4 11:56:57 2014
From: jsw at fnal.gov (Jon Wilson)
Date: Thu, 4 Dec 2014 10:56:57 -0600
Subject: [IPython-dev] Using parallel on a particular cluster
Message-ID: <54809259.1080509@fnal.gov>

Hi,
I would very much like to start using IPython.Parallel on our cluster. 
However, I'm having a hard time getting going.  Let me explain the 
structure of the cluster (as well as I understand it), and my guess 
about where and how things should be run and structured.  Then maybe 
somebody can help me fill in the gaps.

I have a machine that is owned by our research group, located on campus. 
  I have root on this machine, and can install and run anything I like 
(within reason).  No limits on time, CPU, or memory usage beyond the 
limits of the hardware itself.

The cluster has two login nodes, login01 and login02.  Processes on the 
login nodes are killed after about half an hour.  These machines are 
visible from outside the cluster.  Key-pair ssh authentication is 
disabled for some reason, and I think it would be quite a fight to get 
it enabled, even just for my account.

The cluster uses SLURM for scheduling, and jobs are submitted to a 
handful of queues from the login nodes.

The cluster has lots of compute nodes.  Processes that are idle for too 
long are killed.

Network access to most machines outside the cluster is prohibited.  It 
seems that some special cases are whitelisted.  You can talk to the 
login nodes, and a few other sites around the country.  Machines that 
are on-campus but not part of the cluster are NOT in general permitted, 
including our research group's machines.

I do have IPython installed on the cluster, via anaconda.



So, obviously I want to have engines running on the compute nodes.  I 
want the notebook server and the primary kernel to run on our group's 
machine, occasionally spinning up some engines and submitting stuff to 
them, as needed.  Then once the heavy lifting is complete and the 
results returned to the primary kernel, I would shut the engines down 
again and relinquish those compute nodes.

I think that the hub and the schedulers should also run on our group's 
machine.  The login node would be the obvious choice, but long-running 
processes are killed there, so I don't think that it will work.

The engines need to be able to talk to the hub and to the schedulers.  I 
suppose that ssh tunnels are probably the best way to do this.  Since 
our group's machine can't see the compute nodes, and the compute nodes 
aren't allowed to talk to our group's machine, I think I will have to 
request an exception be made to allow them to talk to our group's 
machine.  I hope that this will be granted.

Assuming that it is, the first thing the engines should do is to 
establish ssh tunnels (in both directions) to our machine.

Here is where I get a bit lost.  I don't know how to configure things. 
When I "start" the cluster, I guess the hub and schedulers can just 
start locally, so that's probably easy to configure.  To start the 
engines, I need to ssh to the login node and run "srun" with an 
appropriate SLURM script.  This ssh needs manual intervention: my 
password.  I'm guessing that this is harder to configure.  And where 
will my password be requested?  In the console where the notebook server 
is running?

Then, after waiting in the queue, the engines start one by one.  They 
make the ssh tunnels, and then what: do they attempt to contact the hub 
and schedulers?

Do the hub and schedulers wait for all the engines to become available 
before the cluster is ready to use?  Or can I start submitting work to 
engines as soon as they come online?


This looks like a really useful tool, but I'm struggling to figure out 
how to start using it.
Regards,
Jon


From fperez.net at gmail.com  Thu Dec  4 12:14:25 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 4 Dec 2014 09:14:25 -0800
Subject: [IPython-dev] Deploying and using iPython Notebooks easily on
	the cloud
In-Reply-To: <CAHzBGdiwx1SoRtjNEDC2MJjMnvvuzz6mNtOAqfJg_vjkKCwsxg@mail.gmail.com>
References: <CAHzBGdiwx1SoRtjNEDC2MJjMnvvuzz6mNtOAqfJg_vjkKCwsxg@mail.gmail.com>
Message-ID: <CAHAreOrsfXYEA-VtPF_gUokUFWY7mmEwi=KqcJO3ai4oZfXMKg@mail.gmail.com>

On Thu, Dec 4, 2014 at 3:09 AM, Samuel Cozannet <
samuel.cozannet at canonical.com> wrote:

> Following the example of tmpnb that deploys notebooks in containers, I
> wrote a charm for Juju
> <https://demo.jujucharms.com/~samuel-cozannet/trusty/ipython-notebook-3/>
> to easily consume iPython Notebooks on any cloud and/or locally (LXC on
> Linux, Vagrant/Virtualbox for Mac)
>

Great, thanks for sharing this! Very happy to see the cloud/infrastructure
options getting better.

Cheers

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141204/915f237c/attachment.html>

From rjtskolasinski at gmail.com  Thu Dec  4 12:43:43 2014
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Thu, 4 Dec 2014 18:43:43 +0100
Subject: [IPython-dev] Using IPython Parallel inside a class
In-Reply-To: <938dd195bbb412e9fbf137a27994f4ba.squirrel@webmail.caltech.edu>
References: <CAAiurtf0B79pCxZBr=WoSof-zbHykUJ3+vF7fLfimFOk+ScuMA@mail.gmail.com>
	<CAAusYCjuw=434fmH3rD+Loc9ysKZYO7mAAW0iAUg5ZBsG7dadw@mail.gmail.com>
	<CAAiurtc90doEi_nfrK-J7OT_wUPBry9-5YrEc_5PL=NVj3EEkQ@mail.gmail.com>
	<CAAusYCgpBn6svjWA4QxgO05Lct7GhJPXDpaAAwviw3pMchyr_Q@mail.gmail.com>
	<938dd195bbb412e9fbf137a27994f4ba.squirrel@webmail.caltech.edu>
Message-ID: <CAAiurtcgAjFo9C1_3MhykTHxfWdNHbnwTMZdgDGUTsSt7BhR=A@mail.gmail.com>

Hi All,

I found out that the "Hamiltonian" class I will be using is not good for
pickling, therefore I used little different approach to define my solver as
a separate class (that just gets the momentum value) and create object of
it for every run separately.

My solution:
http://www.codesend.com/view/ed72e945804613a914dbddb0d9d23e06/
http://www.codesend.com/view/87df0f95e98e812398e444750784c907/


I even included passing some additional parameters as a default argument to
the function.

If someone would have better solution I will appreciate any feedback.

Cheers,
Rafal

2014-12-04 2:58 GMT+01:00 Michael McKerns <mmckerns at caltech.edu>:

> Try replacing the serialization engine with `dill` or `cloudpickle`.
> That's probably your best chance at serializing and shipping the
> objects across the wire.  IPython.parallel lets you override the
> serializer with either of these.
>
>
> > On Wed, Dec 3, 2014 at 12:37 PM, Rafa? Skolasi?ski
> > <rjtskolasinski at gmail.com
> >> wrote:
> >
> >> Hi,
> >>
> >> It doesn't look like a solution. I still get bunch of errors. And I
> >> would
> >> like to use self.hamiltonian instance, not create more instances of it.
> >> the
> >> hamiltonian object in my real (not just a test) simulation code enters
> >> as
> >> the argument to the Simulation class constructor.
> >>
> >
> > Your original code gave errors trying to serialize the Hamiltonian
> > instance
> > in order to make copies for each machine in the cluster. Some objects
> > can't
> > be serialized like that. In any event, each machine needs to have access
> > to
> > the data that it will process. So, you need to somehow get that data (in
> > whole or in part) to each machine.
> >
> > Hope that helps,
> >
> > -Doug
> >
> >
> >
> >>
> >> And I would also like to execute hamiltonian.get_matrix method in
> >> parallel. The version of that method I use in my real simulation code is
> >> a
> >> little more complex and also take some time to be executed. Not maybe as
> >> much as diagonalization but still.
> >>
> >> Cheers,
> >> Rafal
> >>
> >> 2014-12-03 18:28 GMT+01:00 Doug Blank <doug.blank at gmail.com>:
> >>
> >>> I haven't done much parallel IPython, but it looks like you just need
> >>> to
> >>> define/import everything on the remote clients, something like:
> >>>
> >>> def get_energy(momentum):
> >>>     import scipy.sparse.linalg as sla
> >>>     from scipy.sparse import coo_matrix
> >>>     from scipy.sparse import csc_matrix
> >>>     import numpy as np
> >>>     class Hamiltonian:
> >>>         def get_matrix(self, k):
> >>>             # sleep(3.0)
> >>>             return k**2 * np.diag(range(100))
> >>>
> >>>     hamiltonian = Hamiltonian()
> >>>     mat = hamiltonian.get_matrix(momentum)
> >>>     mat = csc_matrix(mat)
> >>>     ev = sla.eigsh(mat,k=5, sigma = 0.5, which = 'LM')
> >>>     return ev[0]
> >>>
> >>> def compute_parallel(momenta):
> >>>     rc = parallel.Client()
> >>>     dview = rc[:]
> >>>     spectrum = dview.map_sync(get_energy,momenta)
> >>>     return spectrum
> >>>
> >>> -Doug
> >>>
> >>> On Wed, Dec 3, 2014 at 12:21 PM, Rafa? Skolasi?ski <
> >>> rjtskolasinski at gmail.com> wrote:
> >>>
> >>>> Oh, in the file I attached there should ofc be line
> >>>> spectrum = dview.map_sync(self.get_energy,momenta)
> >>>> instead of
> >>>> spectrum = dview.map_sync(self.get_energy_for_parallel,momenta)
> >>>>
> >>>> Sorry for mistake.
> >>>> Raphael
> >>>>
> >>>>
> >>>> 2014-12-03 18:05 GMT+01:00 Rafa? Skolasi?ski
> >>>> <rjtskolasinski at gmail.com>:
> >>>>
> >>>>> Hi Guys,
> >>>>>
> >>>>> I hope this is the right place to ask this kind of question.
> >>>>>
> >>>>> I would like to use IPython parallel in my physical simulation.
> >>>>> In my mind I've got class Simulation, which contain as a field object
> >>>>> hamiltonian of class Hamiltonian.
> >>>>>
> >>>>> Method get_energy should call  hamiltonian.get_matrix(momentum),
> >>>>> diagonalize it with scipy method and return energies.
> >>>>>
> >>>>> I would like to execute it in parallel for a list of different
> >>>>> momenta.
> >>>>>
> >>>>>
> >>>>> I tested it with a standard python 'map' function and it worked so I
> >>>>> thought that easiest way would be to use 'dview.map_sync' in the same
> >>>>> way
> >>>>> but it doesn't looks like a proper way of doing it.
> >>>>>
> >>>>> I am attaching my testing code. I would be very grateful for any help
> >>>>> and ideas how to do it in a nice and clean way.
> >>>>>
> >>>>> Cheers,
> >>>>> Raphael
> >>>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> IPython-dev mailing list
> >>>> IPython-dev at scipy.org
> >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>>>
> >>>>
> >>>
> >>> _______________________________________________
> >>> IPython-dev mailing list
> >>> IPython-dev at scipy.org
> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>>
> >>>
> >>
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> >>
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
> ---
>
> Mike McKerns
> California Institute of Technology
> TEL: (626)395-5773 or (626)590-8470
> http://www.its.caltech.edu/~mmckerns
> mmckerns at caltech.edu
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141204/92ed58b1/attachment.html>

From andrew.gibiansky at gmail.com  Thu Dec  4 14:41:58 2014
From: andrew.gibiansky at gmail.com (Andrew Gibiansky)
Date: Thu, 4 Dec 2014 11:41:58 -0800
Subject: [IPython-dev] Create New Notebook from Command-line
Message-ID: <CAF-LYKLE8eZkGer+8eNmDTYoWrcc=5AeEn9TxKe334mHNd3LFw@mail.gmail.com>

Hey all,

I like to use `ipython notebook MyNotebook.ipynb` to open an ipynb file.
However, if I use this on a file that doesn't exist, I get:

2014-12-04 11:39:39.952 [NotebookApp] CRITICAL | No such file or directory:
/Users/person/MyNotebook.ipynb

How can I programmatically create a new notebook so that `ipython notebook
TheNB.ipynb` will work? This must be either command-line commands or Python
code, but looking into ipython flags there doesn't seem to be anything.
Does anyone have any suggestions? ("Press the button" does not work in my
case)

thanks!
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141204/a03df181/attachment.html>

From benjaminrk at gmail.com  Thu Dec  4 15:43:33 2014
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 4 Dec 2014 12:43:33 -0800
Subject: [IPython-dev] Parallel magic and exception
In-Reply-To: <1417708629805-5079639.post@n6.nabble.com>
References: <1417708629805-5079639.post@n6.nabble.com>
Message-ID: <CAHNn8BUeJEo97ehMz2o-wgLKWVXw=K0Ogwc1nHxm5T9E3nhfkA@mail.gmail.com>

Do the engine processes actually die, or does the particular execution
stop? Can you provide code that reproduces the error?

CompositeError is a wrapper class on the Client for representing multiple
Exceptions that have been raised on the engines. It is raised when the
Client tries to get a result of multiple executions, and one or more of
them fail, e.g. in blocking %px, or AsyncResult.get.

-MinRK
?

On Thu, Dec 4, 2014 at 7:57 AM, thwiouz <maidos93 at laposte.net> wrote:

> Hi guys,
>
> I'm currently testing the parallel mode of IPython for a project. My
> problem
> is that I need to run a lot of
> simulations (for loop) for a Monte Carlo program with some code I wrote.
> Because I had no idea about the
> computation cost of my program, I totally handled it into a notebook. Thus
> everything is self contained
> (definition of new classes, functions and so on).
> In my last cell I do my loop and then I get the result I want.
>
> In order to parallelize my Notebook without modifying it a lot, I started
> to
> read the doc and use the %autopx
> magic which works well. In order to do so, I first create a client, then a
> view that I put into synchronous mode
> and then I execute my whole notebook.
>
> At the end, I use the view dict to get my variables back to get the results
> of the computations and everything
> should work well. Everything looks like okay, but I have got a f***ing
> error
> in my code, well this is a
> RuntimeWarning (that I filtered to an 'error') for some of my engines.
> Obviously, I do not handle the error
> right because the  engines that encountered the error stop without getting
> any stack of what happened.
>
> I saw something about CompositeError, but how do I manage it into a loop
> into the engine side?
>
> Thanks a lot,
>
>
>
> --
> View this message in context:
> http://python.6.x6.nabble.com/Parallel-magic-and-exception-tp5079639.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141204/0e08ef75/attachment.html>

From maidos93 at laposte.net  Fri Dec  5 03:35:47 2014
From: maidos93 at laposte.net (thwiouz)
Date: Fri, 5 Dec 2014 00:35:47 -0800 (PST)
Subject: [IPython-dev] Parallel magic and exception
In-Reply-To: <CAHNn8BUeJEo97ehMz2o-wgLKWVXw=K0Ogwc1nHxm5T9E3nhfkA@mail.gmail.com>
References: <1417708629805-5079639.post@n6.nabble.com>
	<CAHNn8BUeJEo97ehMz2o-wgLKWVXw=K0Ogwc1nHxm5T9E3nhfkA@mail.gmail.com>
Message-ID: <1417768547912-5079704.post@n6.nabble.com>

Well, it seems the process in the engine stops: when I look at the task
manager, there is a jump of 1 process (100% -> ~80%). I could see a
RuntimeWarning error in the log in the window of the engine process.

Well, in order to provide the code I should post the notebook because as I
said, I've defined a lot of personal things.
But basically the code is the following:

---------------------------------------
from IPython import parallel

rc = parallel.Client()
dview = rc[:]
dview.block = True
dview.activate()
---------------------------------------
%autopx
---------------------------------------
# Definition of many classes I wrote bla bla
# and instanciation of a lot of objects in
# all engines because of autopx
---------------------------------------
# My big loop for that uses the different objects
# I created
---------------------------------------
%autopx
---------------------------------------
# And then, I'm in the client side, so I can handle the results
# But some of my engines began KO, so I stopped the execution here

I hope it could help,

Cheers,



--
View this message in context: http://python.6.x6.nabble.com/Parallel-magic-and-exception-tp5079639p5079704.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From bussonniermatthias at gmail.com  Fri Dec  5 03:57:54 2014
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Fri, 5 Dec 2014 09:57:54 +0100
Subject: [IPython-dev] Create New Notebook from Command-line
In-Reply-To: <CAF-LYKLE8eZkGer+8eNmDTYoWrcc=5AeEn9TxKe334mHNd3LFw@mail.gmail.com>
References: <CAF-LYKLE8eZkGer+8eNmDTYoWrcc=5AeEn9TxKe334mHNd3LFw@mail.gmail.com>
Message-ID: <1D45DCA3-4329-4AB2-91E7-D017213C3810@gmail.com>

Hi Andrew. 

Base on a Fernando example would the following python snippet suit you ?

import IPython.nbformat as nbf
nb = nbf.v4.new_notebook()
text = "This is an auto-generated notebook"
nb['cells'] = [ nbf.v4.new_markdown_cell(text)]
nb['cells'].append(nbf.v4.new_code_cell('#'+text))
fname = 'test.ipynb'

with open(fname, 'w') as f:
    nbf.write(nb, f, 4)

-- 
M

Le 4 d?c. 2014 ? 20:41, Andrew Gibiansky a ?crit :

> Hey all,
> 
> I like to use `ipython notebook MyNotebook.ipynb` to open an ipynb file. However, if I use this on a file that doesn't exist, I get: 
> 
> 2014-12-04 11:39:39.952 [NotebookApp] CRITICAL | No such file or directory: /Users/person/MyNotebook.ipynb
> 
> How can I programmatically create a new notebook so that `ipython notebook TheNB.ipynb` will work? This must be either command-line commands or Python code, but looking into ipython flags there doesn't seem to be anything. Does anyone have any suggestions? ("Press the button" does not work in my case)
> 
> thanks!
> Andrew
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From lucienboillod at gmail.com  Fri Dec  5 08:34:54 2014
From: lucienboillod at gmail.com (Lucien Boillod)
Date: Fri, 5 Dec 2014 14:34:54 +0100
Subject: [IPython-dev] change or interact with the html into ipython profile
Message-ID: <24B5373E-9703-49E0-92DD-DB8F86BDADDA@gmail.com>

Hello,

I currently create a custom ipython notebook profile, and I wonder if there is a way to interact with the source html.
Basically I wanted to change the href of the title ( which is "/tree/? by default ) so I tried to do it with the custom.js like that:

$('.ipython_notebook').each(function() {
    var link = $(this).html();
    $(this).contents().wrap('<a href="www.google.com"></a>');
});

But it seems not work, it not changing the html at all.
Do someone as a method to do that and interact with the html ??

Thank you,

Cheers,

Lucien Boillod

From bussonniermatthias at gmail.com  Fri Dec  5 09:42:11 2014
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Fri, 5 Dec 2014 15:42:11 +0100
Subject: [IPython-dev] change or interact with the html into ipython
	profile
In-Reply-To: <24B5373E-9703-49E0-92DD-DB8F86BDADDA@gmail.com>
References: <24B5373E-9703-49E0-92DD-DB8F86BDADDA@gmail.com>
Message-ID: <9DFC85F2-A877-46AF-8876-EBDD42953C92@gmail.com>


Le 5 d?c. 2014 ? 14:34, Lucien Boillod a ?crit :

> Hello,
> 
> I currently create a custom ipython notebook profile, and I wonder if there is a way to interact with the source html.
> Basically I wanted to change the href of the title ( which is "/tree/? by default ) so I tried to do it with the custom.js like that:
> 
> $('.ipython_notebook').each(function() {
>    var link = $(this).html();
>    $(this).contents().wrap('<a href="www.google.com"></a>');
> });
> 

Not sure what you want to achieve, and what you call 'title'  The IPython logo ?

$('#ipython_notebook >a').attr('href','//www.google.com') 

??

-- 
M


> But it seems not work, it not changing the html at all.
> Do someone as a method to do that and interact with the html ??
> 
> Thank you,
> 
> Cheers,
> 
> Lucien Boillod
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141205/c775e464/attachment.html>

From wes.turner at gmail.com  Fri Dec  5 14:06:25 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Fri, 5 Dec 2014 13:06:25 -0600
Subject: [IPython-dev] Create New Notebook from Command-line
In-Reply-To: <1D45DCA3-4329-4AB2-91E7-D017213C3810@gmail.com>
References: <CAF-LYKLE8eZkGer+8eNmDTYoWrcc=5AeEn9TxKe334mHNd3LFw@mail.gmail.com>
	<1D45DCA3-4329-4AB2-91E7-D017213C3810@gmail.com>
Message-ID: <CACfEFw8E_arH7OSozoWzO4HCyQbJA9YGhpTpY_WCVnOTbkZCUg@mail.gmail.com>

On Fri, Dec 5, 2014 at 2:57 AM, Matthias BUSSONNIER <
bussonniermatthias at gmail.com> wrote:

> Hi Andrew.
>
> Base on a Fernando example would the following python snippet suit you ?
>
> import IPython.nbformat as nbf
> nb = nbf.v4.new_notebook()
> text = "This is an auto-generated notebook"
> nb['cells'] = [ nbf.v4.new_markdown_cell(text)]
> nb['cells'].append(nbf.v4.new_code_cell('#'+text))
> fname = 'test.ipynb'
>
> with open(fname, 'w') as f:
>     nbf.write(nb, f, 4)


1. This could be a useful commandline option.

2. You could accomplish the same with a
https://github.com/audreyr/cookiecutter jinja2 project template.

    * There could be variations on such a template e.g. with default
headings for:
      *
https://en.wikipedia.org/wiki/Scientific_method#Elements_of_the_scientific_method
      * { Abstract, Question, Hypothesis, Experiment, Observations/Data,
Analysis, Conclusion, ... }
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141205/7d8705de/attachment.html>

From wes.turner at gmail.com  Fri Dec  5 14:16:22 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Fri, 5 Dec 2014 13:16:22 -0600
Subject: [IPython-dev] A Reproducible IPython project template (was: Create
 New Notebook from Command-line)
Message-ID: <CACfEFw-+tONV1WcgSXrsKqLZOtgfS3ee4+9ZFMc_W8YrMuuotg@mail.gmail.com>

On Fri, Dec 5, 2014 at 1:06 PM, Wes Turner <wes.turner at gmail.com> wrote:

>
>
> On Fri, Dec 5, 2014 at 2:57 AM, Matthias BUSSONNIER <
> bussonniermatthias at gmail.com> wrote:
>
>> Hi Andrew.
>>
>> Base on a Fernando example would the following python snippet suit you ?
>>
>> import IPython.nbformat as nbf
>> nb = nbf.v4.new_notebook()
>> text = "This is an auto-generated notebook"
>> nb['cells'] = [ nbf.v4.new_markdown_cell(text)]
>> nb['cells'].append(nbf.v4.new_code_cell('#'+text))
>> fname = 'test.ipynb'
>>
>> with open(fname, 'w') as f:
>>     nbf.write(nb, f, 4)
>
>
> 1. This could be a useful commandline option.
>
> 2. You could accomplish the same with a
> https://github.com/audreyr/cookiecutter jinja2 project template.
>
>     * There could be variations on such a template e.g. with default
> headings for:
>       *
> https://en.wikipedia.org/wiki/Scientific_method#Elements_of_the_scientific_method
>       * { Abstract, Question, Hypothesis, Experiment, Observations/Data,
> Analysis, Conclusion, ... }
>

Really, a reproducible project template could have:

* folders for e.g. ./data , ./scripts , ./notebooks {./lib for vendored
libraries)
* a setup.py supporting a necessary directory structure
* a Makefile (that generates an index with links to nbviewer)
  * https://github.com/westurner/notebooks/blob/gh-pages/Makefile
  * https://github.com/westurner/notebooks/blob/gh-pages/makeindex.py
* an install.sh script
  * (miniconda; conda create -n py27 python readline pip)
  * https://github.com/westurner/notebooks/blob/gh-pages/install.sh
* an extension (that I still believe should be included with IPython) that
lists the versions of all installed (on the import path) and utilized
libraries and extensions
  * https://github.com/ipython/ipython/issues/2597
  * https://github.com/rasbt/watermark
  * https://github.com/jrjohansson/version_information
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141205/b93fc307/attachment.html>

From andrew.gibiansky at gmail.com  Fri Dec  5 14:25:10 2014
From: andrew.gibiansky at gmail.com (Andrew Gibiansky)
Date: Fri, 5 Dec 2014 11:25:10 -0800
Subject: [IPython-dev] Create New Notebook from Command-line
In-Reply-To: <CACfEFw8E_arH7OSozoWzO4HCyQbJA9YGhpTpY_WCVnOTbkZCUg@mail.gmail.com>
References: <CAF-LYKLE8eZkGer+8eNmDTYoWrcc=5AeEn9TxKe334mHNd3LFw@mail.gmail.com>
	<1D45DCA3-4329-4AB2-91E7-D017213C3810@gmail.com>
	<CACfEFw8E_arH7OSozoWzO4HCyQbJA9YGhpTpY_WCVnOTbkZCUg@mail.gmail.com>
Message-ID: <CAF-LYK+uaG_Bf6+ZCvtLBq_E2OkB7K-uStyKvj6D711ViM43cw@mail.gmail.com>

Matthias,

That's perfect and exactly the sort of thing I was looking for. I looked
around the IPython API but didn't quite get to the code snippet you showed.

Thanks!

On Fri, Dec 5, 2014 at 11:06 AM, Wes Turner <wes.turner at gmail.com> wrote:

>
>
> On Fri, Dec 5, 2014 at 2:57 AM, Matthias BUSSONNIER <
> bussonniermatthias at gmail.com> wrote:
>
>> Hi Andrew.
>>
>> Base on a Fernando example would the following python snippet suit you ?
>>
>> import IPython.nbformat as nbf
>> nb = nbf.v4.new_notebook()
>> text = "This is an auto-generated notebook"
>> nb['cells'] = [ nbf.v4.new_markdown_cell(text)]
>> nb['cells'].append(nbf.v4.new_code_cell('#'+text))
>> fname = 'test.ipynb'
>>
>> with open(fname, 'w') as f:
>>     nbf.write(nb, f, 4)
>
>
> 1. This could be a useful commandline option.
>
> 2. You could accomplish the same with a
> https://github.com/audreyr/cookiecutter jinja2 project template.
>
>     * There could be variations on such a template e.g. with default
> headings for:
>       *
> https://en.wikipedia.org/wiki/Scientific_method#Elements_of_the_scientific_method
>       * { Abstract, Question, Hypothesis, Experiment, Observations/Data,
> Analysis, Conclusion, ... }
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141205/1a0c9a53/attachment.html>

From fperez.net at gmail.com  Fri Dec  5 16:34:31 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 5 Dec 2014 13:34:31 -0800
Subject: [IPython-dev] Create New Notebook from Command-line
In-Reply-To: <CAF-LYK+uaG_Bf6+ZCvtLBq_E2OkB7K-uStyKvj6D711ViM43cw@mail.gmail.com>
References: <CAF-LYKLE8eZkGer+8eNmDTYoWrcc=5AeEn9TxKe334mHNd3LFw@mail.gmail.com>
	<1D45DCA3-4329-4AB2-91E7-D017213C3810@gmail.com>
	<CACfEFw8E_arH7OSozoWzO4HCyQbJA9YGhpTpY_WCVnOTbkZCUg@mail.gmail.com>
	<CAF-LYK+uaG_Bf6+ZCvtLBq_E2OkB7K-uStyKvj6D711ViM43cw@mail.gmail.com>
Message-ID: <CAHAreOoV1Y6-qQoT7-FN7Thyou0ju2ntJpYSiJF3u7neyTZd4Q@mail.gmail.com>

On Fri, Dec 5, 2014 at 11:25 AM, Andrew Gibiansky <
andrew.gibiansky at gmail.com> wrote:

> That's perfect and exactly the sort of thing I was looking for. I looked
> around the IPython API but didn't quite get to the code snippet you showed.


Here's the old example Matthias referred to, updated to the new API (this
is basically what Matthias posted, as a notebook):

http://nbviewer.ipython.org/gist/fperez/b3ab79a32b0e4777a646

Cheers

f
-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141205/b9bc9530/attachment.html>

From zvoros at gmail.com  Sat Dec  6 16:51:03 2014
From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Sat, 06 Dec 2014 22:51:03 +0100
Subject: [IPython-dev] Create New Notebook from Command-line
In-Reply-To: <CAHAreOoV1Y6-qQoT7-FN7Thyou0ju2ntJpYSiJF3u7neyTZd4Q@mail.gmail.com>
References: <CAF-LYKLE8eZkGer+8eNmDTYoWrcc=5AeEn9TxKe334mHNd3LFw@mail.gmail.com>	<1D45DCA3-4329-4AB2-91E7-D017213C3810@gmail.com>	<CACfEFw8E_arH7OSozoWzO4HCyQbJA9YGhpTpY_WCVnOTbkZCUg@mail.gmail.com>	<CAF-LYK+uaG_Bf6+ZCvtLBq_E2OkB7K-uStyKvj6D711ViM43cw@mail.gmail.com>
	<CAHAreOoV1Y6-qQoT7-FN7Thyou0ju2ntJpYSiJF3u7neyTZd4Q@mail.gmail.com>
Message-ID: <54837A47.9090807@gmail.com>

A comment along similar lines: wouldn't it make sense to allow the user 
to automatically execute python code before a notebook is loaded, 
somewhat similarly to what is done on the javascript side with the 
content of  profile/startup/custom?

Often, I find myself in a similar situation: when the name of a notebook 
is of a particular form, I want to create a data folder for that 
notebook, want to have a specific first cell, etc.

If there was a place somewhere in profile/python/startup, whose python 
code were executed before the notebook is loaded, a lot of these 
problems could easily be solved, or delegated to the user.

Now, I don't know the security implications of such a mechanism, and 
this idea might have already been discussed and rejected by the core 
team, so I might be barking up the wrong tree...

Cheers,
Zolt?n


On 12/05/2014 10:34 PM, Fernando Perez wrote:
>
> On Fri, Dec 5, 2014 at 11:25 AM, Andrew Gibiansky 
> <andrew.gibiansky at gmail.com <mailto:andrew.gibiansky at gmail.com>> wrote:
>
>     That's perfect and exactly the sort of thing I was looking for. I
>     looked around the IPython API but didn't quite get to the code
>     snippet you showed.
>
>
> Here's the old example Matthias referred to, updated to the new API 
> (this is basically what Matthias posted, as a notebook):
>
> http://nbviewer.ipython.org/gist/fperez/b3ab79a32b0e4777a646
>
> Cheers
>
> f
> -- 
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From takowl at gmail.com  Sat Dec  6 17:17:33 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 6 Dec 2014 14:17:33 -0800
Subject: [IPython-dev] Create New Notebook from Command-line
In-Reply-To: <54837A47.9090807@gmail.com>
References: <CAF-LYKLE8eZkGer+8eNmDTYoWrcc=5AeEn9TxKe334mHNd3LFw@mail.gmail.com>
	<1D45DCA3-4329-4AB2-91E7-D017213C3810@gmail.com>
	<CACfEFw8E_arH7OSozoWzO4HCyQbJA9YGhpTpY_WCVnOTbkZCUg@mail.gmail.com>
	<CAF-LYK+uaG_Bf6+ZCvtLBq_E2OkB7K-uStyKvj6D711ViM43cw@mail.gmail.com>
	<CAHAreOoV1Y6-qQoT7-FN7Thyou0ju2ntJpYSiJF3u7neyTZd4Q@mail.gmail.com>
	<54837A47.9090807@gmail.com>
Message-ID: <CAOvn4qhzOmFPXLkqu=--D+Gxpx-Wob8OicCuCufs3qWY_6PuFw@mail.gmail.com>

On 6 December 2014 at 13:51, Zolt?n V?r?s <zvoros at gmail.com> wrote:

> A comment along similar lines: wouldn't it make sense to allow the user
> to automatically execute python code before a notebook is loaded,
> somewhat similarly to what is done on the javascript side with the
> content of  profile/startup/custom?
>
> Often, I find myself in a similar situation: when the name of a notebook
> is of a particular form, I want to create a data folder for that
> notebook, want to have a specific first cell, etc.
>

We have startup files which are executed whenever an IPython kernel is
started, including on opening a new notebook. By design, however, the
kernel does not know the name of the notebook.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141206/a482129d/attachment.html>

From wes.turner at gmail.com  Sat Dec  6 18:46:25 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Sat, 6 Dec 2014 17:46:25 -0600
Subject: [IPython-dev] Create New Notebook from Command-line
In-Reply-To: <CAOvn4qhzOmFPXLkqu=--D+Gxpx-Wob8OicCuCufs3qWY_6PuFw@mail.gmail.com>
References: <CAF-LYKLE8eZkGer+8eNmDTYoWrcc=5AeEn9TxKe334mHNd3LFw@mail.gmail.com>
	<1D45DCA3-4329-4AB2-91E7-D017213C3810@gmail.com>
	<CACfEFw8E_arH7OSozoWzO4HCyQbJA9YGhpTpY_WCVnOTbkZCUg@mail.gmail.com>
	<CAF-LYK+uaG_Bf6+ZCvtLBq_E2OkB7K-uStyKvj6D711ViM43cw@mail.gmail.com>
	<CAHAreOoV1Y6-qQoT7-FN7Thyou0ju2ntJpYSiJF3u7neyTZd4Q@mail.gmail.com>
	<54837A47.9090807@gmail.com>
	<CAOvn4qhzOmFPXLkqu=--D+Gxpx-Wob8OicCuCufs3qWY_6PuFw@mail.gmail.com>
Message-ID: <CACfEFw87Sp7bPzB-fG-C-dgAiMrJuLrij+k7r7b5LTnOdYn+eQ@mail.gmail.com>

On Sat, Dec 6, 2014 at 4:17 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> On 6 December 2014 at 13:51, Zolt?n V?r?s <zvoros at gmail.com> wrote:
>
>> A comment along similar lines: wouldn't it make sense to allow the user
>> to automatically execute python code before a notebook is loaded,
>> somewhat similarly to what is done on the javascript side with the
>> content of  profile/startup/custom?
>>
>> Often, I find myself in a similar situation: when the name of a notebook
>> is of a particular form, I want to create a data folder for that
>> notebook, want to have a specific first cell, etc.
>>
>
> We have startup files which are executed whenever an IPython kernel is
> started, including on opening a new notebook. By design, however, the
> kernel does not know the name of the notebook.
>
>
Are these the docs for this feature:

* http://ipython.org/ipython-doc/dev/config/intro.html
* http://ipython.org/ipython-doc/dev/config/options/index.html

.pythonrc is not present in Python 3:

* https://docs.python.org/2/library/user.html

Would this be reproducible?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141206/5a97c728/attachment.html>

From takowl at gmail.com  Sat Dec  6 19:23:19 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 6 Dec 2014 16:23:19 -0800
Subject: [IPython-dev] Create New Notebook from Command-line
In-Reply-To: <CACfEFw87Sp7bPzB-fG-C-dgAiMrJuLrij+k7r7b5LTnOdYn+eQ@mail.gmail.com>
References: <CAF-LYKLE8eZkGer+8eNmDTYoWrcc=5AeEn9TxKe334mHNd3LFw@mail.gmail.com>
	<1D45DCA3-4329-4AB2-91E7-D017213C3810@gmail.com>
	<CACfEFw8E_arH7OSozoWzO4HCyQbJA9YGhpTpY_WCVnOTbkZCUg@mail.gmail.com>
	<CAF-LYK+uaG_Bf6+ZCvtLBq_E2OkB7K-uStyKvj6D711ViM43cw@mail.gmail.com>
	<CAHAreOoV1Y6-qQoT7-FN7Thyou0ju2ntJpYSiJF3u7neyTZd4Q@mail.gmail.com>
	<54837A47.9090807@gmail.com>
	<CAOvn4qhzOmFPXLkqu=--D+Gxpx-Wob8OicCuCufs3qWY_6PuFw@mail.gmail.com>
	<CACfEFw87Sp7bPzB-fG-C-dgAiMrJuLrij+k7r7b5LTnOdYn+eQ@mail.gmail.com>
Message-ID: <CAOvn4qh3=XhD8p0vK-frNiRrK3yQUypXYd4+-CatXyUL48A2qg@mail.gmail.com>

On 6 December 2014 at 15:46, Wes Turner <wes.turner at gmail.com> wrote:

> Are these the docs for this feature:
>

Nope, what I'm talking about is at the bottom here:
http://ipython.org/ipython-doc/dev/interactive/tutorial.html?highlight=startup%20files#startup-files

Python also has the $PYTHONSTARTUP environment variable, which IPython
respects:
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSTARTUP

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141206/f008b92d/attachment.html>

From zvoros at gmail.com  Mon Dec  8 12:17:01 2014
From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Mon, 08 Dec 2014 18:17:01 +0100
Subject: [IPython-dev] Create New Notebook from Command-line
In-Reply-To: <CAOvn4qhzOmFPXLkqu=--D+Gxpx-Wob8OicCuCufs3qWY_6PuFw@mail.gmail.com>
References: <CAF-LYKLE8eZkGer+8eNmDTYoWrcc=5AeEn9TxKe334mHNd3LFw@mail.gmail.com>	<1D45DCA3-4329-4AB2-91E7-D017213C3810@gmail.com>	<CACfEFw8E_arH7OSozoWzO4HCyQbJA9YGhpTpY_WCVnOTbkZCUg@mail.gmail.com>	<CAF-LYK+uaG_Bf6+ZCvtLBq_E2OkB7K-uStyKvj6D711ViM43cw@mail.gmail.com>	<CAHAreOoV1Y6-qQoT7-FN7Thyou0ju2ntJpYSiJF3u7neyTZd4Q@mail.gmail.com>	<54837A47.9090807@gmail.com>
	<CAOvn4qhzOmFPXLkqu=--D+Gxpx-Wob8OicCuCufs3qWY_6PuFw@mail.gmail.com>
Message-ID: <5485DD0D.3000404@gmail.com>

Hi Thomas,


On 12/06/2014 11:17 PM, Thomas Kluyver wrote:
> On 6 December 2014 at 13:51, Zolt?n V?r?s <zvoros at gmail.com 
> <mailto:zvoros at gmail.com>> wrote:
>
>     A comment along similar lines: wouldn't it make sense to allow the
>     user
>     to automatically execute python code before a notebook is loaded,
>     somewhat similarly to what is done on the javascript side with the
>     content of  profile/startup/custom?
>
>     Often, I find myself in a similar situation: when the name of a
>     notebook
>     is of a particular form, I want to create a data folder for that
>     notebook, want to have a specific first cell, etc.
>
>
> We have startup files which are executed whenever an IPython kernel is 
> started, including on opening a new notebook. By design, however, the 
> kernel does not know the name of the notebook.
>
>
Thanks for the clarification! I didn't know about this design feature.

Cheers,
Zolt?n


From roalexan at microsoft.com  Mon Dec  8 14:56:24 2014
From: roalexan at microsoft.com (Robert Alexander)
Date: Mon, 8 Dec 2014 19:56:24 +0000
Subject: [IPython-dev] programmatically passing in password when running
	IPython notebook
Message-ID: <DM2PR03MB4772A5B149905B7ACB8F55BC3640@DM2PR03MB477.namprd03.prod.outlook.com>

Hi. Is there a way, such as using URL request parameters or perhaps an IPython API, to open an IPython notebook and pass in the password programmatically? The profile for the IPython server is configured with a password, but there are cases where I trust the URL and want to bring up the notebook without requiring the user to enter the password.

Thanks, Robert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141208/bdfeeafb/attachment.html>

From takowl at gmail.com  Mon Dec  8 14:58:06 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 8 Dec 2014 11:58:06 -0800
Subject: [IPython-dev] programmatically passing in password when running
 IPython notebook
In-Reply-To: <DM2PR03MB4772A5B149905B7ACB8F55BC3640@DM2PR03MB477.namprd03.prod.outlook.com>
References: <DM2PR03MB4772A5B149905B7ACB8F55BC3640@DM2PR03MB477.namprd03.prod.outlook.com>
Message-ID: <CAOvn4qiYuqTBB=h4RWEqeDsMn3nq=Swxeq2QXUBYBshE7WFkOg@mail.gmail.com>

On 8 December 2014 at 11:56, Robert Alexander <roalexan at microsoft.com>
wrote:

> Hi. Is there a way, such as using URL request parameters or perhaps an
> IPython API, to open an IPython notebook and pass in the password
> programmatically? The profile for the IPython server is configured with a
> password, but there are cases where I trust the URL and want to bring up
> the notebook without requiring the user to enter the password.


Hi Robert,

There isn't currently any mechanism like that, no. We have thought about
having a one-time-password system when opening the browser, so that we
could enable password authentication by default, but it hasn't happened so
far.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141208/91fbd721/attachment.html>

From adam.is.t521 at gmail.com  Mon Dec  8 19:44:12 2014
From: adam.is.t521 at gmail.com (adamist521)
Date: Mon, 8 Dec 2014 16:44:12 -0800 (PST)
Subject: [IPython-dev] TikZ Magics
In-Reply-To: <6F69D108-582B-44F6-A966-164731753ED0@ipp.mpg.de>
References: <6F69D108-582B-44F6-A966-164731753ED0@ipp.mpg.de>
Message-ID: <1418085852053-5080012.post@n6.nabble.com>

Nice work!!

Any way to include and use the libraries of TikZ?



--
View this message in context: http://python.6.x6.nabble.com/TikZ-Magics-tp5001768p5080012.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From michael.kraus at ipp.mpg.de  Tue Dec  9 03:46:37 2014
From: michael.kraus at ipp.mpg.de (Michael Kraus)
Date: Tue, 9 Dec 2014 09:46:37 +0100
Subject: [IPython-dev] TikZ Magics
In-Reply-To: <1418085852053-5080012.post@n6.nabble.com>
References: <6F69D108-582B-44F6-A966-164731753ED0@ipp.mpg.de>
	<1418085852053-5080012.post@n6.nabble.com>
Message-ID: <1CCA25EA-607D-4F57-B2C8-6D1805190D00@ipp.mpg.de>

Hi,

This is a very good point. I updated the scripts at

http://www.ipp.mpg.de/~mkraus/python/tikzmagic.py
http://www.ipp.mpg.de/~mkraus/python/tikzmagic_test.ipynb

to allow for specifying a list of libraries, e.g., by

%%tikz -l arrows,matrix


Cheers,
Michael



> On 09.12.2014, at 01:44, adamist521 <adam.is.t521 at gmail.com> wrote:
> 
> Nice work!!
> 
> Any way to include and use the libraries of TikZ?
> 
> 
> 
> --
> View this message in context: http://python.6.x6.nabble.com/TikZ-Magics-tp5001768p5080012.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141209/9df02782/attachment.sig>

From michael.kraus at ipp.mpg.de  Tue Dec  9 03:49:26 2014
From: michael.kraus at ipp.mpg.de (Michael Kraus)
Date: Tue, 9 Dec 2014 09:49:26 +0100
Subject: [IPython-dev] Problems with tikzmagic and svg output
In-Reply-To: <CAGXzvifFX74WsQZWyEXKEt_oRHw8WCc2_JeYhhTMtfTBTTc3UA@mail.gmail.com>
References: <CAGXzvie8sHRruQLo09sekHRYJeWkwcWXYeL+-k76wBNABRocaA@mail.gmail.com>
	<69DE00F7-45D0-402F-9CC3-97E4509308B2@gmail.com>
	<CAGXzvifFX74WsQZWyEXKEt_oRHw8WCc2_JeYhhTMtfTBTTc3UA@mail.gmail.com>
Message-ID: <5CBE6C5F-F010-464D-865A-636FD77495D1@ipp.mpg.de>

Hi,

Thanks for the fix. However, while svg works fine now, it seems that png and jpg output is broken.

I inserted a switch to set the isolated tag only for svg output. The updated files can be found at

http://www.ipp.mpg.de/~mkraus/python/tikzmagic.py
http://www.ipp.mpg.de/~mkraus/python/tikzmagic_test.ipynb

The latest version also includes the possibility to specify TikZ libraries.


Cheers,
Michael



> On 22.10.2014, at 00:55, David Powell <DavidAnthonyPowell+python at gmail.com> wrote:
> 
> 
> I suppose you should display it in an iframe, if you display the SVG with the isolated=true metadata,
> IPython should do that for you automatically.
> 
> 
> Thanks, this was a fairly easy modification to the magic
> 
> 
> > On a related point, is there an easy way to export the .svg output of a cell to a file directly from within the notebook? I know that I could use nbconvert, manually hack the html, or find the temporary svg created by the magic, but I was hoping to find a more convenient way.
> 
> No, no easy way. For most content, drag-and drop from Chrome to my desktop seem to work here.
> I was able to do it with SVG at some point, but today it refuses.
> --
> 
> That's a pity, it would be nice to have a general solution to this problem for arbitrary data displayed in the cell output. Perhaps some kind of widget which runs nbconvert and selects the output from one cell would be a solution. For this specific tikz magic, the easiest thing was to add an extra argument to save a copy of the file.
> 
> Anyway, I have created a modified version of this tikz magic with these two fixes, shared here in case it is useful to anybody else:
> 
> https://gist.github.com/DavidPowell/84655e9a87fdfdaf2717 <https://gist.github.com/DavidPowell/84655e9a87fdfdaf2717>
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141209/b085716d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141209/b085716d/attachment.sig>

From roalexan at microsoft.com  Tue Dec  9 14:19:41 2014
From: roalexan at microsoft.com (Robert Alexander)
Date: Tue, 9 Dec 2014 19:19:41 +0000
Subject: [IPython-dev] running cells in notebook programmatically
Message-ID: <DM2PR03MB47762F1CF9D8F1CF9F18CB8C3650@DM2PR03MB477.namprd03.prod.outlook.com>

Hi. Running a script from the command line (using IPython 2.3.0) containing:

from IPython.nbformat import current as nbf
from IPython import get_ipython

nb = nbf.new_notebook()
firstCell = "print hello"
cells = [ nbf.new_code_cell(firstCell) ]
nb['worksheets'].append(nbf.new_worksheet(cells=cells))

ip = get_ipython()
print "ip"
print ip
for cell in nb.worksheets[0].cells:
    if cell.cell_type != 'code':
        continue
    print "cell.input"
    print cell.input
    ip.run_cell(cell.input)

returns the error:

ip
None
cell.input
print hello
Traceback (most recent call last):
  File "//.ipython/createnotebook.py", line 17, in <module>
    ip.run_cell(cell.input)
AttributeError: 'NoneType' object has no attribute 'run_cell'

Is there a correct way to do this?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141209/343afb16/attachment.html>

From nathan12343 at gmail.com  Tue Dec  9 14:22:47 2014
From: nathan12343 at gmail.com (Nathan Goldbaum)
Date: Tue, 09 Dec 2014 19:22:47 +0000
Subject: [IPython-dev] running cells in notebook programmatically
References: <DM2PR03MB47762F1CF9D8F1CF9F18CB8C3650@DM2PR03MB477.namprd03.prod.outlook.com>
Message-ID: <CAJXewOkps0TRcD4sTOHn00m783=NbHqP0ec3EL9Kc7Tq-k-Tnw@mail.gmail.com>

You need to run the script under IPython, not the standard python
interpreter.  Under CPython, get_ipython() will return None.  Under
IPython, it returns an instance of TerminalInteractiveShell.

You might also be interested in the runipy project:
https://github.com/paulgb/runipy

On Tue Dec 09 2014 at 11:19:58 AM Robert Alexander <roalexan at microsoft.com>
wrote:

>  Hi. Running a script from the command line (using IPython 2.3.0)
> containing:
>
>
>
> from IPython.nbformat import current as nbf
>
> *from IPython import get_ipython*
>
>
>
> nb = nbf.new_notebook()
>
> firstCell = "print hello"
>
> cells = [ nbf.new_code_cell(firstCell) ]
>
> nb['worksheets'].append(nbf.new_worksheet(cells=cells))
>
>
>
> *ip = get_ipython()*
>
> print "ip"
>
> print ip
>
> for cell in nb.worksheets[0].cells:
>
>     if cell.cell_type != 'code':
>
>         continue
>
>     print "cell.input"
>
>     print cell.input
>
>     *ip.run_cell(cell.input)*
>
>
>
> returns the error:
>
>
>
> ip
>
> None
>
> cell.input
>
> print hello
>
> Traceback (most recent call last):
>
>   File "//.ipython/createnotebook.py", line 17, in <module>
>
>     ip.run_cell(cell.input)
>
> *AttributeError: 'NoneType' object has no attribute 'run_cell'*
>
>
>
> Is there a correct way to do this?
>  _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141209/ef5d63b0/attachment.html>

From hughesadam87 at gmail.com  Tue Dec  9 19:46:53 2014
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Tue, 9 Dec 2014 19:46:53 -0500
Subject: [IPython-dev] More widget feasibility ideas
In-Reply-To: <CACejjWx5wOXxSVb9rgRME94iS8y2sCdGw=PRryyBZFLTvN6Uvg@mail.gmail.com>
References: <CAMHV+dAptq7vwXb9wm3PvTK9m9Ru3LMR-jBabXvHBs4PFOM9kA@mail.gmail.com>
	<CACejjWygugCD1t8e6hBtiFKhGmD_Ad+mfF=RHLC6vmCTN_Em0g@mail.gmail.com>
	<CAMHV+dABJ+TwAsYBd6VGNHAj6NUJqncacmbO3D_0mDDO9xxGOA@mail.gmail.com>
	<CAMHV+dB0yRPpnjfALhK3eaeoFTDPH-VN6B31u8XzLt9hfLVRgA@mail.gmail.com>
	<CACejjWw6VXNzyjkda_VO4nu6KAC=QNYHdcV0vGkGkhM1=P5Q1A@mail.gmail.com>
	<CAMHV+dBUdePMz4kuPQG06NG8OSoAV7my27Hd51JAqS3vG+Pkjg@mail.gmail.com>
	<CACejjWwpNQzp=j8ZtpC8HjP4CQj97zA9-scUY5uWC8SHZHosHg@mail.gmail.com>
	<CAMHV+dCiVRnc7U-wTwzeUqCoxKR+_fuCxW=rAtrU70-K_Nmx_Q@mail.gmail.com>
	<CACejjWxx1deQx9oT124KYMJ4FRZ+q=fo4U63FDEEuO=55Fb99A@mail.gmail.com>
	<CAMHV+dDHHFoYkmugy4NwLWRDhzmZFpsUy8TG8G4UB5=7CZaejw@mail.gmail.com>
	<CAMHV+dDJY1+9hxKSQuJ9TGTZnVFb=pomLWNNrdpoGTYfQkgJ2w@mail.gmail.com>
	<CACejjWzXtVX6pLmBOhtSu_sb5N8nQjzViE+k-Dnoc0-TEC3E+Q@mail.gmail.com>
	<CAMHV+dALeaJd81FXK5NwRU0=2zE=+0g649AWDeSkgtPYQ58T-g@mail.gmail.com>
	<CACejjWwYP1amUkRZkOLT3_PPoXq4B2_AzNtW0Atd+D=QXPF0Uw@mail.gmail.com>
	<CAMHV+dBjAd3+s2eHEpRxCg3yEvBcpfuB5ttNjy1MZ9Bu28cCyw@mail.gmail.com>
	<CACejjWyaf9vMobggJmRzAdz=VDqkWa=4mhAcDpL5f27NTJ8Bng@mail.gmail.com>
	<CAMHV+dC4XN+pO2E7qOx=4SAuzm2+PKpCg6M8+RZy_V9KFENNdw@mail.gmail.com>
	<CACejjWx5wOXxSVb9rgRME94iS8y2sCdGw=PRryyBZFLTvN6Uvg@mail.gmail.com>
Message-ID: <CAMHV+dBR+v70OqExPTK8-QRRZQQKxb9R=7TgHWgtecAZ1J8J-w@mail.gmail.com>

Nicholas,

Thanks for all of your help on this.  We were able to see it through and
put you in the acknowledgements:

https://www.youtube.com/watch?v=RhMHgQbP__A

On Thu, Oct 30, 2014 at 8:35 PM, Nicholas Bollweg <nick.bollweg at gmail.com>
wrote:

> if there's nothing else going on there, that is correct!
>
> On Thu, Oct 30, 2014 at 1:38 PM, Adam Hughes <hughesadam87 at gmail.com>
> wrote:
>
>> So we can also drop the def displayed() / on_displayed portions of the
>> GUI, right?
>>
>> On Thu, Oct 30, 2014 at 1:05 PM, Nicholas Bollweg <nick.bollweg at gmail.com
>> > wrote:
>>
>>> You specify it when you create the instance:
>>>
>>> w = SomeWidget(_dom_classes=["panel-body"])
>>>
>>> or, in the case of class that should always get a _dom_class:
>>>
>>> class PanelBody(widgets.Box):
>>>     def __init__(self, *args, **kwargs):
>>>         super(PanelBody, self).__init__(*args, **kwargs)
>>>         self._dom_classes += ("panel-body",)
>>>
>>> You could also monkey with the kwargs beforehand, avoiding, under the
>>> covers, two calls to update_classes, but this approach is easy to read,
>>> which i think wins out.
>>>
>>> When I asked about this on gitter, the reasoning for this change was
>>> that the add/remove class couldn't be made stateful, and that it was using
>>> extra, custom comms stuff to do something that traitlets were perfectly
>>> capable of. Thus, this traitlet. Howevever, as you can see from it having
>>> the _ prefix, this is really much more of a widget-developer functionality:
>>> it is suggested that no-fooling custom javascript be done... which is still
>>> not exactly simple.
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141209/4220bbbe/attachment.html>

From adam.is.t521 at gmail.com  Tue Dec  9 20:46:41 2014
From: adam.is.t521 at gmail.com (adamist521)
Date: Tue, 9 Dec 2014 17:46:41 -0800 (PST)
Subject: [IPython-dev] TikZ Magics
In-Reply-To: <1CCA25EA-607D-4F57-B2C8-6D1805190D00@ipp.mpg.de>
References: <6F69D108-582B-44F6-A966-164731753ED0@ipp.mpg.de>
	<1418085852053-5080012.post@n6.nabble.com>
	<1CCA25EA-607D-4F57-B2C8-6D1805190D00@ipp.mpg.de>
Message-ID: <1418176001946-5080092.post@n6.nabble.com>

Awesome!!!!
Thanks a lot!



--
View this message in context: http://python.6.x6.nabble.com/TikZ-Magics-tp5001768p5080092.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From nick.bollweg at gmail.com  Tue Dec  9 21:15:41 2014
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Tue, 9 Dec 2014 21:15:41 -0500
Subject: [IPython-dev] More widget feasibility ideas
In-Reply-To: <CAMHV+dBR+v70OqExPTK8-QRRZQQKxb9R=7TgHWgtecAZ1J8J-w@mail.gmail.com>
References: <CAMHV+dAptq7vwXb9wm3PvTK9m9Ru3LMR-jBabXvHBs4PFOM9kA@mail.gmail.com>
	<CACejjWygugCD1t8e6hBtiFKhGmD_Ad+mfF=RHLC6vmCTN_Em0g@mail.gmail.com>
	<CAMHV+dABJ+TwAsYBd6VGNHAj6NUJqncacmbO3D_0mDDO9xxGOA@mail.gmail.com>
	<CAMHV+dB0yRPpnjfALhK3eaeoFTDPH-VN6B31u8XzLt9hfLVRgA@mail.gmail.com>
	<CACejjWw6VXNzyjkda_VO4nu6KAC=QNYHdcV0vGkGkhM1=P5Q1A@mail.gmail.com>
	<CAMHV+dBUdePMz4kuPQG06NG8OSoAV7my27Hd51JAqS3vG+Pkjg@mail.gmail.com>
	<CACejjWwpNQzp=j8ZtpC8HjP4CQj97zA9-scUY5uWC8SHZHosHg@mail.gmail.com>
	<CAMHV+dCiVRnc7U-wTwzeUqCoxKR+_fuCxW=rAtrU70-K_Nmx_Q@mail.gmail.com>
	<CACejjWxx1deQx9oT124KYMJ4FRZ+q=fo4U63FDEEuO=55Fb99A@mail.gmail.com>
	<CAMHV+dDHHFoYkmugy4NwLWRDhzmZFpsUy8TG8G4UB5=7CZaejw@mail.gmail.com>
	<CAMHV+dDJY1+9hxKSQuJ9TGTZnVFb=pomLWNNrdpoGTYfQkgJ2w@mail.gmail.com>
	<CACejjWzXtVX6pLmBOhtSu_sb5N8nQjzViE+k-Dnoc0-TEC3E+Q@mail.gmail.com>
	<CAMHV+dALeaJd81FXK5NwRU0=2zE=+0g649AWDeSkgtPYQ58T-g@mail.gmail.com>
	<CACejjWwYP1amUkRZkOLT3_PPoXq4B2_AzNtW0Atd+D=QXPF0Uw@mail.gmail.com>
	<CAMHV+dBjAd3+s2eHEpRxCg3yEvBcpfuB5ttNjy1MZ9Bu28cCyw@mail.gmail.com>
	<CACejjWyaf9vMobggJmRzAdz=VDqkWa=4mhAcDpL5f27NTJ8Bng@mail.gmail.com>
	<CAMHV+dC4XN+pO2E7qOx=4SAuzm2+PKpCg6M8+RZy_V9KFENNdw@mail.gmail.com>
	<CACejjWx5wOXxSVb9rgRME94iS8y2sCdGw=PRryyBZFLTvN6Uvg@mail.gmail.com>
	<CAMHV+dBR+v70OqExPTK8-QRRZQQKxb9R=7TgHWgtecAZ1J8J-w@mail.gmail.com>
Message-ID: <CACejjWyt_Q3d7XHZJKbt6ym9pnFMy_0P1mW8G7vOAR-HLyeqHg@mail.gmail.com>

Adam:
Thanks, I appreciate it!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141209/fc51d28f/attachment.html>

From cappy2112 at gmail.com  Fri Dec 12 02:13:48 2014
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Thu, 11 Dec 2014 23:13:48 -0800
Subject: [IPython-dev] What causes the kernel to stay busy?
Message-ID: <CABRXM4=mi0zkeC4b=Aw_YqnZtWAPgdFEi8u-kTnZW3k-J7LuMQ@mail.gmail.com>

Python 2.7.8, iPython 2.2.0

I'm running on Mavericks 10.9.5, and I have one notebook running in Safari.

I'm parsing some html- not a lot of data.
Sometimes the notebook stops responding, even though it gives you the
illusion
that your python code was executed. (this is very misleading)

Your only clue is the tiny black circle in the upper right corner.
Once that shows up, game over. If I restart the kernel, all of the
variables loose their values. Interrupting the kernel doesn't help.

What is causing this and can it be fixed?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141211/e2a0df6a/attachment.html>

From efiring at hawaii.edu  Fri Dec 12 03:53:12 2014
From: efiring at hawaii.edu (Eric Firing)
Date: Thu, 11 Dec 2014 22:53:12 -1000
Subject: [IPython-dev] What causes the kernel to stay busy?
In-Reply-To: <CABRXM4=mi0zkeC4b=Aw_YqnZtWAPgdFEi8u-kTnZW3k-J7LuMQ@mail.gmail.com>
References: <CABRXM4=mi0zkeC4b=Aw_YqnZtWAPgdFEi8u-kTnZW3k-J7LuMQ@mail.gmail.com>
Message-ID: <548AACF8.8010708@hawaii.edu>

On 2014/12/11, 9:13 PM, Tony Cappellini wrote:
> Python 2.7.8, iPython 2.2.0
>
> I'm running on Mavericks 10.9.5, and I have one notebook running in Safari.
>
> I'm parsing some html- not a lot of data.
> Sometimes the notebook stops responding, even though it gives you the
> illusion
> that your python code was executed. (this is very misleading)
>
> Your only clue is the tiny black circle in the upper right corner.
> Once that shows up, game over. If I restart the kernel, all of the
> variables loose their values. Interrupting the kernel doesn't help.
>
> What is causing this and can it be fixed?

Sounds like that pesky app-nap.  I think your problem will be solved with:

conda update ipython

The newer version disables app-nap in ipython.

Eric

>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



From efiring at hawaii.edu  Fri Dec 12 03:55:23 2014
From: efiring at hawaii.edu (Eric Firing)
Date: Thu, 11 Dec 2014 22:55:23 -1000
Subject: [IPython-dev] What causes the kernel to stay busy?
In-Reply-To: <CABRXM4=mi0zkeC4b=Aw_YqnZtWAPgdFEi8u-kTnZW3k-J7LuMQ@mail.gmail.com>
References: <CABRXM4=mi0zkeC4b=Aw_YqnZtWAPgdFEi8u-kTnZW3k-J7LuMQ@mail.gmail.com>
Message-ID: <548AAD7B.7040600@hawaii.edu>

On 2014/12/11, 9:13 PM, Tony Cappellini wrote:
> Python 2.7.8, iPython 2.2.0
>
> I'm running on Mavericks 10.9.5, and I have one notebook running in Safari.
>
> I'm parsing some html- not a lot of data.
> Sometimes the notebook stops responding, even though it gives you the
> illusion
> that your python code was executed. (this is very misleading)
>
> Your only clue is the tiny black circle in the upper right corner.
> Once that shows up, game over. If I restart the kernel, all of the
> variables loose their values. Interrupting the kernel doesn't help.
>
> What is causing this and can it be fixed?

I wasn't paying attention to which list this was addressed to--I thought 
it was anaconda.  Regardless of whether you are using anaconda, the 
answer is the same: you need IPython 2.3 or higher.

Eric

>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



From cappy2112 at gmail.com  Fri Dec 12 13:20:35 2014
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Fri, 12 Dec 2014 10:20:35 -0800
Subject: [IPython-dev] What causes the kernel to stay busy?
Message-ID: <CABRXM4nRG834_XtyKDzpty1tFmsXTPPUsxa1ej33_p-QGO6dMw@mail.gmail.com>

Message: 3
Date: Thu, 11 Dec 2014 22:55:23 -1000
From: Eric Firing <efiring at hawaii.edu>
Subject: Re: [IPython-dev] What causes the kernel to stay busy?
To: ipython-dev at scipy.org
Message-ID: <548AAD7B.7040600 at hawaii.edu>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

I don't use anaconda,  I use Macports. My initial experience with conda
on Windows was bad. The older Enthrought (pre-conda) installers worked much
better.

>>I wasn't paying attention to which list this was addressed to--I thought
>>it was anaconda.  Regardless of whether you are using anaconda, the
>>answer is the same: you need IPython 2.3 or higher.

It looks like Macports does have a package for iPython 2.3.1, so I
will be installing that shortly.

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141212/c31cf807/attachment.html>

From cappy2112 at gmail.com  Fri Dec 12 16:27:26 2014
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Fri, 12 Dec 2014 13:27:26 -0800
Subject: [IPython-dev] What causes the kernel to stay busy?
In-Reply-To: <548B5C54.6000408@hawaii.edu>
References: <CABRXM4nRG834_XtyKDzpty1tFmsXTPPUsxa1ej33_p-QGO6dMw@mail.gmail.com>
	<CABRXM4nZziurARGqySALWhX0AUm+3tNjbg8AzHcM74LaEgYhOw@mail.gmail.com>
	<548B5C54.6000408@hawaii.edu>
Message-ID: <CABRXM4mZ3aLb-VCuv88Hc4RA1xT8up5BDqq87E__zAiPVB2geA@mail.gmail.com>

Message: 3
Date: Thu, 11 Dec 2014 22:55:23 -1000
From: Eric Firing <efiring at hawaii.edu>
Subject: Re: [IPython-dev] What causes the kernel to stay busy?
To: ipython-dev at scipy.org
Message-ID: <548AAD7B.7040600 at hawaii.edu>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed


>>I wasn't paying attention to which list this was addressed to--I thought
>> was anaconda.  Regardless of whether you are using anaconda, the
>>answer is the same: you need IPython 2.3 or higher

Python 2.7.8 (default, Oct 15 2014, 22:04:42)
Type "copyright", "credits" or "license" for more information.

IPython 2.3.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

After updating to iPython 2.3.1 and entering a few python statements
into the notebook, it looks like the kernel busy issue still persists.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141212/91e12e93/attachment.html>

From efiring at hawaii.edu  Fri Dec 12 16:40:58 2014
From: efiring at hawaii.edu (Eric Firing)
Date: Fri, 12 Dec 2014 11:40:58 -1000
Subject: [IPython-dev] What causes the kernel to stay busy?
In-Reply-To: <CABRXM4mZ3aLb-VCuv88Hc4RA1xT8up5BDqq87E__zAiPVB2geA@mail.gmail.com>
References: <CABRXM4nRG834_XtyKDzpty1tFmsXTPPUsxa1ej33_p-QGO6dMw@mail.gmail.com>	<CABRXM4nZziurARGqySALWhX0AUm+3tNjbg8AzHcM74LaEgYhOw@mail.gmail.com>	<548B5C54.6000408@hawaii.edu>
	<CABRXM4mZ3aLb-VCuv88Hc4RA1xT8up5BDqq87E__zAiPVB2geA@mail.gmail.com>
Message-ID: <548B60EA.6010909@hawaii.edu>

On 2014/12/12, 11:27 AM, Tony Cappellini wrote:
> Message: 3
> Date: Thu, 11 Dec 2014 22:55:23 -1000
> From: Eric Firing <efiring at hawaii.edu <mailto:efiring at hawaii.edu>>
> Subject: Re: [IPython-dev] What causes the kernel to stay busy?
> To: ipython-dev at scipy.org <mailto:ipython-dev at scipy.org>
> Message-ID: <548AAD7B.7040600 at hawaii.edu
> <mailto:548AAD7B.7040600 at hawaii.edu>>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>
>>>I wasn't paying attention to which list this was addressed to--I thought
>>> was anaconda.  Regardless of whether you are using anaconda, the
>>>answer is the same: you need IPython 2.3 or higher
>
> Python 2.7.8 (default, Oct 15 2014, 22:04:42)
> Type "copyright", "credits" or "license" for more information.
>
> IPython 2.3.1 -- An enhanced Interactive Python.
> ?         -> Introduction and overview of IPython's features.
> %quickref -> Quick reference.
> help      -> Python's own help system.
> object?   -> Details about 'object', use 'object??' for extra details.
>
> After updating to iPython 2.3.1 and entering a few python statements
> into the notebook, it looks like the kernel busy issue still persists.
>
>
If you enter the same statements at a raw Python terminal prompt, does 
it remain responsive?  And if so, what about if you do it in a plain 
IPython console?  (Is the problem notebook-specific?)

What is the simplest sequence of commands that triggers the hang?

I probably won't be able to offer any suggestions, but if anyone else 
has an idea, they probably will want the answers to the above questions.

Also, it's possible I was wrong about the version at which app-nap 
handling was introduced; but updating to the latest release is a good 
first step in troubleshooting in any case.

Eric


From cappy2112 at gmail.com  Fri Dec 12 17:37:12 2014
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Fri, 12 Dec 2014 14:37:12 -0800
Subject: [IPython-dev] What causes the kernel to stay busy?
In-Reply-To: <548B60EA.6010909@hawaii.edu>
References: <CABRXM4nRG834_XtyKDzpty1tFmsXTPPUsxa1ej33_p-QGO6dMw@mail.gmail.com>
	<CABRXM4nZziurARGqySALWhX0AUm+3tNjbg8AzHcM74LaEgYhOw@mail.gmail.com>
	<548B5C54.6000408@hawaii.edu>
	<CABRXM4mZ3aLb-VCuv88Hc4RA1xT8up5BDqq87E__zAiPVB2geA@mail.gmail.com>
	<548B60EA.6010909@hawaii.edu>
Message-ID: <CABRXM4n2oV=dL6HJjzq=b6Nu28qXAbq8SZgRi_qTH0nQfhHK7g@mail.gmail.com>

On Fri, Dec 12, 2014 at 1:40 PM, Eric Firing <efiring at hawaii.edu> wrote:
>
> On 2014/12/12, 11:27 AM, Tony Cappellini wrote:
>
>> Message: 3
>> Date: Thu, 11 Dec 2014 22:55:23 -1000
>> From: Eric Firing <efiring at hawaii.edu <mailto:efiring at hawaii.edu>>
>> Subject: Re: [IPython-dev] What causes the kernel to stay busy?
>> To: ipython-dev at scipy.org <mailto:ipython-dev at scipy.org>
>> Message-ID: <548AAD7B.7040600 at hawaii.edu
>> <mailto:548AAD7B.7040600 at hawaii.edu>>
>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>
>>
>>
>>
>>  >>If you enter the same statements at a raw Python terminal prompt, does
> it remain responsive?

Haven't tried it- don't think I will be able to, since the regular python
prompt doesn't deal well with multi-line entries.
iPython obviously excels at this, as does the notebook.


> >>And if so, what about if you do it in a plain IPython console?  (Is the
> problem notebook-specific?)
>
Will try this soon. My hunch is it's the notebook, but it could be affected
by the specific browser too. Safari is quite problematic these days, but
most browsers are as well.

>
> >>What is the simplest sequence of commands that triggers the hang?
>
It varies and I'm not always executing the same code.

Will post some answers to your questions later today .

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141212/3c2fce1b/attachment.html>

From cappy2112 at gmail.com  Sat Dec 13 14:03:43 2014
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Sat, 13 Dec 2014 11:03:43 -0800
Subject: [IPython-dev] What causes the kernel to stay busy?
Message-ID: <CABRXM4k8rC6f0jA6-1gZ_RZJnNKDfL3jObF8Pbt4WC_TeRZTFg@mail.gmail.com>

Message: 3
Date: Fri, 12 Dec 2014 11:40:58 -1000
From: Eric Firing <efiring at hawaii.edu>
Subject: Re: [IPython-dev] What causes the kernel to stay busy?
To: cappy2112 at gmail.com, ipython-dev at scipy.org
Message-ID: <548B60EA.6010909 at hawaii.edu>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

>>If you enter the same statements at a raw Python terminal prompt,
I didn't enter them into the interpreter manually, I've just created a
script
an ran it. There are too many lines and too much indenting to enter into
the normal Python interpreter, without making a mistake.

I did paste the script contents into the non-notebook iPython interpreter.
It ran just fine.

>>it remain responsive?  And if so, what about if you do it in a plain
>>IPython console?  (Is the problem notebook-specific?)

It appears that it may be notebook-specific
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141213/ee74095a/attachment.html>

From maplabs at light42.com  Sat Dec 13 19:57:35 2014
From: maplabs at light42.com (=?utf-8?b?QnJpYW4gTSBIYW1saW4=?=)
Date: Sat, 13 Dec 2014 16:57:35 -0800
Subject: [IPython-dev] misc debian packaging recomm.
Message-ID: <20141213165735.o244ap1hnccc0ow4@webmail.light42.com>

    Thomas et al
 ? our?most recent OSGeo-Live "nightly" built on LUbuntu 14.04 reads directly from jtaylor's PPA
 so the versions are.. ?pandas 0.13; ?matplotlib 1.3.1; ?numpy 1.8.2; ? scipy 0.13; ?cartopy 0.11;
 ?IPython 2.3 ? (also rpy2 2.5.1)
 this seems good .. any advice still welcome, and this is likely how it will freeze

 --
Brian M Hamlin
 OSGeo California Chapter
 blog.light42.com

  


From takowl at gmail.com  Mon Dec 15 21:20:15 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 15 Dec 2014 18:20:15 -0800
Subject: [IPython-dev] Storing signatures outside the notebook
Message-ID: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>

Min and I were just talking to Aron Ahmadia and Chris Kees. We covered
several topics, but one of the more straightforward things we discussed was
the problem that storing signatures in the notebook gives for version
control - because any two changes will make the signature conflict, even if
the changes are at opposite ends of the notebook.

We possibly shouldn't try to rework this before 3.0, but I started thinking
about alternatives, and wanted to put a couple out for discussion:

1. Store signatures next to notebooks - maybe a .ipynb.sig file, like you
sometimes see checksum files next to downloads. This has the advantage that
to move the notebook along with its trust, you can just move those two
files together, but the disadvantage that it clutters up your working
directory.
2. An extra subdirectory, like we already use for checkpoints, but for
signatures. Less clutter, but more awkward to move a notebook and its
associated signature together.
3. A per-user database storing hashes of all trusted notebooks. Checking if
a notebook is trusted then becomes: hash it, and check whether that hash is
in the database. The notebook's location on the filesystem is irrelevant.
This may be troublesome for people who sync their IPython directory across
multiple computers, or have home directories on an NFS mount, though.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141215/bb6178b1/attachment.html>

From aron at ahmadia.net  Mon Dec 15 21:43:49 2014
From: aron at ahmadia.net (Aron Ahmadia)
Date: Mon, 15 Dec 2014 18:43:49 -0800
Subject: [IPython-dev] Storing signatures outside the notebook
In-Reply-To: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
References: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
Message-ID: <CAPhiW4go_Vvqbkqe49A2Z0-aDD3CQGyKZRPtP2Kr0Te+oWGNWg@mail.gmail.com>

Another option to consider is

[2b]  Creating a per-notebook/directory subdirectory that handles all such
extra data/metadata, similar to .git.

This subdirectory is easily excluded from version control, organizes
IPython content into a single location on the file system, and lives close
to the original content.

Thanks for at least considering this.  The clashing signatures cause a lot
of merge conflicts that would otherwise be easily avoided.

A

On Mon, Dec 15, 2014 at 6:20 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
> Min and I were just talking to Aron Ahmadia and Chris Kees. We covered
> several topics, but one of the more straightforward things we discussed was
> the problem that storing signatures in the notebook gives for version
> control - because any two changes will make the signature conflict, even if
> the changes are at opposite ends of the notebook.
>
> We possibly shouldn't try to rework this before 3.0, but I started
> thinking about alternatives, and wanted to put a couple out for discussion:
>
> 1. Store signatures next to notebooks - maybe a .ipynb.sig file, like you
> sometimes see checksum files next to downloads. This has the advantage that
> to move the notebook along with its trust, you can just move those two
> files together, but the disadvantage that it clutters up your working
> directory.
> 2. An extra subdirectory, like we already use for checkpoints, but for
> signatures. Less clutter, but more awkward to move a notebook and its
> associated signature together.
> 3. A per-user database storing hashes of all trusted notebooks. Checking
> if a notebook is trusted then becomes: hash it, and check whether that hash
> is in the database. The notebook's location on the filesystem is
> irrelevant. This may be troublesome for people who sync their IPython
> directory across multiple computers, or have home directories on an NFS
> mount, though.
>
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141215/9606103d/attachment.html>

From ellisonbg at gmail.com  Tue Dec 16 00:28:43 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 15 Dec 2014 21:28:43 -0800
Subject: [IPython-dev] Storing signatures outside the notebook
In-Reply-To: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
References: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
Message-ID: <CAH4pYpR-3PssaHUvZYuFkzZJOQPunrnS4TtwU_WvSkydKT3Fgg@mail.gmail.com>

I think these ideas are very interesting, can one of you add this to the
dev meeting agenda for this Th?

On Mon, Dec 15, 2014 at 6:20 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
> Min and I were just talking to Aron Ahmadia and Chris Kees. We covered
> several topics, but one of the more straightforward things we discussed was
> the problem that storing signatures in the notebook gives for version
> control - because any two changes will make the signature conflict, even if
> the changes are at opposite ends of the notebook.
>
> We possibly shouldn't try to rework this before 3.0, but I started
> thinking about alternatives, and wanted to put a couple out for discussion:
>
> 1. Store signatures next to notebooks - maybe a .ipynb.sig file, like you
> sometimes see checksum files next to downloads. This has the advantage that
> to move the notebook along with its trust, you can just move those two
> files together, but the disadvantage that it clutters up your working
> directory.
> 2. An extra subdirectory, like we already use for checkpoints, but for
> signatures. Less clutter, but more awkward to move a notebook and its
> associated signature together.
> 3. A per-user database storing hashes of all trusted notebooks. Checking
> if a notebook is trusted then becomes: hash it, and check whether that hash
> is in the database. The notebook's location on the filesystem is
> irrelevant. This may be troublesome for people who sync their IPython
> directory across multiple computers, or have home directories on an NFS
> mount, though.
>
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>

-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141215/5cbf7d65/attachment.html>

From ellisonbg at gmail.com  Tue Dec 16 00:29:59 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 15 Dec 2014 21:29:59 -0800
Subject: [IPython-dev] Storing signatures outside the notebook
In-Reply-To: <CAH4pYpR-3PssaHUvZYuFkzZJOQPunrnS4TtwU_WvSkydKT3Fgg@mail.gmail.com>
References: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
	<CAH4pYpR-3PssaHUvZYuFkzZJOQPunrnS4TtwU_WvSkydKT3Fgg@mail.gmail.com>
Message-ID: <CAH4pYpQttHerFTxGw9LxjRTUQgNxmpKC9-rJyHBDtXo56E711g@mail.gmail.com>

I should note that "moving the trust" is not just the notebook and its
signature. You would also have to move the users secret key used to
generate the signature. In my mind, this whole model relies on the extreme
difficultly of transferring trust to another person.

On Mon, Dec 15, 2014 at 9:28 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> I think these ideas are very interesting, can one of you add this to the
> dev meeting agenda for this Th?
>
> On Mon, Dec 15, 2014 at 6:20 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>
>> Min and I were just talking to Aron Ahmadia and Chris Kees. We covered
>> several topics, but one of the more straightforward things we discussed was
>> the problem that storing signatures in the notebook gives for version
>> control - because any two changes will make the signature conflict, even if
>> the changes are at opposite ends of the notebook.
>>
>> We possibly shouldn't try to rework this before 3.0, but I started
>> thinking about alternatives, and wanted to put a couple out for discussion:
>>
>> 1. Store signatures next to notebooks - maybe a .ipynb.sig file, like you
>> sometimes see checksum files next to downloads. This has the advantage that
>> to move the notebook along with its trust, you can just move those two
>> files together, but the disadvantage that it clutters up your working
>> directory.
>> 2. An extra subdirectory, like we already use for checkpoints, but for
>> signatures. Less clutter, but more awkward to move a notebook and its
>> associated signature together.
>> 3. A per-user database storing hashes of all trusted notebooks. Checking
>> if a notebook is trusted then becomes: hash it, and check whether that hash
>> is in the database. The notebook's location on the filesystem is
>> irrelevant. This may be troublesome for people who sync their IPython
>> directory across multiple computers, or have home directories on an NFS
>> mount, though.
>>
>> Thomas
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
>


-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141215/9019f3c0/attachment.html>

From takowl at gmail.com  Tue Dec 16 00:36:07 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 15 Dec 2014 21:36:07 -0800
Subject: [IPython-dev] Storing signatures outside the notebook
In-Reply-To: <CAH4pYpQttHerFTxGw9LxjRTUQgNxmpKC9-rJyHBDtXo56E711g@mail.gmail.com>
References: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
	<CAH4pYpR-3PssaHUvZYuFkzZJOQPunrnS4TtwU_WvSkydKT3Fgg@mail.gmail.com>
	<CAH4pYpQttHerFTxGw9LxjRTUQgNxmpKC9-rJyHBDtXo56E711g@mail.gmail.com>
Message-ID: <CAOvn4qj0zByk3Hkxib=phc3WvZPkFdAaDq+yBWyuOhn+oupJdQ@mail.gmail.com>

I meant moving the file on the filesystem for the same user.

Thomas
On Dec 15, 2014 9:30 PM, "Brian Granger" <ellisonbg at gmail.com> wrote:

> I should note that "moving the trust" is not just the notebook and its
> signature. You would also have to move the users secret key used to
> generate the signature. In my mind, this whole model relies on the extreme
> difficultly of transferring trust to another person.
>
> On Mon, Dec 15, 2014 at 9:28 PM, Brian Granger <ellisonbg at gmail.com>
> wrote:
>>
>> I think these ideas are very interesting, can one of you add this to the
>> dev meeting agenda for this Th?
>>
>> On Mon, Dec 15, 2014 at 6:20 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>>
>>> Min and I were just talking to Aron Ahmadia and Chris Kees. We covered
>>> several topics, but one of the more straightforward things we discussed was
>>> the problem that storing signatures in the notebook gives for version
>>> control - because any two changes will make the signature conflict, even if
>>> the changes are at opposite ends of the notebook.
>>>
>>> We possibly shouldn't try to rework this before 3.0, but I started
>>> thinking about alternatives, and wanted to put a couple out for discussion:
>>>
>>> 1. Store signatures next to notebooks - maybe a .ipynb.sig file, like
>>> you sometimes see checksum files next to downloads. This has the advantage
>>> that to move the notebook along with its trust, you can just move those two
>>> files together, but the disadvantage that it clutters up your working
>>> directory.
>>> 2. An extra subdirectory, like we already use for checkpoints, but for
>>> signatures. Less clutter, but more awkward to move a notebook and its
>>> associated signature together.
>>> 3. A per-user database storing hashes of all trusted notebooks. Checking
>>> if a notebook is trusted then becomes: hash it, and check whether that hash
>>> is in the database. The notebook's location on the filesystem is
>>> irrelevant. This may be troublesome for people who sync their IPython
>>> directory across multiple computers, or have home directories on an NFS
>>> mount, though.
>>>
>>> Thomas
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> @ellisonbg on Twitter and GitHub
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141215/bb4eedbc/attachment.html>

From ellisonbg at gmail.com  Tue Dec 16 00:42:05 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 15 Dec 2014 21:42:05 -0800
Subject: [IPython-dev] Storing signatures outside the notebook
In-Reply-To: <CAOvn4qj0zByk3Hkxib=phc3WvZPkFdAaDq+yBWyuOhn+oupJdQ@mail.gmail.com>
References: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
	<CAH4pYpR-3PssaHUvZYuFkzZJOQPunrnS4TtwU_WvSkydKT3Fgg@mail.gmail.com>
	<CAH4pYpQttHerFTxGw9LxjRTUQgNxmpKC9-rJyHBDtXo56E711g@mail.gmail.com>
	<CAOvn4qj0zByk3Hkxib=phc3WvZPkFdAaDq+yBWyuOhn+oupJdQ@mail.gmail.com>
Message-ID: <CAH4pYpRvvssfghvQFaLS7ZFcmn_gMiJGAJj1nB77c2z26o-ZAw@mail.gmail.com>

Ahh, yes that would work

On Mon, Dec 15, 2014 at 9:36 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
> I meant moving the file on the filesystem for the same user.
>
> Thomas
> On Dec 15, 2014 9:30 PM, "Brian Granger" <ellisonbg at gmail.com> wrote:
>
>> I should note that "moving the trust" is not just the notebook and its
>> signature. You would also have to move the users secret key used to
>> generate the signature. In my mind, this whole model relies on the extreme
>> difficultly of transferring trust to another person.
>>
>> On Mon, Dec 15, 2014 at 9:28 PM, Brian Granger <ellisonbg at gmail.com>
>> wrote:
>>>
>>> I think these ideas are very interesting, can one of you add this to the
>>> dev meeting agenda for this Th?
>>>
>>> On Mon, Dec 15, 2014 at 6:20 PM, Thomas Kluyver <takowl at gmail.com>
>>> wrote:
>>>>
>>>> Min and I were just talking to Aron Ahmadia and Chris Kees. We covered
>>>> several topics, but one of the more straightforward things we discussed was
>>>> the problem that storing signatures in the notebook gives for version
>>>> control - because any two changes will make the signature conflict, even if
>>>> the changes are at opposite ends of the notebook.
>>>>
>>>> We possibly shouldn't try to rework this before 3.0, but I started
>>>> thinking about alternatives, and wanted to put a couple out for discussion:
>>>>
>>>> 1. Store signatures next to notebooks - maybe a .ipynb.sig file, like
>>>> you sometimes see checksum files next to downloads. This has the advantage
>>>> that to move the notebook along with its trust, you can just move those two
>>>> files together, but the disadvantage that it clutters up your working
>>>> directory.
>>>> 2. An extra subdirectory, like we already use for checkpoints, but for
>>>> signatures. Less clutter, but more awkward to move a notebook and its
>>>> associated signature together.
>>>> 3. A per-user database storing hashes of all trusted notebooks.
>>>> Checking if a notebook is trusted then becomes: hash it, and check whether
>>>> that hash is in the database. The notebook's location on the filesystem is
>>>> irrelevant. This may be troublesome for people who sync their IPython
>>>> directory across multiple computers, or have home directories on an NFS
>>>> mount, though.
>>>>
>>>> Thomas
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>
>>> --
>>> Brian E. Granger
>>> Cal Poly State University, San Luis Obispo
>>> @ellisonbg on Twitter and GitHub
>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> @ellisonbg on Twitter and GitHub
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>

-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141215/5c0e8a35/attachment.html>

From maplabs at light42.com  Tue Dec 16 00:43:20 2014
From: maplabs at light42.com (=?utf-8?b?QnJpYW4gTSBIYW1saW4=?=)
Date: Mon, 15 Dec 2014 21:43:20 -0800
Subject: [IPython-dev] DOH Test Framework for Javascript and widgets
Message-ID: <20141215214320.9qbpp29u9cs8ok08@webmail.light42.com>

    FWIW - link ?(conversation on testing earlier today)
 ?
 http://dojotoolkit.org/reference-guide/1.10/util/doh.html

 --
Brian M Hamlin
 OSGeo California Chapter
 blog.light42.com

  


From benjaminrk at gmail.com  Tue Dec 16 02:21:50 2014
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 15 Dec 2014 23:21:50 -0800
Subject: [IPython-dev] Storing signatures outside the notebook
In-Reply-To: <CAH4pYpRvvssfghvQFaLS7ZFcmn_gMiJGAJj1nB77c2z26o-ZAw@mail.gmail.com>
References: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
	<CAH4pYpR-3PssaHUvZYuFkzZJOQPunrnS4TtwU_WvSkydKT3Fgg@mail.gmail.com>
	<CAH4pYpQttHerFTxGw9LxjRTUQgNxmpKC9-rJyHBDtXo56E711g@mail.gmail.com>
	<CAOvn4qj0zByk3Hkxib=phc3WvZPkFdAaDq+yBWyuOhn+oupJdQ@mail.gmail.com>
	<CAH4pYpRvvssfghvQFaLS7ZFcmn_gMiJGAJj1nB77c2z26o-ZAw@mail.gmail.com>
Message-ID: <CAHNn8BVQVaapStYfCZ=4-4+89R-52QAFwSjX6upy_5Uooa6yJA@mail.gmail.com>

I've implemented a version of the db ?option in #7244
<https://github.com/ipython/ipython/pull/7244>. Each approach has
advantages and disadvantages, but I don't relish having to chase notebook
files around with more things associated by filename. We do already do that
with checkpoints, though, so this could easily be done in the same way.

I do think we should do this for 3.0, if we decide we want to do it at all,
since we are doing an nbformat revision already.

-MinRK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141215/bafcf8b2/attachment.html>

From rjtskolasinski at gmail.com  Tue Dec 16 16:40:30 2014
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Tue, 16 Dec 2014 22:40:30 +0100
Subject: [IPython-dev] Making custom converter/template with nbconvert
Message-ID: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>

Hi guys,

I am working on making a converter that would translate ipython notebooks
into edX course format (OLX - open learning format).

My idea is to treat one ipython notebook as a subsection of the course
(it's like the whole chapter describing separate topic divided into few
units).

I tested how nbconvert --to html works and it's going into edX format very
smoothly.
So far I would only have to change location of the folders with figures and
remove 'end of paragraph' symbol.

Unfortunately each unit must go into separate html file and here comes my
first question: what is the easiest way to force nbconverter to export one
notebook into few html files (let's say Heading 1 will start every unit and
at the same time will be the name of it)?

I will be very grateful for your help and advice where I should start.

Best,
Rafal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141216/d1e3e5c3/attachment.html>

From takowl at gmail.com  Tue Dec 16 17:13:29 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 16 Dec 2014 14:13:29 -0800
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
Message-ID: <CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>

Hi Rafa?,

On 16 December 2014 at 13:40, Rafa? Skolasi?ski <rjtskolasinski at gmail.com>
wrote:

> Unfortunately each unit must go into separate html file and here comes my
> first question: what is the easiest way to force nbconverter to export one
> notebook into few html files (let's say Heading 1 will start every unit and
> at the same time will be the name of it)?


I think you will need to make a custom exporter - nbconvert supports
different output formats with exporter classes, which you can find here:
https://github.com/ipython/ipython/tree/master/IPython/nbconvert/exporters

Your case will be a bit trickier than most of those examples, because all
the TemplateExporter subclasses assume that they are writing a single main
output file. But there's no reason that an exporter class couldn't write
several files from one notebook.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141216/8a2b1082/attachment.html>

From ondrej.certik at gmail.com  Tue Dec 16 19:14:28 2014
From: ondrej.certik at gmail.com (=?UTF-8?B?T25kxZllaiDEjGVydMOtaw==?=)
Date: Tue, 16 Dec 2014 17:14:28 -0700
Subject: [IPython-dev] Storing signatures outside the notebook
In-Reply-To: <CAHNn8BVQVaapStYfCZ=4-4+89R-52QAFwSjX6upy_5Uooa6yJA@mail.gmail.com>
References: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
	<CAH4pYpR-3PssaHUvZYuFkzZJOQPunrnS4TtwU_WvSkydKT3Fgg@mail.gmail.com>
	<CAH4pYpQttHerFTxGw9LxjRTUQgNxmpKC9-rJyHBDtXo56E711g@mail.gmail.com>
	<CAOvn4qj0zByk3Hkxib=phc3WvZPkFdAaDq+yBWyuOhn+oupJdQ@mail.gmail.com>
	<CAH4pYpRvvssfghvQFaLS7ZFcmn_gMiJGAJj1nB77c2z26o-ZAw@mail.gmail.com>
	<CAHNn8BVQVaapStYfCZ=4-4+89R-52QAFwSjX6upy_5Uooa6yJA@mail.gmail.com>
Message-ID: <CADDwiVC2nCrdBBsoXcff1BQ1ptJcxnWp2N57aD27SpFNGct9eA@mail.gmail.com>

What is the point of the signature in the first place? Is it just to
verify that the .ipynb file is consistent, i.e. no accidental changes?
I just use git for that.

Ondrej

On Tue, Dec 16, 2014 at 12:21 AM, MinRK <benjaminrk at gmail.com> wrote:
> I've implemented a version of the db option in #7244. Each approach has
> advantages and disadvantages, but I don't relish having to chase notebook
> files around with more things associated by filename. We do already do that
> with checkpoints, though, so this could easily be done in the same way.
>
> I do think we should do this for 3.0, if we decide we want to do it at all,
> since we are doing an nbformat revision already.
>
> -MinRK
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From takowl at gmail.com  Wed Dec 17 00:39:23 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 16 Dec 2014 21:39:23 -0800
Subject: [IPython-dev] Storing signatures outside the notebook
In-Reply-To: <CADDwiVC2nCrdBBsoXcff1BQ1ptJcxnWp2N57aD27SpFNGct9eA@mail.gmail.com>
References: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
	<CAH4pYpR-3PssaHUvZYuFkzZJOQPunrnS4TtwU_WvSkydKT3Fgg@mail.gmail.com>
	<CAH4pYpQttHerFTxGw9LxjRTUQgNxmpKC9-rJyHBDtXo56E711g@mail.gmail.com>
	<CAOvn4qj0zByk3Hkxib=phc3WvZPkFdAaDq+yBWyuOhn+oupJdQ@mail.gmail.com>
	<CAH4pYpRvvssfghvQFaLS7ZFcmn_gMiJGAJj1nB77c2z26o-ZAw@mail.gmail.com>
	<CAHNn8BVQVaapStYfCZ=4-4+89R-52QAFwSjX6upy_5Uooa6yJA@mail.gmail.com>
	<CADDwiVC2nCrdBBsoXcff1BQ1ptJcxnWp2N57aD27SpFNGct9eA@mail.gmail.com>
Message-ID: <CAOvn4qh2GteR8qmbp+v4mz4sZ2zaFuN=cSexVA6i7RTUhSiNpw@mail.gmail.com>

On 16 December 2014 at 16:14, Ond?ej ?ert?k <ondrej.certik at gmail.com> wrote:

> What is the point of the signature in the first place? Is it just to
> verify that the .ipynb file is consistent, i.e. no accidental changes?
> I just use git for that.
>

Notebooks can contain rich output, including HTML and Javascript. But if
you open a notebook in a running notebook server, the Javascript is able to
send arbitrary code to your kernel for execution. We don't think users
should have to assume that opening a document means allowing it to
immediately run invisible code, so we sanitise the HTML outputs to exclude
anything that could cause code to be executed. But that will break some
outputs that depend on Javascript (e.g. d3 plots). Signatures are a
compromise to deal with this: once you've run the entire notebook, all the
output has come from cells you've run (and had a chance to inspect first),
so we mark it as trusted. When you open a trusted notebook, the output is
not sanitised, so stored Javascript can run.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141216/441c4c7f/attachment.html>

From benjaminrk at gmail.com  Wed Dec 17 00:54:03 2014
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 16 Dec 2014 21:54:03 -0800
Subject: [IPython-dev] Storing signatures outside the notebook
In-Reply-To: <CADDwiVC2nCrdBBsoXcff1BQ1ptJcxnWp2N57aD27SpFNGct9eA@mail.gmail.com>
References: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
	<CAH4pYpR-3PssaHUvZYuFkzZJOQPunrnS4TtwU_WvSkydKT3Fgg@mail.gmail.com>
	<CAH4pYpQttHerFTxGw9LxjRTUQgNxmpKC9-rJyHBDtXo56E711g@mail.gmail.com>
	<CAOvn4qj0zByk3Hkxib=phc3WvZPkFdAaDq+yBWyuOhn+oupJdQ@mail.gmail.com>
	<CAH4pYpRvvssfghvQFaLS7ZFcmn_gMiJGAJj1nB77c2z26o-ZAw@mail.gmail.com>
	<CAHNn8BVQVaapStYfCZ=4-4+89R-52QAFwSjX6upy_5Uooa6yJA@mail.gmail.com>
	<CADDwiVC2nCrdBBsoXcff1BQ1ptJcxnWp2N57aD27SpFNGct9eA@mail.gmail.com>
Message-ID: <CAHNn8BVq8eu8Pu6k4V6NWCzf8zH4+35CRDw1J1nZ=NNfZgboog@mail.gmail.com>

On Tue, Dec 16, 2014 at 4:14 PM, Ond?ej ?ert?k <ondrej.certik at gmail.com>
wrote:
>
> What is the point of the signature in the first place? Is it just to
> verify that the .ipynb file is consistent, i.e. no accidental changes?
> I just use git for that.
>

The question we want to answer is "Has the user explicitly trusted this
notebook?" We don't want to execute untrusted code in html/js outputs when
the notebook is opened. There are two mechanisms to trust a notebook:

1. manually mark that the notebook is trusted
2. execute the entire notebook yourself

The question is all in how IPython stores and checks whether that notebook
is trusted. Currently, IPython hashes the contents of the notebook plus a
secret stored in the user's IPython profile, and stores this value in the
notebook metadata. We don't care about the signature per se, we just want
to know whether the notebook is identical to one that the user trusted
before. Putting it in the metadata was the easiest way to ensure that the
signature followed the notebook around for the user, but it doesn't really
make sense for the signature to follow the notebook into VCS, or onto other
users' machines. One alternative under discussion here is to just hash
trusted notebooks (no secret key) and store the hashes in a
user-only-readable location.

-MinRK



>
> Ondrej
>
> On Tue, Dec 16, 2014 at 12:21 AM, MinRK <benjaminrk at gmail.com> wrote:
> > I've implemented a version of the db option in #7244. Each approach has
> > advantages and disadvantages, but I don't relish having to chase notebook
> > files around with more things associated by filename. We do already do
> that
> > with checkpoints, though, so this could easily be done in the same way.
> >
> > I do think we should do this for 3.0, if we decide we want to do it at
> all,
> > since we are doing an nbformat revision already.
> >
> > -MinRK
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141216/35615b78/attachment.html>

From bussonniermatthias at gmail.com  Wed Dec 17 03:23:35 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Wed, 17 Dec 2014 09:23:35 +0100
Subject: [IPython-dev] Storing signatures outside the notebook
In-Reply-To: <CAHNn8BVq8eu8Pu6k4V6NWCzf8zH4+35CRDw1J1nZ=NNfZgboog@mail.gmail.com>
References: <CAOvn4qjjVavAbcHTc6Gaymy87VxMwF2=F6N_fCjyTJLmHHHVBg@mail.gmail.com>
	<CAH4pYpR-3PssaHUvZYuFkzZJOQPunrnS4TtwU_WvSkydKT3Fgg@mail.gmail.com>
	<CAH4pYpQttHerFTxGw9LxjRTUQgNxmpKC9-rJyHBDtXo56E711g@mail.gmail.com>
	<CAOvn4qj0zByk3Hkxib=phc3WvZPkFdAaDq+yBWyuOhn+oupJdQ@mail.gmail.com>
	<CAH4pYpRvvssfghvQFaLS7ZFcmn_gMiJGAJj1nB77c2z26o-ZAw@mail.gmail.com>
	<CAHNn8BVQVaapStYfCZ=4-4+89R-52QAFwSjX6upy_5Uooa6yJA@mail.gmail.com>
	<CADDwiVC2nCrdBBsoXcff1BQ1ptJcxnWp2N57aD27SpFNGct9eA@mail.gmail.com>
	<CAHNn8BVq8eu8Pu6k4V6NWCzf8zH4+35CRDw1J1nZ=NNfZgboog@mail.gmail.com>
Message-ID: <906C7D7E-45CC-4DC8-9743-D3F0750D4ED7@gmail.com>


Le 17 d?c. 2014 ? 06:54, MinRK <benjaminrk at gmail.com> a ?crit :

> 
> 
> On Tue, Dec 16, 2014 at 4:14 PM, Ond?ej ?ert?k <ondrej.certik at gmail.com> wrote:
> What is the point of the signature in the first place? Is it just to
> verify that the .ipynb file is consistent, i.e. no accidental changes?
> I just use git for that.
> 
> The question we want to answer is "Has the user explicitly trusted this notebook?" We don't want to execute untrusted code in html/js outputs when the notebook is opened. There are two mechanisms to trust a notebook:
> 
> 1. manually mark that the notebook is trusted
> 2. execute the entire notebook yourself
> 
> The question is all in how IPython stores and checks whether that notebook is trusted. Currently, IPython hashes the contents of the notebook plus a secret stored in the user's IPython profile, and stores this value in the notebook metadata. We don't care about the signature per se, we just want to know whether the notebook is identical to one that the user trusted before. Putting it in the metadata was the easiest way to ensure that the signature followed the notebook around for the user, but it doesn't really make sense for the signature to follow the notebook into VCS, or onto other users' machines. One alternative under discussion here is to just hash trusted notebooks (no secret key) and store the hashes in a user-only-readable location.
> 
> -MinRK



Btw, purely hashing without secret, if one knows the algorithm make forging a notebook that collide easier. 
We should at least have a secrets salt to avoid that. So I would keep the secret. 
-- 
M
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141217/7ee5fbe4/attachment.html>

From tritemio at gmail.com  Wed Dec 17 14:37:55 2014
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Wed, 17 Dec 2014 11:37:55 -0800
Subject: [IPython-dev] Open-file dialog in the notebook for data files
Message-ID: <CANn2QUzUXDAjO7xwpcXbkhcr4UN3vAHco6_1BcV1pmTu7CA46Q@mail.gmail.com>

Dear ipython developers,

I occasionally have the need to get a data file name via a GUI dialog when
using the notebook.

Opening a QT dialog from the notebook involves initializing the QT gui. At
the same time I always use inline plots so I also need to enable the inline
plot backed. This can be done with

    %gui qt
    %matplotlib inline

These commands work when executed manually on different cells.
However, the previous commands will freeze the ipython notebook if they are
in the same cell or if executing the notebook with Run-All (open issue #5798
<https://github.com/ipython/ipython/issues/5798>). The problem is
multi-platform and occurs both with PySide and PyQT.

Is there any workaround for this? It really bothers me to lose the ability
to Run-All when I enable the QT gui.

Alternatively would be possible to use a javascript dialog to select a file
on the filesystem? If yes an example would be nice.

PS: I'm using ipython 2.x

Thanks,
Antonio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141217/ada73526/attachment.html>

From hughesadam87 at gmail.com  Wed Dec 17 14:53:39 2014
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Wed, 17 Dec 2014 14:53:39 -0500
Subject: [IPython-dev] Open-file dialog in the notebook for data files
In-Reply-To: <CANn2QUzUXDAjO7xwpcXbkhcr4UN3vAHco6_1BcV1pmTu7CA46Q@mail.gmail.com>
References: <CANn2QUzUXDAjO7xwpcXbkhcr4UN3vAHco6_1BcV1pmTu7CA46Q@mail.gmail.com>
Message-ID: <CAMHV+dDSzx8C8z_+7swvejGMz08LjhEkfn0b9zxdEnio3v9rwA@mail.gmail.com>

Would you be able to just use an IPython widget to get the file name?

On Wed, Dec 17, 2014 at 2:37 PM, Antonino Ingargiola <tritemio at gmail.com>
wrote:
>
> Dear ipython developers,
>
> I occasionally have the need to get a data file name via a GUI dialog when
> using the notebook.
>
> Opening a QT dialog from the notebook involves initializing the QT gui. At
> the same time I always use inline plots so I also need to enable the inline
> plot backed. This can be done with
>
>     %gui qt
>     %matplotlib inline
>
> These commands work when executed manually on different cells.
> However, the previous commands will freeze the ipython notebook if they
> are in the same cell or if executing the notebook with Run-All (open issue
> #5798 <https://github.com/ipython/ipython/issues/5798>). The problem is
> multi-platform and occurs both with PySide and PyQT.
>
> Is there any workaround for this? It really bothers me to lose the ability
> to Run-All when I enable the QT gui.
>
> Alternatively would be possible to use a javascript dialog to select a
> file on the filesystem? If yes an example would be nice.
>
> PS: I'm using ipython 2.x
>
> Thanks,
> Antonio
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141217/e76099e8/attachment.html>

From tritemio at gmail.com  Wed Dec 17 16:11:39 2014
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Wed, 17 Dec 2014 13:11:39 -0800
Subject: [IPython-dev] Open-file dialog in the notebook for data files
In-Reply-To: <CAMHV+dDSzx8C8z_+7swvejGMz08LjhEkfn0b9zxdEnio3v9rwA@mail.gmail.com>
References: <CANn2QUzUXDAjO7xwpcXbkhcr4UN3vAHco6_1BcV1pmTu7CA46Q@mail.gmail.com>
	<CAMHV+dDSzx8C8z_+7swvejGMz08LjhEkfn0b9zxdEnio3v9rwA@mail.gmail.com>
Message-ID: <CANn2QUwA-62itNHin0F=BahNrsaU7_61axBUiKz7QgR=M5WMkw@mail.gmail.com>

Yes, is there a widget I can use for that? If yes I must have missed it...

Antonio

On Wed, Dec 17, 2014 at 11:53 AM, Adam Hughes <hughesadam87 at gmail.com>
wrote:
>
> Would you be able to just use an IPython widget to get the file name?
>
> On Wed, Dec 17, 2014 at 2:37 PM, Antonino Ingargiola <tritemio at gmail.com>
> wrote:
>
>> Dear ipython developers,
>>
>> I occasionally have the need to get a data file name via a GUI dialog
>> when using the notebook.
>>
>> Opening a QT dialog from the notebook involves initializing the QT gui.
>> At the same time I always use inline plots so I also need to enable the
>> inline plot backed. This can be done with
>>
>>     %gui qt
>>     %matplotlib inline
>>
>> These commands work when executed manually on different cells.
>> However, the previous commands will freeze the ipython notebook if they
>> are in the same cell or if executing the notebook with Run-All (open issue
>> #5798 <https://github.com/ipython/ipython/issues/5798>). The problem is
>> multi-platform and occurs both with PySide and PyQT.
>>
>> Is there any workaround for this? It really bothers me to lose the
>> ability to Run-All when I enable the QT gui.
>>
>> Alternatively would be possible to use a javascript dialog to select a
>> file on the filesystem? If yes an example would be nice.
>>
>> PS: I'm using ipython 2.x
>>
>> Thanks,
>> Antonio
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141217/b094e68a/attachment.html>

From hughesadam87 at gmail.com  Wed Dec 17 16:25:01 2014
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Wed, 17 Dec 2014 16:25:01 -0500
Subject: [IPython-dev] Open-file dialog in the notebook for data files
In-Reply-To: <CANn2QUwA-62itNHin0F=BahNrsaU7_61axBUiKz7QgR=M5WMkw@mail.gmail.com>
References: <CANn2QUzUXDAjO7xwpcXbkhcr4UN3vAHco6_1BcV1pmTu7CA46Q@mail.gmail.com>
	<CAMHV+dDSzx8C8z_+7swvejGMz08LjhEkfn0b9zxdEnio3v9rwA@mail.gmail.com>
	<CANn2QUwA-62itNHin0F=BahNrsaU7_61axBUiKz7QgR=M5WMkw@mail.gmail.com>
Message-ID: <CAMHV+dBijMDkz2tLz1FtFkUGXuYCObfq4MQ1UtLx0gSRYtqaVw@mail.gmail.com>

You can do a widget box and next to it a button, like "Load file" and then
link the button click even to the name of the text in the file.   I can
give you some help later this week if you have trouble finding how to do
that (we are about to pack up computers for a lab move)

On Wed, Dec 17, 2014 at 4:11 PM, Antonino Ingargiola <tritemio at gmail.com>
wrote:
>
> Yes, is there a widget I can use for that? If yes I must have missed it...
>
> Antonio
>
> On Wed, Dec 17, 2014 at 11:53 AM, Adam Hughes <hughesadam87 at gmail.com>
> wrote:
>>
>> Would you be able to just use an IPython widget to get the file name?
>>
>> On Wed, Dec 17, 2014 at 2:37 PM, Antonino Ingargiola <tritemio at gmail.com>
>> wrote:
>>
>>> Dear ipython developers,
>>>
>>> I occasionally have the need to get a data file name via a GUI dialog
>>> when using the notebook.
>>>
>>> Opening a QT dialog from the notebook involves initializing the QT gui.
>>> At the same time I always use inline plots so I also need to enable the
>>> inline plot backed. This can be done with
>>>
>>>     %gui qt
>>>     %matplotlib inline
>>>
>>> These commands work when executed manually on different cells.
>>> However, the previous commands will freeze the ipython notebook if they
>>> are in the same cell or if executing the notebook with Run-All (open issue
>>> #5798 <https://github.com/ipython/ipython/issues/5798>). The problem is
>>> multi-platform and occurs both with PySide and PyQT.
>>>
>>> Is there any workaround for this? It really bothers me to lose the
>>> ability to Run-All when I enable the QT gui.
>>>
>>> Alternatively would be possible to use a javascript dialog to select a
>>> file on the filesystem? If yes an example would be nice.
>>>
>>> PS: I'm using ipython 2.x
>>>
>>> Thanks,
>>> Antonio
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141217/2cd72858/attachment.html>

From tritemio at gmail.com  Wed Dec 17 18:08:13 2014
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Wed, 17 Dec 2014 15:08:13 -0800
Subject: [IPython-dev] Open-file dialog in the notebook for data files
In-Reply-To: <CAMHV+dBijMDkz2tLz1FtFkUGXuYCObfq4MQ1UtLx0gSRYtqaVw@mail.gmail.com>
References: <CANn2QUzUXDAjO7xwpcXbkhcr4UN3vAHco6_1BcV1pmTu7CA46Q@mail.gmail.com>
	<CAMHV+dDSzx8C8z_+7swvejGMz08LjhEkfn0b9zxdEnio3v9rwA@mail.gmail.com>
	<CANn2QUwA-62itNHin0F=BahNrsaU7_61axBUiKz7QgR=M5WMkw@mail.gmail.com>
	<CAMHV+dBijMDkz2tLz1FtFkUGXuYCObfq4MQ1UtLx0gSRYtqaVw@mail.gmail.com>
Message-ID: <CANn2QUz7KaHOrRf7J-r3c2it1r3UwJKwvvCGB2RZ1=66Au75rA@mail.gmail.com>

Hi Adam,

thanks for offering to help. Just to clarify, I need to browse the
filesystem to find the file, not just show a list of files in a folder.

I have some basic familiarity with the standard ipython widgets but I don't
remember having seen something resembling an open-file dialog.

Any pointer is appreciated,
Antonio

On Wed, Dec 17, 2014 at 1:25 PM, Adam Hughes <hughesadam87 at gmail.com> wrote:
>
> You can do a widget box and next to it a button, like "Load file" and then
> link the button click even to the name of the text in the file.   I can
> give you some help later this week if you have trouble finding how to do
> that (we are about to pack up computers for a lab move)
>
> On Wed, Dec 17, 2014 at 4:11 PM, Antonino Ingargiola <tritemio at gmail.com>
> wrote:
>>
>> Yes, is there a widget I can use for that? If yes I must have missed it...
>>
>> Antonio
>>
>> On Wed, Dec 17, 2014 at 11:53 AM, Adam Hughes <hughesadam87 at gmail.com>
>> wrote:
>>>
>>> Would you be able to just use an IPython widget to get the file name?
>>>
>>> On Wed, Dec 17, 2014 at 2:37 PM, Antonino Ingargiola <tritemio at gmail.com
>>> > wrote:
>>>
>>>> Dear ipython developers,
>>>>
>>>> I occasionally have the need to get a data file name via a GUI dialog
>>>> when using the notebook.
>>>>
>>>> Opening a QT dialog from the notebook involves initializing the QT gui.
>>>> At the same time I always use inline plots so I also need to enable the
>>>> inline plot backed. This can be done with
>>>>
>>>>     %gui qt
>>>>     %matplotlib inline
>>>>
>>>> These commands work when executed manually on different cells.
>>>> However, the previous commands will freeze the ipython notebook if they
>>>> are in the same cell or if executing the notebook with Run-All (open issue
>>>> #5798 <https://github.com/ipython/ipython/issues/5798>). The problem
>>>> is multi-platform and occurs both with PySide and PyQT.
>>>>
>>>> Is there any workaround for this? It really bothers me to lose the
>>>> ability to Run-All when I enable the QT gui.
>>>>
>>>> Alternatively would be possible to use a javascript dialog to select a
>>>> file on the filesystem? If yes an example would be nice.
>>>>
>>>> PS: I'm using ipython 2.x
>>>>
>>>> Thanks,
>>>> Antonio
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141217/cefe9a58/attachment.html>

From hughesadam87 at gmail.com  Wed Dec 17 18:36:57 2014
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Wed, 17 Dec 2014 18:36:57 -0500
Subject: [IPython-dev] Open-file dialog in the notebook for data files
In-Reply-To: <CANn2QUz7KaHOrRf7J-r3c2it1r3UwJKwvvCGB2RZ1=66Au75rA@mail.gmail.com>
References: <CANn2QUzUXDAjO7xwpcXbkhcr4UN3vAHco6_1BcV1pmTu7CA46Q@mail.gmail.com>
	<CAMHV+dDSzx8C8z_+7swvejGMz08LjhEkfn0b9zxdEnio3v9rwA@mail.gmail.com>
	<CANn2QUwA-62itNHin0F=BahNrsaU7_61axBUiKz7QgR=M5WMkw@mail.gmail.com>
	<CAMHV+dBijMDkz2tLz1FtFkUGXuYCObfq4MQ1UtLx0gSRYtqaVw@mail.gmail.com>
	<CANn2QUz7KaHOrRf7J-r3c2it1r3UwJKwvvCGB2RZ1=66Au75rA@mail.gmail.com>
Message-ID: <CAMHV+dATy_-yAtVBRGZgy6q_J8Zs3NJ358JEVcTPOLEgBCPZ1A@mail.gmail.com>

Oh, ya sorry, I don't think there's a widget (AFAIK) that could pop up a
file system browser.  While there are pre-existing widgets like that in the
Traits library, they're probably not supported by the lighter Traitlets
library on which Traits is built.

On Wed, Dec 17, 2014 at 6:08 PM, Antonino Ingargiola <tritemio at gmail.com>
wrote:
>
> Hi Adam,
>
> thanks for offering to help. Just to clarify, I need to browse the
> filesystem to find the file, not just show a list of files in a folder.
>
> I have some basic familiarity with the standard ipython widgets but I
> don't remember having seen something resembling an open-file dialog.
>
> Any pointer is appreciated,
> Antonio
>
> On Wed, Dec 17, 2014 at 1:25 PM, Adam Hughes <hughesadam87 at gmail.com>
> wrote:
>>
>> You can do a widget box and next to it a button, like "Load file" and
>> then link the button click even to the name of the text in the file.   I
>> can give you some help later this week if you have trouble finding how to
>> do that (we are about to pack up computers for a lab move)
>>
>> On Wed, Dec 17, 2014 at 4:11 PM, Antonino Ingargiola <tritemio at gmail.com>
>> wrote:
>>>
>>> Yes, is there a widget I can use for that? If yes I must have missed
>>> it...
>>>
>>> Antonio
>>>
>>> On Wed, Dec 17, 2014 at 11:53 AM, Adam Hughes <hughesadam87 at gmail.com>
>>> wrote:
>>>>
>>>> Would you be able to just use an IPython widget to get the file name?
>>>>
>>>> On Wed, Dec 17, 2014 at 2:37 PM, Antonino Ingargiola <
>>>> tritemio at gmail.com> wrote:
>>>>
>>>>> Dear ipython developers,
>>>>>
>>>>> I occasionally have the need to get a data file name via a GUI dialog
>>>>> when using the notebook.
>>>>>
>>>>> Opening a QT dialog from the notebook involves initializing the QT
>>>>> gui. At the same time I always use inline plots so I also need to enable
>>>>> the inline plot backed. This can be done with
>>>>>
>>>>>     %gui qt
>>>>>     %matplotlib inline
>>>>>
>>>>> These commands work when executed manually on different cells.
>>>>> However, the previous commands will freeze the ipython notebook if
>>>>> they are in the same cell or if executing the notebook with Run-All (open
>>>>> issue #5798 <https://github.com/ipython/ipython/issues/5798>). The
>>>>> problem is multi-platform and occurs both with PySide and PyQT.
>>>>>
>>>>> Is there any workaround for this? It really bothers me to lose the
>>>>> ability to Run-All when I enable the QT gui.
>>>>>
>>>>> Alternatively would be possible to use a javascript dialog to select a
>>>>> file on the filesystem? If yes an example would be nice.
>>>>>
>>>>> PS: I'm using ipython 2.x
>>>>>
>>>>> Thanks,
>>>>> Antonio
>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141217/bb61ae0b/attachment.html>

From tritemio at gmail.com  Wed Dec 17 19:15:04 2014
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Wed, 17 Dec 2014 16:15:04 -0800
Subject: [IPython-dev] Open-file dialog in the notebook for data files
In-Reply-To: <CAMHV+dATy_-yAtVBRGZgy6q_J8Zs3NJ358JEVcTPOLEgBCPZ1A@mail.gmail.com>
References: <CANn2QUzUXDAjO7xwpcXbkhcr4UN3vAHco6_1BcV1pmTu7CA46Q@mail.gmail.com>
	<CAMHV+dDSzx8C8z_+7swvejGMz08LjhEkfn0b9zxdEnio3v9rwA@mail.gmail.com>
	<CANn2QUwA-62itNHin0F=BahNrsaU7_61axBUiKz7QgR=M5WMkw@mail.gmail.com>
	<CAMHV+dBijMDkz2tLz1FtFkUGXuYCObfq4MQ1UtLx0gSRYtqaVw@mail.gmail.com>
	<CANn2QUz7KaHOrRf7J-r3c2it1r3UwJKwvvCGB2RZ1=66Au75rA@mail.gmail.com>
	<CAMHV+dATy_-yAtVBRGZgy6q_J8Zs3NJ358JEVcTPOLEgBCPZ1A@mail.gmail.com>
Message-ID: <CANn2QUxndNLwEkZe7FY6+OrWZmG8nhJ=ZpkO9f8f=ULMy4Mu5w@mail.gmail.com>

I found that I can trigger an open file dialog simply with

<input type="file" id="theFile" />

in a markdown cell.

The question is, is there a way to put the file name in a python variable?

Antonio


On Wed, Dec 17, 2014 at 3:36 PM, Adam Hughes <hughesadam87 at gmail.com> wrote:
>
> Oh, ya sorry, I don't think there's a widget (AFAIK) that could pop up a
> file system browser.  While there are pre-existing widgets like that in the
> Traits library, they're probably not supported by the lighter Traitlets
> library on which Traits is built.
>
> On Wed, Dec 17, 2014 at 6:08 PM, Antonino Ingargiola <tritemio at gmail.com>
> wrote:
>>
>> Hi Adam,
>>
>> thanks for offering to help. Just to clarify, I need to browse the
>> filesystem to find the file, not just show a list of files in a folder.
>>
>> I have some basic familiarity with the standard ipython widgets but I
>> don't remember having seen something resembling an open-file dialog.
>>
>> Any pointer is appreciated,
>> Antonio
>>
>> On Wed, Dec 17, 2014 at 1:25 PM, Adam Hughes <hughesadam87 at gmail.com>
>> wrote:
>>>
>>> You can do a widget box and next to it a button, like "Load file" and
>>> then link the button click even to the name of the text in the file.   I
>>> can give you some help later this week if you have trouble finding how to
>>> do that (we are about to pack up computers for a lab move)
>>>
>>> On Wed, Dec 17, 2014 at 4:11 PM, Antonino Ingargiola <tritemio at gmail.com
>>> > wrote:
>>>>
>>>> Yes, is there a widget I can use for that? If yes I must have missed
>>>> it...
>>>>
>>>> Antonio
>>>>
>>>> On Wed, Dec 17, 2014 at 11:53 AM, Adam Hughes <hughesadam87 at gmail.com>
>>>> wrote:
>>>>>
>>>>> Would you be able to just use an IPython widget to get the file name?
>>>>>
>>>>> On Wed, Dec 17, 2014 at 2:37 PM, Antonino Ingargiola <
>>>>> tritemio at gmail.com> wrote:
>>>>>
>>>>>> Dear ipython developers,
>>>>>>
>>>>>> I occasionally have the need to get a data file name via a GUI dialog
>>>>>> when using the notebook.
>>>>>>
>>>>>> Opening a QT dialog from the notebook involves initializing the QT
>>>>>> gui. At the same time I always use inline plots so I also need to enable
>>>>>> the inline plot backed. This can be done with
>>>>>>
>>>>>>     %gui qt
>>>>>>     %matplotlib inline
>>>>>>
>>>>>> These commands work when executed manually on different cells.
>>>>>> However, the previous commands will freeze the ipython notebook if
>>>>>> they are in the same cell or if executing the notebook with Run-All (open
>>>>>> issue #5798 <https://github.com/ipython/ipython/issues/5798>). The
>>>>>> problem is multi-platform and occurs both with PySide and PyQT.
>>>>>>
>>>>>> Is there any workaround for this? It really bothers me to lose the
>>>>>> ability to Run-All when I enable the QT gui.
>>>>>>
>>>>>> Alternatively would be possible to use a javascript dialog to select
>>>>>> a file on the filesystem? If yes an example would be nice.
>>>>>>
>>>>>> PS: I'm using ipython 2.x
>>>>>>
>>>>>> Thanks,
>>>>>> Antonio
>>>>>>
>>>>>> _______________________________________________
>>>>>> IPython-dev mailing list
>>>>>> IPython-dev at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141217/3cc07dd4/attachment.html>

From zvoros at gmail.com  Thu Dec 18 02:36:29 2014
From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Thu, 18 Dec 2014 08:36:29 +0100
Subject: [IPython-dev] Open-file dialog in the notebook for data files
In-Reply-To: <CANn2QUz7KaHOrRf7J-r3c2it1r3UwJKwvvCGB2RZ1=66Au75rA@mail.gmail.com>
References: <CANn2QUzUXDAjO7xwpcXbkhcr4UN3vAHco6_1BcV1pmTu7CA46Q@mail.gmail.com>	<CAMHV+dDSzx8C8z_+7swvejGMz08LjhEkfn0b9zxdEnio3v9rwA@mail.gmail.com>	<CANn2QUwA-62itNHin0F=BahNrsaU7_61axBUiKz7QgR=M5WMkw@mail.gmail.com>	<CAMHV+dBijMDkz2tLz1FtFkUGXuYCObfq4MQ1UtLx0gSRYtqaVw@mail.gmail.com>
	<CANn2QUz7KaHOrRf7J-r3c2it1r3UwJKwvvCGB2RZ1=66Au75rA@mail.gmail.com>
Message-ID: <549283FD.5060301@gmail.com>

Antonio,

On 12/18/2014 12:08 AM, Antonino Ingargiola wrote:
> Hi Adam,
>
> thanks for offering to help. Just to clarify, I need to browse the 
> filesystem to find the file, not just show a list of files in a folder.
>
But, as far as I know, browsing the filesystem will give you only the 
filename, and not its path. In order to get the full path, you'll need a 
dedicated file manager library. There are tons of ready-to-use jquery 
plugins on the net.

Cheers,
Zolt?n


From apps.embedded at gmail.com  Thu Dec 18 07:04:42 2014
From: apps.embedded at gmail.com (Apps Embedded)
Date: Thu, 18 Dec 2014 13:04:42 +0100
Subject: [IPython-dev] IPython notebook available under the Android platform
Message-ID: <CAO=GJvxiFa5Wsmrbw2UjYAXYqET4r9WcOLZ7D-ddSY7OK1si9w@mail.gmail.com>

Hi,

We developped an Android app called "LabPy Console" using Python and the
SciPy packages (Python is a trademark of the Python Software Foundation).
We had the trademark agreement from the PSF for posting such a work under
the Play Store.

Within this app, we are using the IPython package.

Our app is under the GPL v3 licence and our source code is available
through our support web site :
https://code.google.com/p/labpyconsoleapp/

Using the IPython notebook extension is a new feature on the Android
platform.
>From your point of view, we would like to know if there is no legal issue
from a licence point of view for instance.
Does the LabPy Console app name is ok for you too?

If you have any remarks or question or suggestions (even bugs...), our
contact mail is apps.embedded at gmail.com

Best regards.

Apps Embedded Team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141218/922dfc9b/attachment.html>

From rjtskolasinski at gmail.com  Thu Dec 18 09:49:43 2014
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Thu, 18 Dec 2014 15:49:43 +0100
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
	<CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
Message-ID: <CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>

Hi Thomas,

Yes, that sounds like a logical way. But I still have few question about
how to do it.

First, do I also need to have custom preprocessor? At which point of
conversion it will be the most convenient to make changes?

Should my custom exporter class inherit from a TemplateExporter? Or maybe
separation into files should happen at this level?

And last, once I will have my exporter class will it be enough to specify
which exporter should be used in config file and provide config to
nbconvert via --config mycfg.py ?

Thanks for all your help!

Best,
Rafal

P.S. If there are any useful guides or FAQs about writing custom converters
that would be very helpful. I looked for some on my own but didn't have
much luck.

2014-12-16 23:13 GMT+01:00 Thomas Kluyver <takowl at gmail.com>:
>
> Hi Rafa?,
>
> On 16 December 2014 at 13:40, Rafa? Skolasi?ski <rjtskolasinski at gmail.com>
> wrote:
>
>> Unfortunately each unit must go into separate html file and here comes my
>> first question: what is the easiest way to force nbconverter to export one
>> notebook into few html files (let's say Heading 1 will start every unit and
>> at the same time will be the name of it)?
>
>
> I think you will need to make a custom exporter - nbconvert supports
> different output formats with exporter classes, which you can find here:
> https://github.com/ipython/ipython/tree/master/IPython/nbconvert/exporters
>
> Your case will be a bit trickier than most of those examples, because all
> the TemplateExporter subclasses assume that they are writing a single main
> output file. But there's no reason that an exporter class couldn't write
> several files from one notebook.
>
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141218/ab8573d5/attachment.html>

From wes.turner at gmail.com  Thu Dec 18 11:07:33 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Thu, 18 Dec 2014 10:07:33 -0600
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
	<CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
	<CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
Message-ID: <CACfEFw9WgrCyUONqsCTwP3E7A1PQg0QJFEMzHBY7G2fMDEQmxQ@mail.gmail.com>

Great idea!

Are there imports which must/should be repeated in each output file?

What is the advantage of splitting one notebook into multiple output
documents?
(Why not just create multiple documents?)

It looks like:

 ipython/IPython/nbconvert/exporters/htmlexporter.py:HTMLExporter

https://github.com/ipython/ipython/blob/master/IPython/nbconvert/exporters/html.py#L61
 extends
 ipython/IPython/nbconvert/exporters/templateexporter.py:TemplateExporter
 extends
 ipython/IPython/nbconvert/exporters/exporter.py:Exporter

https://github.com/ipython/ipython/blob/master/IPython/nbconvert/exporters/exporter.py#L45

If each document is to be the same, you probably don't need to reimplement
_load_template.

I could be wrong, but it would probably be easiest to split nb_copy by
iterating through the notebook cells.


https://github.com/ipython/ipython/blob/master/IPython/nbconvert/exporters/templateexporter.py#L196


class EDXExporter(HTMLExporter):
    def from_notebook_node(nb, resources, **kw):
        nb_copy, resources = super(EDXExporter,
self).from_notebook_node(nb, resources, **kw)
        output = collections.OrderedDict()
        chunks = split_notebook_by_headings?(nb_copy)
        for section in chunks:
            output[section['name']] = self.template.render(nb=nb_copy,
resources=resources)
        return output, resources

https://github.com/ipython/ipython/blob/master/IPython/nbconvert/nbconvertapp.py#L308

https://github.com/ipython/ipython/blob/master/IPython/nbconvert/writers/files.py#L46

class MultipleFilesWriter(FilesWriter):
    def write(self, output, resources, notebook_name=None, **kw):
        # ...
        for section_name, _output in output.iteritems():
             # ...


On Thu, Dec 18, 2014 at 8:49 AM, Rafa? Skolasi?ski <rjtskolasinski at gmail.com
> wrote:
>
> Hi Thomas,
>
> Yes, that sounds like a logical way. But I still have few question about
> how to do it.
>
> First, do I also need to have custom preprocessor? At which point of
> conversion it will be the most convenient to make changes?
>
> Should my custom exporter class inherit from a TemplateExporter? Or maybe
> separation into files should happen at this level?
>
> And last, once I will have my exporter class will it be enough to specify
> which exporter should be used in config file and provide config to
> nbconvert via --config mycfg.py ?
>
> Thanks for all your help!
>
> Best,
> Rafal
>
> P.S. If there are any useful guides or FAQs about writing custom
> converters that would be very helpful. I looked for some on my own but
> didn't have much luck.
>
> 2014-12-16 23:13 GMT+01:00 Thomas Kluyver <takowl at gmail.com>:
>
>> Hi Rafa?,
>>
>> On 16 December 2014 at 13:40, Rafa? Skolasi?ski <rjtskolasinski at gmail.com
>> > wrote:
>>
>>> Unfortunately each unit must go into separate html file and here comes
>>> my first question: what is the easiest way to force nbconverter to export
>>> one notebook into few html files (let's say Heading 1 will start every unit
>>> and at the same time will be the name of it)?
>>
>>
>> I think you will need to make a custom exporter - nbconvert supports
>> different output formats with exporter classes, which you can find here:
>> https://github.com/ipython/ipython/tree/master/IPython/nbconvert/exporters
>>
>> Your case will be a bit trickier than most of those examples, because all
>> the TemplateExporter subclasses assume that they are writing a single main
>> output file. But there's no reason that an exporter class couldn't write
>> several files from one notebook.
>>
>> Thomas
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141218/5b5924eb/attachment.html>

From bussonniermatthias at gmail.com  Thu Dec 18 11:09:08 2014
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 18 Dec 2014 17:09:08 +0100
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
	<CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
	<CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
Message-ID: <17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com>


Le 18 d?c. 2014 ? 15:49, Rafa? Skolasi?ski a ?crit :

> Hi Thomas,
> 
> Yes, that sounds like a logical way. But I still have few question about how to do it.
> 
> First, do I also need to have custom preprocessor? At which point of conversion it will be the most convenient to make changes? 

that's up to how you want to design your exporter. 
You can either have a preprocessor that split the notebooks in chunks and do the conversion after. 
So virtually running nbconvert N times on n chucks of the notebook.

> Should my custom exporter class inherit from a TemplateExporter? Or maybe separation into files should happen at this level?

You are not obliged to, but I think you should at least inherit Exporter if you like your custom one to be accepted by nbconvert. 
The architecture is not really meant to export into many files, except  for the resources file (Png, Svg) that do into a separate folder. 

If you write your custom writer then your Exporter can most likely produce many file that would be written at once. 

> 
> And last, once I will have my exporter class will it be enough to specify which exporter should be used in config file and provide config to nbconvert via --config mycfg.py ?

Yes that should be enough., any trait lets of your custom class should be automatically set to the value of the config.

> 
> Thanks for all your help!

Feel free to send us feedback on what you think of the APi or what limitation there is. 

Thanks !

-- 
M


> 
> Best,
> Rafal
> 
> P.S. If there are any useful guides or FAQs about writing custom converters that would be very helpful. I looked for some on my own but didn't have much luck.
> 
> 2014-12-16 23:13 GMT+01:00 Thomas Kluyver <takowl at gmail.com>:
> Hi Rafa?,
> 
> On 16 December 2014 at 13:40, Rafa? Skolasi?ski <rjtskolasinski at gmail.com> wrote:
> Unfortunately each unit must go into separate html file and here comes my first question: what is the easiest way to force nbconverter to export one notebook into few html files (let's say Heading 1 will start every unit and at the same time will be the name of it)? 
> 
> I think you will need to make a custom exporter - nbconvert supports different output formats with exporter classes, which you can find here: https://github.com/ipython/ipython/tree/master/IPython/nbconvert/exporters
> 
> Your case will be a bit trickier than most of those examples, because all the TemplateExporter subclasses assume that they are writing a single main output file. But there's no reason that an exporter class couldn't write several files from one notebook.
> 
> Thomas
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141218/cb779c65/attachment.html>

From rjtskolasinski at gmail.com  Thu Dec 18 11:58:39 2014
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Thu, 18 Dec 2014 17:58:39 +0100
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com>
References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
	<CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
	<CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
	<17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com>
Message-ID: <CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com>

Hi guys,

Thanks for extensive response!

In my output there is not need to be any imports. Output files will be only
htmls to be put into edX course.

The need of splitting notebook into few output files comes from making it
as friendly as possible to creators of the course content.
In principle they could write every unit in a separate notebook - but,
because units together build a subsection of the course which describes one
topic, it's more convenient to write them in one notebooks and separate by
the largest headings.

I tested how converting to html, that is already provided in nbconvert, is
accepted by edX and it went quite well. I only needed to change two things
in the output file ( change path to the folders with figures and remove
'end of paragraph' symbol ) .


In this situation maybe what Matthias proposed is a way to go:


> that's up to how you want to design your exporter.
> You can either have a preprocessor that split the notebooks in chunks and
> do the conversion after.
> So virtually running nbconvert N times on n chucks of the notebook.
>


Then for example I could have in the notebook to export sth like that:

<Heading 1 cell> first_unit_name </Heading 1 cell>

Cells with stuff for first unit


<Heading 2 cell> second_unit_name </Heading 2 cell>

Cells with stuff for second unit

etc...

And with the output I would see first_unit_name.html and
second_unit_name.html
where content of each of them would look like it would be created only from
notebook
containing cells with corresponding content.


Later I would like for example kick out every cells with python code
leaving only their output (but this is for a later stage of development,
after having the splitting going well).





Oh, I have one question about the preprocessors. I found little old post
https://blog.safaribooksonline.com/2013/12/18/ipython-notebook-plugins/
about how to write custom plugins to nbconvert. Despite the fact that
Jonathan was using transformers instead of preprocessors it worked when I
changed config file only from:

C = get_config()
c.Exporter.transformers = [?exclude_transformer.ExcludeTransformer?]

to:
c = get_config()
c.Exporter.preprocessors = ["exclude_transformer.ExcludeTransformer"]



But when I tried to change usage of Transformer into Preprocessor:
from http://pastebin.com/rQufqzUr into http://pastebin.com/Q8mgaYAJ

(correcting ofc line in config into c.Exporter.preprocessors =
["exclude_preprocessor.ExcludePreprocessor"])

I got the following error: http://pastebin.com/MSAHR2PP

Any idea why?


Oh... And one more question :). While writing some code I love to test
things interactively (this is why I loved ipython so much).
Is there any easy way to play in ipython notebook with other notebook that
goes as argument into all converter arguments?

Like I would like to see by hand what every worksheet in code like here
http://pastebin.com/rQufqzUr contains.


Best,
Rafal



> Should my custom exporter class inherit from a TemplateExporter? Or maybe
> separation into files should happen at this level?
>
>
> You are not obliged to, but I think you should at least inherit Exporter
> if you like your custom one to be accepted by nbconvert.
> The architecture is not really meant to export into many files, except
>  for the resources file (Png, Svg) that do into a separate folder.
>
> If you write your custom writer then your Exporter can most likely produce
> many file that would be written at once.
>
>
> And last, once I will have my exporter class will it be enough to specify
> which exporter should be used in config file and provide config to
> nbconvert via --config mycfg.py ?
>
>
> Yes that should be enough., any trait lets of your custom class should be
> automatically set to the value of the config.
>
>
> Thanks for all your help!
>
>
> Feel free to send us feedback on what you think of the APi or what
> limitation there is.
>
> Thanks !
>
> --
> M
>
>
>
> Best,
> Rafal
>
> P.S. If there are any useful guides or FAQs about writing custom
> converters that would be very helpful. I looked for some on my own but
> didn't have much luck.
>
> 2014-12-16 23:13 GMT+01:00 Thomas Kluyver <takowl at gmail.com>:
>>
>> Hi Rafa?,
>>
>> On 16 December 2014 at 13:40, Rafa? Skolasi?ski <rjtskolasinski at gmail.com
>> > wrote:
>>
>>> Unfortunately each unit must go into separate html file and here comes
>>> my first question: what is the easiest way to force nbconverter to export
>>> one notebook into few html files (let's say Heading 1 will start every unit
>>> and at the same time will be the name of it)?
>>
>>
>> I think you will need to make a custom exporter - nbconvert supports
>> different output formats with exporter classes, which you can find here:
>> https://github.com/ipython/ipython/tree/master/IPython/nbconvert/exporters
>>
>> Your case will be a bit trickier than most of those examples, because all
>> the TemplateExporter subclasses assume that they are writing a single main
>> output file. But there's no reason that an exporter class couldn't write
>> several files from one notebook.
>>
>> Thomas
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141218/40da4623/attachment.html>

From rjtskolasinski at gmail.com  Thu Dec 18 12:06:42 2014
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Thu, 18 Dec 2014 18:06:42 +0100
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com>
References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
	<CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
	<CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
	<17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com>
	<CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com>
Message-ID: <CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com>

Ok, For last question I think I found answer here:

http://nbviewer.ipython.org/github/ipython/ipython/blob/2.x/examples/Notebook/Importing%20Notebooks.ipynb


I will let you know if I will encounter any problems.

Best,
Rafal

2014-12-18 17:58 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>:
>
> Hi guys,
>
> Thanks for extensive response!
>
> In my output there is not need to be any imports. Output files will be
> only htmls to be put into edX course.
>
> The need of splitting notebook into few output files comes from making it
> as friendly as possible to creators of the course content.
> In principle they could write every unit in a separate notebook - but,
> because units together build a subsection of the course which describes one
> topic, it's more convenient to write them in one notebooks and separate by
> the largest headings.
>
> I tested how converting to html, that is already provided in nbconvert, is
> accepted by edX and it went quite well. I only needed to change two things
> in the output file ( change path to the folders with figures and remove
> 'end of paragraph' symbol ) .
>
>
> In this situation maybe what Matthias proposed is a way to go:
>
>
>> that's up to how you want to design your exporter.
>> You can either have a preprocessor that split the notebooks in chunks and
>> do the conversion after.
>> So virtually running nbconvert N times on n chucks of the notebook.
>>
>
>
> Then for example I could have in the notebook to export sth like that:
>
> <Heading 1 cell> first_unit_name </Heading 1 cell>
>
> Cells with stuff for first unit
>
>
> <Heading 2 cell> second_unit_name </Heading 2 cell>
>
> Cells with stuff for second unit
>
> etc...
>
> And with the output I would see first_unit_name.html and
> second_unit_name.html
> where content of each of them would look like it would be created only
> from notebook
> containing cells with corresponding content.
>
>
> Later I would like for example kick out every cells with python code
> leaving only their output (but this is for a later stage of development,
> after having the splitting going well).
>
>
>
>
>
> Oh, I have one question about the preprocessors. I found little old post
> https://blog.safaribooksonline.com/2013/12/18/ipython-notebook-plugins/
> about how to write custom plugins to nbconvert. Despite the fact that
> Jonathan was using transformers instead of preprocessors it worked when I
> changed config file only from:
>
> C = get_config()
> c.Exporter.transformers = [?exclude_transformer.ExcludeTransformer?]
>
> to:
> c = get_config()
> c.Exporter.preprocessors = ["exclude_transformer.ExcludeTransformer"]
>
>
>
> But when I tried to change usage of Transformer into Preprocessor:
> from http://pastebin.com/rQufqzUr into http://pastebin.com/Q8mgaYAJ
>
> (correcting ofc line in config into c.Exporter.preprocessors =
> ["exclude_preprocessor.ExcludePreprocessor"])
>
> I got the following error: http://pastebin.com/MSAHR2PP
>
> Any idea why?
>
>
> Oh... And one more question :). While writing some code I love to test
> things interactively (this is why I loved ipython so much).
> Is there any easy way to play in ipython notebook with other notebook that
> goes as argument into all converter arguments?
>
> Like I would like to see by hand what every worksheet in code like here
> http://pastebin.com/rQufqzUr contains.
>
>
> Best,
> Rafal
>
>
>
>> Should my custom exporter class inherit from a TemplateExporter? Or maybe
>> separation into files should happen at this level?
>>
>>
>> You are not obliged to, but I think you should at least inherit Exporter
>> if you like your custom one to be accepted by nbconvert.
>> The architecture is not really meant to export into many files, except
>>  for the resources file (Png, Svg) that do into a separate folder.
>>
>> If you write your custom writer then your Exporter can most likely
>> produce many file that would be written at once.
>>
>>
>> And last, once I will have my exporter class will it be enough to specify
>> which exporter should be used in config file and provide config to
>> nbconvert via --config mycfg.py ?
>>
>>
>> Yes that should be enough., any trait lets of your custom class should be
>> automatically set to the value of the config.
>>
>>
>> Thanks for all your help!
>>
>>
>> Feel free to send us feedback on what you think of the APi or what
>> limitation there is.
>>
>> Thanks !
>>
>> --
>> M
>>
>>
>>
>> Best,
>> Rafal
>>
>> P.S. If there are any useful guides or FAQs about writing custom
>> converters that would be very helpful. I looked for some on my own but
>> didn't have much luck.
>>
>> 2014-12-16 23:13 GMT+01:00 Thomas Kluyver <takowl at gmail.com>:
>>>
>>> Hi Rafa?,
>>>
>>> On 16 December 2014 at 13:40, Rafa? Skolasi?ski <
>>> rjtskolasinski at gmail.com> wrote:
>>>
>>>> Unfortunately each unit must go into separate html file and here comes
>>>> my first question: what is the easiest way to force nbconverter to export
>>>> one notebook into few html files (let's say Heading 1 will start every unit
>>>> and at the same time will be the name of it)?
>>>
>>>
>>> I think you will need to make a custom exporter - nbconvert supports
>>> different output formats with exporter classes, which you can find here:
>>> https://github.com/ipython/ipython/tree/master/IPython/nbconvert/exporters
>>>
>>> Your case will be a bit trickier than most of those examples, because
>>> all the TemplateExporter subclasses assume that they are writing a single
>>> main output file. But there's no reason that an exporter class couldn't
>>> write several files from one notebook.
>>>
>>> Thomas
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141218/9cb7c690/attachment.html>

From ssanderson at quantopian.com  Thu Dec 18 13:19:53 2014
From: ssanderson at quantopian.com (ssanderson)
Date: Thu, 18 Dec 2014 10:19:53 -0800 (PST)
Subject: [IPython-dev] Database-backed persistence for IPython Notebooks
Message-ID: <1418926793351-5080923.post@n6.nabble.com>

Hi All,

I want to announce a new open source project, "pgcontents", that's in
development at  Quantopian <http://www.quantopian.com>   for use in our
hosted research environment.

pgcontents is a PostgreSQL-backed implementation of IPython's  Contents API
<https://github.com/ipython/ipython/wiki/IPEP-27:-Contents-Service>  .  This
means that it provides an alternative storage backend for the files managed
by IPython.  The immediate goal is to use this to build a service that can
host notebooks without having to worry about the various pitfalls of
file-based storage in the cloud, but it also opens the door to more
sophisticated schemes for checkpointing and sharing notebooks than what's
currently easy or possible with the default manager.

You can find the source and installation instructions for pgcontents at
https://github.com/quantopian/pgcontents.  It's currently under heavy
development, but the version on PyPI is a serviceable (if probably less
efficient than it could be) drop-in replacement for the default manager.

Thanks,
-Scott



--
View this message in context: http://python.6.x6.nabble.com/Database-backed-persistence-for-IPython-Notebooks-tp5080923.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From jgomezdans at gmail.com  Thu Dec 18 14:13:08 2014
From: jgomezdans at gmail.com (Jose Gomez-Dans)
Date: Thu, 18 Dec 2014 19:13:08 +0000
Subject: [IPython-dev] IPython parallel "education"
Message-ID: <CAMWde5opz5G1ud400Q6V39zhp63NO6O8vXe3CoNXR3ArGz2O8g@mail.gmail.com>

Hello,

I'm trying to figure out the use of the IPython parallel scheme for some of
the processing we are doing. In effect, what we have is a bunch of
calculations on a vector, and we have tons of these vectors, stored in an
(N,11) matrix, where N is typically >10E6. The calculations involve an
euclidean distance and a few vector/matix, matrix/vector products (these
are typically reasonable matrices, say 200x200 or so). First idea is to
approach the problem by splitting the starting (N,11) matrix into (N/s, 11)
matrices (s being the number of nodes) and evaluate them in parallel.

from IPython.parallel import *
import numpy as np

client = Client(profile="default")
ids    = client.ids
nodes  = len(ids)
view   = client.load_balanced_view()
dview  = client[:]

with dview.sync_imports():
    import numpy as np


def the_func ( x ):
    """Does some numerical stuff with a vector `x`"""


@dview.parallel(block=True)
def run_func ( inputx ):
    (x,f) = inputx
    return f(x)


X=np.random.randn(8000000,2)
# parallel run
Y = run_func.map ( [(X[(2000000*i):(i+1)*2000000], the_func) for i in
xrange(nodes)] )

# serial run
YY = the_func(X)



The above code works OK. htop tells me it's using all cores on my laptop.
If I just run the_func(X), it appears to use only one core (htop dixit),
and it takes a little bit longer (~10-15%), which maybe is a lot to do with
parallel linear algebra inside of the local BLAS version or internal
optimisations on scipy's euclidean distance calculations.

We plan to use this on an IPython cluster connected with SSH, to which I
don't yet have access. Are there some obvious guidelines about optimising
for this new setup that I need to know? And yes, hoping to get a bigger
speed up as we increase the size of our problem and nodes!

Thanks!
Jose
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141218/f6c6d29b/attachment.html>

From aron at ahmadia.net  Thu Dec 18 15:22:21 2014
From: aron at ahmadia.net (Aron Ahmadia)
Date: Thu, 18 Dec 2014 12:22:21 -0800
Subject: [IPython-dev] IPython parallel "education"
In-Reply-To: <CAMWde5opz5G1ud400Q6V39zhp63NO6O8vXe3CoNXR3ArGz2O8g@mail.gmail.com>
References: <CAMWde5opz5G1ud400Q6V39zhp63NO6O8vXe3CoNXR3ArGz2O8g@mail.gmail.com>
Message-ID: <CAPhiW4jG8TuHyHh=CJxZvoON5r08CxCyHKM3xpH9ae52_QB=bw@mail.gmail.com>

On Thu, Dec 18, 2014 at 11:13 AM, Jose Gomez-Dans <jgomezdans at gmail.com>
wrote:
>
> # parallel run
> Y = run_func.map ( [(X[(2000000*i):(i+1)*2000000], the_func) for i in
> xrange(nodes)] )
>

What happens if instead of partitioning the data, you create a list of work
units and map those?
Something like:

def apply_the_func(i):
      return the_func(X[N*i):X[(i+1)*N])

Y = run_func.map ( [xrange(i), apply_the_func) for i in range(nodes)] )

You'd instantiate X ahead of time on your nodes.

I suggest this because if "the_func" is not particularly complex, a good
fraction of your time will be spent just on sending input and results back
and forth.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141218/2963f0c2/attachment.html>

From fperez.net at gmail.com  Thu Dec 18 16:36:06 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 18 Dec 2014 13:36:06 -0800
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com>
References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
	<CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
	<CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
	<17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com>
	<CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com>
	<CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com>
Message-ID: <CAHAreOpPmg2bVUmeRMEYPOFdRMvkU_A1nfvgTxz=kcQ=T9ghjg@mail.gmail.com>

On Thu, Dec 18, 2014 at 9:06 AM, Rafa? Skolasi?ski <rjtskolasinski at gmail.com
> wrote:
>
> I will let you know if I will encounter any problems.


And please post here on the list once you have something up and running.
It's very important for the project to show how others use it not only as
an interactive environment, but also as infrastructure to build upon.
That's the kind of clear value that helps us secure resources to sustain
its development.

Best

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141218/7a5db98e/attachment.html>

From rjtskolasinski at gmail.com  Thu Dec 18 16:38:27 2014
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Thu, 18 Dec 2014 22:38:27 +0100
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <CAHAreOpPmg2bVUmeRMEYPOFdRMvkU_A1nfvgTxz=kcQ=T9ghjg@mail.gmail.com>
References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
	<CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
	<CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
	<17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com>
	<CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com>
	<CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com>
	<CAHAreOpPmg2bVUmeRMEYPOFdRMvkU_A1nfvgTxz=kcQ=T9ghjg@mail.gmail.com>
Message-ID: <CAAiurtc0hT5muWuupzym20nNDrZ3TswH2t7d0YAuXUeSN54ipw@mail.gmail.com>

I will ; - )

2014-12-18 22:36 GMT+01:00 Fernando Perez <fperez.net at gmail.com>:
>
>
> On Thu, Dec 18, 2014 at 9:06 AM, Rafa? Skolasi?ski <
> rjtskolasinski at gmail.com> wrote:
>>
>> I will let you know if I will encounter any problems.
>
>
> And please post here on the list once you have something up and running.
> It's very important for the project to show how others use it not only as
> an interactive environment, but also as infrastructure to build upon.
> That's the kind of clear value that helps us secure resources to sustain
> its development.
>
> Best
>
> f
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141218/0dcb00c3/attachment.html>

From fperez.net at gmail.com  Thu Dec 18 16:38:10 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 18 Dec 2014 13:38:10 -0800
Subject: [IPython-dev] IPython notebook available under the Android
	platform
In-Reply-To: <CAO=GJvxiFa5Wsmrbw2UjYAXYqET4r9WcOLZ7D-ddSY7OK1si9w@mail.gmail.com>
References: <CAO=GJvxiFa5Wsmrbw2UjYAXYqET4r9WcOLZ7D-ddSY7OK1si9w@mail.gmail.com>
Message-ID: <CAHAreOquXpTkiaeV4SL1fG_M3KMkvG6dkz3Vf+3Bz76Ftu7mrw@mail.gmail.com>

Hi

On Thu, Dec 18, 2014 at 4:04 AM, Apps Embedded <apps.embedded at gmail.com>
wrote:
>
> Hi,
>
> We developped an Android app called "LabPy Console" using Python and the
> SciPy packages (Python is a trademark of the Python Software Foundation).
> We had the trademark agreement from the PSF for posting such a work under
> the Play Store.
>
> Within this app, we are using the IPython package.
>
> Our app is under the GPL v3 licence and our source code is available
> through our support web site :
> https://code.google.com/p/labpyconsoleapp/
>
> Using the IPython notebook extension is a new feature on the Android
> platform.
> From your point of view, we would like to know if there is no legal issue
> from a licence point of view for instance.
> Does the LabPy Console app name is ok for you too?
>

Sure, no problem from this side.

On behalf of the dev team, I wanted to thank you for approaching us in this
manner and giving us an opportunity for feedback about the use of the
project's name.  Yours is the kind of effort  we hope will succeed, as it's
one more way for users to get to the scientific python tools. We wish you
the best, and keep us posted on how it all goes.

Regards,

f
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141218/4911228f/attachment.html>

From david.froger.ml at mailoo.org  Fri Dec 19 04:04:46 2014
From: david.froger.ml at mailoo.org (David Froger)
Date: Fri, 19 Dec 2014 10:04:46 +0100
Subject: [IPython-dev] status of ipnbdoctest.py
Message-ID: <20141219090446.10096.87594@otis.rocq.inria.fr>

Dear all,

I'm look for a tool like ipnbdoctest.py [0] which
description is:

    simple example script for running and testing notebooks.

    Usage: `ipnbdoctest.py foo.ipynb [bar.ipynb [...]]`

    Each cell is submitted to the kernel, and the outputs are compared with those
    stored in the notebook. 

ipnbdoctest.py looks great, but seems to be not actively developped. Forks
added new functionnalities, which have not been merged back.

Do you make use of such a tool?

Is there other ways to automatically test IPython notebook outputs?

Would it be worth to maintain and develop ipnbdoctest.py?

[0] https://gist.github.com/minrk/2620735

Thanks,
David


From nathan12343 at gmail.com  Fri Dec 19 04:21:23 2014
From: nathan12343 at gmail.com (Nathan Goldbaum)
Date: Fri, 19 Dec 2014 01:21:23 -0800
Subject: [IPython-dev] status of ipnbdoctest.py
In-Reply-To: <20141219090446.10096.87594@otis.rocq.inria.fr>
References: <20141219090446.10096.87594@otis.rocq.inria.fr>
Message-ID: <CAJXewO=wbi9Onf_41WJD8JtQD7bpSkvtS=M8CHarJ9gLV_aYNg@mail.gmail.com>

You can use runipy to script notebook evaluation:

https://github.com/paulgb/runipy

I don't think you can verify that the output isn't changing, but you can at
least check that cells aren't raising exceptions and view a converted
version of the evaluated notebook.  I think adding functionality to verify
that cell output isn't changing would be a neat addition to runipy.

On Fri, Dec 19, 2014 at 1:04 AM, David Froger <david.froger.ml at mailoo.org>
wrote:
>
> Dear all,
>
> I'm look for a tool like ipnbdoctest.py [0] which
> description is:
>
>     simple example script for running and testing notebooks.
>
>     Usage: `ipnbdoctest.py foo.ipynb [bar.ipynb [...]]`
>
>     Each cell is submitted to the kernel, and the outputs are compared
> with those
>     stored in the notebook.
>
> ipnbdoctest.py looks great, but seems to be not actively developped. Forks
> added new functionnalities, which have not been merged back.
>
> Do you make use of such a tool?
>
> Is there other ways to automatically test IPython notebook outputs?
>
> Would it be worth to maintain and develop ipnbdoctest.py?
>
> [0] https://gist.github.com/minrk/2620735
>
> Thanks,
> David
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141219/7abfdaef/attachment.html>

From zvoros at gmail.com  Fri Dec 19 05:02:53 2014
From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Fri, 19 Dec 2014 11:02:53 +0100
Subject: [IPython-dev] status of ipnbdoctest.py
In-Reply-To: <CAJXewO=wbi9Onf_41WJD8JtQD7bpSkvtS=M8CHarJ9gLV_aYNg@mail.gmail.com>
References: <20141219090446.10096.87594@otis.rocq.inria.fr>
	<CAJXewO=wbi9Onf_41WJD8JtQD7bpSkvtS=M8CHarJ9gLV_aYNg@mail.gmail.com>
Message-ID: <5493F7CD.3030205@gmail.com>


On 12/19/2014 10:21 AM, Nathan Goldbaum wrote:
> You can use runipy to script notebook evaluation:
>
> https://github.com/paulgb/runipy
>
> I don't think you can verify that the output isn't changing, but you 
> can at least check that cells aren't raising exceptions and view a 
> converted version of the evaluated notebook.  I think adding 
> functionality to verify that cell output isn't changing would be a 
> neat addition to runipy.
>
But there are cases, when the output changes, even if you run the same 
piece of code. Two examples that come to my mind are cells that involve 
the time/date in some way, and cells that operate on random numbers. It 
might also happen that some underlying data file that is plotted is 
modified between two executions. It might be really tricky to check for 
these cases.

Cheers,
Zolt?n


From david.froger.ml at mailoo.org  Fri Dec 19 05:20:11 2014
From: david.froger.ml at mailoo.org (David Froger)
Date: Fri, 19 Dec 2014 11:20:11 +0100
Subject: [IPython-dev] status of ipnbdoctest.py
In-Reply-To: <CAJXewO=wbi9Onf_41WJD8JtQD7bpSkvtS=M8CHarJ9gLV_aYNg@mail.gmail.com>
References: <20141219090446.10096.87594@otis.rocq.inria.fr>
	<CAJXewO=wbi9Onf_41WJD8JtQD7bpSkvtS=M8CHarJ9gLV_aYNg@mail.gmail.com>
Message-ID: <20141219102011.10096.32757@otis.rocq.inria.fr>

> You can use runipy to script notebook evaluation:
> 
> https://github.com/paulgb/runipy
> 
> I don't think you can verify that the output isn't changing, but you can at
> least check that cells aren't raising exceptions and view a converted version
> of the evaluated notebook.? I think adding functionality to verify that cell
> output isn't changing would be a neat addition to runipy.

Thank for the reply, I will have a look if ipnbdoctest.py functionnalities can
be added in runipy.


From bussonniermatthias at gmail.com  Fri Dec 19 06:26:32 2014
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Fri, 19 Dec 2014 12:26:32 +0100
Subject: [IPython-dev] status of ipnbdoctest.py
In-Reply-To: <20141219102011.10096.32757@otis.rocq.inria.fr>
References: <20141219090446.10096.87594@otis.rocq.inria.fr>
	<CAJXewO=wbi9Onf_41WJD8JtQD7bpSkvtS=M8CHarJ9gLV_aYNg@mail.gmail.com>
	<20141219102011.10096.32757@otis.rocq.inria.fr>
Message-ID: <C0285E6D-85E5-4D3D-9116-618C5A6A76E0@gmail.com>

We probably have such a piece of code in IPython itself in testing of notebooks I guess. 

Check in nbgrader too: 

https://github.com/jupyter/nbgrader

Don't forget to update your git package with the recent security vulnerability.

-- 
M


Le 19 d?c. 2014 ? 11:20, David Froger a ?crit :

>> You can use runipy to script notebook evaluation:
>> 
>> https://github.com/paulgb/runipy
>> 
>> I don't think you can verify that the output isn't changing, but you can at
>> least check that cells aren't raising exceptions and view a converted version
>> of the evaluated notebook.  I think adding functionality to verify that cell
>> output isn't changing would be a neat addition to runipy.
> 
> Thank for the reply, I will have a look if ipnbdoctest.py functionnalities can
> be added in runipy.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141219/426b31a6/attachment.html>

From bussonniermatthias at gmail.com  Fri Dec 19 06:36:12 2014
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Fri, 19 Dec 2014 12:36:12 +0100
Subject: [IPython-dev] Database-backed persistence for IPython Notebooks
In-Reply-To: <1418926793351-5080923.post@n6.nabble.com>
References: <1418926793351-5080923.post@n6.nabble.com>
Message-ID: <88C2D384-0644-4E46-8CA1-16BBC2E9A8B4@gmail.com>

Hi Scott, 


Le 18 d?c. 2014 ? 19:19, ssanderson a ?crit :

> Hi All,
> 
> I want to announce a new open source project, "pgcontents", that's in
> development at  Quantopian <http://www.quantopian.com>   for use in our
> hosted research environment.
> 
> pgcontents is a PostgreSQL-backed implementation of IPython's  Contents API
> <https://github.com/ipython/ipython/wiki/IPEP-27:-Contents-Service>  .  This
> means that it provides an alternative storage backend for the files managed
> by IPython.  The immediate goal is to use this to build a service that can
> host notebooks without having to worry about the various pitfalls of
> file-based storage in the cloud, but it also opens the door to more
> sophisticated schemes for checkpointing and sharing notebooks than what's
> currently easy or possible with the default manager.


Thanks a lot for that, really appreciate ! it is really great. 
This is typically the kind of things that will also work for non-python notebook (so II cc the jupyter ml) 
And that help us clean our codebase. 




> You can find the source and installation instructions for pgcontents at
> https://github.com/quantopian/pgcontents.  It's currently under heavy
> development, but the version on PyPI is a serviceable (if probably less
> efficient than it could be) drop-in replacement for the default manager.
> 
> Thanks,
> -Scott
> 

Thanks !
-- 
M



From rjtskolasinski at gmail.com  Fri Dec 19 11:12:36 2014
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Fri, 19 Dec 2014 17:12:36 +0100
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <CAAiurtc0hT5muWuupzym20nNDrZ3TswH2t7d0YAuXUeSN54ipw@mail.gmail.com>
References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
	<CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
	<CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
	<17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com>
	<CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com>
	<CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com>
	<CAHAreOpPmg2bVUmeRMEYPOFdRMvkU_A1nfvgTxz=kcQ=T9ghjg@mail.gmail.com>
	<CAAiurtc0hT5muWuupzym20nNDrZ3TswH2t7d0YAuXUeSN54ipw@mail.gmail.com>
Message-ID: <CAAiurtcXF5FRcXd7GiO5Jsd3+mHEGHB81cKfME1ypezEYcT-7w@mail.gmail.com>

Hi guys,

I got some kind of working version. It almost do what I want from it (at
least for first working version).
http://pastebin.com/bi0003LS

Of course later there will come more functionality and polishing.

Although I got strange problem. It looks like my converter ignores fact I
ask it for using 'basic' template.

Does the syntax for it changed? I was basing on this old tutorial:

http://nbviewer.ipython.org/github/ipython/ipython-in-depth/blob/master/examples/Notebook/Using%20nbconvert%20as%20a%20Library.ipynb

Best,
Rafal

2014-12-18 22:38 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>:
>
> I will ; - )
>
> 2014-12-18 22:36 GMT+01:00 Fernando Perez <fperez.net at gmail.com>:
>
>>
>> On Thu, Dec 18, 2014 at 9:06 AM, Rafa? Skolasi?ski <
>> rjtskolasinski at gmail.com> wrote:
>>>
>>> I will let you know if I will encounter any problems.
>>
>>
>> And please post here on the list once you have something up and running.
>> It's very important for the project to show how others use it not only as
>> an interactive environment, but also as infrastructure to build upon.
>> That's the kind of clear value that helps us secure resources to sustain
>> its development.
>>
>> Best
>>
>> f
>>
>>
>> --
>> Fernando Perez (@fperez_org; http://fperez.org)
>> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>> fernando.perez-at-berkeley: contact me here for any direct mail
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141219/877e4805/attachment.html>

From rjtskolasinski at gmail.com  Fri Dec 19 11:57:19 2014
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Fri, 19 Dec 2014 17:57:19 +0100
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <CAAiurtcXF5FRcXd7GiO5Jsd3+mHEGHB81cKfME1ypezEYcT-7w@mail.gmail.com>
References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
	<CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
	<CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
	<17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com>
	<CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com>
	<CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com>
	<CAHAreOpPmg2bVUmeRMEYPOFdRMvkU_A1nfvgTxz=kcQ=T9ghjg@mail.gmail.com>
	<CAAiurtc0hT5muWuupzym20nNDrZ3TswH2t7d0YAuXUeSN54ipw@mail.gmail.com>
	<CAAiurtcXF5FRcXd7GiO5Jsd3+mHEGHB81cKfME1ypezEYcT-7w@mail.gmail.com>
Message-ID: <CAAiurtdW4k1Ne0CUvz54TkLpdi+iiJYJR=Yrp7ADemXfnEDzFQ@mail.gmail.com>

Ok. I found it.
Now the corresponding line should be
exportHtml =
HTMLExporter(config=Config({'HTMLExporter':{'template_file':'basic'}}))
instead of
exportHtml =
HTMLExporter(config=Config({'HTMLExporter':{'default_template':'basic'}}))


2014-12-19 17:12 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>:
>
> Hi guys,
>
> I got some kind of working version. It almost do what I want from it (at
> least for first working version).
> http://pastebin.com/bi0003LS
>
> Of course later there will come more functionality and polishing.
>
> Although I got strange problem. It looks like my converter ignores fact I
> ask it for using 'basic' template.
>
> Does the syntax for it changed? I was basing on this old tutorial:
>
> http://nbviewer.ipython.org/github/ipython/ipython-in-depth/blob/master/examples/Notebook/Using%20nbconvert%20as%20a%20Library.ipynb
>
> Best,
> Rafal
>
> 2014-12-18 22:38 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>:
>>
>> I will ; - )
>>
>> 2014-12-18 22:36 GMT+01:00 Fernando Perez <fperez.net at gmail.com>:
>>
>>>
>>> On Thu, Dec 18, 2014 at 9:06 AM, Rafa? Skolasi?ski <
>>> rjtskolasinski at gmail.com> wrote:
>>>>
>>>> I will let you know if I will encounter any problems.
>>>
>>>
>>> And please post here on the list once you have something up and running.
>>> It's very important for the project to show how others use it not only as
>>> an interactive environment, but also as infrastructure to build upon.
>>> That's the kind of clear value that helps us secure resources to sustain
>>> its development.
>>>
>>> Best
>>>
>>> f
>>>
>>>
>>> --
>>> Fernando Perez (@fperez_org; http://fperez.org)
>>> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>>> fernando.perez-at-berkeley: contact me here for any direct mail
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141219/e52bb0fa/attachment.html>

From zach at zacharyjones.us  Fri Dec 19 18:03:25 2014
From: zach at zacharyjones.us (Zachary Jones)
Date: Fri, 19 Dec 2014 18:03:25 -0500
Subject: [IPython-dev] Behavior of Default Kernel
Message-ID: <CAL2pmi_JT2fQ+SyRLupsyD5WyRyA6AXSrnRmi1aw2jV+V9vGcA@mail.gmail.com>

Hi everyone,

I have a question about the behavior of default kernels

I was testing IPython from github today (commit hash: dc60d34) and wanted
to understand if the behavior I am seeing is expected.

The IPython notebook server is running under Python3.4, but I would the
default kernel to be a custom Python 2 stack.

Running "ipython kernelspec list" from the CLI returns:
Available kernels:
  python3
  my_python2

I invoked IPython notebook using the default_kernel_name:
'--MappingKernelManager.default_kernel_name=my_python2'

The API call to http://localhost:8000/api/kernelspecs returns:
{
    "default": "my_python2",
    "kernelspecs": {
        "my_python2": {
             "env": {},
             "argv": [*omitted*],
             "name": "my_python2",
             "display_name": "My Python2 Stack"
        },
        "python3": {
             "env": {},
             "argv": [*omitted*],
             "name": "python3",
             "display_name": "IPython (Python 3)"
        }
    }
}

With this configuration, I observed the following:

   - When I go to to the notebook dashboard, next to the "New Notebook"
   button is "IPython (Python 3)" and the hover over says "IPython (Python 3)
   is the default kernel for new notebooks".
   - When I click on "New Notebook", the notebook opened uses "IPython
   (Python 3)" kernel.
   - When I open existing notebooks that are being converted from v3 to v4,
   the notebooks open using "My Python 2 Stack" kernel.
   - When I open an existing notebook that has had the kernel spec inserted
   into metadata, that kernel is used.


*Finally to my questions..*
Is the default_kernel_name supposed to be used when the "New Notebook"
button is pressed? and is it suppose to be shown next to the "New Notebook"
button?

If the answers are yes, should I open an Issue in GitHub? I tried looking
for an existing issue, but did not turn up anything.


Thanks for your help,
Zach Jones
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141219/d5d242cb/attachment.html>

From python at elbonia.de  Sat Dec 20 08:29:46 2014
From: python at elbonia.de (Juergen Hasch)
Date: Sat, 20 Dec 2014 14:29:46 +0100
Subject: [IPython-dev] Getting the IPython path in a config file
Message-ID: <549579CA.5080408@elbonia.de>

Hi,
I am using IPython with custom templates, as well as pre- and postprocessors.

For custom templates I need to specify the path to the template, whereas pre- or postprocessors need to be in the Python
path.

Now I would like to keep the custom stuff together with the rest of the IPython data, so this is what my
"ipython_nbconvert_config.py" looks like:

	import sys
	import os
	ipythondir = sys.path[-1]
	extensions = os.path.join(ipythondir,'extensions')
	sys.path.append( extensions )

	c = get_config()

	c.Exporter.template_file = 'hide_input_output.tpl'
	c.Exporter.template_path = [ os.path.join(ipythondir,'templates') ]
	c.Exporter.preprocessors = ['codefolding.CodeFoldingPreprocessor', 'pymdpreprocessor.PyMarkdownPreprocessor' ]
	c.Exporter.postprocessors = ['embed.EmbedPostProcessor']

Is there a better way than to look at sys.path[-1] to get the path to the IPython directory inside a config file ?
I know I could set a custom IPYTHONDIR environment variable, but that kind of defeats the purpose of using the current
IPython directory.

  Juergen


From andrew.gibiansky at gmail.com  Sat Dec 20 16:44:12 2014
From: andrew.gibiansky at gmail.com (Andrew Gibiansky)
Date: Sat, 20 Dec 2014 16:44:12 -0500
Subject: [IPython-dev] Announce: IPython Notebook Mac App (Europa Notebook)
Message-ID: <CAF-LYKKR8e1RYj5MUg3x=GEhKU-mT-nJaON5vY6p9upgcMiQCg@mail.gmail.com>

Hey all,

I'd like to announce the first release of an experiment I've been working
on: an IPython Notebook mac app. This is similar in spirit to [1] and [2],
but I've tried to iron out a few kinks people have had in installation as
well as bundle some of my favorite extensions.

You can download it here: http://www.europanotebook.com

Features:
- Completely bundled Python, separate from system python
- No external dependencies whatsoever ? *completely* bundled
- Includes matplotlib, numpy, and scipy
- Includes a few extra JS extensions, such as cell hiding and the wonderful
Calico extensions [3]
- Integrated with Finder

This is part of my ongoing work to make IHaskell (and Haskell) more
accessible ? now that IPython is neatly packaged, I plan on adding the
IHaskell kernel and platform to the package. However, I figured that the
intermediate may also be useful to someone.

Note: I was considering experimenting with making this a paid Mac
application. I don't know if this is worth money to anyone ? if you have
opinions on this matter, let me know, I'm still trying to figure out what I
think about all of this.

Thanks to all the IPython devs for creating such a great platform!
-- Andrew

[1] https://github.com/mangecoeur/ipython-desktop
[2] https://github.com/liyanage/ipython-notebook/wiki
[3] http://wiki.roboteducation.org/ICalico
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141220/6f1a9345/attachment.html>

From ellisonbg at gmail.com  Sat Dec 20 16:51:24 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 20 Dec 2014 13:51:24 -0800
Subject: [IPython-dev] Announce: IPython Notebook Mac App (Europa
	Notebook)
In-Reply-To: <CAF-LYKKR8e1RYj5MUg3x=GEhKU-mT-nJaON5vY6p9upgcMiQCg@mail.gmail.com>
References: <CAF-LYKKR8e1RYj5MUg3x=GEhKU-mT-nJaON5vY6p9upgcMiQCg@mail.gmail.com>
Message-ID: <CAH4pYpSq9mZqq925oK1NjsOsYB+pkKtQDuL4SWqeX3UmZQQZZQ@mail.gmail.com>

Andrew,

Very exciting! If you aren't worried about making $, you might want to look
at integrating with conda for all of the packaging things. I would hate to
see you reinvent yet another packaging system.

Cheers,

Brian

On Sat, Dec 20, 2014 at 1:44 PM, Andrew Gibiansky <
andrew.gibiansky at gmail.com> wrote:
>
> Hey all,
>
> I'd like to announce the first release of an experiment I've been working
> on: an IPython Notebook mac app. This is similar in spirit to [1] and [2],
> but I've tried to iron out a few kinks people have had in installation as
> well as bundle some of my favorite extensions.
>
> You can download it here: http://www.europanotebook.com
>
> Features:
> - Completely bundled Python, separate from system python
> - No external dependencies whatsoever ? *completely* bundled
> - Includes matplotlib, numpy, and scipy
> - Includes a few extra JS extensions, such as cell hiding and the
> wonderful Calico extensions [3]
> - Integrated with Finder
>
> This is part of my ongoing work to make IHaskell (and Haskell) more
> accessible ? now that IPython is neatly packaged, I plan on adding the
> IHaskell kernel and platform to the package. However, I figured that the
> intermediate may also be useful to someone.
>
> Note: I was considering experimenting with making this a paid Mac
> application. I don't know if this is worth money to anyone ? if you have
> opinions on this matter, let me know, I'm still trying to figure out what I
> think about all of this.
>
> Thanks to all the IPython devs for creating such a great platform!
> -- Andrew
>
> [1] https://github.com/mangecoeur/ipython-desktop
> [2] https://github.com/liyanage/ipython-notebook/wiki
> [3] http://wiki.roboteducation.org/ICalico
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>

-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141220/7564a19a/attachment.html>

From ellisonbg at gmail.com  Sat Dec 20 16:56:04 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 20 Dec 2014 13:56:04 -0800
Subject: [IPython-dev] Announce: IPython Notebook Mac App (Europa
	Notebook)
In-Reply-To: <CAH4pYpSq9mZqq925oK1NjsOsYB+pkKtQDuL4SWqeX3UmZQQZZQ@mail.gmail.com>
References: <CAF-LYKKR8e1RYj5MUg3x=GEhKU-mT-nJaON5vY6p9upgcMiQCg@mail.gmail.com>
	<CAH4pYpSq9mZqq925oK1NjsOsYB+pkKtQDuL4SWqeX3UmZQQZZQ@mail.gmail.com>
Message-ID: <CAH4pYpSjaNh9eYdcDMC0d_vpq2Djtq_8yfpi+m38yV-+PioCwg@mail.gmail.com>

Ohh, but more importantly, I think the name of this is too close to the
ideas and abstractions present in the new name of the notebook "Jupyter"
and will run afoul of trademark issues. The potential for confusion would
hurt both projects. I would, politely ask you to consider changing the name
to something that has more distance from our trademarks. I know it may seem
silly for an open source project to worry about these types of things, but
we have run into issues in the past, and we need to be careful about this.
I should emphasize that at this point, this is just my own personal
opinion....this may be the type of thing that we need to bring up to the
project's steering council.

This is still very cool though!

Cheers,

Brian

On Sat, Dec 20, 2014 at 1:51 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> Andrew,
>
> Very exciting! If you aren't worried about making $, you might want to
> look at integrating with conda for all of the packaging things. I would
> hate to see you reinvent yet another packaging system.
>
> Cheers,
>
> Brian
>
> On Sat, Dec 20, 2014 at 1:44 PM, Andrew Gibiansky <
> andrew.gibiansky at gmail.com> wrote:
>
>> Hey all,
>>
>> I'd like to announce the first release of an experiment I've been working
>> on: an IPython Notebook mac app. This is similar in spirit to [1] and [2],
>> but I've tried to iron out a few kinks people have had in installation as
>> well as bundle some of my favorite extensions.
>>
>> You can download it here: http://www.europanotebook.com
>>
>> Features:
>> - Completely bundled Python, separate from system python
>> - No external dependencies whatsoever ? *completely* bundled
>> - Includes matplotlib, numpy, and scipy
>> - Includes a few extra JS extensions, such as cell hiding and the
>> wonderful Calico extensions [3]
>> - Integrated with Finder
>>
>> This is part of my ongoing work to make IHaskell (and Haskell) more
>> accessible ? now that IPython is neatly packaged, I plan on adding the
>> IHaskell kernel and platform to the package. However, I figured that the
>> intermediate may also be useful to someone.
>>
>> Note: I was considering experimenting with making this a paid Mac
>> application. I don't know if this is worth money to anyone ? if you have
>> opinions on this matter, let me know, I'm still trying to figure out what I
>> think about all of this.
>>
>> Thanks to all the IPython devs for creating such a great platform!
>> -- Andrew
>>
>> [1] https://github.com/mangecoeur/ipython-desktop
>> [2] https://github.com/liyanage/ipython-notebook/wiki
>> [3] http://wiki.roboteducation.org/ICalico
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
>


-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141220/baabb7ff/attachment.html>

From andrew.gibiansky at gmail.com  Sat Dec 20 17:06:39 2014
From: andrew.gibiansky at gmail.com (Andrew Gibiansky)
Date: Sat, 20 Dec 2014 17:06:39 -0500
Subject: [IPython-dev] Announce: IPython Notebook Mac App (Europa
	Notebook)
In-Reply-To: <CAH4pYpSjaNh9eYdcDMC0d_vpq2Djtq_8yfpi+m38yV-+PioCwg@mail.gmail.com>
References: <CAF-LYKKR8e1RYj5MUg3x=GEhKU-mT-nJaON5vY6p9upgcMiQCg@mail.gmail.com>
	<CAH4pYpSq9mZqq925oK1NjsOsYB+pkKtQDuL4SWqeX3UmZQQZZQ@mail.gmail.com>
	<CAH4pYpSjaNh9eYdcDMC0d_vpq2Djtq_8yfpi+m38yV-+PioCwg@mail.gmail.com>
Message-ID: <CAF-LYK+EGZSSx_U0L4fiMkkhFE1=znSfmYScyXndRUUVGL2JNg@mail.gmail.com>

Brian,

Sure, no problem! Let's have an offline discussion further about this ? I
intentionally avoided using "IPython" or "Jupyter" in the name anywhere,
but am happy to change things.

-- Andrew

On Sat, Dec 20, 2014 at 4:56 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Ohh, but more importantly, I think the name of this is too close to the
> ideas and abstractions present in the new name of the notebook "Jupyter"
> and will run afoul of trademark issues. The potential for confusion would
> hurt both projects. I would, politely ask you to consider changing the name
> to something that has more distance from our trademarks. I know it may seem
> silly for an open source project to worry about these types of things, but
> we have run into issues in the past, and we need to be careful about this.
> I should emphasize that at this point, this is just my own personal
> opinion....this may be the type of thing that we need to bring up to the
> project's steering council.
>
> This is still very cool though!
>
> Cheers,
>
> Brian
>
> On Sat, Dec 20, 2014 at 1:51 PM, Brian Granger <ellisonbg at gmail.com>
> wrote:
>>
>> Andrew,
>>
>> Very exciting! If you aren't worried about making $, you might want to
>> look at integrating with conda for all of the packaging things. I would
>> hate to see you reinvent yet another packaging system.
>>
>> Cheers,
>>
>> Brian
>>
>> On Sat, Dec 20, 2014 at 1:44 PM, Andrew Gibiansky <
>> andrew.gibiansky at gmail.com> wrote:
>>
>>> Hey all,
>>>
>>> I'd like to announce the first release of an experiment I've been
>>> working on: an IPython Notebook mac app. This is similar in spirit to [1]
>>> and [2], but I've tried to iron out a few kinks people have had in
>>> installation as well as bundle some of my favorite extensions.
>>>
>>> You can download it here: http://www.europanotebook.com
>>>
>>> Features:
>>> - Completely bundled Python, separate from system python
>>> - No external dependencies whatsoever ? *completely* bundled
>>> - Includes matplotlib, numpy, and scipy
>>> - Includes a few extra JS extensions, such as cell hiding and the
>>> wonderful Calico extensions [3]
>>> - Integrated with Finder
>>>
>>> This is part of my ongoing work to make IHaskell (and Haskell) more
>>> accessible ? now that IPython is neatly packaged, I plan on adding the
>>> IHaskell kernel and platform to the package. However, I figured that the
>>> intermediate may also be useful to someone.
>>>
>>> Note: I was considering experimenting with making this a paid Mac
>>> application. I don't know if this is worth money to anyone ? if you have
>>> opinions on this matter, let me know, I'm still trying to figure out what I
>>> think about all of this.
>>>
>>> Thanks to all the IPython devs for creating such a great platform!
>>> -- Andrew
>>>
>>> [1] https://github.com/mangecoeur/ipython-desktop
>>> [2] https://github.com/liyanage/ipython-notebook/wiki
>>> [3] http://wiki.roboteducation.org/ICalico
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> @ellisonbg on Twitter and GitHub
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141220/abb80bcc/attachment.html>

From ellisonbg at gmail.com  Sat Dec 20 17:09:31 2014
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 20 Dec 2014 14:09:31 -0800
Subject: [IPython-dev] Announce: IPython Notebook Mac App (Europa
	Notebook)
In-Reply-To: <CAF-LYK+EGZSSx_U0L4fiMkkhFE1=znSfmYScyXndRUUVGL2JNg@mail.gmail.com>
References: <CAF-LYKKR8e1RYj5MUg3x=GEhKU-mT-nJaON5vY6p9upgcMiQCg@mail.gmail.com>
	<CAH4pYpSq9mZqq925oK1NjsOsYB+pkKtQDuL4SWqeX3UmZQQZZQ@mail.gmail.com>
	<CAH4pYpSjaNh9eYdcDMC0d_vpq2Djtq_8yfpi+m38yV-+PioCwg@mail.gmail.com>
	<CAF-LYK+EGZSSx_U0L4fiMkkhFE1=znSfmYScyXndRUUVGL2JNg@mail.gmail.com>
Message-ID: <CAH4pYpSaEoaHbwmAXMN6+c54nK=3FanFj7-D-_Uy9WrnhVaxTw@mail.gmail.com>

Thanks for understanding. More than willing to talk offline about this. I
am going to be out shopping and getting ready for Christmas much of the
weekend, but feel free to private chat me on gitter this week (ellisonbg).

Cheers,

Brian

On Sat, Dec 20, 2014 at 2:06 PM, Andrew Gibiansky <
andrew.gibiansky at gmail.com> wrote:
>
> Brian,
>
> Sure, no problem! Let's have an offline discussion further about this ? I
> intentionally avoided using "IPython" or "Jupyter" in the name anywhere,
> but am happy to change things.
>
> -- Andrew
>
> On Sat, Dec 20, 2014 at 4:56 PM, Brian Granger <ellisonbg at gmail.com>
> wrote:
>
>> Ohh, but more importantly, I think the name of this is too close to the
>> ideas and abstractions present in the new name of the notebook "Jupyter"
>> and will run afoul of trademark issues. The potential for confusion would
>> hurt both projects. I would, politely ask you to consider changing the name
>> to something that has more distance from our trademarks. I know it may seem
>> silly for an open source project to worry about these types of things, but
>> we have run into issues in the past, and we need to be careful about this.
>> I should emphasize that at this point, this is just my own personal
>> opinion....this may be the type of thing that we need to bring up to the
>> project's steering council.
>>
>> This is still very cool though!
>>
>> Cheers,
>>
>> Brian
>>
>> On Sat, Dec 20, 2014 at 1:51 PM, Brian Granger <ellisonbg at gmail.com>
>> wrote:
>>>
>>> Andrew,
>>>
>>> Very exciting! If you aren't worried about making $, you might want to
>>> look at integrating with conda for all of the packaging things. I would
>>> hate to see you reinvent yet another packaging system.
>>>
>>> Cheers,
>>>
>>> Brian
>>>
>>> On Sat, Dec 20, 2014 at 1:44 PM, Andrew Gibiansky <
>>> andrew.gibiansky at gmail.com> wrote:
>>>
>>>> Hey all,
>>>>
>>>> I'd like to announce the first release of an experiment I've been
>>>> working on: an IPython Notebook mac app. This is similar in spirit to [1]
>>>> and [2], but I've tried to iron out a few kinks people have had in
>>>> installation as well as bundle some of my favorite extensions.
>>>>
>>>> You can download it here: http://www.europanotebook.com
>>>>
>>>> Features:
>>>> - Completely bundled Python, separate from system python
>>>> - No external dependencies whatsoever ? *completely* bundled
>>>> - Includes matplotlib, numpy, and scipy
>>>> - Includes a few extra JS extensions, such as cell hiding and the
>>>> wonderful Calico extensions [3]
>>>> - Integrated with Finder
>>>>
>>>> This is part of my ongoing work to make IHaskell (and Haskell) more
>>>> accessible ? now that IPython is neatly packaged, I plan on adding the
>>>> IHaskell kernel and platform to the package. However, I figured that the
>>>> intermediate may also be useful to someone.
>>>>
>>>> Note: I was considering experimenting with making this a paid Mac
>>>> application. I don't know if this is worth money to anyone ? if you have
>>>> opinions on this matter, let me know, I'm still trying to figure out what I
>>>> think about all of this.
>>>>
>>>> Thanks to all the IPython devs for creating such a great platform!
>>>> -- Andrew
>>>>
>>>> [1] https://github.com/mangecoeur/ipython-desktop
>>>> [2] https://github.com/liyanage/ipython-notebook/wiki
>>>> [3] http://wiki.roboteducation.org/ICalico
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>
>>> --
>>> Brian E. Granger
>>> Cal Poly State University, San Luis Obispo
>>> @ellisonbg on Twitter and GitHub
>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> @ellisonbg on Twitter and GitHub
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>

-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141220/341499d5/attachment.html>

From bussonniermatthias at gmail.com  Sat Dec 20 17:11:38 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Sat, 20 Dec 2014 23:11:38 +0100
Subject: [IPython-dev] Announce: IPython Notebook Mac App (Europa
	Notebook)
In-Reply-To: <CAF-LYK+EGZSSx_U0L4fiMkkhFE1=znSfmYScyXndRUUVGL2JNg@mail.gmail.com>
References: <CAF-LYKKR8e1RYj5MUg3x=GEhKU-mT-nJaON5vY6p9upgcMiQCg@mail.gmail.com>
	<CAH4pYpSq9mZqq925oK1NjsOsYB+pkKtQDuL4SWqeX3UmZQQZZQ@mail.gmail.com>
	<CAH4pYpSjaNh9eYdcDMC0d_vpq2Djtq_8yfpi+m38yV-+PioCwg@mail.gmail.com>
	<CAF-LYK+EGZSSx_U0L4fiMkkhFE1=znSfmYScyXndRUUVGL2JNg@mail.gmail.com>
Message-ID: <4CFA58C7-B360-4CCB-B404-34CB5B91A2BC@gmail.com>

Super awesome. Moving from one flat to another so only internet on mobile for ? week or so.

Looking forward to try out!
-- 
M

Envoy? de mon iPhone

> Le 20 d?c. 2014 ? 23:06, Andrew Gibiansky <andrew.gibiansky at gmail.com> a ?crit :
> 
> Brian,
> 
> Sure, no problem! Let's have an offline discussion further about this ? I intentionally avoided using "IPython" or "Jupyter" in the name anywhere, but am happy to change things.
> 
> -- Andrew
> 
>> On Sat, Dec 20, 2014 at 4:56 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>> Ohh, but more importantly, I think the name of this is too close to the ideas and abstractions present in the new name of the notebook "Jupyter" and will run afoul of trademark issues. The potential for confusion would hurt both projects. I would, politely ask you to consider changing the name to something that has more distance from our trademarks. I know it may seem silly for an open source project to worry about these types of things, but we have run into issues in the past, and we need to be careful about this. I should emphasize that at this point, this is just my own personal opinion....this may be the type of thing that we need to bring up to the project's steering council.
>> 
>> This is still very cool though!
>> 
>> Cheers,
>> 
>> Brian
>> 
>>> On Sat, Dec 20, 2014 at 1:51 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>> Andrew,
>>> 
>>> Very exciting! If you aren't worried about making $, you might want to look at integrating with conda for all of the packaging things. I would hate to see you reinvent yet another packaging system.
>>> 
>>> Cheers,
>>> 
>>> Brian
>>> 
>>>> On Sat, Dec 20, 2014 at 1:44 PM, Andrew Gibiansky <andrew.gibiansky at gmail.com> wrote:
>>>> Hey all,
>>>> 
>>>> I'd like to announce the first release of an experiment I've been working on: an IPython Notebook mac app. This is similar in spirit to [1] and [2], but I've tried to iron out a few kinks people have had in installation as well as bundle some of my favorite extensions. 
>>>> 
>>>> You can download it here: http://www.europanotebook.com
>>>> 
>>>> Features:
>>>> - Completely bundled Python, separate from system python
>>>> - No external dependencies whatsoever ? *completely* bundled
>>>> - Includes matplotlib, numpy, and scipy
>>>> - Includes a few extra JS extensions, such as cell hiding and the wonderful Calico extensions [3]
>>>> - Integrated with Finder
>>>> 
>>>> This is part of my ongoing work to make IHaskell (and Haskell) more accessible ? now that IPython is neatly packaged, I plan on adding the IHaskell kernel and platform to the package. However, I figured that the intermediate may also be useful to someone.
>>>> 
>>>> Note: I was considering experimenting with making this a paid Mac application. I don't know if this is worth money to anyone ? if you have opinions on this matter, let me know, I'm still trying to figure out what I think about all of this.
>>>> 
>>>> Thanks to all the IPython devs for creating such a great platform!
>>>> -- Andrew
>>>> 
>>>> [1] https://github.com/mangecoeur/ipython-desktop
>>>> [2] https://github.com/liyanage/ipython-notebook/wiki
>>>> [3] http://wiki.roboteducation.org/ICalico
>>>> 
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> 
>>> 
>>> -- 
>>> Brian E. Granger
>>> Cal Poly State University, San Luis Obispo
>>> @ellisonbg on Twitter and GitHub
>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> 
>> 
>> -- 
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> @ellisonbg on Twitter and GitHub
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141220/f64f0df3/attachment.html>

From takowl at gmail.com  Sat Dec 20 19:40:51 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 20 Dec 2014 16:40:51 -0800
Subject: [IPython-dev] Getting the IPython path in a config file
In-Reply-To: <549579CA.5080408@elbonia.de>
References: <549579CA.5080408@elbonia.de>
Message-ID: <CAOvn4qhA8NoRgWQ5EF3w3x76NimG0RNQQprOdKA0cTon8VHuMA@mail.gmail.com>

Hi Juergen,

On 20 December 2014 at 05:29, Juergen Hasch <python at elbonia.de> wrote:

> Is there a better way than to look at sys.path[-1] to get the path to the
> IPython directory inside a config file ?


You can import IPython.utils.path and call get_ipython_dir():
http://ipython.org/ipython-doc/dev/api/generated/IPython.utils.path.html#IPython.utils.path.get_ipython_dir

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141220/b63f07ec/attachment.html>

From python at elbonia.de  Sun Dec 21 07:12:26 2014
From: python at elbonia.de (Juergen Hasch)
Date: Sun, 21 Dec 2014 13:12:26 +0100
Subject: [IPython-dev] Getting the IPython path in a config file
In-Reply-To: <CAOvn4qhA8NoRgWQ5EF3w3x76NimG0RNQQprOdKA0cTon8VHuMA@mail.gmail.com>
References: <549579CA.5080408@elbonia.de>
	<CAOvn4qhA8NoRgWQ5EF3w3x76NimG0RNQQprOdKA0cTon8VHuMA@mail.gmail.com>
Message-ID: <5496B92A.6080106@elbonia.de>

Hi Thomas,
thanks. This is exactly what I was looking for.

  Juergen

Am 21.12.2014 um 01:40 schrieb Thomas Kluyver:
> Hi Juergen,
> 
> On 20 December 2014 at 05:29, Juergen Hasch <python at elbonia.de <mailto:python at elbonia.de>> wrote:
> 
>     Is there a better way than to look at sys.path[-1] to get the path to the IPython directory inside a config file ?
> 
> 
> You can import IPython.utils.path and call get_ipython_dir():
> http://ipython.org/ipython-doc/dev/api/generated/IPython.utils.path.html#IPython.utils.path.get_ipython_dir
> 
> Thomas
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 



From anton.akhmerov at gmail.com  Mon Dec 22 09:32:59 2014
From: anton.akhmerov at gmail.com (Anton Akhmerov)
Date: Mon, 22 Dec 2014 15:32:59 +0100
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
Message-ID: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>

Hi everyone,

I am a part of the team developing an EdX online course on topology in
condensed matter (see http://tiny.cc/topocm), and I'd like to ask
several IPython notebook questions.

We decided to develop the course predominantly using IPython/Jupyter
notebooks for several reasons. First and foremost, the notebooks are
really awesome; thanks to the team and community for developing this
great tool.
On a smaller scale, the notebooks allow to share content with people
not willing to register on EdX, to tightly integrate the programming
elements of the course with the rest of the content, and finally to
decouple the course development from the EdX studio.
We intend to develop all of the course as a bunch of IPython notebooks
and develop a custom tool that uses nbconvert to generate EdX folder +
xml + content course structure.
The reason why that last part is an advantage is because there is no
notion of version control in the studio, and it's a relatively
rudimentary tool with a heavy GUI.
In a longer term perspective we could also run a jupyterhub instance
to relieve the course users from the need to setup their own
programming environment (this doesn't seem to be an option right now
due to the time constraints).

So here go the questions:

* Is the rough release date for v3.0 known? There are several
important modifications to the notebook format that make this change
relevant for us. I also hear that the release is planned soon, so do
you aim for a release some time in January?
* What are the best practices for keeping notebooks in git? We've
found nbdiff, but it seems limited: for example it doesn't do anything
to diff markdown cells. Is there a better way?
* What are the current options for creating rich graphic content that
doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
bokeh. For us ipywidgets seem like the best solution so far, but it's
a pet project of Jake Vanderplas that didn't see any action in around
half a year. So I wonder if there is anything more alive in that
respect, that'd be great. Perhaps in v3?
* In general, do you have any further advice?

Thanks,
Anton Akhmerov


From bussonniermatthias at gmail.com  Mon Dec 22 11:21:59 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Mon, 22 Dec 2014 17:21:59 +0100
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
Message-ID: <9D3B8EDD-00B3-4DEC-B2CB-171352F3AB30@gmail.com>

Hi Anton, 

Le 22 d?c. 2014 ? 15:32, Anton Akhmerov <anton.akhmerov at gmail.com> a ?crit :

> Hi everyone,
> 
> I am a part of the team developing an EdX online course on topology in
> condensed matter (see http://tiny.cc/topocm), and I'd like to ask
> several IPython notebook questions.
> 
> We decided to develop the course predominantly using IPython/Jupyter
> notebooks for several reasons. First and foremost, the notebooks are
> really awesome; thanks to the team and community for developing this
> great tool.

Thanks !

> On a smaller scale, the notebooks allow to share content with people
> not willing to register on EdX, to tightly integrate the programming
> elements of the course with the rest of the content, and finally to
> decouple the course development from the EdX studio.
> We intend to develop all of the course as a bunch of IPython notebooks
> and develop a custom tool that uses nbconvert to generate EdX folder +
> xml + content course structure.
> The reason why that last part is an advantage is because there is no
> notion of version control in the studio, and it's a relatively
> rudimentary tool with a heavy GUI.
> In a longer term perspective we could also run a jupyterhub instance
> to relieve the course users from the need to setup their own
> programming environment (this doesn't seem to be an option right now
> due to the time constraints).

Great all this is good news, thanks for sharing ! 

> * Is the rough release date for v3.0 known? There are several
> important modifications to the notebook format that make this change
> relevant for us. I also hear that the release is planned soon, so do
> you aim for a release some time in January?

We are hopping for January, Yes. 

> * What are the best practices for keeping notebooks in git? We've
> found nbdiff, but it seems limited: for example it doesn't do anything
> to diff markdown cells. Is there a better way?

There is nbflatten from Thomas Kluyver:
https://gist.github.com/takluyver/bc8f3275c7d34abb68bf

though it is not perfect. We don't have good solution for now.

> * What are the current options for creating rich graphic content that
> doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
> bokeh. For us ipywidgets seem like the best solution so far, but it's
> a pet project of Jake Vanderplas that didn't see any action in around
> half a year. So I wonder if there is anything more alive in that
> respect, that'd be great. Perhaps in v3?

We have a draft of static widgets that record the cross product of all the state your 
widget can take and could reply it on a static version, still this is pretty experimental
and require ill the computation to be done in advance by the kernel and stored, 
which is limited.

Maybe other will have more input. 


> * In general, do you have any further advice?

Did you had a look at jupyter/nbgrader in the context of classes/courses it might be useful. 
Keep  sending us feedback and use cases that will help us a lot in the future development. 

I'm sure other will have more things to say, sorry not to have been of much help, 
-- 
Matthias


> 
> Thanks,
> Anton Akhmerov
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141222/b5b1fc3e/attachment.html>

From cyrille.rossant at gmail.com  Mon Dec 22 11:47:22 2014
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Mon, 22 Dec 2014 17:47:22 +0100
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
Message-ID: <CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>

Hi Anton,

> * What are the best practices for keeping notebooks in git? We've
> found nbdiff, but it seems limited: for example it doesn't do anything
> to diff markdown cells. Is there a better way?

I am currently working on a solution that basically replaces JSON
.ipynb by regular Markdown .md. You loose all metadata and outputs,
but you keep Markdown cells and the input of code cells. It is still
highly experimental and unstable, you'll find the code here:
https://github.com/rossant/ipymd/tree/support-atlas -- let me know if
you're interested in trying it out or contributing.

> * What are the current options for creating rich graphic content that
> doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
> bokeh. For us ipywidgets seem like the best solution so far, but it's
> a pet project of Jake Vanderplas that didn't see any action in around
> half a year. So I wonder if there is anything more alive in that
> respect, that'd be great. Perhaps in v3?

I'm also very interested in this use case, but as far as I know
there's nothing ready yet... We plan to support this in VisPy, but we
haven't started to work on it yet.

I suppose you could work something out based on tmpnb.org -- you get a
temporary live kernel in the cloud launched just for you.

Cyrille


From bussonniermatthias at gmail.com  Mon Dec 22 11:57:01 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Mon, 22 Dec 2014 17:57:01 +0100
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
Message-ID: <2BE8DE52-5F9F-44E8-BF95-7EE702DCE0BC@gmail.com>


> I suppose you could work something out based on tmpnb.org -- you get a
> temporary live kernel in the cloud launched just for you.

This seem to be down just now, DNS issues, we are investigating. 
-- 
M

> 
> Cyrille
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From jgomezdans at gmail.com  Mon Dec 22 12:19:27 2014
From: jgomezdans at gmail.com (Jose Gomez-Dans)
Date: Mon, 22 Dec 2014 17:19:27 +0000
Subject: [IPython-dev] IPython parallel "education"
In-Reply-To: <CAPhiW4jG8TuHyHh=CJxZvoON5r08CxCyHKM3xpH9ae52_QB=bw@mail.gmail.com>
References: <CAMWde5opz5G1ud400Q6V39zhp63NO6O8vXe3CoNXR3ArGz2O8g@mail.gmail.com>
	<CAPhiW4jG8TuHyHh=CJxZvoON5r08CxCyHKM3xpH9ae52_QB=bw@mail.gmail.com>
Message-ID: <CAMWde5qJnWM_7x_EWoLYibKCS+g4PgRa49MRZPRjEufmt+c=Hg@mail.gmail.com>

Hi Aron,

On 18 December 2014 at 20:22, Aron Ahmadia <aron at ahmadia.net> wrote:

> What happens if instead of partitioning the data, you create a list of
> work units and map those?
> Something like:
>
> def apply_the_func(i):
>       return the_func(X[N*i):X[(i+1)*N])
>
> Y = run_func.map ( [xrange(i), apply_the_func) for i in range(nodes)] )
>

This provides a substantial speed-up. I also tested other approaches
(scatter&gather), but all in all, "pushing" X to the engines seems & using
your suggestion seems to work. A question I have is what is going on behind
the scenes when I push X around: do all the engines get a copy of the full
X? In my case, X can be quite large, and it seems expensive to send lots
and lots of data to engines that will only operate on a small fraction of
the data...

Thanks for your help
Jose
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141222/49eef8ca/attachment.html>

From odewahn at oreilly.com  Mon Dec 22 12:53:20 2014
From: odewahn at oreilly.com (Andrew Odewahn)
Date: Mon, 22 Dec 2014 12:53:20 -0500
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
Message-ID: <CAO5K8EvwouhSRrqmXu18-rj=7Ybuyuz9A3PuLsTvwzsAieAneg@mail.gmail.com>

Hi, Anton.  I've been working on something along a similar line: running a
Notebook kernel in a Docker container via tmpnb and then connecting to it
via a widget.  This presentation has some of the details:

http://odewahn.github.io/publishing-workflows-for-jupyter/#1

Like Cyrille Rossant's work, it's still very experimental, but I've been
pleased with the results so far.  The biggest issues have been around
getting it running for version 2 of the Notebook (currently it only works
for version 1) and getting the CORS headers set up for tmpnb.

Andrew

ps -- here are a few examples of the output.  (It only works in Chrome
right now due to the CORS issues):

http://sites.oreilly.com/odewahn/ipython-tutorial2/ch01.html

http://sites.oreilly.com/odewahn/jem-test/ch01.html



On Mon, Dec 22, 2014 at 11:47 AM, Cyrille Rossant <cyrille.rossant at gmail.com
> wrote:

> Hi Anton,
>
> > * What are the best practices for keeping notebooks in git? We've
> > found nbdiff, but it seems limited: for example it doesn't do anything
> > to diff markdown cells. Is there a better way?
>
> I am currently working on a solution that basically replaces JSON
> .ipynb by regular Markdown .md. You loose all metadata and outputs,
> but you keep Markdown cells and the input of code cells. It is still
> highly experimental and unstable, you'll find the code here:
> https://github.com/rossant/ipymd/tree/support-atlas -- let me know if
> you're interested in trying it out or contributing.
>
> > * What are the current options for creating rich graphic content that
> > doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
> > bokeh. For us ipywidgets seem like the best solution so far, but it's
> > a pet project of Jake Vanderplas that didn't see any action in around
> > half a year. So I wonder if there is anything more alive in that
> > respect, that'd be great. Perhaps in v3?
>
> I'm also very interested in this use case, but as far as I know
> there's nothing ready yet... We plan to support this in VisPy, but we
> haven't started to work on it yet.
>
> I suppose you could work something out based on tmpnb.org -- you get a
> temporary live kernel in the cloud launched just for you.
>
> Cyrille
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141222/68566f0e/attachment.html>

From wstein at gmail.com  Mon Dec 22 13:39:36 2014
From: wstein at gmail.com (William Stein)
Date: Mon, 22 Dec 2014 10:39:36 -0800
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CAO5K8EvwouhSRrqmXu18-rj=7Ybuyuz9A3PuLsTvwzsAieAneg@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
	<CAO5K8EvwouhSRrqmXu18-rj=7Ybuyuz9A3PuLsTvwzsAieAneg@mail.gmail.com>
Message-ID: <CACLE5GAJF=vXKUApf4Yh1vdLo4RbcjWrRu7QRmxAcnVhRGa1iA@mail.gmail.com>

Hi,

I just (seconds ago) added the ability to publish IPython notebooks
from SageMathCloud (https://cloud.sagemath.com).  I've posted a quick
video I just made for you here illustrating how it works:

  http://youtu.be/QcOK-LRCW38

Basically:
   1. Create a SageMathCloud account, a new project, and upload or
create an ipython notebook.
   2. Click the "Publish" button in the upper right corner of the
IPython notebook.
   3. Share the link.
... or ...
  Since SageMathCloud projects *are* just full Linux accounts, you can
use the Terminal via +New --> Terminal, or just ssh in (as explained
in project settings), then directly run nbconvert with whatever
options you want.    Click the "i" to the left of any file or
directory tree to make it public.

SageMathCloud is kind of like http://tmpnb.org/, except it is a lot
less "tmp" -- every project is persistent, snapshotted every few
minutes (using bup), and there is chat, collaborators, simultaneous
editing for documents, etc.   It's all open source now [1], though the
free hosted environment itself is very powerful (with 19 dedicated
servers plus cloud compute resources).  We frequently had around 800
simultaneous running projects around finals recently...

[1] https://github.com/sagemath/cloud

William

On Mon, Dec 22, 2014 at 9:53 AM, Andrew Odewahn <odewahn at oreilly.com> wrote:
> Hi, Anton.  I've been working on something along a similar line: running a
> Notebook kernel in a Docker container via tmpnb and then connecting to it
> via a widget.  This presentation has some of the details:
>
> http://odewahn.github.io/publishing-workflows-for-jupyter/#1
>
> Like Cyrille Rossant's work, it's still very experimental, but I've been
> pleased with the results so far.  The biggest issues have been around
> getting it running for version 2 of the Notebook (currently it only works
> for version 1) and getting the CORS headers set up for tmpnb.
>
> Andrew
>
> ps -- here are a few examples of the output.  (It only works in Chrome right
> now due to the CORS issues):
>
> http://sites.oreilly.com/odewahn/ipython-tutorial2/ch01.html
>
> http://sites.oreilly.com/odewahn/jem-test/ch01.html
>
>
>
> On Mon, Dec 22, 2014 at 11:47 AM, Cyrille Rossant
> <cyrille.rossant at gmail.com> wrote:
>>
>> Hi Anton,
>>
>> > * What are the best practices for keeping notebooks in git? We've
>> > found nbdiff, but it seems limited: for example it doesn't do anything
>> > to diff markdown cells. Is there a better way?
>>
>> I am currently working on a solution that basically replaces JSON
>> .ipynb by regular Markdown .md. You loose all metadata and outputs,
>> but you keep Markdown cells and the input of code cells. It is still
>> highly experimental and unstable, you'll find the code here:
>> https://github.com/rossant/ipymd/tree/support-atlas -- let me know if
>> you're interested in trying it out or contributing.
>>
>> > * What are the current options for creating rich graphic content that
>> > doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
>> > bokeh. For us ipywidgets seem like the best solution so far, but it's
>> > a pet project of Jake Vanderplas that didn't see any action in around
>> > half a year. So I wonder if there is anything more alive in that
>> > respect, that'd be great. Perhaps in v3?
>>
>> I'm also very interested in this use case, but as far as I know
>> there's nothing ready yet... We plan to support this in VisPy, but we
>> haven't started to work on it yet.
>>
>> I suppose you could work something out based on tmpnb.org -- you get a
>> temporary live kernel in the cloud launched just for you.
>>
>> Cyrille
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From moritz.beber at gmail.com  Mon Dec 22 15:50:04 2014
From: moritz.beber at gmail.com (Moritz Beber)
Date: Mon, 22 Dec 2014 21:50:04 +0100
Subject: [IPython-dev] IPython parallel "education"
In-Reply-To: <CAMWde5qJnWM_7x_EWoLYibKCS+g4PgRa49MRZPRjEufmt+c=Hg@mail.gmail.com>
References: <CAMWde5opz5G1ud400Q6V39zhp63NO6O8vXe3CoNXR3ArGz2O8g@mail.gmail.com>
	<CAPhiW4jG8TuHyHh=CJxZvoON5r08CxCyHKM3xpH9ae52_QB=bw@mail.gmail.com>
	<CAMWde5qJnWM_7x_EWoLYibKCS+g4PgRa49MRZPRjEufmt+c=Hg@mail.gmail.com>
Message-ID: <CAFOFTpQHPM3BU3Pj0roBeVA2_X4fUzGcyMLkmCNCitNSmKZm-g@mail.gmail.com>

Hi Jose,

Just wanted to share my experience with parallel:

On Mon, Dec 22, 2014 at 6:19 PM, Jose Gomez-Dans <jgomezdans at gmail.com>
wrote:

> Hi Aron,
>
> On 18 December 2014 at 20:22, Aron Ahmadia <aron at ahmadia.net> wrote:
>
>> What happens if instead of partitioning the data, you create a list of
>> work units and map those?
>> Something like:
>>
>> def apply_the_func(i):
>>       return the_func(X[N*i):X[(i+1)*N])
>>
>> Y = run_func.map ( [xrange(i), apply_the_func) for i in range(nodes)] )
>>
>
> This provides a substantial speed-up. I also tested other approaches
> (scatter&gather), but all in all, "pushing" X to the engines seems & using
> your suggestion seems to work. A question I have is what is going on behind
> the scenes when I push X around: do all the engines get a copy of the full
> X? In my case, X can be quite large, and it seems expensive to send lots
> and lots of data to engines that will only operate on a small fraction of
> the data...
>


I've been working with up to 2 GB of data and using the push mechanism is
not really feasible at that size. Also, the transmission time increases
linearly (super linearly?) with more target engines. So I've tried a few
solutions:

1.) If you're working on the same host and don't expect to expand that
switch to multiprocessing. It's very fast in transmitting data.
2.) Store your data on the file system and have each engine access that.
Either you have a shared file system for the remote kernels to access or
you'll need to copy the data beforehand/use paramiko.
3.) Having a database server is quite a bit of work to invest at the
beginning (especially if you don't know how) but really lends itself to
this sort of task. A database server usually has a connection pool so that
it can automatically handle many workers accessing it concurrently.


>
> Thanks for your help
> Jose
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
Just my thoughts/experience. Best of luck with your project,
Moritz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141222/1380479d/attachment.html>

From wstein at gmail.com  Mon Dec 22 16:30:21 2014
From: wstein at gmail.com (William Stein)
Date: Mon, 22 Dec 2014 13:30:21 -0800
Subject: [IPython-dev] [sage-cloud] Re: Developing a MOOC with IPython
	notebooks
In-Reply-To: <30e94084-9d9a-4558-af8c-e73bb82d4fa2@googlegroups.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
	<CAO5K8EvwouhSRrqmXu18-rj=7Ybuyuz9A3PuLsTvwzsAieAneg@mail.gmail.com>
	<CACLE5GAJF=vXKUApf4Yh1vdLo4RbcjWrRu7QRmxAcnVhRGa1iA@mail.gmail.com>
	<30e94084-9d9a-4558-af8c-e73bb82d4fa2@googlegroups.com>
Message-ID: <CACLE5GDj7Ko_H_haCRkLuEJs5kT+qH+_NmP-H9wV-QHJSBNfmQ@mail.gmail.com>

On Mon, Dec 22, 2014 at 12:21 PM, c.d. mclean <cdmclean.cec at gmail.com> wrote:
>
> tmpnb.org also serves its notebooks with the option of connecting to a Julia
> kernel ...
>
> i have not yet watched the video yet, but i am hoping this is an option
> available with
> a few clicks.

Sadly, not quite yet...

>
> tmpnb.org also serves terminals with a few clicks ...
>
>
> some of us knew that SMC was already effectively ahead of the curve
> on this stuff; nicely done, William !!!

Thanks for the encouragement.

>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-cloud" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-cloud+unsubscribe at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sage-cloud/30e94084-9d9a-4558-af8c-e73bb82d4fa2%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From zach at zacharyjones.us  Mon Dec 22 16:30:52 2014
From: zach at zacharyjones.us (Zachary Jones)
Date: Mon, 22 Dec 2014 16:30:52 -0500
Subject: [IPython-dev] Behavior of Default Kernel
In-Reply-To: <CAL2pmi_JT2fQ+SyRLupsyD5WyRyA6AXSrnRmi1aw2jV+V9vGcA@mail.gmail.com>
References: <CAL2pmi_JT2fQ+SyRLupsyD5WyRyA6AXSrnRmi1aw2jV+V9vGcA@mail.gmail.com>
Message-ID: <CAL2pmi8zGgetxq8iHLhtR89W0+qTaoWmvxNN_nQCJWeZT2M=Xg@mail.gmail.com>

To follow up on this...
After stepping through code, I found my way to this URL:
http://localhost:80000/api/config/tree

Which returns: {"NewNotebookWidget": {"default_kernel": "python3"}}

While  http://localhost:8000/api/kernelspecs

returns {"default": "na_python2", ...}

So it looks like something is off with the code path for '/api/config/tree'

Thanks,
Zach

On Fri, Dec 19, 2014 at 6:03 PM, Zachary Jones <zach at zacharyjones.us> wrote:

> Hi everyone,
>
> I have a question about the behavior of default kernels
>
> I was testing IPython from github today (commit hash: dc60d34) and wanted
> to understand if the behavior I am seeing is expected.
>
> The IPython notebook server is running under Python3.4, but I would the
> default kernel to be a custom Python 2 stack.
>
> Running "ipython kernelspec list" from the CLI returns:
> Available kernels:
>   python3
>   my_python2
>
> I invoked IPython notebook using the default_kernel_name:
> '--MappingKernelManager.default_kernel_name=my_python2'
>
> The API call to http://localhost:8000/api/kernelspecs returns:
> {
>     "default": "my_python2",
>     "kernelspecs": {
>         "my_python2": {
>              "env": {},
>              "argv": [*omitted*],
>              "name": "my_python2",
>              "display_name": "My Python2 Stack"
>         },
>         "python3": {
>              "env": {},
>              "argv": [*omitted*],
>              "name": "python3",
>              "display_name": "IPython (Python 3)"
>         }
>     }
> }
>
> With this configuration, I observed the following:
>
>    - When I go to to the notebook dashboard, next to the "New Notebook"
>    button is "IPython (Python 3)" and the hover over says "IPython (Python 3)
>    is the default kernel for new notebooks".
>    - When I click on "New Notebook", the notebook opened uses "IPython
>    (Python 3)" kernel.
>    - When I open existing notebooks that are being converted from v3 to
>    v4, the notebooks open using "My Python 2 Stack" kernel.
>    - When I open an existing notebook that has had the kernel spec
>    inserted into metadata, that kernel is used.
>
>
> *Finally to my questions..*
> Is the default_kernel_name supposed to be used when the "New Notebook"
> button is pressed? and is it suppose to be shown next to the "New Notebook"
> button?
>
> If the answers are yes, should I open an Issue in GitHub? I tried looking
> for an existing issue, but did not turn up anything.
>
>
> Thanks for your help,
> Zach Jones
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141222/575d9d6d/attachment.html>

From zach at zacharyjones.us  Mon Dec 22 17:04:59 2014
From: zach at zacharyjones.us (Zachary Jones)
Date: Mon, 22 Dec 2014 17:04:59 -0500
Subject: [IPython-dev] Behavior of Default Kernel
In-Reply-To: <CAL2pmi8zGgetxq8iHLhtR89W0+qTaoWmvxNN_nQCJWeZT2M=Xg@mail.gmail.com>
References: <CAL2pmi_JT2fQ+SyRLupsyD5WyRyA6AXSrnRmi1aw2jV+V9vGcA@mail.gmail.com>
	<CAL2pmi8zGgetxq8iHLhtR89W0+qTaoWmvxNN_nQCJWeZT2M=Xg@mail.gmail.com>
Message-ID: <CAL2pmi_pQivG5RGXgsj5-cXAq_7+zSdnsbs15A8Br1JxGW8NEw@mail.gmail.com>

Figured it out, with help on Gitter.  I had an old tree.json file in
~/.ipython/<profile>nbconfig

--Zach

On Mon, Dec 22, 2014 at 4:30 PM, Zachary Jones <zach at zacharyjones.us> wrote:

> To follow up on this...
> After stepping through code, I found my way to this URL:
> http://localhost:80000/api/config/tree
>
> Which returns: {"NewNotebookWidget": {"default_kernel": "python3"}}
>
> While  http://localhost:8000/api/kernelspecs
>
> returns {"default": "na_python2", ...}
>
> So it looks like something is off with the code path for '/api/config/tree'
>
> Thanks,
> Zach
>
> On Fri, Dec 19, 2014 at 6:03 PM, Zachary Jones <zach at zacharyjones.us>
> wrote:
>
>> Hi everyone,
>>
>> I have a question about the behavior of default kernels
>>
>> I was testing IPython from github today (commit hash: dc60d34) and wanted
>> to understand if the behavior I am seeing is expected.
>>
>> The IPython notebook server is running under Python3.4, but I would the
>> default kernel to be a custom Python 2 stack.
>>
>> Running "ipython kernelspec list" from the CLI returns:
>> Available kernels:
>>   python3
>>   my_python2
>>
>> I invoked IPython notebook using the default_kernel_name:
>> '--MappingKernelManager.default_kernel_name=my_python2'
>>
>> The API call to http://localhost:8000/api/kernelspecs returns:
>> {
>>     "default": "my_python2",
>>     "kernelspecs": {
>>         "my_python2": {
>>              "env": {},
>>              "argv": [*omitted*],
>>              "name": "my_python2",
>>              "display_name": "My Python2 Stack"
>>         },
>>         "python3": {
>>              "env": {},
>>              "argv": [*omitted*],
>>              "name": "python3",
>>              "display_name": "IPython (Python 3)"
>>         }
>>     }
>> }
>>
>> With this configuration, I observed the following:
>>
>>    - When I go to to the notebook dashboard, next to the "New Notebook"
>>    button is "IPython (Python 3)" and the hover over says "IPython (Python 3)
>>    is the default kernel for new notebooks".
>>    - When I click on "New Notebook", the notebook opened uses "IPython
>>    (Python 3)" kernel.
>>    - When I open existing notebooks that are being converted from v3 to
>>    v4, the notebooks open using "My Python 2 Stack" kernel.
>>    - When I open an existing notebook that has had the kernel spec
>>    inserted into metadata, that kernel is used.
>>
>>
>> *Finally to my questions..*
>> Is the default_kernel_name supposed to be used when the "New Notebook"
>> button is pressed? and is it suppose to be shown next to the "New Notebook"
>> button?
>>
>> If the answers are yes, should I open an Issue in GitHub? I tried looking
>> for an existing issue, but did not turn up anything.
>>
>>
>> Thanks for your help,
>> Zach Jones
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141222/15fafa76/attachment.html>

From ice.rikh at gmail.com  Mon Dec 22 17:15:09 2014
From: ice.rikh at gmail.com (Erik Hvatum)
Date: Mon, 22 Dec 2014 16:15:09 -0600
Subject: [IPython-dev] IPython parallel "education"
In-Reply-To: <CAFOFTpQHPM3BU3Pj0roBeVA2_X4fUzGcyMLkmCNCitNSmKZm-g@mail.gmail.com>
References: <CAMWde5opz5G1ud400Q6V39zhp63NO6O8vXe3CoNXR3ArGz2O8g@mail.gmail.com>
	<CAPhiW4jG8TuHyHh=CJxZvoON5r08CxCyHKM3xpH9ae52_QB=bw@mail.gmail.com>
	<CAMWde5qJnWM_7x_EWoLYibKCS+g4PgRa49MRZPRjEufmt+c=Hg@mail.gmail.com>
	<CAFOFTpQHPM3BU3Pj0roBeVA2_X4fUzGcyMLkmCNCitNSmKZm-g@mail.gmail.com>
Message-ID: <CAJOi7ysfp9Uyiv4jo8DYMfu1YdEf1GbJsK9deYUcMZLnQWtVQg@mail.gmail.com>

On Mon, Dec 22, 2014 at 2:50 PM, Moritz Beber <moritz.beber at gmail.com>
wrote:

> Hi Jose,
>
> Just wanted to share my experience with parallel:
>
> On Mon, Dec 22, 2014 at 6:19 PM, Jose Gomez-Dans <jgomezdans at gmail.com>
> wrote:
>
>> Hi Aron,
>>
>> On 18 December 2014 at 20:22, Aron Ahmadia <aron at ahmadia.net> wrote:
>>
>>> What happens if instead of partitioning the data, you create a list of
>>> work units and map those?
>>> Something like:
>>>
>>> def apply_the_func(i):
>>>       return the_func(X[N*i):X[(i+1)*N])
>>>
>>> Y = run_func.map ( [xrange(i), apply_the_func) for i in range(nodes)] )
>>>
>>
>> This provides a substantial speed-up. I also tested other approaches
>> (scatter&gather), but all in all, "pushing" X to the engines seems & using
>> your suggestion seems to work. A question I have is what is going on behind
>> the scenes when I push X around: do all the engines get a copy of the full
>> X? In my case, X can be quite large, and it seems expensive to send lots
>> and lots of data to engines that will only operate on a small fraction of
>> the data...
>>
>
>
> I've been working with up to 2 GB of data and using the push mechanism is
> not really feasible at that size. Also, the transmission time increases
> linearly (super linearly?) with more target engines. So I've tried a few
> solutions:
>
> 1.) If you're working on the same host and don't expect to expand that
> switch to multiprocessing. It's very fast in transmitting data.
> 2.) Store your data on the file system and have each engine access that.
> Either you have a shared file system for the remote kernels to access or
> you'll need to copy the data beforehand/use paramiko.
> 3.) Having a database server is quite a bit of work to invest at the
> beginning (especially if you don't know how) but really lends itself to
> this sort of task. A database server usually has a connection pool so that
> it can automatically handle many workers accessing it concurrently.
>
>
>>
>> Thanks for your help
>> Jose
>>
>>
> Just my thoughts/experience. Best of luck with your project,
> Moritz
>
>

It is very informative to see that other folks are interested in getting
Python-multiprocessing-style shared memory in other contexts.  I think it
would be a massive benefit for IPython clustering to opportunistically
share data backing Numpy arrays between processes, defaulting to either
mutex locking or copy-on-modify, with the ability to specify the desired
behavior.

This is actually quite doable.

As Moritz noted, Python multiprocessing transmission is extremely fast.
That's because it simply stashes shared array data in anonymous,
non-file-backed mmap regions shared between processes, rather than
transmitting it.  The thing about the anonymous, non-file-backed mmap
regions used by the Python multiprocessing module is that they are
anonymous - they are identified strictly by file descriptor.  This is no
problem in the case where the other process was forked: it inherits the
file descriptor.  Informing an unrelated process of such a region requires
piping the file descriptor through a Unix domain socket (the Python
multiprocessing module does, in fact, do this for you when you request
spawn rather than fork).

The main advantage of keeping the regions anonymous: the OS maintains a
reference count and automatically destroys and deallocates the region when
the final process with that handle open closes it or otherwise stops
existing (the region is destroyed even if that final process segfaults or
is kill -9ed).  A mmaped named region using shm must be explicitly
destroyed, requiring a watchdog process in order to ensure destruction in
the case of ungraceful exit, even if a reference count is kept in the
shared region.  This duty could perhaps be assumed by the IPython cluster
controller.

Zach Pincus and I put together a Numpy shared memory thingy along these
lines, using named, shared, non-file-backed regions with a locked reference
count and automatic deallocation:
https://github.com/erikhvatum/py_interprocess_shared_memory_blob  It's MIT
licensed, so any and all are welcome to use it for any purpose, without
limitation.

I know how to go about doing the Windows side of the implementation
(currently, only POSIX and Darwin support are complete), but haven't had a
need thus far.  If Windows support is something you want (even just for
reference), drop me a line and I'll finish up the Windows parts.

Cheers,
Erik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141222/99cdbd96/attachment.html>

From j.davidgriffiths at gmail.com  Tue Dec 23 02:40:28 2014
From: j.davidgriffiths at gmail.com (John Griffiths)
Date: Tue, 23 Dec 2014 18:40:28 +1100
Subject: [IPython-dev] magic aliases
Message-ID: <CACcz1g0EP7oSObKDj4BVdd=3nSegw8PnZn_UY5w5jfXE15k7wg@mail.gmail.com>

Simple Q I think: is it possible to add a second alias for a magic function?

Basically, I'm using %%matlab and %matlab magics a lot, and
I think my notebooks would look just a tad nicer if I could reduce those
to  %%mb and %mb.

So is it possible to register '%%mb' and '%mb' as shorthands for '%%matlab'
and '%matlab' ?


Ta,


john


-- 

Dr. John Griffiths

Post-Doctoral Research Fellow

Rotman Research Institute, Baycrest

Toronto, Canada

and

Visiting Research Fellow

School of Physics

University of Sydney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141223/65738bfb/attachment.html>

From python at elbonia.de  Tue Dec 23 02:59:10 2014
From: python at elbonia.de (Juergen Hasch)
Date: Tue, 23 Dec 2014 08:59:10 +0100
Subject: [IPython-dev] magic aliases
In-Reply-To: <CACcz1g0EP7oSObKDj4BVdd=3nSegw8PnZn_UY5w5jfXE15k7wg@mail.gmail.com>
References: <CACcz1g0EP7oSObKDj4BVdd=3nSegw8PnZn_UY5w5jfXE15k7wg@mail.gmail.com>
Message-ID: <549920CE.6040405@elbonia.de>

Try %alias

Example:
	%alias_magic t timeit
Gives:
	Created `%t` as an alias for `%timeit`.
	Created `%%t` as an alias for `%%timeit`.

  Juergen

Am 23.12.2014 um 08:40 schrieb John Griffiths:
> 
> Simple Q I think: is it possible to add a second alias for a magic function?
> 
> Basically, I'm using %%matlab and %matlab magics a lot, and
> I think my notebooks would look just a tad nicer if I could reduce those to  %%mb and %mb.
> 
> So is it possible to register '%%mb' and '%mb' as shorthands for '%%matlab' and '%matlab' ?
> 
> 
> Ta,
> 
> 
> john
> 
> 
> -- 
> 
> Dr. John Griffiths
> 
> Post-Doctoral Research Fellow
> 
> Rotman Research Institute, Baycrest
> 
> Toronto, Canada
> 
> and
> 
> Visiting Research Fellow
> 
> School of Physics
> 
> University of Sydney
> 
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 



From alimanfoo at googlemail.com  Tue Dec 23 06:06:53 2014
From: alimanfoo at googlemail.com (Alistair Miles)
Date: Tue, 23 Dec 2014 11:06:53 +0000
Subject: [IPython-dev] status of ipnbdoctest.py
In-Reply-To: <5493F7CD.3030205@gmail.com>
References: <20141219090446.10096.87594@otis.rocq.inria.fr>
	<CAJXewO=wbi9Onf_41WJD8JtQD7bpSkvtS=M8CHarJ9gLV_aYNg@mail.gmail.com>
	<5493F7CD.3030205@gmail.com>
Message-ID: <CAMr-JwbOfxU7cLoPoKE+_gV898EF1zDXxnbbqALP5S2TXzM5yA@mail.gmail.com>

Just to say I'd also find this very useful. Dealing with random numbers can
be worked around by setting the seed at the top of any notebooks you know
you want to use as doc tests. Matplotlib plots would need some special
handling as I believe the figure object ID ends up in the notebook output
and this obviously changes each run. I've seen people work around this by
stripping out object IDs before comparison, don't know what the best
general solution would be but would be very useful for testing libraries
for making plots.

Cheers,
Alistair

On Friday, 19 December 2014, Zolt?n V?r?s <zvoros at gmail.com> wrote:

>
> On 12/19/2014 10:21 AM, Nathan Goldbaum wrote:
> > You can use runipy to script notebook evaluation:
> >
> > https://github.com/paulgb/runipy
> >
> > I don't think you can verify that the output isn't changing, but you
> > can at least check that cells aren't raising exceptions and view a
> > converted version of the evaluated notebook.  I think adding
> > functionality to verify that cell output isn't changing would be a
> > neat addition to runipy.
> >
> But there are cases, when the output changes, even if you run the same
> piece of code. Two examples that come to my mind are cells that involve
> the time/date in some way, and cells that operate on random numbers. It
> might also happen that some underlying data file that is plotted is
> modified between two executions. It might be really tricky to check for
> these cases.
>
> Cheers,
> Zolt?n
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org <javascript:;>
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


-- 
Alistair Miles
Head of Epidemiological Informatics
Centre for Genomics and Global Health <http://cggh.org>
The Wellcome Trust Centre for Human Genetics
Roosevelt Drive
Oxford
OX3 7BN
United Kingdom
Web: http://purl.org/net/aliman
Email: alimanfoo at gmail.com
Tel: +44 (0)1865 287721
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141223/986a6589/attachment.html>

From bussonniermatthias at gmail.com  Tue Dec 23 06:46:52 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Tue, 23 Dec 2014 12:46:52 +0100
Subject: [IPython-dev] status of ipnbdoctest.py
In-Reply-To: <CAMr-JwbOfxU7cLoPoKE+_gV898EF1zDXxnbbqALP5S2TXzM5yA@mail.gmail.com>
References: <20141219090446.10096.87594@otis.rocq.inria.fr>
	<CAJXewO=wbi9Onf_41WJD8JtQD7bpSkvtS=M8CHarJ9gLV_aYNg@mail.gmail.com>
	<5493F7CD.3030205@gmail.com>
	<CAMr-JwbOfxU7cLoPoKE+_gV898EF1zDXxnbbqALP5S2TXzM5yA@mail.gmail.com>
Message-ID: <4EE548A4-97C9-40E2-BC9A-2A3EABCAC769@gmail.com>


Le 23 d?c. 2014 ? 12:06, Alistair Miles <alimanfoo at googlemail.com> a ?crit :

> Just to say I'd also find this very useful. Dealing with random numbers can be worked around by setting the seed at the top of any notebooks you know you want to use as doc tests. Matplotlib plots would need some special handling as I believe the figure object ID ends up in the notebook output and this obviously changes each run. I've seen people work around this by stripping out object IDs before comparison, don't know what the best general solution would be but would be very useful for testing libraries for making plots. 



You can use that :

https://github.com/ipython/ipython/blob/master/IPython/nbconvert/preprocessors/tests/test_execute.py#L26-L45

and in general, if you yes ipynb, then you probably have IPython and nbconvert install, so you can use our infrastructure to 
test that re-executing notebook does not change the content. 
-- 
M

> 
> Cheers,
> Alistair
> 
> On Friday, 19 December 2014, Zolt?n V?r?s <zvoros at gmail.com> wrote:
> 
> On 12/19/2014 10:21 AM, Nathan Goldbaum wrote:
> > You can use runipy to script notebook evaluation:
> >
> > https://github.com/paulgb/runipy
> >
> > I don't think you can verify that the output isn't changing, but you
> > can at least check that cells aren't raising exceptions and view a
> > converted version of the evaluated notebook.  I think adding
> > functionality to verify that cell output isn't changing would be a
> > neat addition to runipy.
> >
> But there are cases, when the output changes, even if you run the same
> piece of code. Two examples that come to my mind are cells that involve
> the time/date in some way, and cells that operate on random numbers. It
> might also happen that some underlying data file that is plotted is
> modified between two executions. It might be really tricky to check for
> these cases.
> 
> Cheers,
> Zolt?n
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> -- 
> Alistair Miles
> Head of Epidemiological Informatics
> Centre for Genomics and Global Health <http://cggh.org>
> The Wellcome Trust Centre for Human Genetics
> Roosevelt Drive
> Oxford
> OX3 7BN
> United Kingdom
> Web: http://purl.org/net/aliman
> Email: alimanfoo at gmail.com
> Tel: +44 (0)1865 287721
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141223/250a7259/attachment.html>

From wes.turner at gmail.com  Tue Dec 23 06:55:52 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Tue, 23 Dec 2014 05:55:52 -0600
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
Message-ID: <CACfEFw8zhgzm9SLXLVjiLqvrjcyznHYGGFsv8ziMAvHJLg8-fA@mail.gmail.com>

There was a thread a few days ago about "[IPython-dev] Making custom
converter/template with nbconvert"
specifically in regards to generating edX OLX:
http://mail.scipy.org/pipermail/ipython-dev/2014-December/015576.html

On Mon, Dec 22, 2014 at 8:32 AM, Anton Akhmerov <anton.akhmerov at gmail.com>
wrote:

> Hi everyone,
>
> I am a part of the team developing an EdX online course on topology in
> condensed matter (see http://tiny.cc/topocm), and I'd like to ask
> several IPython notebook questions.
>
> We decided to develop the course predominantly using IPython/Jupyter
> notebooks for several reasons. First and foremost, the notebooks are
> really awesome; thanks to the team and community for developing this
> great tool.
> On a smaller scale, the notebooks allow to share content with people
> not willing to register on EdX, to tightly integrate the programming
> elements of the course with the rest of the content, and finally to
> decouple the course development from the EdX studio.
> We intend to develop all of the course as a bunch of IPython notebooks
> and develop a custom tool that uses nbconvert to generate EdX folder +
> xml + content course structure.
> The reason why that last part is an advantage is because there is no
> notion of version control in the studio, and it's a relatively
> rudimentary tool with a heavy GUI.
> In a longer term perspective we could also run a jupyterhub instance
> to relieve the course users from the need to setup their own
> programming environment (this doesn't seem to be an option right now
> due to the time constraints).
>
> So here go the questions:
>
> * Is the rough release date for v3.0 known? There are several
> important modifications to the notebook format that make this change
> relevant for us. I also hear that the release is planned soon, so do
> you aim for a release some time in January?
> * What are the best practices for keeping notebooks in git? We've
> found nbdiff, but it seems limited: for example it doesn't do anything
> to diff markdown cells. Is there a better way?
> * What are the current options for creating rich graphic content that
> doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
> bokeh. For us ipywidgets seem like the best solution so far, but it's
> a pet project of Jake Vanderplas that didn't see any action in around
> half a year. So I wonder if there is anything more alive in that
> respect, that'd be great. Perhaps in v3?
> * In general, do you have any further advice?
>
> Thanks,
> Anton Akhmerov
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141223/dfd93ac2/attachment.html>

From anton.akhmerov at gmail.com  Tue Dec 23 06:59:16 2014
From: anton.akhmerov at gmail.com (Anton Akhmerov)
Date: Tue, 23 Dec 2014 12:59:16 +0100
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CACfEFw8zhgzm9SLXLVjiLqvrjcyznHYGGFsv8ziMAvHJLg8-fA@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<CACfEFw8zhgzm9SLXLVjiLqvrjcyznHYGGFsv8ziMAvHJLg8-fA@mail.gmail.com>
Message-ID: <CADr1H-8Z0Byy1AYL_y5paa_9pcLd=Z4qb5+Q_5ifbmiLMmgdPQ@mail.gmail.com>

On Tue, Dec 23, 2014 at 12:55 PM, Wes Turner <wes.turner at gmail.com> wrote:
> There was a thread a few days ago about "[IPython-dev] Making custom
> converter/template with nbconvert"
> specifically in regards to generating edX OLX:
> http://mail.scipy.org/pipermail/ipython-dev/2014-December/015576.html

Right, that's another one of us; my questions are about other
features. It seems the converter is going to be working fine
eventually.

Anton

> On Mon, Dec 22, 2014 at 8:32 AM, Anton Akhmerov <anton.akhmerov at gmail.com>
> wrote:
>>
>> Hi everyone,
>>
>> I am a part of the team developing an EdX online course on topology in
>> condensed matter (see http://tiny.cc/topocm), and I'd like to ask
>> several IPython notebook questions.
>>
>> We decided to develop the course predominantly using IPython/Jupyter
>> notebooks for several reasons. First and foremost, the notebooks are
>> really awesome; thanks to the team and community for developing this
>> great tool.
>> On a smaller scale, the notebooks allow to share content with people
>> not willing to register on EdX, to tightly integrate the programming
>> elements of the course with the rest of the content, and finally to
>> decouple the course development from the EdX studio.
>> We intend to develop all of the course as a bunch of IPython notebooks
>> and develop a custom tool that uses nbconvert to generate EdX folder +
>> xml + content course structure.
>> The reason why that last part is an advantage is because there is no
>> notion of version control in the studio, and it's a relatively
>> rudimentary tool with a heavy GUI.
>> In a longer term perspective we could also run a jupyterhub instance
>> to relieve the course users from the need to setup their own
>> programming environment (this doesn't seem to be an option right now
>> due to the time constraints).
>>
>> So here go the questions:
>>
>> * Is the rough release date for v3.0 known? There are several
>> important modifications to the notebook format that make this change
>> relevant for us. I also hear that the release is planned soon, so do
>> you aim for a release some time in January?
>> * What are the best practices for keeping notebooks in git? We've
>> found nbdiff, but it seems limited: for example it doesn't do anything
>> to diff markdown cells. Is there a better way?
>> * What are the current options for creating rich graphic content that
>> doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
>> bokeh. For us ipywidgets seem like the best solution so far, but it's
>> a pet project of Jake Vanderplas that didn't see any action in around
>> half a year. So I wonder if there is anything more alive in that
>> respect, that'd be great. Perhaps in v3?
>> * In general, do you have any further advice?
>>
>> Thanks,
>> Anton Akhmerov
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From anton.akhmerov at gmail.com  Tue Dec 23 07:15:46 2014
From: anton.akhmerov at gmail.com (Anton Akhmerov)
Date: Tue, 23 Dec 2014 13:15:46 +0100
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <9D3B8EDD-00B3-4DEC-B2CB-171352F3AB30@gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<9D3B8EDD-00B3-4DEC-B2CB-171352F3AB30@gmail.com>
Message-ID: <CADr1H-8VTc1M1_vva1kOxc1jGBwC5ck6mdUno3SbQNyA8oFoQw@mail.gmail.com>

On Mon, Dec 22, 2014 at 5:21 PM, Matthias Bussonnier
<bussonniermatthias at gmail.com> wrote:

> There is nbflatten from Thomas Kluyver:
> https://gist.github.com/takluyver/bc8f3275c7d34abb68bf
>
> though it is not perfect. We don't have good solution for now.

Yeah, it suggests nbdif for merging, so I guess we'll just use nbdiff for now.

> We have a draft of static widgets that record the cross product of all the
> state your
> widget can take and could reply it on a static version, still this is pretty
> experimental
> and require ill the computation to be done in advance by the kernel and
> stored,
> which is limited.

That's actually quite ok, do you have any pointers? Are there already PR's?

> Did you had a look at jupyter/nbgrader in the context of classes/courses it
> might be useful.

I know of nbgrader, but I don't think it will play nicely with EdX.

> Keep  sending us feedback and use cases that will help us a lot in the
> future development.

Will do, certainly.

Best,
Anton

> I'm sure other will have more things to say, sorry not to have been of much
> help,
> --
> Matthias
>
>
>
> Thanks,
> Anton Akhmerov
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From anton.akhmerov at gmail.com  Tue Dec 23 07:24:21 2014
From: anton.akhmerov at gmail.com (Anton Akhmerov)
Date: Tue, 23 Dec 2014 13:24:21 +0100
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CAO5K8EvwouhSRrqmXu18-rj=7Ybuyuz9A3PuLsTvwzsAieAneg@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
	<CAO5K8EvwouhSRrqmXu18-rj=7Ybuyuz9A3PuLsTvwzsAieAneg@mail.gmail.com>
Message-ID: <CADr1H--Leeev6mcxF-PKr+j22ou29VRJ9bHfac9_uUbUx=xObQ@mail.gmail.com>

Hi Andrew,

On Mon, Dec 22, 2014 at 6:53 PM, Andrew Odewahn <odewahn at oreilly.com> wrote:
> Hi, Anton.  I've been working on something along a similar line: running a
> Notebook kernel in a Docker container via tmpnb and then connecting to it
> via a widget.  This presentation has some of the details:

I checked out the stuff. Looks very promising, but by the looks of it,
we won't be able to set up anything like this with reasonable
stability and within our time scale.
Is there a repository, or any other source to stay up to date on the
status of your work?

Anton

> http://odewahn.github.io/publishing-workflows-for-jupyter/#1
>
> Like Cyrille Rossant's work, it's still very experimental, but I've been
> pleased with the results so far.  The biggest issues have been around
> getting it running for version 2 of the Notebook (currently it only works
> for version 1) and getting the CORS headers set up for tmpnb.
>
> Andrew
>
> ps -- here are a few examples of the output.  (It only works in Chrome right
> now due to the CORS issues):
>
> http://sites.oreilly.com/odewahn/ipython-tutorial2/ch01.html
>
> http://sites.oreilly.com/odewahn/jem-test/ch01.html
>
>
>
> On Mon, Dec 22, 2014 at 11:47 AM, Cyrille Rossant
> <cyrille.rossant at gmail.com> wrote:
>>
>> Hi Anton,
>>
>> > * What are the best practices for keeping notebooks in git? We've
>> > found nbdiff, but it seems limited: for example it doesn't do anything
>> > to diff markdown cells. Is there a better way?
>>
>> I am currently working on a solution that basically replaces JSON
>> .ipynb by regular Markdown .md. You loose all metadata and outputs,
>> but you keep Markdown cells and the input of code cells. It is still
>> highly experimental and unstable, you'll find the code here:
>> https://github.com/rossant/ipymd/tree/support-atlas -- let me know if
>> you're interested in trying it out or contributing.
>>
>> > * What are the current options for creating rich graphic content that
>> > doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
>> > bokeh. For us ipywidgets seem like the best solution so far, but it's
>> > a pet project of Jake Vanderplas that didn't see any action in around
>> > half a year. So I wonder if there is anything more alive in that
>> > respect, that'd be great. Perhaps in v3?
>>
>> I'm also very interested in this use case, but as far as I know
>> there's nothing ready yet... We plan to support this in VisPy, but we
>> haven't started to work on it yet.
>>
>> I suppose you could work something out based on tmpnb.org -- you get a
>> temporary live kernel in the cloud launched just for you.
>>
>> Cyrille
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From zvoros at gmail.com  Tue Dec 23 07:43:02 2014
From: zvoros at gmail.com (=?UTF-8?B?Wm9sdMOhbiBWw7Zyw7Zz?=)
Date: Tue, 23 Dec 2014 13:43:02 +0100
Subject: [IPython-dev] newest ipython does not start
Message-ID: <54996356.9070703@gmail.com>

Hi all,

I have just pulled the latest code from master, and it seems to me that 
there is a bug of some sort in start_ipython. Here is my traceback:

v923z at tux:~$ ipython
Traceback (most recent call last):
   File "/usr/local/bin/ipython", line 4, in <module>
     from IPython import start_ipython
   File "/usr/local/lib/python2.7/dist-packages/IPython/__init__.py", 
line 49, in <module>
     from .terminal.embed import embed
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/terminal/embed.py", line 
16, in <module>
     from IPython.core.interactiveshell import DummyMod
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", 
line 3045
     exec(code_obj, self.user_global_ns, self.user_ns)
SyntaxError: unqualified exec is not allowed in function 'run_code' it 
contains a nested function with free variables

I have also done a clean install (cloned the code from github), but the 
problem still persists. Is there something wrong on my side, or this is 
a new issue?

Thanks,
Zolt?n


From anton.akhmerov at gmail.com  Tue Dec 23 07:46:30 2014
From: anton.akhmerov at gmail.com (Anton Akhmerov)
Date: Tue, 23 Dec 2014 13:46:30 +0100
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CACLE5GAJF=vXKUApf4Yh1vdLo4RbcjWrRu7QRmxAcnVhRGa1iA@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
	<CAO5K8EvwouhSRrqmXu18-rj=7Ybuyuz9A3PuLsTvwzsAieAneg@mail.gmail.com>
	<CACLE5GAJF=vXKUApf4Yh1vdLo4RbcjWrRu7QRmxAcnVhRGa1iA@mail.gmail.com>
Message-ID: <CADr1H-_f2j1Ujo5nDG43b9wDbFXRZRE=C1FK1ReF2=WYSCoALQ@mail.gmail.com>

 On Mon, Dec 22, 2014 at 7:39 PM, William Stein <wstein at gmail.com> wrote:
> Hi,
>
> I just (seconds ago) added the ability to publish IPython notebooks
> from SageMathCloud (https://cloud.sagemath.com).

Actually I should've considered SMC. Looking at the FAQ, it seems that
it is suitable for most our needs, the only unclear thing being the
software setup.
Basically in addition to the regular scipy-stack we also need the
kwant package (http://kwant-project.org), which is in pypi.
Unfortunately it also requires the MUMPS linear algebra package, so
overall the installation process would be somewhat involved I imagine.

Do you know if we would be able to set it up, so that the course users
wouldn't have a need to install Kwant separately?

Some extra questions: Is it possible use single cell servers with
custom code pre-executed? Is it possible to embed those in an IFrame?

Thanks,
Anton

> On Mon, Dec 22, 2014 at 9:53 AM, Andrew Odewahn <odewahn at oreilly.com> wrote:
>> Hi, Anton.  I've been working on something along a similar line: running a
>> Notebook kernel in a Docker container via tmpnb and then connecting to it
>> via a widget.  This presentation has some of the details:
>>
>> http://odewahn.github.io/publishing-workflows-for-jupyter/#1
>>
>> Like Cyrille Rossant's work, it's still very experimental, but I've been
>> pleased with the results so far.  The biggest issues have been around
>> getting it running for version 2 of the Notebook (currently it only works
>> for version 1) and getting the CORS headers set up for tmpnb.
>>
>> Andrew
>>
>> ps -- here are a few examples of the output.  (It only works in Chrome right
>> now due to the CORS issues):
>>
>> http://sites.oreilly.com/odewahn/ipython-tutorial2/ch01.html
>>
>> http://sites.oreilly.com/odewahn/jem-test/ch01.html
>>
>>
>>
>> On Mon, Dec 22, 2014 at 11:47 AM, Cyrille Rossant
>> <cyrille.rossant at gmail.com> wrote:
>>>
>>> Hi Anton,
>>>
>>> > * What are the best practices for keeping notebooks in git? We've
>>> > found nbdiff, but it seems limited: for example it doesn't do anything
>>> > to diff markdown cells. Is there a better way?
>>>
>>> I am currently working on a solution that basically replaces JSON
>>> .ipynb by regular Markdown .md. You loose all metadata and outputs,
>>> but you keep Markdown cells and the input of code cells. It is still
>>> highly experimental and unstable, you'll find the code here:
>>> https://github.com/rossant/ipymd/tree/support-atlas -- let me know if
>>> you're interested in trying it out or contributing.
>>>
>>> > * What are the current options for creating rich graphic content that
>>> > doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
>>> > bokeh. For us ipywidgets seem like the best solution so far, but it's
>>> > a pet project of Jake Vanderplas that didn't see any action in around
>>> > half a year. So I wonder if there is anything more alive in that
>>> > respect, that'd be great. Perhaps in v3?
>>>
>>> I'm also very interested in this use case, but as far as I know
>>> there's nothing ready yet... We plan to support this in VisPy, but we
>>> haven't started to work on it yet.
>>>
>>> I suppose you could work something out based on tmpnb.org -- you get a
>>> temporary live kernel in the cloud launched just for you.
>>>
>>> Cyrille
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From takowl at gmail.com  Tue Dec 23 08:25:19 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 23 Dec 2014 13:25:19 +0000
Subject: [IPython-dev] newest ipython does not start
In-Reply-To: <54996356.9070703@gmail.com>
References: <54996356.9070703@gmail.com>
Message-ID: <CAOvn4qhfXE2qDoctjWhXfXWL9k2BE_rXYnT_LCaXmNJ5m1_rFQ@mail.gmail.com>

Hi Zolt?n,

On 23 December 2014 at 12:43, Zolt?n V?r?s <zvoros at gmail.com> wrote:

> I have also done a clean install (cloned the code from github), but the
> problem still persists. Is there something wrong on my side, or this is
> a new issue?


That sounds like it may have been introduced by my changes in #7262, but I
don't see it, and it didn't cause a test failure. Can you file a Github
issue and show the output from print(IPython.sys_info())?

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141223/5e79fd23/attachment.html>

From zvoros at gmail.com  Tue Dec 23 08:50:04 2014
From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Tue, 23 Dec 2014 14:50:04 +0100
Subject: [IPython-dev] newest ipython does not start
In-Reply-To: <CAOvn4qhfXE2qDoctjWhXfXWL9k2BE_rXYnT_LCaXmNJ5m1_rFQ@mail.gmail.com>
References: <54996356.9070703@gmail.com>
	<CAOvn4qhfXE2qDoctjWhXfXWL9k2BE_rXYnT_LCaXmNJ5m1_rFQ@mail.gmail.com>
Message-ID: <5499730C.4050501@gmail.com>

Hi Thomas,

Thanks for the prompt feedback!

On 12/23/2014 02:25 PM, Thomas Kluyver wrote:
> Hi Zolt?n,
>
> On 23 December 2014 at 12:43, Zolt?n V?r?s <zvoros at gmail.com 
> <mailto:zvoros at gmail.com>> wrote:
>
>     I have also done a clean install (cloned the code from github),
>     but the
>     problem still persists. Is there something wrong on my side, or
>     this is
>     a new issue?
>
>
> That sounds like it may have been introduced by my changes in #7262, 
> but I don't see it, and it didn't cause a test failure. Can you file a 
> Github issue and show the output from print(IPython.sys_info())?
I would gladly do that, but how do I get to the ipython prompt in the 
first place? At the moment, I can't start ipython at all. I could check 
out an earlier version that worked, and then get the print-out from 
there. Would that be helpful?

Cheers,
Zolt?n


From takowl at gmail.com  Tue Dec 23 09:36:50 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 23 Dec 2014 14:36:50 +0000
Subject: [IPython-dev] newest ipython does not start
In-Reply-To: <5499730C.4050501@gmail.com>
References: <54996356.9070703@gmail.com>
	<CAOvn4qhfXE2qDoctjWhXfXWL9k2BE_rXYnT_LCaXmNJ5m1_rFQ@mail.gmail.com>
	<5499730C.4050501@gmail.com>
Message-ID: <CAOvn4qhANcE638Me9KvsUqt3_cN25FiJ1px9-3QbcW1GDAQ6JQ@mail.gmail.com>

On 23 December 2014 at 13:50, Zolt?n V?r?s <zvoros at gmail.com> wrote:

> I would gladly do that, but how do I get to the ipython prompt in the
> first place? At the moment, I can't start ipython at all. I could check
> out an earlier version that worked, and then get the print-out from
> there. Would that be helpful?
>

Yes, most of the information would be the same, though mention along with
it that it's using an older commit of IPython. You could also do it from
the regular Python interactive prompt - IPython is a standard package that
you can import.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141223/81470474/attachment.html>

From takowl at gmail.com  Tue Dec 23 09:54:27 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 23 Dec 2014 14:54:27 +0000
Subject: [IPython-dev] newest ipython does not start
In-Reply-To: <54996356.9070703@gmail.com>
References: <54996356.9070703@gmail.com>
Message-ID: <CAOvn4qgMxG6rJ=e7-ucmiFnOiaB3G0Bm-m70g=00DG4p0n=zGA@mail.gmail.com>

On 23 December 2014 at 12:43, Zolt?n V?r?s <zvoros at gmail.com> wrote:

> SyntaxError: unqualified exec is not allowed in function 'run_code' it
> contains a nested function with free variables
>

I can now reproduce this on a computer that's still using Python 2.7.3.
Digging into Python bugs, I think we've fallen foul of issue 21591 (
http://bugs.python.org/issue21591 ), which was only fixed in Python
2.7.9... which was only released two weeks ago. I guess my system and the
Travis bots got updated very quickly.

I guess it's probably not reasonable to depend on a new bugfix release that
soon. It should be an easy, if annoying, fix.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141223/d8427815/attachment.html>

From takowl at gmail.com  Tue Dec 23 10:01:47 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 23 Dec 2014 15:01:47 +0000
Subject: [IPython-dev] newest ipython does not start
In-Reply-To: <CAOvn4qgMxG6rJ=e7-ucmiFnOiaB3G0Bm-m70g=00DG4p0n=zGA@mail.gmail.com>
References: <54996356.9070703@gmail.com>
	<CAOvn4qgMxG6rJ=e7-ucmiFnOiaB3G0Bm-m70g=00DG4p0n=zGA@mail.gmail.com>
Message-ID: <CAOvn4qjHQmodd2_tQmhG7ROVFw7EyPRzMfJFVCorNQQBeCpHxw@mail.gmail.com>

On 23 December 2014 at 14:54, Thomas Kluyver <takowl at gmail.com> wrote:

> I guess it's probably not reasonable to depend on a new bugfix release
> that soon. It should be an easy, if annoying, fix.


https://github.com/ipython/ipython/pull/7290
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141223/dcf9ebc9/attachment.html>

From jgomezdans at gmail.com  Tue Dec 23 10:04:27 2014
From: jgomezdans at gmail.com (Jose Gomez-Dans)
Date: Tue, 23 Dec 2014 15:04:27 +0000
Subject: [IPython-dev] IPython parallel "education"
In-Reply-To: <CAFOFTpQHPM3BU3Pj0roBeVA2_X4fUzGcyMLkmCNCitNSmKZm-g@mail.gmail.com>
References: <CAMWde5opz5G1ud400Q6V39zhp63NO6O8vXe3CoNXR3ArGz2O8g@mail.gmail.com>
	<CAPhiW4jG8TuHyHh=CJxZvoON5r08CxCyHKM3xpH9ae52_QB=bw@mail.gmail.com>
	<CAMWde5qJnWM_7x_EWoLYibKCS+g4PgRa49MRZPRjEufmt+c=Hg@mail.gmail.com>
	<CAFOFTpQHPM3BU3Pj0roBeVA2_X4fUzGcyMLkmCNCitNSmKZm-g@mail.gmail.com>
Message-ID: <CAMWde5oSvQ+4wnT2nzbHQx6xSR9Ftp6-+-UPRW-w9pV0QWCX_Q@mail.gmail.com>

Hi Moritz,

Many thanks for your comments, they're really helpful!

On 22 December 2014 at 20:50, Moritz Beber <moritz.beber at gmail.com> wrote:

> Hi Jose,
>
> Just wanted to share my experience with parallel:
>
>
> I've been working with up to 2 GB of data and using the push mechanism is
> not really feasible at that size. Also, the transmission time increases
> linearly (super linearly?) with more target engines. So I've tried a few
> solutions:
>
> 1.) If you're working on the same host and don't expect to expand that
> switch to multiprocessing. It's very fast in transmitting data.
>

For some work, I have used multiprocessing with shared memory. This worked
very well


> 2.) Store your data on the file system and have each engine access that.
> Either you have a shared file system for the remote kernels to access or
> you'll need to copy the data beforehand/use paramiko.
>

There's a significant overhead in using NFS in our system (access to the
disk server is our bottleneck, which are heavily used). This will probably
limit things. I was hoping there was some way to only transmit the chunks
you need to the engines that require them. Using a file store requires
copying the entire file to all slaves, maybe this will be a problem.

3.) Having a database server is quite a bit of work to invest at the
> beginning (especially if you don't know how) but really lends itself to
> this sort of task. A database server usually has a connection pool so that
> it can automatically handle many workers accessing it concurrently.
>

I'm not familiar with databases at all, and all our stuff is stored as
arrays. I was thinking about using HDF5 (well PyTables), given that it's
compressed and a fairly transparent numpy array "transport", but I was
hoping that i didn't have to go about using NFS!

Eriks's email is waaaay over my competence level on these things!!!

I'll keep experimenting and trying things, and hopefully report back (or
more likely, ask more questions)
Thanks!
Jose
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141223/c23b7f8a/attachment.html>

From zvoros at gmail.com  Tue Dec 23 10:29:27 2014
From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Tue, 23 Dec 2014 16:29:27 +0100
Subject: [IPython-dev] newest ipython does not start
In-Reply-To: <CAOvn4qjHQmodd2_tQmhG7ROVFw7EyPRzMfJFVCorNQQBeCpHxw@mail.gmail.com>
References: <54996356.9070703@gmail.com>	<CAOvn4qgMxG6rJ=e7-ucmiFnOiaB3G0Bm-m70g=00DG4p0n=zGA@mail.gmail.com>
	<CAOvn4qjHQmodd2_tQmhG7ROVFw7EyPRzMfJFVCorNQQBeCpHxw@mail.gmail.com>
Message-ID: <54998A57.90806@gmail.com>

Thomas,

Many thanks for tracking down the issue! I guess, the moral is, perhaps, 
that I should switch to python3.

Cheers,
Zolt?n

On 12/23/2014 04:01 PM, Thomas Kluyver wrote:
> On 23 December 2014 at 14:54, Thomas Kluyver <takowl at gmail.com 
> <mailto:takowl at gmail.com>> wrote:
>
>     I guess it's probably not reasonable to depend on a new bugfix
>     release that soon. It should be an easy, if annoying, fix.
>
>
> https://github.com/ipython/ipython/pull/7290
>



From wstein at gmail.com  Tue Dec 23 11:00:37 2014
From: wstein at gmail.com (William Stein)
Date: Tue, 23 Dec 2014 08:00:37 -0800
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CADr1H-_f2j1Ujo5nDG43b9wDbFXRZRE=C1FK1ReF2=WYSCoALQ@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
	<CAO5K8EvwouhSRrqmXu18-rj=7Ybuyuz9A3PuLsTvwzsAieAneg@mail.gmail.com>
	<CACLE5GAJF=vXKUApf4Yh1vdLo4RbcjWrRu7QRmxAcnVhRGa1iA@mail.gmail.com>
	<CADr1H-_f2j1Ujo5nDG43b9wDbFXRZRE=C1FK1ReF2=WYSCoALQ@mail.gmail.com>
Message-ID: <CACLE5GBosMSfUnnoPPavdWq+73Q6uLR4Xj4pqfpwXVEOZoOCew@mail.gmail.com>

On Tue, Dec 23, 2014 at 4:46 AM, Anton Akhmerov
<anton.akhmerov at gmail.com> wrote:
>  On Mon, Dec 22, 2014 at 7:39 PM, William Stein <wstein at gmail.com> wrote:
>> Hi,
>>
>> I just (seconds ago) added the ability to publish IPython notebooks
>> from SageMathCloud (https://cloud.sagemath.com).
>
> Actually I should've considered SMC. Looking at the FAQ, it seems that
> it is suitable for most our needs, the only unclear thing being the
> software setup.
> Basically in addition to the regular scipy-stack we also need the
> kwant package (http://kwant-project.org), which is in pypi.

Since it was so easy, kwant is now installed for all SMC projects:

  https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-a18c706e8bcd/files/support/2014-12-23-kwant.html


Note: on import of kwant, it says "RuntimeWarning: The installed SciPy
does not use UMFPACK. Instead, SciPy will use the version of SuperLu
it is shipped with. Performance can be very poor in this case."   Can
you run tests and let me know if this is a problem?  If you know how
to rebuild scipy so it uses UMFPACK, I can fix this, of course.  (With
Sage -- which I know much better than scipy -- there are a million
subtle performance dependencies like this, and it's best to get
feedback from an expert when one hits one of them.)


> Unfortunately it also requires the MUMPS linear algebra package, so
> overall the installation process would be somewhat involved I imagine.
>
> Do you know if we would be able to set it up, so that the course users
> wouldn't have a need to install Kwant separately?

Regarding kwant -- done!

Regarding MUMPS, is that this somewhat strange seeming "public-domain"
library, where you fill out a form and get an email link?

    http://mumps.enseeiht.fr/index.php?page=dwnld#license

Anyway, my model with SMC is to pre-install a very wide range of
packages, available for everybody, motivated by what people need.    I
have a script [1] that installs everything into a custom Python
install (=Sage), each time I upgrade to a new version (though I leave
the old versions in case people need them).

> Some extra questions: Is it possible use single cell servers with
> custom code pre-executed?
>  Is it possible to embed those in an IFrame?

I don't know -- I've cc'd Andrey and Jason, the single cell server
devs, and the sage-cell mailng list.

[1] https://github.com/sagemath/cloud/blob/master/build.py

William

>
> Thanks,
> Anton
>
>> On Mon, Dec 22, 2014 at 9:53 AM, Andrew Odewahn <odewahn at oreilly.com> wrote:
>>> Hi, Anton.  I've been working on something along a similar line: running a
>>> Notebook kernel in a Docker container via tmpnb and then connecting to it
>>> via a widget.  This presentation has some of the details:
>>>
>>> http://odewahn.github.io/publishing-workflows-for-jupyter/#1
>>>
>>> Like Cyrille Rossant's work, it's still very experimental, but I've been
>>> pleased with the results so far.  The biggest issues have been around
>>> getting it running for version 2 of the Notebook (currently it only works
>>> for version 1) and getting the CORS headers set up for tmpnb.
>>>
>>> Andrew
>>>
>>> ps -- here are a few examples of the output.  (It only works in Chrome right
>>> now due to the CORS issues):
>>>
>>> http://sites.oreilly.com/odewahn/ipython-tutorial2/ch01.html
>>>
>>> http://sites.oreilly.com/odewahn/jem-test/ch01.html
>>>
>>>
>>>
>>> On Mon, Dec 22, 2014 at 11:47 AM, Cyrille Rossant
>>> <cyrille.rossant at gmail.com> wrote:
>>>>
>>>> Hi Anton,
>>>>
>>>> > * What are the best practices for keeping notebooks in git? We've
>>>> > found nbdiff, but it seems limited: for example it doesn't do anything
>>>> > to diff markdown cells. Is there a better way?
>>>>
>>>> I am currently working on a solution that basically replaces JSON
>>>> .ipynb by regular Markdown .md. You loose all metadata and outputs,
>>>> but you keep Markdown cells and the input of code cells. It is still
>>>> highly experimental and unstable, you'll find the code here:
>>>> https://github.com/rossant/ipymd/tree/support-atlas -- let me know if
>>>> you're interested in trying it out or contributing.
>>>>
>>>> > * What are the current options for creating rich graphic content that
>>>> > doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
>>>> > bokeh. For us ipywidgets seem like the best solution so far, but it's
>>>> > a pet project of Jake Vanderplas that didn't see any action in around
>>>> > half a year. So I wonder if there is anything more alive in that
>>>> > respect, that'd be great. Perhaps in v3?
>>>>
>>>> I'm also very interested in this use case, but as far as I know
>>>> there's nothing ready yet... We plan to support this in VisPy, but we
>>>> haven't started to work on it yet.
>>>>
>>>> I suppose you could work something out based on tmpnb.org -- you get a
>>>> temporary live kernel in the cloud launched just for you.
>>>>
>>>> Cyrille
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>>
>> --
>> William Stein
>> Professor of Mathematics
>> University of Washington
>> http://wstein.org
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From takowl at gmail.com  Tue Dec 23 11:06:56 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 23 Dec 2014 16:06:56 +0000
Subject: [IPython-dev] newest ipython does not start
In-Reply-To: <54998A57.90806@gmail.com>
References: <54996356.9070703@gmail.com>
	<CAOvn4qgMxG6rJ=e7-ucmiFnOiaB3G0Bm-m70g=00DG4p0n=zGA@mail.gmail.com>
	<CAOvn4qjHQmodd2_tQmhG7ROVFw7EyPRzMfJFVCorNQQBeCpHxw@mail.gmail.com>
	<54998A57.90806@gmail.com>
Message-ID: <CAOvn4qgyHOi+X1e+7F9o2GXdg6+H-3359sgGn_rH2APJ7rqRGg@mail.gmail.com>

On 23 December 2014 at 15:29, Zolt?n V?r?s <zvoros at gmail.com> wrote:

> Many thanks for tracking down the issue! I guess, the moral is, perhaps,
> that I should switch to python3.
>

That also doesn't exhibit the problem.

The Python 2 'exec code in ns' syntax became a function 'exec(code, ns)' in
Python 3. But the latter syntax also works in Python 2 - it sees it as an
exec statement with a tuple, and treats the tuple like the equivalent
function call in Python 3. So we're relying on that and only using the
function call form to call exec. But there was evidently a bug in exactly
how Python 2 handled the new syntax.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141223/4c2a34aa/attachment.html>

From wstein at gmail.com  Tue Dec 23 11:48:24 2014
From: wstein at gmail.com (William Stein)
Date: Tue, 23 Dec 2014 08:48:24 -0800
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CACLE5GBosMSfUnnoPPavdWq+73Q6uLR4Xj4pqfpwXVEOZoOCew@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
	<CAO5K8EvwouhSRrqmXu18-rj=7Ybuyuz9A3PuLsTvwzsAieAneg@mail.gmail.com>
	<CACLE5GAJF=vXKUApf4Yh1vdLo4RbcjWrRu7QRmxAcnVhRGa1iA@mail.gmail.com>
	<CADr1H-_f2j1Ujo5nDG43b9wDbFXRZRE=C1FK1ReF2=WYSCoALQ@mail.gmail.com>
	<CACLE5GBosMSfUnnoPPavdWq+73Q6uLR4Xj4pqfpwXVEOZoOCew@mail.gmail.com>
Message-ID: <CACLE5GBQpa1c62NxjZ3pkREKgOW-y37RvNbVDM=Q8P6RRzgF-Q@mail.gmail.com>

On Tue, Dec 23, 2014 at 8:00 AM, William Stein <wstein at gmail.com> wrote:
> On Tue, Dec 23, 2014 at 4:46 AM, Anton Akhmerov
> <anton.akhmerov at gmail.com> wrote:
>>  On Mon, Dec 22, 2014 at 7:39 PM, William Stein <wstein at gmail.com> wrote:
>>> Hi,
>>>
>>> I just (seconds ago) added the ability to publish IPython notebooks
>>> from SageMathCloud (https://cloud.sagemath.com).
>>
>> Actually I should've considered SMC. Looking at the FAQ, it seems that
>> it is suitable for most our needs, the only unclear thing being the
>> software setup.
>> Basically in addition to the regular scipy-stack we also need the
>> kwant package (http://kwant-project.org), which is in pypi.
>
> Since it was so easy, kwant is now installed for all SMC projects:
>
>   https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-a18c706e8bcd/files/support/2014-12-23-kwant.html
>
>
> Note: on import of kwant, it says "RuntimeWarning: The installed SciPy
> does not use UMFPACK. Instead, SciPy will use the version of SuperLu
> it is shipped with. Performance can be very poor in this case."   Can
> you run tests and let me know if this is a problem?  If you know how
> to rebuild scipy so it uses UMFPACK, I can fix this, of course.  (With
> Sage -- which I know much better than scipy -- there are a million
> subtle performance dependencies like this, and it's best to get
> feedback from an expert when one hits one of them.)

Hi,

I removed the pip version and installed Kwant system-wide using the
Ubuntu Packages suggested at http://kwant-project.org/install

Now kwant works fine without the UMFPACK warning.

 --  William

>
>
>> Unfortunately it also requires the MUMPS linear algebra package, so
>> overall the installation process would be somewhat involved I imagine.
>>
>> Do you know if we would be able to set it up, so that the course users
>> wouldn't have a need to install Kwant separately?
>
> Regarding kwant -- done!
>
> Regarding MUMPS, is that this somewhat strange seeming "public-domain"
> library, where you fill out a form and get an email link?
>
>     http://mumps.enseeiht.fr/index.php?page=dwnld#license
>
> Anyway, my model with SMC is to pre-install a very wide range of
> packages, available for everybody, motivated by what people need.    I
> have a script [1] that installs everything into a custom Python
> install (=Sage), each time I upgrade to a new version (though I leave
> the old versions in case people need them).
>
>> Some extra questions: Is it possible use single cell servers with
>> custom code pre-executed?
>>  Is it possible to embed those in an IFrame?
>
> I don't know -- I've cc'd Andrey and Jason, the single cell server
> devs, and the sage-cell mailng list.
>
> [1] https://github.com/sagemath/cloud/blob/master/build.py
>
> William
>
>>
>> Thanks,
>> Anton
>>
>>> On Mon, Dec 22, 2014 at 9:53 AM, Andrew Odewahn <odewahn at oreilly.com> wrote:
>>>> Hi, Anton.  I've been working on something along a similar line: running a
>>>> Notebook kernel in a Docker container via tmpnb and then connecting to it
>>>> via a widget.  This presentation has some of the details:
>>>>
>>>> http://odewahn.github.io/publishing-workflows-for-jupyter/#1
>>>>
>>>> Like Cyrille Rossant's work, it's still very experimental, but I've been
>>>> pleased with the results so far.  The biggest issues have been around
>>>> getting it running for version 2 of the Notebook (currently it only works
>>>> for version 1) and getting the CORS headers set up for tmpnb.
>>>>
>>>> Andrew
>>>>
>>>> ps -- here are a few examples of the output.  (It only works in Chrome right
>>>> now due to the CORS issues):
>>>>
>>>> http://sites.oreilly.com/odewahn/ipython-tutorial2/ch01.html
>>>>
>>>> http://sites.oreilly.com/odewahn/jem-test/ch01.html
>>>>
>>>>
>>>>
>>>> On Mon, Dec 22, 2014 at 11:47 AM, Cyrille Rossant
>>>> <cyrille.rossant at gmail.com> wrote:
>>>>>
>>>>> Hi Anton,
>>>>>
>>>>> > * What are the best practices for keeping notebooks in git? We've
>>>>> > found nbdiff, but it seems limited: for example it doesn't do anything
>>>>> > to diff markdown cells. Is there a better way?
>>>>>
>>>>> I am currently working on a solution that basically replaces JSON
>>>>> .ipynb by regular Markdown .md. You loose all metadata and outputs,
>>>>> but you keep Markdown cells and the input of code cells. It is still
>>>>> highly experimental and unstable, you'll find the code here:
>>>>> https://github.com/rossant/ipymd/tree/support-atlas -- let me know if
>>>>> you're interested in trying it out or contributing.
>>>>>
>>>>> > * What are the current options for creating rich graphic content that
>>>>> > doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
>>>>> > bokeh. For us ipywidgets seem like the best solution so far, but it's
>>>>> > a pet project of Jake Vanderplas that didn't see any action in around
>>>>> > half a year. So I wonder if there is anything more alive in that
>>>>> > respect, that'd be great. Perhaps in v3?
>>>>>
>>>>> I'm also very interested in this use case, but as far as I know
>>>>> there's nothing ready yet... We plan to support this in VisPy, but we
>>>>> haven't started to work on it yet.
>>>>>
>>>>> I suppose you could work something out based on tmpnb.org -- you get a
>>>>> temporary live kernel in the cloud launched just for you.
>>>>>
>>>>> Cyrille
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>
>>>
>>>
>>> --
>>> William Stein
>>> Professor of Mathematics
>>> University of Washington
>>> http://wstein.org
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From anton.akhmerov at gmail.com  Tue Dec 23 11:53:16 2014
From: anton.akhmerov at gmail.com (Anton Akhmerov)
Date: Tue, 23 Dec 2014 17:53:16 +0100
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CACLE5GBQpa1c62NxjZ3pkREKgOW-y37RvNbVDM=Q8P6RRzgF-Q@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
	<CAO5K8EvwouhSRrqmXu18-rj=7Ybuyuz9A3PuLsTvwzsAieAneg@mail.gmail.com>
	<CACLE5GAJF=vXKUApf4Yh1vdLo4RbcjWrRu7QRmxAcnVhRGa1iA@mail.gmail.com>
	<CADr1H-_f2j1Ujo5nDG43b9wDbFXRZRE=C1FK1ReF2=WYSCoALQ@mail.gmail.com>
	<CACLE5GBosMSfUnnoPPavdWq+73Q6uLR4Xj4pqfpwXVEOZoOCew@mail.gmail.com>
	<CACLE5GBQpa1c62NxjZ3pkREKgOW-y37RvNbVDM=Q8P6RRzgF-Q@mail.gmail.com>
Message-ID: <CADr1H-9FAWiLXvjjbTj2FPjgP=UR8fY8HZOGRix+3NpiWnAFxA@mail.gmail.com>

Hi,

On Tue, Dec 23, 2014 at 5:48 PM, William Stein <wstein at gmail.com> wrote:
> I removed the pip version and installed Kwant system-wide using the
> Ubuntu Packages suggested at http://kwant-project.org/install
>
> Now kwant works fine without the UMFPACK warning.

Perfect, this means that also MUMPS is installed through the Ubuntu
repos (it's actually indeed public domain, they just want to collect
user emails).
Sadly Umfpack in Scipy is now gone due to software rot and GPL <->
Python world incompatibility.

Thanks a lot for providing Kwant in SMC. This makes SMC really useful
both for the Kwant team and the MOOC I'm organizing.

Anton

>  --  William
>
>>
>>
>>> Unfortunately it also requires the MUMPS linear algebra package, so
>>> overall the installation process would be somewhat involved I imagine.
>>>
>>> Do you know if we would be able to set it up, so that the course users
>>> wouldn't have a need to install Kwant separately?
>>
>> Regarding kwant -- done!
>>
>> Regarding MUMPS, is that this somewhat strange seeming "public-domain"
>> library, where you fill out a form and get an email link?
>>
>>     http://mumps.enseeiht.fr/index.php?page=dwnld#license
>>
>> Anyway, my model with SMC is to pre-install a very wide range of
>> packages, available for everybody, motivated by what people need.    I
>> have a script [1] that installs everything into a custom Python
>> install (=Sage), each time I upgrade to a new version (though I leave
>> the old versions in case people need them).
>>
>>> Some extra questions: Is it possible use single cell servers with
>>> custom code pre-executed?
>>>  Is it possible to embed those in an IFrame?
>>
>> I don't know -- I've cc'd Andrey and Jason, the single cell server
>> devs, and the sage-cell mailng list.
>>
>> [1] https://github.com/sagemath/cloud/blob/master/build.py
>>
>> William
>>
>>>
>>> Thanks,
>>> Anton
>>>
>>>> On Mon, Dec 22, 2014 at 9:53 AM, Andrew Odewahn <odewahn at oreilly.com> wrote:
>>>>> Hi, Anton.  I've been working on something along a similar line: running a
>>>>> Notebook kernel in a Docker container via tmpnb and then connecting to it
>>>>> via a widget.  This presentation has some of the details:
>>>>>
>>>>> http://odewahn.github.io/publishing-workflows-for-jupyter/#1
>>>>>
>>>>> Like Cyrille Rossant's work, it's still very experimental, but I've been
>>>>> pleased with the results so far.  The biggest issues have been around
>>>>> getting it running for version 2 of the Notebook (currently it only works
>>>>> for version 1) and getting the CORS headers set up for tmpnb.
>>>>>
>>>>> Andrew
>>>>>
>>>>> ps -- here are a few examples of the output.  (It only works in Chrome right
>>>>> now due to the CORS issues):
>>>>>
>>>>> http://sites.oreilly.com/odewahn/ipython-tutorial2/ch01.html
>>>>>
>>>>> http://sites.oreilly.com/odewahn/jem-test/ch01.html
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Dec 22, 2014 at 11:47 AM, Cyrille Rossant
>>>>> <cyrille.rossant at gmail.com> wrote:
>>>>>>
>>>>>> Hi Anton,
>>>>>>
>>>>>> > * What are the best practices for keeping notebooks in git? We've
>>>>>> > found nbdiff, but it seems limited: for example it doesn't do anything
>>>>>> > to diff markdown cells. Is there a better way?
>>>>>>
>>>>>> I am currently working on a solution that basically replaces JSON
>>>>>> .ipynb by regular Markdown .md. You loose all metadata and outputs,
>>>>>> but you keep Markdown cells and the input of code cells. It is still
>>>>>> highly experimental and unstable, you'll find the code here:
>>>>>> https://github.com/rossant/ipymd/tree/support-atlas -- let me know if
>>>>>> you're interested in trying it out or contributing.
>>>>>>
>>>>>> > * What are the current options for creating rich graphic content that
>>>>>> > doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
>>>>>> > bokeh. For us ipywidgets seem like the best solution so far, but it's
>>>>>> > a pet project of Jake Vanderplas that didn't see any action in around
>>>>>> > half a year. So I wonder if there is anything more alive in that
>>>>>> > respect, that'd be great. Perhaps in v3?
>>>>>>
>>>>>> I'm also very interested in this use case, but as far as I know
>>>>>> there's nothing ready yet... We plan to support this in VisPy, but we
>>>>>> haven't started to work on it yet.
>>>>>>
>>>>>> I suppose you could work something out based on tmpnb.org -- you get a
>>>>>> temporary live kernel in the cloud launched just for you.
>>>>>>
>>>>>> Cyrille
>>>>>> _______________________________________________
>>>>>> IPython-dev mailing list
>>>>>> IPython-dev at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> William Stein
>>>> Professor of Mathematics
>>>> University of Washington
>>>> http://wstein.org
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> William Stein
>> Professor of Mathematics
>> University of Washington
>> http://wstein.org
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From wstein at gmail.com  Tue Dec 23 11:57:38 2014
From: wstein at gmail.com (William Stein)
Date: Tue, 23 Dec 2014 08:57:38 -0800
Subject: [IPython-dev] Developing a MOOC with IPython notebooks
In-Reply-To: <CADr1H-9FAWiLXvjjbTj2FPjgP=UR8fY8HZOGRix+3NpiWnAFxA@mail.gmail.com>
References: <CADr1H--5K8cgDXkr9Wc6GGFn6kvBQ=q3S+A_0_N9Cee6dvR7_A@mail.gmail.com>
	<CA+-1RQTQUD6NOWtXQKXugFKQucKTLHdLBp7AqtMsX04nEjDDRw@mail.gmail.com>
	<CAO5K8EvwouhSRrqmXu18-rj=7Ybuyuz9A3PuLsTvwzsAieAneg@mail.gmail.com>
	<CACLE5GAJF=vXKUApf4Yh1vdLo4RbcjWrRu7QRmxAcnVhRGa1iA@mail.gmail.com>
	<CADr1H-_f2j1Ujo5nDG43b9wDbFXRZRE=C1FK1ReF2=WYSCoALQ@mail.gmail.com>
	<CACLE5GBosMSfUnnoPPavdWq+73Q6uLR4Xj4pqfpwXVEOZoOCew@mail.gmail.com>
	<CACLE5GBQpa1c62NxjZ3pkREKgOW-y37RvNbVDM=Q8P6RRzgF-Q@mail.gmail.com>
	<CADr1H-9FAWiLXvjjbTj2FPjgP=UR8fY8HZOGRix+3NpiWnAFxA@mail.gmail.com>
Message-ID: <CACLE5GDUmaVAZ8+AgQUS3CFQSyaWPsNw2SkDmirZXMW4ECf5hA@mail.gmail.com>

On Tue, Dec 23, 2014 at 8:53 AM, Anton Akhmerov
<anton.akhmerov at gmail.com> wrote:
> Hi,
>
> On Tue, Dec 23, 2014 at 5:48 PM, William Stein <wstein at gmail.com> wrote:
>> I removed the pip version and installed Kwant system-wide using the
>> Ubuntu Packages suggested at http://kwant-project.org/install
>>
>> Now kwant works fine without the UMFPACK warning.
>
> Perfect, this means that also MUMPS is installed through the Ubuntu
> repos (it's actually indeed public domain, they just want to collect
> user emails).
> Sadly Umfpack in Scipy is now gone due to software rot and GPL <->
> Python world incompatibility.
>
> Thanks a lot for providing Kwant in SMC. This makes SMC really useful
> both for the Kwant team and the MOOC I'm organizing.

You're very welcome.  Please don't hesitate to ask if you have any
additional SMC-related needs!

 -- William

>
> Anton
>
>>  --  William
>>
>>>
>>>
>>>> Unfortunately it also requires the MUMPS linear algebra package, so
>>>> overall the installation process would be somewhat involved I imagine.
>>>>
>>>> Do you know if we would be able to set it up, so that the course users
>>>> wouldn't have a need to install Kwant separately?
>>>
>>> Regarding kwant -- done!
>>>
>>> Regarding MUMPS, is that this somewhat strange seeming "public-domain"
>>> library, where you fill out a form and get an email link?
>>>
>>>     http://mumps.enseeiht.fr/index.php?page=dwnld#license
>>>
>>> Anyway, my model with SMC is to pre-install a very wide range of
>>> packages, available for everybody, motivated by what people need.    I
>>> have a script [1] that installs everything into a custom Python
>>> install (=Sage), each time I upgrade to a new version (though I leave
>>> the old versions in case people need them).
>>>
>>>> Some extra questions: Is it possible use single cell servers with
>>>> custom code pre-executed?
>>>>  Is it possible to embed those in an IFrame?
>>>
>>> I don't know -- I've cc'd Andrey and Jason, the single cell server
>>> devs, and the sage-cell mailng list.
>>>
>>> [1] https://github.com/sagemath/cloud/blob/master/build.py
>>>
>>> William
>>>
>>>>
>>>> Thanks,
>>>> Anton
>>>>
>>>>> On Mon, Dec 22, 2014 at 9:53 AM, Andrew Odewahn <odewahn at oreilly.com> wrote:
>>>>>> Hi, Anton.  I've been working on something along a similar line: running a
>>>>>> Notebook kernel in a Docker container via tmpnb and then connecting to it
>>>>>> via a widget.  This presentation has some of the details:
>>>>>>
>>>>>> http://odewahn.github.io/publishing-workflows-for-jupyter/#1
>>>>>>
>>>>>> Like Cyrille Rossant's work, it's still very experimental, but I've been
>>>>>> pleased with the results so far.  The biggest issues have been around
>>>>>> getting it running for version 2 of the Notebook (currently it only works
>>>>>> for version 1) and getting the CORS headers set up for tmpnb.
>>>>>>
>>>>>> Andrew
>>>>>>
>>>>>> ps -- here are a few examples of the output.  (It only works in Chrome right
>>>>>> now due to the CORS issues):
>>>>>>
>>>>>> http://sites.oreilly.com/odewahn/ipython-tutorial2/ch01.html
>>>>>>
>>>>>> http://sites.oreilly.com/odewahn/jem-test/ch01.html
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Dec 22, 2014 at 11:47 AM, Cyrille Rossant
>>>>>> <cyrille.rossant at gmail.com> wrote:
>>>>>>>
>>>>>>> Hi Anton,
>>>>>>>
>>>>>>> > * What are the best practices for keeping notebooks in git? We've
>>>>>>> > found nbdiff, but it seems limited: for example it doesn't do anything
>>>>>>> > to diff markdown cells. Is there a better way?
>>>>>>>
>>>>>>> I am currently working on a solution that basically replaces JSON
>>>>>>> .ipynb by regular Markdown .md. You loose all metadata and outputs,
>>>>>>> but you keep Markdown cells and the input of code cells. It is still
>>>>>>> highly experimental and unstable, you'll find the code here:
>>>>>>> https://github.com/rossant/ipymd/tree/support-atlas -- let me know if
>>>>>>> you're interested in trying it out or contributing.
>>>>>>>
>>>>>>> > * What are the current options for creating rich graphic content that
>>>>>>> > doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
>>>>>>> > bokeh. For us ipywidgets seem like the best solution so far, but it's
>>>>>>> > a pet project of Jake Vanderplas that didn't see any action in around
>>>>>>> > half a year. So I wonder if there is anything more alive in that
>>>>>>> > respect, that'd be great. Perhaps in v3?
>>>>>>>
>>>>>>> I'm also very interested in this use case, but as far as I know
>>>>>>> there's nothing ready yet... We plan to support this in VisPy, but we
>>>>>>> haven't started to work on it yet.
>>>>>>>
>>>>>>> I suppose you could work something out based on tmpnb.org -- you get a
>>>>>>> temporary live kernel in the cloud launched just for you.
>>>>>>>
>>>>>>> Cyrille
>>>>>>> _______________________________________________
>>>>>>> IPython-dev mailing list
>>>>>>> IPython-dev at scipy.org
>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> IPython-dev mailing list
>>>>>> IPython-dev at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> William Stein
>>>>> Professor of Mathematics
>>>>> University of Washington
>>>>> http://wstein.org
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>>
>>> --
>>> William Stein
>>> Professor of Mathematics
>>> University of Washington
>>> http://wstein.org
>>
>>
>>
>> --
>> William Stein
>> Professor of Mathematics
>> University of Washington
>> http://wstein.org
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From rawlins at gmail.com  Wed Dec 24 16:52:19 2014
From: rawlins at gmail.com (Kyle Rawlins)
Date: Wed, 24 Dec 2014 16:52:19 -0500
Subject: [IPython-dev] line magic in middle of line
In-Reply-To: <mailman.5.1419357601.26870.ipython-dev@scipy.org>
References: <mailman.5.1419357601.26870.ipython-dev@scipy.org>
Message-ID: <BEFF2971-0B2A-4B19-BDBB-566B294C682F@gmail.com>

Hi all,
To my surprise I just noticed that assignment of a line magic that returns something to a variable works.  That is, I have a line magic I?ve written called `te` that parses something and returns a python object, and code like the following (specific to this magic, sorry) seems to do exactly what I would?ve hoped (but didn?t expect).

	x = %te x_e

Is this documented anywhere / actually supported behavior?  Under what conditions can line magics be mixed with python code?  Some rudimentary testing suggests that this might be specific to statements.  It is actually super useful to me that this works, but I couldn?t find any documentation that talked about this (or even any built in magics where this behavior would make much sense), so it would be helpful to know the limits.

Happy holidays,
-kyle

-- 
Kyle Rawlins
http://sites.krieger.jhu.edu/rawlins/
Department of Cognitive Science
Johns Hopkins University

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141224/7b896a61/attachment.html>

From takowl at gmail.com  Wed Dec 24 17:28:09 2014
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 24 Dec 2014 22:28:09 +0000
Subject: [IPython-dev] line magic in middle of line
In-Reply-To: <BEFF2971-0B2A-4B19-BDBB-566B294C682F@gmail.com>
References: <mailman.5.1419357601.26870.ipython-dev@scipy.org>
	<BEFF2971-0B2A-4B19-BDBB-566B294C682F@gmail.com>
Message-ID: <CAOvn4qhVYeM5_Nys6RaZ2degweHu5r400ide6pmSuOGj_pHtXw@mail.gmail.com>

Hi Kyle,

On 24 December 2014 at 21:52, Kyle Rawlins <rawlins at gmail.com> wrote:

> Is this documented anywhere / actually supported behavior?  Under what
> conditions can line magics be mixed with python code?


It only works in simple assignments, and you can do it with both magics and
!system escapes - the normal demo of that feature is to do "files = !ls".
The timeit magic also returns a useful results object.

It is documented for system commands:
http://ipython.org/ipython-doc/dev/interactive/tutorial.html#system-shell-commands
http://ipython.org/ipython-doc/dev/interactive/reference.html#manual-capture-of-command-output

But I can't currently see anywhere where this is documented for magic
functions.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141224/00240b92/attachment.html>

From damianavila at gmail.com  Wed Dec 24 17:35:48 2014
From: damianavila at gmail.com (=?UTF-8?Q?Dami=C3=A1n_Avila?=)
Date: Wed, 24 Dec 2014 19:35:48 -0300
Subject: [IPython-dev] line magic in middle of line
In-Reply-To: <CAOvn4qhVYeM5_Nys6RaZ2degweHu5r400ide6pmSuOGj_pHtXw@mail.gmail.com>
References: <mailman.5.1419357601.26870.ipython-dev@scipy.org>
	<BEFF2971-0B2A-4B19-BDBB-566B294C682F@gmail.com>
	<CAOvn4qhVYeM5_Nys6RaZ2degweHu5r400ide6pmSuOGj_pHtXw@mail.gmail.com>
Message-ID: <CAH+mRR2hW2uw=oCHsUEzwr_r7zrt_cQ4yKJFoTqyFDzTBm1v5w@mail.gmail.com>

>But I can't currently see anywhere where this is documented for magic
functions.

It is poorly "documented" (in fact, shown) in the examples [1] and, IIRC,
in the IPython-in-depth tutorial...

[1]
http://nbviewer.ipython.org/github/ipython/ipython/blob/master/examples/IPython%20Kernel/Beyond%20Plain%20Python.ipynb

Cheers


2014-12-24 19:28 GMT-03:00 Thomas Kluyver <takowl at gmail.com>:

> Hi Kyle,
>
> On 24 December 2014 at 21:52, Kyle Rawlins <rawlins at gmail.com> wrote:
>
>> Is this documented anywhere / actually supported behavior?  Under what
>> conditions can line magics be mixed with python code?
>
>
> It only works in simple assignments, and you can do it with both magics
> and !system escapes - the normal demo of that feature is to do "files =
> !ls". The timeit magic also returns a useful results object.
>
> It is documented for system commands:
>
> http://ipython.org/ipython-doc/dev/interactive/tutorial.html#system-shell-commands
>
> http://ipython.org/ipython-doc/dev/interactive/reference.html#manual-capture-of-command-output
>
> But I can't currently see anywhere where this is documented for magic
> functions.
>
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
*Dami?n*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141224/a3115d69/attachment.html>

From rawlins at gmail.com  Fri Dec 26 18:00:29 2014
From: rawlins at gmail.com (Kyle Rawlins)
Date: Fri, 26 Dec 2014 18:00:29 -0500
Subject: [IPython-dev] IPython-dev Digest, Vol 131, Issue 37
In-Reply-To: <mailman.3.1419530401.7778.ipython-dev@scipy.org>
References: <mailman.3.1419530401.7778.ipython-dev@scipy.org>
Message-ID: <1914FBE7-3693-4BA7-AB2C-93D7483CE9CC@gmail.com>


On Dec 25, 2014, at 1:00 PM, ipython-dev-request at scipy.org wrote:
> Date: Wed, 24 Dec 2014 22:28:09 +0000
> From: Thomas Kluyver <takowl at gmail.com>
> Subject: Re: [IPython-dev] line magic in middle of line
> To: IPython developers list <ipython-dev at scipy.org>
> Message-ID:
> 	<CAOvn4qhVYeM5_Nys6RaZ2degweHu5r400ide6pmSuOGj_pHtXw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Hi Kyle,
> 
> On 24 December 2014 at 21:52, Kyle Rawlins <rawlins at gmail.com> wrote:
> 
>> Is this documented anywhere / actually supported behavior?  Under what
>> conditions can line magics be mixed with python code?
> 
> 
> It only works in simple assignments, and you can do it with both magics and
> !system escapes - the normal demo of that feature is to do "files = !ls".
> The timeit magic also returns a useful results object.
> 
> It is documented for system commands:
> http://ipython.org/ipython-doc/dev/interactive/tutorial.html#system-shell-commands
> http://ipython.org/ipython-doc/dev/interactive/reference.html#manual-capture-of-command-output
> 
> But I can't currently see anywhere where this is documented for magic
> functions.

Great, thanks, glad to hear this is intended!  I thought this might be pretty useful for people to know about, so I opened a small pull request updating the docs based on the example `files = %sx ls`, which I discovered does exactly the same thing as the above ! example.

Best,
-kyle



-- 
Kyle Rawlins
http://sites.krieger.jhu.edu/rawlins/
Department of Cognitive Science
Johns Hopkins University



From dsdale24 at gmail.com  Sat Dec 27 15:31:36 2014
From: dsdale24 at gmail.com (Darren Dale)
Date: Sat, 27 Dec 2014 20:31:36 +0000
Subject: [IPython-dev] embedding ipython, namespace question
Message-ID: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>

Hello,

(sorry for double-posting on ipython-user. I was advised to post here
instead.)

I'm working on embedding an ipython qtconsole in my applilcation as a dock
widget. The
https://github.com/ipython/ipython/blob/master/examples/Embedding/inprocess_qtconsole.py
example
was a helpful first step. As variables are created or updated in the main
application, I'd like to be able to inspect them in the qtconsole. Is it
possible for the qtconsole's namespace to be shared or linked with that of
the main application? It seems like this should be possible, based on the
documentation for
http://ipython.org/ipython-doc/1/api/generated/IPython.terminal.embed.html#IPython.terminal.embed.InteractiveShellEmbed
.
I'm creating my kernel and client at the beginning of my module, and then
wiring up the widget when my application's main window is about to show.
But if I run `dir()` in the embedded ipython qt console, the list is
identical to what I get if I run `dir()` in the standalone `ipython
qtconsole` program.

I've seen some comments along the lines of "pass a user_ns", but I have no
idea what this means, or specifically how to do it.

Thanks,
Darren
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141227/4632fc49/attachment.html>

From rosborn at anl.gov  Sat Dec 27 16:42:32 2014
From: rosborn at anl.gov (Osborn, Raymond)
Date: Sat, 27 Dec 2014 21:42:32 +0000
Subject: [IPython-dev] embedding ipython, namespace question
In-Reply-To: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
Message-ID: <AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov>

We synchronize an IPython shell to a PySide QTreeView in NeXpy, an application for viewing and analyzing HDF5 files (http://nexpy.github.io/nexpy/). If it?s any help, you can find the source code at https://github.com/nexpy/nexpy, which was, I think, based on the same example you cite. The application is initialized in nexpy/gui/consoleapp.py, which subclasses IPythonConsoleApp, but the main PySide windows are defined in mainwindow.py.

Here are some of the critical lines in mainwindow.py:

class MainWindow(QtGui.QMainWindow):

    def __init__(self, app, tree, config=None):

        self.console = RichIPythonWidget(config=self.config, parent=rightpane)
        self.console.kernel_manager = QtInProcessKernelManager(config=self.config)
        self.console.kernel_manager.start_kernel()
        self.console.kernel_manager.kernel.gui = 'qt4'
        self.console.kernel_client = self.console.kernel_manager.client()
        self.console.kernel_client.start_channels()

        self.shell = self.console.kernel_manager.kernel.shell
        self.user_ns = self.console.kernel_manager.kernel.shell.user_ns

The last line makes the dictionary of the IPython shell, i.e., user_ns, available to the rest of the GUI. That must be what the ?pass a user_ns? refers to.

Best regards,
Ray

On Dec 27, 2014, at 2:31 PM, Darren Dale <dsdale24 at gmail.com<mailto:dsdale24 at gmail.com>> wrote:

Hello,

(sorry for double-posting on ipython-user. I was advised to post here instead.)

I'm working on embedding an ipython qtconsole in my applilcation as a dock widget. The https://github.com/ipython/ipython/blob/master/examples/Embedding/inprocess_qtconsole.py example was a helpful first step. As variables are created or updated in the main application, I'd like to be able to inspect them in the qtconsole. Is it possible for the qtconsole's namespace to be shared or linked with that of the main application? It seems like this should be possible, based on the documentation for http://ipython.org/ipython-doc/1/api/generated/IPython.terminal.embed.html#IPython.terminal.embed.InteractiveShellEmbed . I'm creating my kernel and client at the beginning of my module, and then wiring up the widget when my application's main window is about to show. But if I run `dir()` in the embedded ipython qt console, the list is identical to what I get if I run `dir()` in the standalone `ipython qtconsole` program.

I've seen some comments along the lines of "pass a user_ns", but I have no idea what this means, or specifically how to do it.

Thanks,
Darren
_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org<mailto:IPython-dev at scipy.org>
http://mail.scipy.org/mailman/listinfo/ipython-dev

--
Ray Osborn, Senior Scientist
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov<mailto:ROsborn at anl.gov>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141227/895860ce/attachment.html>

From dsdale24 at gmail.com  Sat Dec 27 17:43:10 2014
From: dsdale24 at gmail.com (Darren Dale)
Date: Sat, 27 Dec 2014 22:43:10 +0000
Subject: [IPython-dev] embedding ipython, namespace question
References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
	<AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov>
Message-ID: <CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com>

Thanks Ray. Unfortunately, I want the ipython console's namespace to
reflect that of the application in which it is embedded, not a specific
widget. There is an example that appears to do this at
https://github.com/ipython/ipython/blob/master/examples/Embedding/embed_function.py
, but I'm trying to do the same with the Qt widget. At the top of my main
gui module, I do:

try:
    from IPython.qt.console.rich_ipython_widget import RichIPythonWidget
    from IPython.qt.inprocess import QtInProcessKernelManager

    # Create an in-process kernel
    kernel_manager = QtInProcessKernelManager(user_ns=None)
    kernel_manager.start_kernel()
    kernel = kernel_manager.kernel
    kernel.gui = 'qt4'

    kernel_client = kernel_manager.client()
    kernel_client.start_channels()
except ImportError:
    pass

and then when I initialize my QMainWindow, I call:

    def _load_ipython(self):
        def stop():
            kernel_client.stop_channels()
            kernel_manager.shutdown_kernel()

        control = RichIPythonWidget()
        self.ipythonDockWidget.setWidget(control)
        control.kernel_manager = kernel_manager
        control.kernel_client = kernel_client
        control.exit_requested.connect(stop)

That gets me almost all of the way there. I just need the console to share
the application's namespace, but don't know where or how to pass the
appropriate config arguments.

Darren


On Sat Dec 27 2014 at 4:42:38 PM Osborn, Raymond <rosborn at anl.gov> wrote:

>  We synchronize an IPython shell to a PySide QTreeView in NeXpy, an
> application for viewing and analyzing HDF5 files (
> http://nexpy.github.io/nexpy/). If it?s any help, you can find the source
> code at https://github.com/nexpy/nexpy, which was, I think, based on the
> same example you cite. The application is initialized in
> nexpy/gui/consoleapp.py, which subclasses IPythonConsoleApp, but the main
> PySide windows are defined in mainwindow.py.
>
>  Here are some of the critical lines in mainwindow.py:
>
>  class MainWindow(QtGui.QMainWindow):
>
>      def __init__(self, app, tree, config=None):
>
>          self.console = RichIPythonWidget(config=self.config,
> parent=rightpane)
>         self.console.kernel_manager = QtInProcessKernelManager(
> config=self.config)
>         self.console.kernel_manager.start_kernel()
>         self.console.kernel_manager.kernel.gui = 'qt4'
>         self.console.kernel_client = self.console.kernel_manager.client()
>         self.console.kernel_client.start_channels()
>
>          self.shell = self.console.kernel_manager.kernel.shell
>         self.user_ns = self.console.kernel_manager.kernel.shell.user_ns
>
>  The last line makes the dictionary of the IPython shell, i.e., user_ns,
> available to the rest of the GUI. That must be what the ?pass a user_ns?
> refers to.
>
>  Best regards,
> Ray
>
>  On Dec 27, 2014, at 2:31 PM, Darren Dale <dsdale24 at gmail.com> wrote:
>
> Hello,
>
>  (sorry for double-posting on ipython-user. I was advised to post here
> instead.)
>
>  I'm working on embedding an ipython qtconsole in my applilcation as a
> dock widget. The https://github.com/ipython/ipython/blob/master/
> examples/Embedding/inprocess_qtconsole.py example was a helpful first
> step. As variables are created or updated in the main application, I'd like
> to be able to inspect them in the qtconsole. Is it possible for the
> qtconsole's namespace to be shared or linked with that of the main
> application? It seems like this should be possible, based on the
> documentation for http://ipython.org/ipython-doc/1/api/generated/
> IPython.terminal.embed.html#IPython.terminal.embed.InteractiveShellEmbed .
> I'm creating my kernel and client at the beginning of my module, and then
> wiring up the widget when my application's main window is about to show.
> But if I run `dir()` in the embedded ipython qt console, the list is
> identical to what I get if I run `dir()` in the standalone `ipython
> qtconsole` program.
>
>  I've seen some comments along the lines of "pass a user_ns", but I have
> no idea what this means, or specifically how to do it.
>
>  Thanks,
> Darren
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> --
> Ray Osborn, Senior Scientist
> Materials Science Division
> Argonne National Laboratory
> Argonne, IL 60439, USA
> Phone: +1 (630) 252-9011
> Email: ROsborn at anl.gov
>
>
>   _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141227/a4a2881d/attachment.html>

From bussonniermatthias at gmail.com  Sat Dec 27 18:02:45 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Sun, 28 Dec 2014 00:02:45 +0100
Subject: [IPython-dev] embedding ipython, namespace question
In-Reply-To: <CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com>
References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
	<AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov>
	<CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com>
Message-ID: <FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com>


Le 27 d?c. 2014 ? 23:43, Darren Dale <dsdale24 at gmail.com> a ?crit :

> Thanks Ray. Unfortunately, I want the ipython console's namespace to reflect that of the application in which it is embedded, not a specific widget. There is an example that appears to do this at https://github.com/ipython/ipython/blob/master/examples/Embedding/embed_function.py , but I'm trying to do the same with the Qt widget. At the top of my main gui module, I do:
> 
> try:
>     from IPython.qt.console.rich_ipython_widget import RichIPythonWidget
>     from IPython.qt.inprocess import QtInProcessKernelManager
> 
>     # Create an in-process kernel
>     kernel_manager = QtInProcessKernelManager(user_ns=None)

Don't you want to just pass `globals()` instead of `None` here? 
user_ns kwarg here is **the mutable namespace** (i.e. a dict) you want your kernel to work on.  If you pass None, 
it will create a new empty object for you. If you pass `{"a":1}` then your kernel will see a variable `a` with a value of 1 (mutable) 

So I guess what you want to pass here is either globals(), locals(), or a dict you constructed yourself, 
from the bit you want available. 

You can also do as raymond said, and inject things into user_ns after getting the reference to it. 
-- 
M

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141228/20e6f5b9/attachment.html>

From dsdale24 at gmail.com  Sat Dec 27 18:24:16 2014
From: dsdale24 at gmail.com (Darren Dale)
Date: Sat, 27 Dec 2014 23:24:16 +0000
Subject: [IPython-dev] embedding ipython, namespace question
References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
	<AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov>
	<CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com>
	<FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com>
Message-ID: <CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com>

I passed None because
http://ipython.org/ipython-doc/stable/api/generated/IPython.terminal.embed.html?highlight=embed#IPython.terminal.embed.InteractiveShellEmbed
states: "If given as None, they are automatically taken from the scope
where the shell was called, so that program variables become visible." That
is precisely what I want, but can't figure out how to do it with the Qt
console.

I don't want to inject variables to an isolated namespace, I want the
console to share the application's namespace.

On Sat Dec 27 2014 at 6:02:54 PM Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

>
> Le 27 d?c. 2014 ? 23:43, Darren Dale <dsdale24 at gmail.com> a ?crit :
>
> Thanks Ray. Unfortunately, I want the ipython console's namespace to
> reflect that of the application in which it is embedded, not a specific
> widget. There is an example that appears to do this at
> https://github.com/ipython/ipython/blob/master/examples/Embedding/embed_function.py
> , but I'm trying to do the same with the Qt widget. At the top of my main
> gui module, I do:
>
> try:
>     from IPython.qt.console.rich_ipython_widget import RichIPythonWidget
>     from IPython.qt.inprocess import QtInProcessKernelManager
>
>     # Create an in-process kernel
>     kernel_manager = QtInProcessKernelManager(user_ns=None)
>
>
> Don't you want to just pass `globals()` instead of `None` here?
> user_ns kwarg here is **the mutable namespace** (i.e. a dict) you want
> your kernel to work on.  If you pass None,
> it will create a new empty object for you. If you pass `{"a":1}` then your
> kernel will see a variable `a` with a value of 1 (mutable)
>
> So I guess what you want to pass here is either globals(), locals(), or a
> dict you constructed yourself,
> from the bit you want available.
>
> You can also do as raymond said, and inject things into user_ns after
> getting the reference to it.
> --
> M
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141227/4dabef80/attachment.html>

From bussonniermatthias at gmail.com  Sat Dec 27 18:58:12 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Sun, 28 Dec 2014 00:58:12 +0100
Subject: [IPython-dev] embedding ipython, namespace question
In-Reply-To: <CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com>
References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
	<AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov>
	<CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com>
	<FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com>
	<CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com>
Message-ID: <2F8411A5-50CA-421E-85B3-4665B3B8ECA5@gmail.com>


Le 28 d?c. 2014 ? 00:24, Darren Dale <dsdale24 at gmail.com> a ?crit :

> I passed None because http://ipython.org/ipython-doc/stable/api/generated/IPython.terminal.embed.html?highlight=embed#IPython.terminal.embed.InteractiveShellEmbed states: "If given as None, they are automatically taken from the scope where the shell was called, so that program variables become visible." That is precisely what I want, but can't figure out how to do it with the Qt console.

As far as I am concerned, you are not using InteractiveShellEmbed, I don't see why you apply docs of one function to 
another. 

> I don't want to inject variables to an isolated namespace, I want the console to share the application's namespace.

I did not tell you to pass an isolated namespace, I did tell you to pass `globals()` it is not isolated is is a reference to 
current global namespace so it should be shared :

In [1]: a
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-60b725f10c9c> in <module>()
----> 1 a

NameError: name 'a' is not defined

In [2]: def fun(ns):
   ...:     ns['a'] =1
   ...:

In [3]: fun(globals())

In [4]: a
Out[4]: 1

IIRC `embed` get globals/local by sys.get_frame and pulling from caller (where we can know the stack depth).
Like that:

In [1]: import sys

In [2]: b
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-2-3b5d5c371295> in <module>()
----> 1 b

NameError: name 'b' is not defined

In [3]: def fun(): sys._getframe(0).f_back.f_globals['b']=12

In [4]: fun()

In [5]: b
Out[5]: 12


So here, you just need to do it yourself (example a), as the kernel is potentially on a different process
we don't do any magic (as in b) I guess we could, but this part of the code start to look like some dark magic.

-- 
M




> 
> On Sat Dec 27 2014 at 6:02:54 PM Matthias Bussonnier <bussonniermatthias at gmail.com> wrote:
> 
> Le 27 d?c. 2014 ? 23:43, Darren Dale <dsdale24 at gmail.com> a ?crit :
> 
>> Thanks Ray. Unfortunately, I want the ipython console's namespace to reflect that of the application in which it is embedded, not a specific widget. There is an example that appears to do this at https://github.com/ipython/ipython/blob/master/examples/Embedding/embed_function.py , but I'm trying to do the same with the Qt widget. At the top of my main gui module, I do:
>> 
>> try:
>>     from IPython.qt.console.rich_ipython_widget import RichIPythonWidget
>>     from IPython.qt.inprocess import QtInProcessKernelManager
>> 
>>     # Create an in-process kernel
>>     kernel_manager = QtInProcessKernelManager(user_ns=None)
> 
> Don't you want to just pass `globals()` instead of `None` here? 
> user_ns kwarg here is **the mutable namespace** (i.e. a dict) you want your kernel to work on.  If you pass None, 
> it will create a new empty object for you. If you pass `{"a":1}` then your kernel will see a variable `a` with a value of 1 (mutable) 
> 
> So I guess what you want to pass here is either globals(), locals(), or a dict you constructed yourself, 
> from the bit you want available. 
> 
> You can also do as raymond said, and inject things into user_ns after getting the reference to it. 
> -- 
> M
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141228/5ac145fe/attachment.html>

From dsdale24 at gmail.com  Sat Dec 27 19:22:26 2014
From: dsdale24 at gmail.com (Darren Dale)
Date: Sun, 28 Dec 2014 00:22:26 +0000
Subject: [IPython-dev] embedding ipython, namespace question
References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
	<AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov>
	<CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com>
	<FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com>
	<CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com>
	<2F8411A5-50CA-421E-85B3-4665B3B8ECA5@gmail.com>
Message-ID: <CAK6O52kvY1noUXu6tZfokMT7x68_UPLSstQML9fV6OywBM_SBg@mail.gmail.com>

On Sat Dec 27 2014 at 6:58:23 PM Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

>
> Le 28 d?c. 2014 ? 00:24, Darren Dale <dsdale24 at gmail.com> a ?crit :
>
> I passed None because
> http://ipython.org/ipython-doc/stable/api/generated/IPython.terminal.embed.html?highlight=embed#IPython.terminal.embed.InteractiveShellEmbed
> states: "If given as None, they are automatically taken from the scope
> where the shell was called, so that program variables become visible." That
> is precisely what I want, but can't figure out how to do it with the Qt
> console.
>
>
> As far as I am concerned, you are not using InteractiveShellEmbed, I don't
> see why you apply docs of one function to
> another.
>

Well, there is not much to be gleaned from the qt in-process documentation
at
http://ipython.org/ipython-doc/stable/api/generated/IPython.qt.inprocess.html#module-IPython.qt.inprocess
. If I call `embed` after I start the kernel and client, I recall getting
an error about attempting multiple instantiations of InteractiveShellEmbed,
so it seemed reasonable to assume that the **kwargs listed in the
in-process docs might map to those documented for InteractiveShellEmbed.


> I don't want to inject variables to an isolated namespace, I want the
> console to share the application's namespace.
>
>
> I did not tell you to pass an isolated namespace, I did tell you to pass
> `globals()` it is not isolated is is a reference to
> current global namespace so it should be shared :
>

My mistake. I guess I misunderstood your comment "You can also do as
raymond said, and inject things into user_ns after getting the reference to
it." That sounds like it injecting variables into an isolated namespace.

I'd try passing globals() instead of None, but I don't know how that should
be done.
http://ipython.org/ipython-doc/stable/api/generated/IPython.qt.inprocess.html#module-IPython.qt.inprocess
only lists **kwargs, and it sounds like the arguments listed for
InteractiveShellEmbed do not necessarily apply to the qt.inprocess
functions/methods.

[...]

> So here, you just need to do it yourself (example a), as the kernel is
> potentially on a different process
>

qt.inprocess is in-process, isn't it?


>
> On Sat Dec 27 2014 at 6:02:54 PM Matthias Bussonnier <
> bussonniermatthias at gmail.com> wrote:
>
>>
>> Le 27 d?c. 2014 ? 23:43, Darren Dale <dsdale24 at gmail.com> a ?crit :
>>
>> Thanks Ray. Unfortunately, I want the ipython console's namespace to
>> reflect that of the application in which it is embedded, not a specific
>> widget. There is an example that appears to do this at
>> https://github.com/ipython/ipython/blob/master/examples/Embedding/embed_function.py
>> , but I'm trying to do the same with the Qt widget. At the top of my main
>> gui module, I do:
>>
>> try:
>>     from IPython.qt.console.rich_ipython_widget import RichIPythonWidget
>>     from IPython.qt.inprocess import QtInProcessKernelManager
>>
>>     # Create an in-process kernel
>>     kernel_manager = QtInProcessKernelManager(user_ns=None)
>>
>>
>> Don't you want to just pass `globals()` instead of `None` here?
>> user_ns kwarg here is **the mutable namespace** (i.e. a dict) you want
>> your kernel to work on.  If you pass None,
>> it will create a new empty object for you. If you pass `{"a":1}` then
>> your kernel will see a variable `a` with a value of 1 (mutable)
>>
>> So I guess what you want to pass here is either globals(), locals(), or a
>> dict you constructed yourself,
>> from the bit you want available.
>>
>> You can also do as raymond said, and inject things into user_ns after
>> getting the reference to it.
>> --
>> M
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141228/ffde8e28/attachment.html>

From rosborn at anl.gov  Sat Dec 27 20:48:13 2014
From: rosborn at anl.gov (Osborn, Raymond)
Date: Sun, 28 Dec 2014 01:48:13 +0000
Subject: [IPython-dev] embedding ipython, namespace question
In-Reply-To: <CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com>
References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
	<AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov>
	<CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com>
	<FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com>
	<CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com>
Message-ID: <DD2429D4-4BE1-49FC-A891-BF153FBA942F@anl.gov>

I don?t really understand what you are trying to achieve, but the ?user_ns? dictionary isn?t an isolated namespace - it?s the namespace that is used by the console, and I would have thought you would have to do some kind of injection to add other objects from within the application. The following code is added shortly after the example I gave in the MainWindow class:

        self.user_ns['plotview'] = self.plotview
        self.user_ns['treeview'] = self.treeview
        self.user_ns['mainwindow'] = self

so that other parts of the application are accessible within the console. You could add all the items in the globals() dictionary if you wanted:

        for k, v in globals().items():
            self.user_ns[k] = v

which is presumably equivalent to what Matthias suggested, but you would still have to inject new variables as they are created.

Ray

On Dec 27, 2014, at 5:24 PM, Darren Dale <dsdale24 at gmail.com<mailto:dsdale24 at gmail.com>> wrote:

I passed None because http://ipython.org/ipython-doc/stable/api/generated/IPython.terminal.embed.html?highlight=embed#IPython.terminal.embed.InteractiveShellEmbed states: "If given as None, they are automatically taken from the scope where the shell was called, so that program variables become visible." That is precisely what I want, but can't figure out how to do it with the Qt console.

I don't want to inject variables to an isolated namespace, I want the console to share the application's namespace.

On Sat Dec 27 2014 at 6:02:54 PM Matthias Bussonnier <bussonniermatthias at gmail.com<mailto:bussonniermatthias at gmail.com>> wrote:

Le 27 d?c. 2014 ? 23:43, Darren Dale <dsdale24 at gmail.com<mailto:dsdale24 at gmail.com>> a ?crit :

Thanks Ray. Unfortunately, I want the ipython console's namespace to reflect that of the application in which it is embedded, not a specific widget. There is an example that appears to do this at https://github.com/ipython/ipython/blob/master/examples/Embedding/embed_function.py , but I'm trying to do the same with the Qt widget. At the top of my main gui module, I do:

try:
    from IPython.qt.console.rich_ipython_widget import RichIPythonWidget
    from IPython.qt.inprocess import QtInProcessKernelManager

    # Create an in-process kernel
    kernel_manager = QtInProcessKernelManager(user_ns=None)

Don't you want to just pass `globals()` instead of `None` here?
user_ns kwarg here is **the mutable namespace** (i.e. a dict) you want your kernel to work on.  If you pass None,
it will create a new empty object for you. If you pass `{"a":1}` then your kernel will see a variable `a` with a value of 1 (mutable)

So I guess what you want to pass here is either globals(), locals(), or a dict you constructed yourself,
from the bit you want available.

You can also do as raymond said, and inject things into user_ns after getting the reference to it.
--
M

_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org<mailto:IPython-dev at scipy.org>
http://mail.scipy.org/mailman/listinfo/ipython-dev
_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org<mailto:IPython-dev at scipy.org>
http://mail.scipy.org/mailman/listinfo/ipython-dev

--
Ray Osborn, Senior Scientist
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov<mailto:ROsborn at anl.gov>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141228/eee46bbe/attachment.html>

From doug.blank at gmail.com  Tue Dec 30 12:57:50 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Tue, 30 Dec 2014 12:57:50 -0500
Subject: [IPython-dev] Installing kernels and libs under a virtual env
Message-ID: <CAAusYCin9fs-GBW1JfHeXd6X0miTObb78kjoBwi9a3o0TokbKA@mail.gmail.com>

Having some trouble installing a kernel and python library under a Python
virtual environment (specifically under Travis). Perhaps the root of this
is caused by the new API to install_kernel_spec. Here is the situation:

We have a Python file and kernel that we would like to install via
setup.py. If we install like:

python setup.py install

where we call:

install_kernel_spec(td, 'my_kernel', user=self.user)

from inside a class, then this works (without the venv) when called by
root, or if called with the --user flag.

However, when called from inside a virtual environment without --user, it
attempts to install the kernel in /usr/local/share/jupyter/kernels. If one
uses the --user flag in the virtual env, then it installs in ~/.local/
rather than in the virtual env's local folder. This causes issues later
because ~/.local is not in the Python search path.

I think that if "python setup.py install --user" under a venv went to the
virtual env's local, then everything would be fine. But maybe I'm doing
something wrong (need to use pip?).

Any help appreciated,

-Doug
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141230/06a3925d/attachment.html>

From fperez.net at gmail.com  Tue Dec 30 14:28:30 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 30 Dec 2014 11:28:30 -0800
Subject: [IPython-dev] Installing kernels and libs under a virtual env
In-Reply-To: <CAAusYCin9fs-GBW1JfHeXd6X0miTObb78kjoBwi9a3o0TokbKA@mail.gmail.com>
References: <CAAusYCin9fs-GBW1JfHeXd6X0miTObb78kjoBwi9a3o0TokbKA@mail.gmail.com>
Message-ID: <CAHAreOpYsSmdcf=TK68dTA6k-EExDvTOdTvW8cv5yheihMOzzg@mail.gmail.com>

Hi Doug,

On Tue, Dec 30, 2014 at 9:57 AM, Doug Blank <doug.blank at gmail.com> wrote:

> I think that if "python setup.py install --user" under a venv went to the
> virtual env's local, then everything would be fine. But maybe I'm doing
> something wrong (need to use pip?).


The issue is that the list of targets is hardcoded here:

https://github.com/ipython/ipython/blob/master/IPython/kernel/kernelspec.py#L21

that code isn't currently aware in any way of virtualenvs.

I'm fuzzy on the details of how we ended up deciding we'd interact with
venvs, other than saying that we'll try to bake as little knowledge of
'environment managers' (venv, conda, etc) directly into our kernel
installation tools directly.  That discussion has lots of subtly
interacting moving parts, and I'm afraid to say something misleading right
now.

Min and Thomas were deep in those details, and I'd wait for further
feedback from them (or anyone else who remembers well).  But in the
meantime, you may want to play with that code in a local copy: you may find
a minimally intrusive improvement for venv support that could be a good
solution for a PR.

Cheers,

f



-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141230/48f1b2fe/attachment.html>

From aron at ahmadia.net  Tue Dec 30 14:37:38 2014
From: aron at ahmadia.net (Aron Ahmadia)
Date: Tue, 30 Dec 2014 14:37:38 -0500
Subject: [IPython-dev] Installing kernels and libs under a virtual env
In-Reply-To: <CAHAreOpYsSmdcf=TK68dTA6k-EExDvTOdTvW8cv5yheihMOzzg@mail.gmail.com>
References: <CAAusYCin9fs-GBW1JfHeXd6X0miTObb78kjoBwi9a3o0TokbKA@mail.gmail.com>
	<CAHAreOpYsSmdcf=TK68dTA6k-EExDvTOdTvW8cv5yheihMOzzg@mail.gmail.com>
Message-ID: <CAPhiW4hYVNaSnQ3wttHkx86KMbZxux5f1H+ayOVkTskH7s7xZA@mail.gmail.com>

Any reason in particular you're not using the system prefix to decide where
to install things?  That hardcode looks like it's going to cause all sorts
of misery in the future.

It seems like you could reasonably construct locations from
os.path.join(sys.prefix, 'share', 'jupyter', 'kernels') and avoid the
hardcode.

A

On Tue, Dec 30, 2014 at 2:28 PM, Fernando Perez <fperez.net at gmail.com>
wrote:

> Hi Doug,
>
> On Tue, Dec 30, 2014 at 9:57 AM, Doug Blank <doug.blank at gmail.com> wrote:
>
>> I think that if "python setup.py install --user" under a venv went to the
>> virtual env's local, then everything would be fine. But maybe I'm doing
>> something wrong (need to use pip?).
>
>
> The issue is that the list of targets is hardcoded here:
>
>
> https://github.com/ipython/ipython/blob/master/IPython/kernel/kernelspec.py#L21
>
> that code isn't currently aware in any way of virtualenvs.
>
> I'm fuzzy on the details of how we ended up deciding we'd interact with
> venvs, other than saying that we'll try to bake as little knowledge of
> 'environment managers' (venv, conda, etc) directly into our kernel
> installation tools directly.  That discussion has lots of subtly
> interacting moving parts, and I'm afraid to say something misleading right
> now.
>
> Min and Thomas were deep in those details, and I'd wait for further
> feedback from them (or anyone else who remembers well).  But in the
> meantime, you may want to play with that code in a local copy: you may find
> a minimally intrusive improvement for venv support that could be a good
> solution for a PR.
>
> Cheers,
>
> f
>
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141230/31be77e9/attachment.html>

From fperez.net at gmail.com  Tue Dec 30 14:44:43 2014
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 30 Dec 2014 11:44:43 -0800
Subject: [IPython-dev] Installing kernels and libs under a virtual env
In-Reply-To: <CAPhiW4hYVNaSnQ3wttHkx86KMbZxux5f1H+ayOVkTskH7s7xZA@mail.gmail.com>
References: <CAAusYCin9fs-GBW1JfHeXd6X0miTObb78kjoBwi9a3o0TokbKA@mail.gmail.com>
	<CAHAreOpYsSmdcf=TK68dTA6k-EExDvTOdTvW8cv5yheihMOzzg@mail.gmail.com>
	<CAPhiW4hYVNaSnQ3wttHkx86KMbZxux5f1H+ayOVkTskH7s7xZA@mail.gmail.com>
Message-ID: <CAHAreOp74YoOAWQUVQiWcXv-6pyGLEdDCgUzCmVO1_jR9iOUgw@mail.gmail.com>

On Tue, Dec 30, 2014 at 11:37 AM, Aron Ahmadia <aron at ahmadia.net> wrote:

> It seems like you could reasonably construct locations from
> os.path.join(sys.prefix, 'share', 'jupyter', 'kernels') and avoid the
> hardcode.


I don't really recall, I think Thomas wrote that code so I'll defer to him
for more feedback. At first pass, I'd agree with you, but I might be
missing something (that discussion was, as all things related to
cross-platform python packaging and installation, head-spinning and
nausea-inducing).

Cheers,

f
-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141230/bdd9ed3e/attachment.html>

From doug.blank at gmail.com  Tue Dec 30 15:51:03 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Tue, 30 Dec 2014 15:51:03 -0500
Subject: [IPython-dev] Installing kernels and libs under a virtual env
In-Reply-To: <CAHAreOpYsSmdcf=TK68dTA6k-EExDvTOdTvW8cv5yheihMOzzg@mail.gmail.com>
References: <CAAusYCin9fs-GBW1JfHeXd6X0miTObb78kjoBwi9a3o0TokbKA@mail.gmail.com>
	<CAHAreOpYsSmdcf=TK68dTA6k-EExDvTOdTvW8cv5yheihMOzzg@mail.gmail.com>
Message-ID: <CAAusYCgUNR2vqZG_vm-3cF3nKQTd-mVob7-d8HeJVSA-RQWikw@mail.gmail.com>

On Tue, Dec 30, 2014 at 2:28 PM, Fernando Perez <fperez.net at gmail.com>
wrote:

> Hi Doug,
>
> On Tue, Dec 30, 2014 at 9:57 AM, Doug Blank <doug.blank at gmail.com> wrote:
>
>> I think that if "python setup.py install --user" under a venv went to the
>> virtual env's local, then everything would be fine. But maybe I'm doing
>> something wrong (need to use pip?).
>
>
> The issue is that the list of targets is hardcoded here:
>
>
> https://github.com/ipython/ipython/blob/master/IPython/kernel/kernelspec.py#L21
>
> that code isn't currently aware in any way of virtualenvs.
>
> I'm fuzzy on the details of how we ended up deciding we'd interact with
> venvs, other than saying that we'll try to bake as little knowledge of
> 'environment managers' (venv, conda, etc) directly into our kernel
> installation tools directly.  That discussion has lots of subtly
> interacting moving parts, and I'm afraid to say something misleading right
> now.
>
> Min and Thomas were deep in those details, and I'd wait for further
> feedback from them (or anyone else who remembers well).  But in the
> meantime, you may want to play with that code in a local copy: you may find
> a minimally intrusive improvement for venv support that could be a good
> solution for a PR.
>

Actually, the kernel is installed fine, and is found in the user's kernel
area. It is the installed Python code with "--user" that causes the issue.
Maybe this is just a venv/setup interaction bug.

-Doug


>
> Cheers,
>
> f
>
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141230/23e25817/attachment.html>

From doug.blank at gmail.com  Tue Dec 30 16:31:45 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Tue, 30 Dec 2014 16:31:45 -0500
Subject: [IPython-dev] Installing kernels and libs under a virtual env
In-Reply-To: <CAAusYCgUNR2vqZG_vm-3cF3nKQTd-mVob7-d8HeJVSA-RQWikw@mail.gmail.com>
References: <CAAusYCin9fs-GBW1JfHeXd6X0miTObb78kjoBwi9a3o0TokbKA@mail.gmail.com>
	<CAHAreOpYsSmdcf=TK68dTA6k-EExDvTOdTvW8cv5yheihMOzzg@mail.gmail.com>
	<CAAusYCgUNR2vqZG_vm-3cF3nKQTd-mVob7-d8HeJVSA-RQWikw@mail.gmail.com>
Message-ID: <CAAusYCjBQkbmSDA2NM-zdhtUEPGZOo7KXNrFHdj6-1LPePnZ3w@mail.gmail.com>

On Tue, Dec 30, 2014 at 3:51 PM, Doug Blank <doug.blank at gmail.com> wrote:

> On Tue, Dec 30, 2014 at 2:28 PM, Fernando Perez <fperez.net at gmail.com>
> wrote:
>
>> Hi Doug,
>>
>> On Tue, Dec 30, 2014 at 9:57 AM, Doug Blank <doug.blank at gmail.com> wrote:
>>
>>> I think that if "python setup.py install --user" under a venv went to
>>> the virtual env's local, then everything would be fine. But maybe I'm doing
>>> something wrong (need to use pip?).
>>
>>
>> The issue is that the list of targets is hardcoded here:
>>
>>
>> https://github.com/ipython/ipython/blob/master/IPython/kernel/kernelspec.py#L21
>>
>> that code isn't currently aware in any way of virtualenvs.
>>
>> I'm fuzzy on the details of how we ended up deciding we'd interact with
>> venvs, other than saying that we'll try to bake as little knowledge of
>> 'environment managers' (venv, conda, etc) directly into our kernel
>> installation tools directly.  That discussion has lots of subtly
>> interacting moving parts, and I'm afraid to say something misleading right
>> now.
>>
>> Min and Thomas were deep in those details, and I'd wait for further
>> feedback from them (or anyone else who remembers well).  But in the
>> meantime, you may want to play with that code in a local copy: you may find
>> a minimally intrusive improvement for venv support that could be a good
>> solution for a PR.
>>
>
> Actually, the kernel is installed fine, and is found in the user's kernel
> area. It is the installed Python code with "--user" that causes the issue.
> Maybe this is just a venv/setup interaction bug.
>

I see that distutils can use either a --user flag or a --prefix flag, but
not both at the same time. Adding a "prefix" kwarg option could also work
for install_kernel_spec, and would solve the problem that I am currently
running into with venv with --user.

Min and/or Thomas: does that sound like a viable option?

-Doug


> -Doug
>
>
>>
>> Cheers,
>>
>> f
>>
>>
>>
>> --
>> Fernando Perez (@fperez_org; http://fperez.org)
>> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>> fernando.perez-at-berkeley: contact me here for any direct mail
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141230/a69d6287/attachment.html>

From bussonniermatthias at gmail.com  Tue Dec 30 16:42:46 2014
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Tue, 30 Dec 2014 22:42:46 +0100
Subject: [IPython-dev] Installing kernels and libs under a virtual env
In-Reply-To: <CAAusYCjBQkbmSDA2NM-zdhtUEPGZOo7KXNrFHdj6-1LPePnZ3w@mail.gmail.com>
References: <CAAusYCin9fs-GBW1JfHeXd6X0miTObb78kjoBwi9a3o0TokbKA@mail.gmail.com>
	<CAHAreOpYsSmdcf=TK68dTA6k-EExDvTOdTvW8cv5yheihMOzzg@mail.gmail.com>
	<CAAusYCgUNR2vqZG_vm-3cF3nKQTd-mVob7-d8HeJVSA-RQWikw@mail.gmail.com>
	<CAAusYCjBQkbmSDA2NM-zdhtUEPGZOo7KXNrFHdj6-1LPePnZ3w@mail.gmail.com>
Message-ID: <99531199-48C7-4311-B923-EDBC9E787924@gmail.com>

From what I can tell it is perfectly normal to install a vent kernel to be use globally, and we try not to have any venv specific code for the Jupyter part. 
Installing was the most problematic part in last discussion, and there is no "right" answer so the only logical one is to try to do as least as possible.

-- 
M

Envoy? de mon iPhone

> Le 30 d?c. 2014 ? 22:31, Doug Blank <doug.blank at gmail.com> a ?crit :
> 
>> On Tue, Dec 30, 2014 at 3:51 PM, Doug Blank <doug.blank at gmail.com> wrote:
>>> On Tue, Dec 30, 2014 at 2:28 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>>> Hi Doug,
>>> 
>>>> On Tue, Dec 30, 2014 at 9:57 AM, Doug Blank <doug.blank at gmail.com> wrote:
>>>> I think that if "python setup.py install --user" under a venv went to the virtual env's local, then everything would be fine. But maybe I'm doing something wrong (need to use pip?).
>>> 
>>> 
>>> The issue is that the list of targets is hardcoded here:
>>> 
>>> https://github.com/ipython/ipython/blob/master/IPython/kernel/kernelspec.py#L21
>>> 
>>> that code isn't currently aware in any way of virtualenvs.
>>> 
>>> I'm fuzzy on the details of how we ended up deciding we'd interact with venvs, other than saying that we'll try to bake as little knowledge of 'environment managers' (venv, conda, etc) directly into our kernel installation tools directly.  That discussion has lots of subtly interacting moving parts, and I'm afraid to say something misleading right now.
>>> 
>>> Min and Thomas were deep in those details, and I'd wait for further feedback from them (or anyone else who remembers well).  But in the meantime, you may want to play with that code in a local copy: you may find a minimally intrusive improvement for venv support that could be a good solution for a PR.
>> 
>> Actually, the kernel is installed fine, and is found in the user's kernel area. It is the installed Python code with "--user" that causes the issue. Maybe this is just a venv/setup interaction bug.
> 
> I see that distutils can use either a --user flag or a --prefix flag, but not both at the same time. Adding a "prefix" kwarg option could also work for install_kernel_spec, and would solve the problem that I am currently running into with venv with --user.
> 
> Min and/or Thomas: does that sound like a viable option?
> 
> -Doug
> 
>> 
>> -Doug
>>  
>>> 
>>> Cheers,
>>> 
>>> f
>>> 
>>> 
>>> 
>>> -- 
>>> Fernando Perez (@fperez_org; http://fperez.org)
>>> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>>> fernando.perez-at-berkeley: contact me here for any direct mail
>>> 
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141230/8396bda2/attachment.html>

From doug.blank at gmail.com  Tue Dec 30 17:54:11 2014
From: doug.blank at gmail.com (Doug Blank)
Date: Tue, 30 Dec 2014 17:54:11 -0500
Subject: [IPython-dev] Installing kernels and libs under a virtual env
In-Reply-To: <99531199-48C7-4311-B923-EDBC9E787924@gmail.com>
References: <CAAusYCin9fs-GBW1JfHeXd6X0miTObb78kjoBwi9a3o0TokbKA@mail.gmail.com>
	<CAHAreOpYsSmdcf=TK68dTA6k-EExDvTOdTvW8cv5yheihMOzzg@mail.gmail.com>
	<CAAusYCgUNR2vqZG_vm-3cF3nKQTd-mVob7-d8HeJVSA-RQWikw@mail.gmail.com>
	<CAAusYCjBQkbmSDA2NM-zdhtUEPGZOo7KXNrFHdj6-1LPePnZ3w@mail.gmail.com>
	<99531199-48C7-4311-B923-EDBC9E787924@gmail.com>
Message-ID: <CAAusYCg1gbrHtj-R_eRUy3oRbRQp8BhWWuL_mpnNb09VT4Bg-A@mail.gmail.com>

On Tue, Dec 30, 2014 at 4:42 PM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

> From what I can tell it is perfectly normal to install a vent kernel to be
> use globally, and we try not to have any venv specific code for the Jupyter
> part.
> Installing was the most problematic part in last discussion, and there is
> no "right" answer so the only logical one is to try to do as least as
> possible.
>

Perhaps the issue isn't clear. Currently (as I understand it), if one is
trying to install a Python file *AND* kernelspecs in a virtual env using
"python setup.py install" then:

1) if you use the "--user" flag, then it installs the kernel into the
user's ~/.ipython/kernels (ok), and installs the Python code into ~/.local/
(not ok, because the virtual environment doesn't look there... it has its
own local folder). [This sounds like a bug in either venv or distutils, or
just a disagreement between them.]

2) if you don't use the --user flag, then you need to use root/sudo,
because the kernel installs into the system directory (ok), but installing
the Python code fails because it can't find IPython (because it is in the
virtual env).

It appears that others get around this problem by using the --prefix
flag. If install_kernel_spec also accepted an optional prefix argument,
then it could work seamlessly with distutils.

There are other possibilities (if the above is correct):

a) separate the kernelspec and Python code installs... one to work with
--prefix (for the Python code), and one to work with --user (for the
kernelspec).

b) make a new flag in setup.py to handle these two cases.

c) don't use --user, check permissions to install kernel into system
directory (regardless of self.user), and fallback to user = True. This
works because the code will go into the virtual env, and kernel will go
into wherever it can be written.

Perhaps this will be more clear if someone writes a setup.py for a
kernelspec and tries to test it under Travis.

-Doug


>
> --
> M
>
> Envoy? de mon iPhone
>
> Le 30 d?c. 2014 ? 22:31, Doug Blank <doug.blank at gmail.com> a ?crit :
>
> On Tue, Dec 30, 2014 at 3:51 PM, Doug Blank <doug.blank at gmail.com> wrote:
>
>> On Tue, Dec 30, 2014 at 2:28 PM, Fernando Perez <fperez.net at gmail.com>
>> wrote:
>>
>>> Hi Doug,
>>>
>>> On Tue, Dec 30, 2014 at 9:57 AM, Doug Blank <doug.blank at gmail.com>
>>> wrote:
>>>
>>>> I think that if "python setup.py install --user" under a venv went to
>>>> the virtual env's local, then everything would be fine. But maybe I'm doing
>>>> something wrong (need to use pip?).
>>>
>>>
>>> The issue is that the list of targets is hardcoded here:
>>>
>>>
>>> https://github.com/ipython/ipython/blob/master/IPython/kernel/kernelspec.py#L21
>>>
>>> that code isn't currently aware in any way of virtualenvs.
>>>
>>> I'm fuzzy on the details of how we ended up deciding we'd interact with
>>> venvs, other than saying that we'll try to bake as little knowledge of
>>> 'environment managers' (venv, conda, etc) directly into our kernel
>>> installation tools directly.  That discussion has lots of subtly
>>> interacting moving parts, and I'm afraid to say something misleading right
>>> now.
>>>
>>> Min and Thomas were deep in those details, and I'd wait for further
>>> feedback from them (or anyone else who remembers well).  But in the
>>> meantime, you may want to play with that code in a local copy: you may find
>>> a minimally intrusive improvement for venv support that could be a good
>>> solution for a PR.
>>>
>>
>> Actually, the kernel is installed fine, and is found in the user's kernel
>> area. It is the installed Python code with "--user" that causes the issue.
>> Maybe this is just a venv/setup interaction bug.
>>
>
> I see that distutils can use either a --user flag or a --prefix flag, but
> not both at the same time. Adding a "prefix" kwarg option could also work
> for install_kernel_spec, and would solve the problem that I am currently
> running into with venv with --user.
>
> Min and/or Thomas: does that sound like a viable option?
>
> -Doug
>
>
>> -Doug
>>
>>
>>>
>>> Cheers,
>>>
>>> f
>>>
>>>
>>>
>>> --
>>> Fernando Perez (@fperez_org; http://fperez.org)
>>> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>>> fernando.perez-at-berkeley: contact me here for any direct mail
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141230/f732a5b5/attachment.html>