From martijn at vermaat.name Sun Dec 1 02:50:43 2013 From: martijn at vermaat.name (Martijn Vermaat) Date: Sun, 1 Dec 2013 08:50:43 +0100 Subject: [IPython-dev] Running IPython Notebook against Python 2.7 or Python 3.3 In-Reply-To: References: Message-ID: For a small course, we recently used this [1] approach on our Ubuntu 12.04 systems to get a Python 2.7 environment with IPython Notebook setup. I never tried to copy this verbatim to Python 3.3, but you could give it a shot. cheers, Martijn [1] https://humgenprojects.lumc.nl/trac/humgenprojects/wiki/ProgrammingCourse/SoftwareInstallation 2013/12/1 Eric Matthes : > Hello, > > I have loved working with IPython Notebook, but I've become totally confused > about how I set up my main system. I love how it is set up now; I can start > a virtualenv with ipython, that either uses python 2.7 or python 3.3. But if > I try to build a similar setup on a clean install of 12.04 from virtualbox, > I can't replicate my environment. > > I am not tied to using virtualenv for ipython, but I do need a reliable > setup that lets me choose to use ipython notebook with either Python 2.7 or > 3.3. I've played with virtualenvs, with pip and easy_install, with symlinks > from python and python3.3 directories, with dist-packages and site-packages, > and it's left me pretty confused. > > Can someone suggest a straightforward way to set up an Ubuntu 12.04 system > to use IPython Notebook, so that I can choose to have the notebook using a > Python 2.7 or Python 3.3 interpreter at different times? For context, I am > working on some tutorials that reference both 2.7 code samples and 3.3 code > samples, and I'd like to be able to run both. I have an open question on > stackoverflow about installing ipython notebook to a virtualenv, but I don't > need that particular question answered necessarily at this point if there is > a better alternative. > > (so question) - > http://stackoverflow.com/questions/20290357/ipython-notebook-in-a-virtualenv-using-python-3-3 > > Eric Matthes > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From damontallen at gmail.com Sun Dec 1 09:08:42 2013 From: damontallen at gmail.com (Damon Allen) Date: Sun, 1 Dec 2013 09:08:42 -0500 Subject: [IPython-dev] Running IPython Notebook against Python 2.7 or Python 3.3 In-Reply-To: References: Message-ID: I recently did a clean install and I just now grabbed the apt-get and pip commands from .bash_history and posted them in the Update here. I don't know is everything is needed since I was just in a rush to up and running again. Damon On Sun, Dec 1, 2013 at 2:50 AM, Martijn Vermaat wrote: > For a small course, we recently used this [1] approach on our Ubuntu > 12.04 systems to get a Python 2.7 environment with IPython Notebook > setup. > > I never tried to copy this verbatim to Python 3.3, but you could give it a > shot. > > cheers, > Martijn > > > [1] > https://humgenprojects.lumc.nl/trac/humgenprojects/wiki/ProgrammingCourse/SoftwareInstallation > > > > 2013/12/1 Eric Matthes : > > Hello, > > > > I have loved working with IPython Notebook, but I've become totally > confused > > about how I set up my main system. I love how it is set up now; I can > start > > a virtualenv with ipython, that either uses python 2.7 or python 3.3. > But if > > I try to build a similar setup on a clean install of 12.04 from > virtualbox, > > I can't replicate my environment. > > > > I am not tied to using virtualenv for ipython, but I do need a reliable > > setup that lets me choose to use ipython notebook with either Python 2.7 > or > > 3.3. I've played with virtualenvs, with pip and easy_install, with > symlinks > > from python and python3.3 directories, with dist-packages and > site-packages, > > and it's left me pretty confused. > > > > Can someone suggest a straightforward way to set up an Ubuntu 12.04 > system > > to use IPython Notebook, so that I can choose to have the notebook using > a > > Python 2.7 or Python 3.3 interpreter at different times? For context, I > am > > working on some tutorials that reference both 2.7 code samples and 3.3 > code > > samples, and I'd like to be able to run both. I have an open question on > > stackoverflow about installing ipython notebook to a virtualenv, but I > don't > > need that particular question answered necessarily at this point if > there is > > a better alternative. > > > > (so question) - > > > http://stackoverflow.com/questions/20290357/ipython-notebook-in-a-virtualenv-using-python-3-3 > > > > Eric Matthes > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Sun Dec 1 12:56:10 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 1 Dec 2013 09:56:10 -0800 Subject: [IPython-dev] Running IPython Notebook against Python 2.7 or Python 3.3 In-Reply-To: References: Message-ID: On 1 December 2013 06:08, Damon Allen wrote: > I recently did a clean install and I just now grabbed the apt-get and pip > commands from .bash_history and posted them in the Update here. > I don't know is everything is needed since I was just in a rush to up and > running again. > I think you'll actually end up with IPython running on Python 3.2, not 3.3, because all the python3-foo packages in Ubuntu 12.04 are compiled for Python 3.2. If people need Python 3.3, I'd advise them to use a newer release of Ubuntu which includes 3.3. Another way would be to install Anaconda (or Miniconda: http://repo.continuum.io/miniconda/ ) and create a Python 3 environment. That comes with its own Python, so you can use Python 3.3 even if your system doesn't have it. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From damontallen at gmail.com Sun Dec 1 13:13:09 2013 From: damontallen at gmail.com (Damon Allen) Date: Sun, 1 Dec 2013 13:13:09 -0500 Subject: [IPython-dev] Running IPython Notebook against Python 2.7 or Python 3.3 In-Reply-To: References: Message-ID: When I'm in IPython and click on Help -> Python I get the Python 3.3.3 documentation page. Since the help page seems to pull up the version of Python that you are using I am assuming that I am running Python 3.3. That being said I did install from the source. sudo python3 setup.py install > However, I only use nbconvert with ipython and not ipython3. On Sun, Dec 1, 2013 at 12:56 PM, Thomas Kluyver wrote: > On 1 December 2013 06:08, Damon Allen wrote: > >> I recently did a clean install and I just now grabbed the apt-get and pip >> commands from .bash_history and posted them in the Update here. >> I don't know is everything is needed since I was just in a rush to up and >> running again. >> > > I think you'll actually end up with IPython running on Python 3.2, not > 3.3, because all the python3-foo packages in Ubuntu 12.04 are compiled for > Python 3.2. If people need Python 3.3, I'd advise them to use a newer > release of Ubuntu which includes 3.3. > > Another way would be to install Anaconda (or Miniconda: > http://repo.continuum.io/miniconda/ ) and create a Python 3 environment. > That comes with its own Python, so you can use Python 3.3 even if your > system doesn't have it. > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Sun Dec 1 13:15:19 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 1 Dec 2013 10:15:19 -0800 Subject: [IPython-dev] Running IPython Notebook against Python 2.7 or Python 3.3 In-Reply-To: References: Message-ID: On 1 December 2013 10:13, Damon Allen wrote: > When I'm in IPython and click on Help -> Python I get the Python 3.3.3 > documentation page. Since the help page seems to pull up the version of > Python that you are using I am assuming that I am running Python 3.3. That > being said I did install from the source. > That just points to http://docs.python.org , whichever version of Python you're running. Check sys.version to see the Python version. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From damontallen at gmail.com Sun Dec 1 13:19:29 2013 From: damontallen at gmail.com (Damon Allen) Date: Sun, 1 Dec 2013 13:19:29 -0500 Subject: [IPython-dev] Running IPython Notebook against Python 2.7 or Python 3.3 In-Reply-To: References: Message-ID: You are right. > 3.2.3 (default, Sep 25 2013, 18:22:43) > > > [GCC 4.6.3] > > Damon -------------- next part -------------- An HTML attachment was scrubbed... URL: From damontallen at gmail.com Mon Dec 2 08:14:38 2013 From: damontallen at gmail.com (Damon Allen) Date: Mon, 2 Dec 2013 08:14:38 -0500 Subject: [IPython-dev] nbviewer.ipython.org Message-ID: I've been having issues with using nbviewer.ipython.org over that last week or so. I've been getting an Application Error page more and more and I was wondering is there something wrong with the servers? I don't seem to be having problems with any other sites so I don't think it is an issue on my end but I wanted to ask to see if anyone else is having similar problems. Thank you, Damon -------------- next part -------------- An HTML attachment was scrubbed... URL: From damianavila at gmail.com Mon Dec 2 08:23:25 2013 From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=) Date: Mon, 2 Dec 2013 11:23:25 -0200 Subject: [IPython-dev] nbviewer.ipython.org In-Reply-To: References: Message-ID: Hi Damon, Last week, it has been deployed a new version of nbviewer with lot of improvements. If you have any problem, please report it here opening an issue: https://github.com/ipython/nbviewer/issues Thanks. 2013/12/2 Damon Allen > I've been having issues with using nbviewer.ipython.org over that last > week or so. I've been getting an Application Error page more and more and > I was wondering is there something wrong with the servers? I don't seem to > be having problems with any other sites so I don't think it is an issue on > my end but I wanted to ask to see if anyone else is having similar problems. > > Thank you, > > Damon > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Dami?n Avila Scientific Python Developer Quantitative Finance Analyst Statistics, Biostatistics and Econometrics Consultant Biochemist -------------- next part -------------- An HTML attachment was scrubbed... URL: From damontallen at gmail.com Mon Dec 2 08:26:14 2013 From: damontallen at gmail.com (Damon Allen) Date: Mon, 2 Dec 2013 08:26:14 -0500 Subject: [IPython-dev] nbviewer.ipython.org In-Reply-To: References: Message-ID: Thanks Dami?n. I'll collect a few notes about my experience and report it. Damon T. Allen Ph.D. Adjunct Professor (352) 234-3266 damontallen at gmail.com 311 Rinker Hall College of Building Construction University of Florida On Mon, Dec 2, 2013 at 8:23 AM, Dami?n Avila wrote: > Hi Damon, > > Last week, it has been deployed a new version of nbviewer with lot of > improvements. > If you have any problem, please report it here opening an issue: > https://github.com/ipython/nbviewer/issues > > Thanks. > > > > 2013/12/2 Damon Allen > >> I've been having issues with using nbviewer.ipython.org over that last >> week or so. I've been getting an Application Error page more and more and >> I was wondering is there something wrong with the servers? I don't seem to >> be having problems with any other sites so I don't think it is an issue on >> my end but I wanted to ask to see if anyone else is having similar problems. >> >> Thank you, >> >> Damon >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > > -- > Dami?n Avila > Scientific Python Developer > Quantitative Finance Analyst > Statistics, Biostatistics and Econometrics Consultant > Biochemist > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Dec 2 08:34:29 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Mon, 2 Dec 2013 14:34:29 +0100 Subject: [IPython-dev] nbviewer.ipython.org In-Reply-To: References: Message-ID: <6F528905-8B77-4663-B8C4-1C285E173B95@gmail.com> Yes, if you can send us which notebook is throwing error and what kind of error it would be useful. Did you get the nice 404 or 500 error that look like nbviewer or the ugly purple one. Also you can alway try alternate/dev versions of nbviewer if you need it : http://nbtest.herokuapp.com/ http://gistpynb.herokuapp.com/ -- Matthias Le 2 d?c. 2013 ? 14:26, Damon Allen a ?crit : > Thanks Dami?n. I'll collect a few notes about my experience and report it. > > On Mon, Dec 2, 2013 at 8:23 AM, Dami?n Avila wrote: > Hi Damon, > > Last week, it has been deployed a new version of nbviewer with lot of improvements. > If you have any problem, please report it here opening an issue: https://github.com/ipython/nbviewer/issues > > Thanks. > > > 2013/12/2 Damon Allen > I've been having issues with using nbviewer.ipython.org over that last week or so. I've been getting an Application Error page more and more and I was wondering is there something wrong with the servers? I don't seem to be having problems with any other sites so I don't think it is an issue on my end but I wanted to ask to see if anyone else is having similar problems. > > Thank you, > > Damon > -------------- next part -------------- An HTML attachment was scrubbed... URL: From damontallen at gmail.com Mon Dec 2 08:37:31 2013 From: damontallen at gmail.com (Damon Allen) Date: Mon, 2 Dec 2013 08:37:31 -0500 Subject: [IPython-dev] nbviewer.ipython.org In-Reply-To: <6F528905-8B77-4663-B8C4-1C285E173B95@gmail.com> References: <6F528905-8B77-4663-B8C4-1C285E173B95@gmail.com> Message-ID: I've mainly been getting "Application Error"but I have seen other messages. I'll have to experiment to try to get different ones to show up. Damon T. Allen Ph.D. Adjunct Professor (352) 234-3266 damontallen at gmail.com 311 Rinker Hall College of Building Construction University of Florida On Mon, Dec 2, 2013 at 8:34 AM, Matthias BUSSONNIER < bussonniermatthias at gmail.com> wrote: > Yes, if you can send us which notebook is throwing error > and what kind of error it would be useful. > > Did you get the nice 404 or 500 error that look like nbviewer > or the ugly purple one. > > Also you can alway try alternate/dev versions of nbviewer if you need it : > > http://nbtest.herokuapp.com/ > http://gistpynb.herokuapp.com/ > > -- > Matthias > > Le 2 d?c. 2013 ? 14:26, Damon Allen a ?crit : > > Thanks Dami?n. I'll collect a few notes about my experience and report it. > > On Mon, Dec 2, 2013 at 8:23 AM, Dami?n Avila wrote: > >> Hi Damon, >> >> Last week, it has been deployed a new version of nbviewer with lot of >> improvements. >> If you have any problem, please report it here opening an issue: >> https://github.com/ipython/nbviewer/issues >> >> Thanks. >> >> >> 2013/12/2 Damon Allen >> >>> I've been having issues with using nbviewer.ipython.org over that last >>> week or so. I've been getting an Application Error page more and more and >>> I was wondering is there something wrong with the servers? I don't seem to >>> be having problems with any other sites so I don't think it is an issue on >>> my end but I wanted to ask to see if anyone else is having similar problems. >>> >>> Thank you, >>> >>> Damon >>> >>> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo at sifflez.org Mon Dec 2 10:45:03 2013 From: pablo at sifflez.org (Pablo Oliveira) Date: Mon, 02 Dec 2013 16:45:03 +0100 Subject: [IPython-dev] nbviewer.ipython.org In-Reply-To: <6F528905-8B77-4663-B8C4-1C285E173B95@gmail.com> References: <6F528905-8B77-4663-B8C4-1C285E173B95@gmail.com> Message-ID: <529CAAFF.1090800@sifflez.org> On 12/02/2013 02:34 PM, Matthias BUSSONNIER wrote: > Yes, if you can send us which notebook is throwing error > and what kind of error it would be useful. I'm observing the same behavior with the following notebook (one of the example notebooks on the front page), http://nbviewer.ipython.org/github/fperez/nipy-notebooks/blob/master/exploring_r_formula.ipynb > Did you get the nice 404 or 500 error that look like nbviewer > or the ugly purple one. It throws a purple "Application Error" error page. > Also you can alway try alternate/dev versions of nbviewer if you need it : > > http://nbtest.herokuapp.com/ > http://gistpynb.herokuapp.com/ > On nbtest.herokuapp.com, it works. On gistpynb.herokuapp.com, it throws an "Error 400: Bad Request". Regards, Pablo From bussonniermatthias at gmail.com Mon Dec 2 11:02:29 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Mon, 2 Dec 2013 17:02:29 +0100 Subject: [IPython-dev] nbviewer.ipython.org In-Reply-To: <529CAAFF.1090800@sifflez.org> References: <6F528905-8B77-4663-B8C4-1C285E173B95@gmail.com> <529CAAFF.1090800@sifflez.org> Message-ID: <82B5B8A3-9FAC-485E-89FF-A498F2609FE3@gmail.com> Le 2 d?c. 2013 ? 16:45, Pablo Oliveira a ?crit : > On 12/02/2013 02:34 PM, Matthias BUSSONNIER wrote: >> Yes, if you can send us which notebook is throwing error >> and what kind of error it would be useful. > > I'm observing the same behavior with the following notebook (one of the > example notebooks on the front page), > http://nbviewer.ipython.org/github/fperez/nipy-notebooks/blob/master/exploring_r_formula.ipynb > >> Did you get the nice 404 or 500 error that look like nbviewer >> or the ugly purple one. > > It throws a purple "Application Error" error page. > >> Also you can alway try alternate/dev versions of nbviewer if you need it : >> >> http://nbtest.herokuapp.com/ >> http://gistpynb.herokuapp.com/ >> > On nbtest.herokuapp.com, it works. > On gistpynb.herokuapp.com, it throws an "Error 400: Bad Request". Indeed, each of thoses red line represent a period of 2 min where nbviewer is down. Strange thing is, new-relic did not alert us. Will try to investigate. Thanks. -- Matthias > > Regards, > > Pablo > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Capture d??cran 2013-12-02 ? 17.00.29.png Type: image/png Size: 48738 bytes Desc: not available URL: From ehmatthes at gmail.com Mon Dec 2 11:30:22 2013 From: ehmatthes at gmail.com (Eric Matthes) Date: Mon, 2 Dec 2013 07:30:22 -0900 Subject: [IPython-dev] Running IPython Notebook against Python 2.7 or Python 3.3 In-Reply-To: References: Message-ID: I ended up with IPython running on Python 3.3, confirmed with sys.version, because I ran into a conflict with markupsafe during installation with Python3.2: https://github.com/mitsuhiko/markupsafe/pull/13 Eric On Sun, Dec 1, 2013 at 8:56 AM, Thomas Kluyver wrote: > On 1 December 2013 06:08, Damon Allen wrote: > >> I recently did a clean install and I just now grabbed the apt-get and pip >> commands from .bash_history and posted them in the Update here. >> I don't know is everything is needed since I was just in a rush to up and >> running again. >> > > I think you'll actually end up with IPython running on Python 3.2, not > 3.3, because all the python3-foo packages in Ubuntu 12.04 are compiled for > Python 3.2. If people need Python 3.3, I'd advise them to use a newer > release of Ubuntu which includes 3.3. > > Another way would be to install Anaconda (or Miniconda: > http://repo.continuum.io/miniconda/ ) and create a Python 3 environment. > That comes with its own Python, so you can use Python 3.3 even if your > system doesn't have it. > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehmatthes at gmail.com Mon Dec 2 11:31:51 2013 From: ehmatthes at gmail.com (Eric Matthes) Date: Mon, 2 Dec 2013 07:31:51 -0900 Subject: [IPython-dev] Running IPython Notebook against Python 2.7 or Python 3.3 In-Reply-To: References: Message-ID: Thank you everyone, these responses were really helpful, particularly as I start to help more people get set up for the first time with IPython. Eric On Sat, Nov 30, 2013 at 3:45 PM, Eric Matthes wrote: > Hello, > > I have loved working with IPython Notebook, but I've become totally > confused about how I set up my main system. I love how it is set up now; I > can start a virtualenv with ipython, that either uses python 2.7 or python > 3.3. But if I try to build a similar setup on a clean install of 12.04 from > virtualbox, I can't replicate my environment. > > I am not tied to using virtualenv for ipython, but I do need a reliable > setup that lets me choose to use ipython notebook with either Python 2.7 or > 3.3. I've played with virtualenvs, with pip and easy_install, with symlinks > from python and python3.3 directories, with dist-packages and > site-packages, and it's left me pretty confused. > > Can someone suggest a straightforward way to set up an Ubuntu 12.04 system > to use IPython Notebook, so that I can choose to have the notebook using a > Python 2.7 or Python 3.3 interpreter at different times? For context, I am > working on some tutorials that reference both 2.7 code samples and 3.3 code > samples, and I'd like to be able to run both. I have an open question on > stackoverflow about installing ipython notebook to a virtualenv, but I > don't need that particular question answered necessarily at this point if > there is a better alternative. > > (so question) - > http://stackoverflow.com/questions/20290357/ipython-notebook-in-a-virtualenv-using-python-3-3 > > Eric Matthes > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehmatthes at gmail.com Mon Dec 2 11:37:07 2013 From: ehmatthes at gmail.com (Eric Matthes) Date: Mon, 2 Dec 2013 07:37:07 -0900 Subject: [IPython-dev] Highlight lines of code in code cells? Message-ID: Hello, I often use IPython Notebook to write tutorials. In tutorials, I find it helpful to highlight the lines of code that are being changed in each iteration of a project. Is there a straightforward way to highlight individual lines of code within a code cell? I came up with one approach, but it feels like an ugly hack. I write a comment before the line[s] I want to highlight, and a comment after. Then when converting to html I convert these comment lines to and . This works, but it clutters the raw notebook file with formatting comments. ##bold print("I'd like to highlight this line.") ##/bold becomes: print("I'd like to highlight this line.") Is there a better way? (formatting I'd like to have: http://peak5390.wordpress.com/2012/12/08/matplotlib-basemap-tutorial-making-a-simple-map/ ) Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Dec 2 12:01:02 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Mon, 2 Dec 2013 18:01:02 +0100 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: References: Message-ID: Le 2 d?c. 2013 ? 17:37, Eric Matthes a ?crit : > Hello, > > I often use IPython Notebook to write tutorials. In tutorials, I find it helpful to highlight the lines of code that are being changed in each iteration of a project. Is there a straightforward way to highlight individual lines of code within a code cell? > > I came up with one approach, but it feels like an ugly hack. I write a comment before the line[s] I want to highlight, and a comment after. Then when converting to html I convert these comment lines to and . This works, but it clutters the raw notebook file with formatting comments. > > ##bold > print("I'd like to highlight this line.") > ##/bold > > becomes: > > print("I'd like to highlight this line.") > > Is there a better way? One could bind code mirror gutter event (click on line number) to store metadata into the cells, and write a custom highlighter for the templates in nbconvert. This is more or less the only thing I see. I would say that it's a good long afternoon hack if you are familiar with the codebase/codemirror and Pygment. It would have the advantage that you can easily toggle this highlight on and of with a flag. -- Matthias From ehmatthes at gmail.com Mon Dec 2 12:26:46 2013 From: ehmatthes at gmail.com (Eric Matthes) Date: Mon, 2 Dec 2013 08:26:46 -0900 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: References: Message-ID: I am not very familiar with the codebase yet, and I have few "good long afternoons" at this time of the year. I don't want to clutter my notebooks in a way that will be laborious to clean, so I think will use the commenting approach I described above, but also write a quick script that strips the notebooks of these formatting comments. This way anyone who is using the raw notebooks for their own purposes can choose to use my hack, or have a set of clean, unhighlighted notebooks to work with. I like the efficiency of your approach, though, so when I do get a nice afternoon to hack I will play with that approach. If anyone else has a good suggestion, I'd love to hear it as well. Thanks. Eric On Mon, Dec 2, 2013 at 8:01 AM, Matthias BUSSONNIER < bussonniermatthias at gmail.com> wrote: > > Le 2 d?c. 2013 ? 17:37, Eric Matthes a ?crit : > > > Hello, > > > > I often use IPython Notebook to write tutorials. In tutorials, I find it > helpful to highlight the lines of code that are being changed in each > iteration of a project. Is there a straightforward way to highlight > individual lines of code within a code cell? > > > > I came up with one approach, but it feels like an ugly hack. I write a > comment before the line[s] I want to highlight, and a comment after. Then > when converting to html I convert these comment lines to and . This > works, but it clutters the raw notebook file with formatting comments. > > > > ##bold > > print("I'd like to highlight this line.") > > ##/bold > > > > becomes: > > > > print("I'd like to highlight this line.") > > > > Is there a better way? > > One could bind code mirror gutter event (click on line number) to store > metadata into the cells, > and write a custom highlighter for the templates in nbconvert. > This is more or less the only thing I see. I would say that it's a good > long afternoon hack if you > are familiar with the codebase/codemirror and Pygment. > > It would have the advantage that you can easily toggle this highlight on > and of with a flag. > > -- > Matthias > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jakevdp at cs.washington.edu Mon Dec 2 12:44:27 2013 From: jakevdp at cs.washington.edu (Jacob Vanderplas) Date: Mon, 2 Dec 2013 09:44:27 -0800 Subject: [IPython-dev] IPython notebook/parallel relation Message-ID: Hi, I'm giving a lecture on IPython parallel this afternoon, and I'm anticipating the usual question of "why does IPython include parallel tools?" I know that the standard response is that IPython seeks to provide seamless tools for the entire life-cycle of the scientific idea, as well as the fact that it's a natural fit because the parallelism requires the same message passing as the notebook. I'd like to go into a bit more detail on the architecture overlap during the lecture. Are there any resources on that subject that I could look through and condense into a ~5 minute discussion? Thanks, Jake -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.voorhies at ucsf.edu Mon Dec 2 17:03:00 2013 From: mark.voorhies at ucsf.edu (Mark Voorhies) Date: Mon, 2 Dec 2013 14:03:00 -0800 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: References: Message-ID: <529D0394.6030004@ucsf.edu> On 12/02/2013 09:26 AM, Eric Matthes wrote: > I am not very familiar with the codebase yet, and I have few "good long > afternoons" at this time of the year. I don't want to clutter my notebooks > in a way that will be laborious to clean, so I think will use the > commenting approach I described above, but also write a quick script that > strips the notebooks of these formatting comments. This way anyone who is > using the raw notebooks for their own purposes can choose to use my hack, > or have a set of clean, unhighlighted notebooks to work with. > > I like the efficiency of your approach, though, so when I do get a nice > afternoon to hack I will play with that approach. If anyone else has a good > suggestion, I'd love to hear it as well. > > Thanks. > > Eric > > > On Mon, Dec 2, 2013 at 8:01 AM, Matthias BUSSONNIER < > bussonniermatthias at gmail.com> wrote: > >> >> Le 2 d?c. 2013 ? 17:37, Eric Matthes a ?crit : >> >>> Hello, >>> >>> I often use IPython Notebook to write tutorials. In tutorials, I find it >> helpful to highlight the lines of code that are being changed in each >> iteration of a project. Is there a straightforward way to highlight >> individual lines of code within a code cell? >>> >>> I came up with one approach, but it feels like an ugly hack. I write a >> comment before the line[s] I want to highlight, and a comment after. Then >> when converting to html I convert these comment lines to and . This >> works, but it clutters the raw notebook file with formatting comments. >>> >>> ##bold >>> print("I'd like to highlight this line.") >>> ##/bold >>> >>> becomes: >>> >>> print("I'd like to highlight this line.") >>> >>> Is there a better way? >> >> One could bind code mirror gutter event (click on line number) to store >> metadata into the cells, >> and write a custom highlighter for the templates in nbconvert. >> This is more or less the only thing I see. I would say that it's a good >> long afternoon hack if you >> are familiar with the codebase/codemirror and Pygment. >> >> It would have the advantage that you can easily toggle this highlight on >> and of with a flag. >> >> -- >> Matthias It would be very nice to have this as a cell magic that would highlight lines not in a reference cell (or render as a diff relative to the reference cell). E.g., [23] def Hello(): print "Hello, World" [24] %%diff 23 def Hello(s) print "Hello, %s" % s No idea how to do this, though =) --Mark >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From benjaminrk at gmail.com Mon Dec 2 17:56:14 2013 From: benjaminrk at gmail.com (MinRK) Date: Mon, 2 Dec 2013 14:56:14 -0800 Subject: [IPython-dev] IPython notebook/parallel relation In-Reply-To: References: Message-ID: Sorry, this is probably too late, but I didn't notice this until now. The gist is that IPython is a collection of tools for interactive computing. One of the major pieces of that is abstracting the REPL over the network with our message spec. Once your execution is remote, it's a very short leap to have multiple remote executions in parallel. IPython.parallel is just basic tools for coordinating multiple remote executions, using the exact same messaging architecture as the notebook, etc.* The most important piece of this that people often don't realize is that the IPython 'engine' used in IPython.parallel is the exact same object as the Kernel used by the Notebook. The only difference is that its sockets connect instead of bind, which is mostly insignificant due to zeromq magic. * with a few extra messages that only make sense when the client is also an interactive Python session, most significantly `apply`. -MinRK On Mon, Dec 2, 2013 at 9:44 AM, Jacob Vanderplas wrote: > Hi, > I'm giving a lecture on IPython parallel this afternoon, and I'm > anticipating the usual question of "why does IPython include parallel > tools?" I know that the standard response is that IPython seeks to provide > seamless tools for the entire life-cycle of the scientific idea, as well as > the fact that it's a natural fit because the parallelism requires the same > message passing as the notebook. I'd like to go into a bit more detail on > the architecture overlap during the lecture. Are there any resources on > that subject that I could look through and condense into a ~5 minute > discussion? > > Thanks, > Jake > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gvwilson at third-bit.com Mon Dec 2 18:19:05 2013 From: gvwilson at third-bit.com (Greg Wilson) Date: Mon, 02 Dec 2013 18:19:05 -0500 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: <529D0394.6030004@ucsf.edu> References: <529D0394.6030004@ucsf.edu> Message-ID: <529D1569.2070206@third-bit.com> On 2013-12-02 5:03 PM, Mark Voorhies wrote: > On 12/02/2013 09:26 AM, Eric Matthes wrote: >> I am not very familiar with the codebase yet, and I have few "good long >> afternoons" at this time of the year. I don't want to clutter my notebooks >> in a way that will be laborious to clean, so I think will use the >> commenting approach I described above, but also write a quick script that >> strips the notebooks of these formatting comments. This way anyone who is >> using the raw notebooks for their own purposes can choose to use my hack, >> or have a set of clean, unhighlighted notebooks to work with. >> >> I like the efficiency of your approach, though, so when I do get a nice >> afternoon to hack I will play with that approach. If anyone else has a good >> suggestion, I'd love to hear it as well. >> >> Thanks. >> >> Eric >> >> >> On Mon, Dec 2, 2013 at 8:01 AM, Matthias BUSSONNIER < >> bussonniermatthias at gmail.com> wrote: >> >>> Le 2 d?c. 2013 ? 17:37, Eric Matthes a ?crit : >>> >>>> Hello, >>>> >>>> I often use IPython Notebook to write tutorials. In tutorials, I find it >>> helpful to highlight the lines of code that are being changed in each >>> iteration of a project. Is there a straightforward way to highlight >>> individual lines of code within a code cell? >>>> I came up with one approach, but it feels like an ugly hack. I write a >>> comment before the line[s] I want to highlight, and a comment after. Then >>> when converting to html I convert these comment lines to and . This >>> works, but it clutters the raw notebook file with formatting comments. >>>> ##bold >>>> print("I'd like to highlight this line.") >>>> ##/bold >>>> >>>> becomes: >>>> >>>> print("I'd like to highlight this line.") >>>> >>>> Is there a better way? >>> One could bind code mirror gutter event (click on line number) to store >>> metadata into the cells, >>> and write a custom highlighter for the templates in nbconvert. >>> This is more or less the only thing I see. I would say that it's a good >>> long afternoon hack if you >>> are familiar with the codebase/codemirror and Pygment. >>> >>> It would have the advantage that you can easily toggle this highlight on >>> and of with a flag. >>> >>> -- >>> Matthias > It would be very nice to have this as a cell magic that would highlight lines not in a reference > cell (or render as a diff relative to the reference cell). > > E.g., > > [23] def Hello(): > print "Hello, World" > > [24] %%diff 23 > def Hello(s) > print "Hello, %s" % s > > No idea how to do this, though =) > > --Mark > Software Carpentry would really like this feature as well, but I'd prefer not to tie it to cell numbers, since they change as the notebook is re-executed. Instead, could we use cell tagging to give cells names, then display the diff against a named cell? Thanks, Greg From aron at ahmadia.net Mon Dec 2 18:31:37 2013 From: aron at ahmadia.net (Aron Ahmadia) Date: Mon, 2 Dec 2013 18:31:37 -0500 Subject: [IPython-dev] IPython notebook/parallel relation In-Reply-To: References: Message-ID: I would also add that the %magic syntax lends itself very conveniently to Single Program Multiple Data programming models, where you issue the same set of instructions to a group of processors, and they all go to work on them. A On Mon, Dec 2, 2013 at 5:56 PM, MinRK wrote: > Sorry, this is probably too late, but I didn't notice this until now. > > The gist is that IPython is a collection of tools for interactive > computing. > One of the major pieces of that is abstracting the REPL over the network > with our message spec. > Once your execution is remote, it's a very short leap to have multiple > remote executions in parallel. > IPython.parallel is just basic tools for coordinating multiple remote > executions, > using the exact same messaging architecture as the notebook, etc.* > > The most important piece of this that people often don't realize is that > the IPython 'engine' used in IPython.parallel is the exact same object as > the Kernel used by the Notebook. > The only difference is that its sockets connect instead of bind, which is > mostly insignificant due to zeromq magic. > > * with a few extra messages that only make sense when the client is also > an interactive Python session, most significantly `apply`. > > -MinRK > > > > On Mon, Dec 2, 2013 at 9:44 AM, Jacob Vanderplas < > jakevdp at cs.washington.edu> wrote: > >> Hi, >> I'm giving a lecture on IPython parallel this afternoon, and I'm >> anticipating the usual question of "why does IPython include parallel >> tools?" I know that the standard response is that IPython seeks to provide >> seamless tools for the entire life-cycle of the scientific idea, as well as >> the fact that it's a natural fit because the parallelism requires the same >> message passing as the notebook. I'd like to go into a bit more detail on >> the architecture overlap during the lecture. Are there any resources on >> that subject that I could look through and condense into a ~5 minute >> discussion? >> >> Thanks, >> Jake >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.voorhies at ucsf.edu Mon Dec 2 18:54:12 2013 From: mark.voorhies at ucsf.edu (Mark Voorhies) Date: Mon, 2 Dec 2013 15:54:12 -0800 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: <529D1569.2070206@third-bit.com> References: <529D0394.6030004@ucsf.edu> <529D1569.2070206@third-bit.com> Message-ID: <529D1DA4.3050302@ucsf.edu> On 12/02/2013 03:19 PM, Greg Wilson wrote: > On 2013-12-02 5:03 PM, Mark Voorhies wrote: >> On 12/02/2013 09:26 AM, Eric Matthes wrote: >>> I am not very familiar with the codebase yet, and I have few "good long >>> afternoons" at this time of the year. I don't want to clutter my notebooks >>> in a way that will be laborious to clean, so I think will use the >>> commenting approach I described above, but also write a quick script that >>> strips the notebooks of these formatting comments. This way anyone who is >>> using the raw notebooks for their own purposes can choose to use my hack, >>> or have a set of clean, unhighlighted notebooks to work with. >>> >>> I like the efficiency of your approach, though, so when I do get a nice >>> afternoon to hack I will play with that approach. If anyone else has a good >>> suggestion, I'd love to hear it as well. >>> >>> Thanks. >>> >>> Eric >>> >>> >>> On Mon, Dec 2, 2013 at 8:01 AM, Matthias BUSSONNIER < >>> bussonniermatthias at gmail.com> wrote: >>> >>>> Le 2 d?c. 2013 ? 17:37, Eric Matthes a ?crit : >>>> >>>>> Hello, >>>>> >>>>> I often use IPython Notebook to write tutorials. In tutorials, I find it >>>> helpful to highlight the lines of code that are being changed in each >>>> iteration of a project. Is there a straightforward way to highlight >>>> individual lines of code within a code cell? >>>>> I came up with one approach, but it feels like an ugly hack. I write a >>>> comment before the line[s] I want to highlight, and a comment after. Then >>>> when converting to html I convert these comment lines to and . This >>>> works, but it clutters the raw notebook file with formatting comments. >>>>> ##bold >>>>> print("I'd like to highlight this line.") >>>>> ##/bold >>>>> >>>>> becomes: >>>>> >>>>> print("I'd like to highlight this line.") >>>>> >>>>> Is there a better way? >>>> One could bind code mirror gutter event (click on line number) to store >>>> metadata into the cells, >>>> and write a custom highlighter for the templates in nbconvert. >>>> This is more or less the only thing I see. I would say that it's a good >>>> long afternoon hack if you >>>> are familiar with the codebase/codemirror and Pygment. >>>> >>>> It would have the advantage that you can easily toggle this highlight on >>>> and of with a flag. >>>> >>>> -- >>>> Matthias >> It would be very nice to have this as a cell magic that would highlight lines not in a reference >> cell (or render as a diff relative to the reference cell). >> >> E.g., >> >> [23] def Hello(): >> print "Hello, World" >> >> [24] %%diff 23 >> def Hello(s) >> print "Hello, %s" % s >> >> No idea how to do this, though =) >> >> --Mark >> > Software Carpentry would really like this feature as well, but I'd > prefer not to tie it to cell numbers, since they change as the notebook > is re-executed. Instead, could we use cell tagging to give cells names, > then display the diff against a named cell? > Thanks, > Greg Just started fiddling with this, using rmagic.py and cythonmagic.py as references. Can anyone point me at documentation for getting at retrieving cell content by number and/or tag via the IPython shell instance? Thanks, Mark From ehmatthes at gmail.com Mon Dec 2 18:54:59 2013 From: ehmatthes at gmail.com (Eric Matthes) Date: Mon, 2 Dec 2013 14:54:59 -0900 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: <529D0394.6030004@ucsf.edu> References: <529D0394.6030004@ucsf.edu> Message-ID: It took me a couple readings of your response to understand what you were suggesting. At first I thought [23] and [24] were line numbers, not cell numbers. What you are describing would be incredibly useful for building tutorials. Eric On Mon, Dec 2, 2013 at 1:03 PM, Mark Voorhies wrote: > On 12/02/2013 09:26 AM, Eric Matthes wrote: > > I am not very familiar with the codebase yet, and I have few "good long > > afternoons" at this time of the year. I don't want to clutter my > notebooks > > in a way that will be laborious to clean, so I think will use the > > commenting approach I described above, but also write a quick script that > > strips the notebooks of these formatting comments. This way anyone who is > > using the raw notebooks for their own purposes can choose to use my hack, > > or have a set of clean, unhighlighted notebooks to work with. > > > > I like the efficiency of your approach, though, so when I do get a nice > > afternoon to hack I will play with that approach. If anyone else has a > good > > suggestion, I'd love to hear it as well. > > > > Thanks. > > > > Eric > > > > > > On Mon, Dec 2, 2013 at 8:01 AM, Matthias BUSSONNIER < > > bussonniermatthias at gmail.com> wrote: > > > >> > >> Le 2 d?c. 2013 ? 17:37, Eric Matthes a ?crit : > >> > >>> Hello, > >>> > >>> I often use IPython Notebook to write tutorials. In tutorials, I find > it > >> helpful to highlight the lines of code that are being changed in each > >> iteration of a project. Is there a straightforward way to highlight > >> individual lines of code within a code cell? > >>> > >>> I came up with one approach, but it feels like an ugly hack. I write a > >> comment before the line[s] I want to highlight, and a comment after. > Then > >> when converting to html I convert these comment lines to and . > This > >> works, but it clutters the raw notebook file with formatting comments. > >>> > >>> ##bold > >>> print("I'd like to highlight this line.") > >>> ##/bold > >>> > >>> becomes: > >>> > >>> print("I'd like to highlight this line.") > >>> > >>> Is there a better way? > >> > >> One could bind code mirror gutter event (click on line number) to store > >> metadata into the cells, > >> and write a custom highlighter for the templates in nbconvert. > >> This is more or less the only thing I see. I would say that it's a good > >> long afternoon hack if you > >> are familiar with the codebase/codemirror and Pygment. > >> > >> It would have the advantage that you can easily toggle this highlight on > >> and of with a flag. > >> > >> -- > >> Matthias > > It would be very nice to have this as a cell magic that would highlight > lines not in a reference > cell (or render as a diff relative to the reference cell). > > E.g., > > [23] def Hello(): > print "Hello, World" > > [24] %%diff 23 > def Hello(s) > print "Hello, %s" % s > > No idea how to do this, though =) > > --Mark > > >> > >> > >> > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehmatthes at gmail.com Mon Dec 2 18:57:32 2013 From: ehmatthes at gmail.com (Eric Matthes) Date: Mon, 2 Dec 2013 14:57:32 -0900 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: <529D1569.2070206@third-bit.com> References: <529D0394.6030004@ucsf.edu> <529D1569.2070206@third-bit.com> Message-ID: Are you thinking of one name for a series of cells, or an individual name for each cell? It would be nice to give a series of cells the same name (or the same tag), and then the notebook looks through the series of cells that have the same name. Each successive cell with the same name gets diff'ed against the previous cell of the same name/ tag, and the diff's get highlighted. Eric On Mon, Dec 2, 2013 at 2:19 PM, Greg Wilson wrote: > On 2013-12-02 5:03 PM, Mark Voorhies wrote: > > On 12/02/2013 09:26 AM, Eric Matthes wrote: > >> I am not very familiar with the codebase yet, and I have few "good long > >> afternoons" at this time of the year. I don't want to clutter my > notebooks > >> in a way that will be laborious to clean, so I think will use the > >> commenting approach I described above, but also write a quick script > that > >> strips the notebooks of these formatting comments. This way anyone who > is > >> using the raw notebooks for their own purposes can choose to use my > hack, > >> or have a set of clean, unhighlighted notebooks to work with. > >> > >> I like the efficiency of your approach, though, so when I do get a nice > >> afternoon to hack I will play with that approach. If anyone else has a > good > >> suggestion, I'd love to hear it as well. > >> > >> Thanks. > >> > >> Eric > >> > >> > >> On Mon, Dec 2, 2013 at 8:01 AM, Matthias BUSSONNIER < > >> bussonniermatthias at gmail.com> wrote: > >> > >>> Le 2 d?c. 2013 ? 17:37, Eric Matthes a ?crit : > >>> > >>>> Hello, > >>>> > >>>> I often use IPython Notebook to write tutorials. In tutorials, I find > it > >>> helpful to highlight the lines of code that are being changed in each > >>> iteration of a project. Is there a straightforward way to highlight > >>> individual lines of code within a code cell? > >>>> I came up with one approach, but it feels like an ugly hack. I write a > >>> comment before the line[s] I want to highlight, and a comment after. > Then > >>> when converting to html I convert these comment lines to and . > This > >>> works, but it clutters the raw notebook file with formatting comments. > >>>> ##bold > >>>> print("I'd like to highlight this line.") > >>>> ##/bold > >>>> > >>>> becomes: > >>>> > >>>> print("I'd like to highlight this line.") > >>>> > >>>> Is there a better way? > >>> One could bind code mirror gutter event (click on line number) to store > >>> metadata into the cells, > >>> and write a custom highlighter for the templates in nbconvert. > >>> This is more or less the only thing I see. I would say that it's a > good > >>> long afternoon hack if you > >>> are familiar with the codebase/codemirror and Pygment. > >>> > >>> It would have the advantage that you can easily toggle this highlight > on > >>> and of with a flag. > >>> > >>> -- > >>> Matthias > > It would be very nice to have this as a cell magic that would highlight > lines not in a reference > > cell (or render as a diff relative to the reference cell). > > > > E.g., > > > > [23] def Hello(): > > print "Hello, World" > > > > [24] %%diff 23 > > def Hello(s) > > print "Hello, %s" % s > > > > No idea how to do this, though =) > > > > --Mark > > > Software Carpentry would really like this feature as well, but I'd > prefer not to tie it to cell numbers, since they change as the notebook > is re-executed. Instead, could we use cell tagging to give cells names, > then display the diff against a named cell? > Thanks, > Greg > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmbecker at ucdavis.edu Mon Dec 2 19:10:26 2013 From: gmbecker at ucdavis.edu (Gabriel Becker) Date: Mon, 2 Dec 2013 16:10:26 -0800 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: <529D1DA4.3050302@ucsf.edu> References: <529D0394.6030004@ucsf.edu> <529D1569.2070206@third-bit.com> <529D1DA4.3050302@ucsf.edu> Message-ID: On Mon, Dec 2, 2013 at 3:54 PM, Mark Voorhies wrote: > On 12/02/2013 03:19 PM, Greg Wilson wrote: > > > Thanks, > > Greg > > Just started fiddling with this, using rmagic.py and cythonmagic.py as > references. > > Can anyone point me at documentation for getting at retrieving cell > content by > number and/or tag via the IPython shell instance? > I could be entirely off base here, but my understanding is that the IPython kernel (which provides the magics) used by the notebook doesn't have any concept of cells/the whole notebook in the way you are going to need them. On the other hand, you could look at implementing this as a js extension on top of the notebook via, e.g. https://github.com/cemerick/jsdifflib (I have never used this library, but it came up in a quick google search for js diffing tools). The design of the tool would need to be different, as magics don't live in js, but it seems like you could get where you want to go. ~G > > Thanks, > > Mark > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Gabriel Becker Graduate Student Statistics Department University of California, Davis -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.voorhies at ucsf.edu Mon Dec 2 19:22:27 2013 From: mark.voorhies at ucsf.edu (Mark Voorhies) Date: Mon, 2 Dec 2013 16:22:27 -0800 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: References: <529D0394.6030004@ucsf.edu> <529D1569.2070206@third-bit.com> <529D1DA4.3050302@ucsf.edu> Message-ID: <529D2443.3020004@ucsf.edu> On 12/02/2013 04:10 PM, Gabriel Becker wrote: > On Mon, Dec 2, 2013 at 3:54 PM, Mark Voorhies wrote: > >> On 12/02/2013 03:19 PM, Greg Wilson wrote: >> >>> Thanks, >>> Greg >> >> Just started fiddling with this, using rmagic.py and cythonmagic.py as >> references. >> >> Can anyone point me at documentation for getting at retrieving cell >> content by >> number and/or tag via the IPython shell instance? >> > > I could be entirely off base here, but my understanding is that the IPython > kernel (which provides the magics) used by the notebook doesn't have any > concept of cells/the whole notebook in the way you are going to need them. Looks like shell.history_manager.get_range works for cell numbers (and that the "tags" that Greg mentioned aren't yet implemented). --Mark From gmbecker at ucdavis.edu Mon Dec 2 20:41:40 2013 From: gmbecker at ucdavis.edu (Gabriel Becker) Date: Mon, 2 Dec 2013 17:41:40 -0800 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: <529D2443.3020004@ucsf.edu> References: <529D0394.6030004@ucsf.edu> <529D1569.2070206@third-bit.com> <529D1DA4.3050302@ucsf.edu> <529D2443.3020004@ucsf.edu> Message-ID: Unless I completely miss my guess, shell.history_manager is going to look at the code that has been executed. This is related to what is in the cells, of course, but it isn't the same. Also, the concept of "cell number" if I am correct in assuming you are referring to the numbers that appear next code cells after the run, is that they are not persistently connected to current cell content and don't have a 1-to-1 correspondence to the actual cells during non-headless execution. i.e. if a code cell is executed, then edited and executed again, but the cell number targeted for diffing isn't changed, a magic using shell.history_manager will grab the contents of the cell *when it was run the first time*, as opposed to what I think is conceptually intended, which is what is in the cell now. The metadata tag/JS extension approach can look at the actual contents of any cell at any time, though I'm not claiming it is without its own drawbacks. ~G On Mon, Dec 2, 2013 at 4:22 PM, Mark Voorhies wrote: > On 12/02/2013 04:10 PM, Gabriel Becker wrote: > > On Mon, Dec 2, 2013 at 3:54 PM, Mark Voorhies >wrote: > > > >> On 12/02/2013 03:19 PM, Greg Wilson wrote: > >> > >>> Thanks, > >>> Greg > >> > >> Just started fiddling with this, using rmagic.py and cythonmagic.py as > >> references. > >> > >> Can anyone point me at documentation for getting at retrieving cell > >> content by > >> number and/or tag via the IPython shell instance? > >> > > > > I could be entirely off base here, but my understanding is that the > IPython > > kernel (which provides the magics) used by the notebook doesn't have any > > concept of cells/the whole notebook in the way you are going to need > them. > > Looks like shell.history_manager.get_range works for cell numbers > (and that the "tags" that Greg mentioned aren't yet implemented). > > --Mark > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Gabriel Becker Graduate Student Statistics Department University of California, Davis -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.voorhies at ucsf.edu Mon Dec 2 20:44:49 2013 From: mark.voorhies at ucsf.edu (Mark Voorhies) Date: Mon, 2 Dec 2013 17:44:49 -0800 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: <529D2443.3020004@ucsf.edu> References: <529D0394.6030004@ucsf.edu> <529D1569.2070206@third-bit.com> <529D1DA4.3050302@ucsf.edu> <529D2443.3020004@ucsf.edu> Message-ID: <529D3791.6040401@ucsf.edu> On 12/02/2013 04:22 PM, Mark Voorhies wrote: > On 12/02/2013 04:10 PM, Gabriel Becker wrote: >> On Mon, Dec 2, 2013 at 3:54 PM, Mark Voorhies wrote: >> >>> On 12/02/2013 03:19 PM, Greg Wilson wrote: >>> >>>> Thanks, >>>> Greg >>> >>> Just started fiddling with this, using rmagic.py and cythonmagic.py as >>> references. >>> >>> Can anyone point me at documentation for getting at retrieving cell >>> content by >>> number and/or tag via the IPython shell instance? >>> >> >> I could be entirely off base here, but my understanding is that the IPython >> kernel (which provides the magics) used by the notebook doesn't have any >> concept of cells/the whole notebook in the way you are going to need them. > > Looks like shell.history_manager.get_range works for cell numbers > (and that the "tags" that Greg mentioned aren't yet implemented). Here's a simple prototype: https://github.com/markvoorhies/ipython/tree/diffmagic (https://github.com/markvoorhies/ipython/commit/d24bbb8cd76ba420c61b428b2c9802651f815c6d) "%diff 3 5" Gives a simple text rendering of a line-based diff of cells 3 and 5, e.g.: - def Hello(): + def Hello(s): ? + # A simple function - print "Hello, world" ? ^^^^^ + print "Hello, %s" % s ? ^^ ++++ --Mark > > --Mark > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > . > From mark.voorhies at ucsf.edu Mon Dec 2 23:23:01 2013 From: mark.voorhies at ucsf.edu (Mark Voorhies) Date: Mon, 2 Dec 2013 20:23:01 -0800 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: References: <529D0394.6030004@ucsf.edu> <529D1569.2070206@third-bit.com> <529D1DA4.3050302@ucsf.edu> <529D2443.3020004@ucsf.edu> Message-ID: <529D5CA5.5030604@ucsf.edu> On 12/02/2013 05:41 PM, Gabriel Becker wrote: > Unless I completely miss my guess, shell.history_manager is going to look > at the code that has been executed. This is related to what is in the > cells, of course, but it isn't the same. > > Also, the concept of "cell number" if I am correct in assuming you are > referring to the numbers that appear next code cells after the run, is that > they are not persistently connected to current cell content and don't have > a 1-to-1 correspondence to the actual cells during non-headless execution. > > i.e. if a code cell is executed, then edited and executed again, but the > cell number targeted for diffing isn't changed, a magic using > shell.history_manager will grab the contents of the cell *when it was run > the first time* All of this is correct. > as opposed to what I think is conceptually intended, which > is what is in the cell now. It is true that my approach won't work for anything that the kernel hasn't seen, so it can't do live updates of the diff as cells are edited (and JS _would_ be the right approach for that sort of thing). Outside of that, I think most behaviors can be achieved given the right interface. Here's what the current iteration (https://github.com/markvoorhies/ipython/commit/216ea233413a20061e6c77863798d449be6200c9) can handle: %diff 6 7 --> gives diff of cells [6] and [7]. If [7] is edited (e.g., becoming [9]), the %diff output does not change, even if re-run. %diff 6 9 _does_ diff the new content. %diff 6 --> gives diff of the last cell (e.g., [7]) with cell [6]. If cell [7] is edited and re-run (e.g., becoming [9]), re-running the %diff _does_ diff [6] and [9]. %diff --> gives diff of the last two cells (e.g. [6] and [7]). This is not useful for re-running (because the %diff cell itself becomes part of the history), but is a nice shorthand for the most common use case. %diff A B --> diffs the most recent cells with first lines "##label:A" and "##label:B" respectively. If either cell is edited and re-run, then re-running the %diff _does_ diff the latest content. (This is a place-holder, pending an IPythonic mechanism for labeling cells) The last example allows this work flow: 1) Create cell "##label:A" 2) Create cell "##label:B" (via M-C,M-V in cell A and editing) 3) %diff A B 4) Fix up A and B with little edits as needed, re-running the edited cell and "%diff A B" with each edit. 5) Create cell "##label:C" (via M-C,M-V in cell B and editing) 6) %diff B C 7) Fix up C with little edits as needed, re-running the edited cell and "%diff B C" with each edit. 8) etc. For teaching, I would tend to skip the "little edits" steps, in which case (3) can become "%diff A" and (6) can become "%diff B". --Mark From zvoros at gmail.com Tue Dec 3 06:35:53 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Tue, 03 Dec 2013 12:35:53 +0100 Subject: [IPython-dev] defining latex commands in notebook In-Reply-To: References: <5298AD2A.5080305@gmail.com> <3F65B9E7-48DE-4A8C-BAC2-16432F6CE88E@gmail.com> <5298D43F.5010706@gmail.com> <5298E781.3030900@gmail.com> Message-ID: <529DC219.2060804@gmail.com> The problem is somewhere else. sphinx version 1.2b3 behaves in the same way, raw cells are hashed. On 29/11/13 21:33, Damon Allen wrote: > I have1.2b1. > > Damon > > On Fri, Nov 29, 2013 at 2:14 PM, Zolt?n V?r?s > wrote: > > > On 29/11/13 19:58, Damon Allen wrote: >> My version and yours may be different. I'm just using the 1.1 >> release rather than >> the current 2.0 version. Alternatively, it might be an issue >> with sphinx. I have installed sphinx and tornado using pip >> rather than using what is in the repositories. > I suspect sphinx then. (I guess, tornado doesn't matter in this > case.) My version is 1.1.3. Is yours the same? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zvoros at gmail.com Tue Dec 3 07:06:08 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Tue, 03 Dec 2013 13:06:08 +0100 Subject: [IPython-dev] defining latex commands in notebook In-Reply-To: References: <5298AD2A.5080305@gmail.com> <3F65B9E7-48DE-4A8C-BAC2-16432F6CE88E@gmail.com> <5298D43F.5010706@gmail.com> <5298E781.3030900@gmail.com> Message-ID: <529DC930.5060708@gmail.com> Sorry for the noise, it turned out that sphinx version 1.2b3 does not place hash marks at the beginning of lines. However, this doesn't solve the original problem, namely, that \newcommand cannot be used in a consistent way in both the notebook, and the latex output: if I place $\newcommand{}$ in a markdown cell, then it will be passed to latex as $\newcommand{}$, while if I put that in a raw cell, it is not processed by mathjax. For now I can just run change the type of the cell before converting, but this is obviously not an ideal solution... Cheers, Zolt?n On 29/11/13 21:33, Damon Allen wrote: > I have1.2b1. > > Damon > > On Fri, Nov 29, 2013 at 2:14 PM, Zolt?n V?r?s > wrote: > > > On 29/11/13 19:58, Damon Allen wrote: >> My version and yours may be different. I'm just using the 1.1 >> release rather than >> the current 2.0 version. Alternatively, it might be an issue >> with sphinx. I have installed sphinx and tornado using pip >> rather than using what is in the repositories. > I suspect sphinx then. (I guess, tornado doesn't matter in this > case.) My version is 1.1.3. Is yours the same? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zvoros at gmail.com Tue Dec 3 08:46:50 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Tue, 03 Dec 2013 14:46:50 +0100 Subject: [IPython-dev] old latex templates for nbconvert? Message-ID: <529DE0CA.8090806@gmail.com> Hi all, Is there some sort of repository for old nbconverter templates? I used to use the "basic" template that produced very nice output with coloured frames, but the new article/report templates are really stripped down and their output is rather flat. I think it would actually make sense to have a collection of various templates (in the wiki, perhaps?) for the benefit of everyone. Cheers, Zolt?n From konrad.hinsen at fastmail.net Tue Dec 3 11:34:01 2013 From: konrad.hinsen at fastmail.net (Konrad Hinsen) Date: Tue, 3 Dec 2013 17:34:01 +0100 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition Message-ID: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> Thomas Kluyver writes: > Have any of us come across ActivePapers? It combines code, data and > documentation into an HDF5 file to be uploaded as supplementary info for a > journal article. It looks like it focuses more on the storage and > reusability than the presentation and interactivity of IPython notebooks - > so there could be interesting opportunities for integration, such as an > IPython notebook backed by code and data stored in this form. > > https://bitbucket.org/khinsen/active_papers_py/wiki/Home I have had exactly that idea as well and I was planning to discuss it here. You were a bit faster :-) Note that ActivePapers already has a very basic IPython interface (try "aptool ipython"), but that doesn't include notebooks, just an IPython prompt for exploring the data in an ActivePaper. Note also that ActivePapers has move to Github and has its own Web site: http://www.activepapers.org/ My idea for integrating IPython notebooks into ActivePapers is to store notebooks inside the ActivePapers HDF5 files and treat each cell in a notebook like a calclet. Apart from writing all the interface code, the main challenge is how and where to store provenance tracking information. In ActivePapers, a calclet is an HDF5 dataset (containing a string) whose metadata contains the dependency information. One approach would eb to keep this representation and thus turn a notebook into a sequence of HDF5 datasets. At the other extreme, one could store notebooks in their native JSON representation inside an ActivePaper and insert the dependency information in the JSON code, or in a supplementary HDF5 dataset. If anyone here is interested in contributing to this kind of work, I'd be happy to discuss the details. Konrad. From damontallen at gmail.com Tue Dec 3 11:51:42 2013 From: damontallen at gmail.com (Damon Allen) Date: Tue, 3 Dec 2013 11:51:42 -0500 Subject: [IPython-dev] defining latex commands in notebook In-Reply-To: <529DC219.2060804@gmail.com> References: <5298AD2A.5080305@gmail.com> <3F65B9E7-48DE-4A8C-BAC2-16432F6CE88E@gmail.com> <5298D43F.5010706@gmail.com> <5298E781.3030900@gmail.com> <529DC219.2060804@gmail.com> Message-ID: What is the exact command you are using to do the conversion? I always use something like: ipython nbconvert "My_Notebook.ipynb" --to latex > and I have found that if I use ipython3 instead of ipython I have conversion issues. This seems to be related to that fact that I haven't been able to update sphinx for Python 3. Damon On Tue, Dec 3, 2013 at 6:35 AM, Zolt?n V?r?s wrote: > The problem is somewhere else. sphinx version 1.2b3 behaves in the same > way, raw cells are hashed. > > > > On 29/11/13 21:33, Damon Allen wrote: > > I have 1.2b1. > > Damon > > On Fri, Nov 29, 2013 at 2:14 PM, Zolt?n V?r?s wrote: > >> >> On 29/11/13 19:58, Damon Allen wrote: >> >> My version and yours may be different. I'm just using the 1.1 releaserather than the current 2.0 version. Alternatively, it might be an issue >> with sphinx. I have installed sphinx and tornado using pip rather than >> using what is in the repositories. >> >> I suspect sphinx then. (I guess, tornado doesn't matter in this case.) >> My version is 1.1.3. Is yours the same? >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Tue Dec 3 13:12:51 2013 From: benjaminrk at gmail.com (MinRK) Date: Tue, 3 Dec 2013 10:12:51 -0800 Subject: [IPython-dev] old latex templates for nbconvert? In-Reply-To: <529DE0CA.8090806@gmail.com> References: <529DE0CA.8090806@gmail.com> Message-ID: Yup, it?s here On Tue, Dec 3, 2013 at 5:46 AM, Zolt?n V?r?s wrote: > Hi all, > > Is there some sort of repository for old nbconverter templates? I used > to use the "basic" template that produced very nice output with coloured > frames, but the new article/report templates are really stripped down > and their output is rather flat. I think it would actually make sense to > have a collection of various templates (in the wiki, perhaps?) for the > benefit of everyone. > > Cheers, > Zolt?n > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zvoros at gmail.com Tue Dec 3 13:47:31 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Tue, 03 Dec 2013 19:47:31 +0100 Subject: [IPython-dev] defining latex commands in notebook In-Reply-To: References: <5298AD2A.5080305@gmail.com> <3F65B9E7-48DE-4A8C-BAC2-16432F6CE88E@gmail.com> <5298D43F.5010706@gmail.com> <5298E781.3030900@gmail.com> <529DC219.2060804@gmail.com> Message-ID: <529E2743.4080102@gmail.com> As I said in my e-mail, I had sorted out the issue with the hash marks, 1.2b3 doesn't produce that. The problem is that I don't see how latex definitions can be made to work in both the notebook, and the pdf output. If I put the definition in a raw cell, that won't be evaluated by mathjax, but it will be exported correctly. But if I define the macro in a markdown cell, then it won't be exported properly. I can't believe that I am the only person with this problem:) Zolt?n On 03/12/13 17:51, Damon Allen wrote: > What is the exact command you are using to do the conversion? I > always use something like: > > ipython nbconvert "My_Notebook.ipynb" --to latex > > > and I have found that if I use ipython3 instead of ipython I have > conversion issues. This seems to be related to that fact that I > haven't been able to update sphinx for Python 3. > > Damon > > > > > On Tue, Dec 3, 2013 at 6:35 AM, Zolt?n V?r?s > wrote: > > The problem is somewhere else. sphinx version 1.2b3 behaves in the > same way, raw cells are hashed. > > > > On 29/11/13 21:33, Damon Allen wrote: >> I have1.2b1. >> >> Damon >> >> On Fri, Nov 29, 2013 at 2:14 PM, Zolt?n V?r?s > > wrote: >> >> >> On 29/11/13 19:58, Damon Allen wrote: >>> My version and yours may be different. I'm just using the >>> 1.1 release >>> rather than the current 2.0 version. Alternatively, it >>> might be an issue with sphinx. I have installed sphinx and >>> tornado using pip rather than using what is in the repositories. >> I suspect sphinx then. (I guess, tornado doesn't matter in >> this case.) My version is 1.1.3. Is yours the same? >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From zvoros at gmail.com Tue Dec 3 13:48:08 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Tue, 03 Dec 2013 19:48:08 +0100 Subject: [IPython-dev] old latex templates for nbconvert? In-Reply-To: References: <529DE0CA.8090806@gmail.com> Message-ID: <529E2768.8060800@gmail.com> OK, thanks! Should I just try to write it up in the wiki, or you think that it's useless? On 03/12/13 19:12, MinRK wrote: > > Yup, it's here > > > > On Tue, Dec 3, 2013 at 5:46 AM, Zolt?n V?r?s > wrote: > > Hi all, > > Is there some sort of repository for old nbconverter templates? I used > to use the "basic" template that produced very nice output with > coloured > frames, but the new article/report templates are really stripped down > and their output is rather flat. I think it would actually make > sense to > have a collection of various templates (in the wiki, perhaps?) for the > benefit of everyone. > > Cheers, > Zolt?n > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Dec 3 19:10:05 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 3 Dec 2013 16:10:05 -0800 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> Message-ID: Hi Konrad, On 3 December 2013 08:34, Konrad Hinsen wrote: > My idea for integrating IPython notebooks into ActivePapers is to > store notebooks inside the ActivePapers HDF5 files and treat each cell > in a notebook like a calclet. Apart from writing all the interface > code, the main challenge is how and where to store provenance tracking > information. In ActivePapers, a calclet is an HDF5 dataset (containing > a string) whose metadata contains the dependency information. One > approach would eb to keep this representation and thus turn a notebook > into a sequence of HDF5 datasets. At the other extreme, one could > store notebooks in their native JSON representation inside an > ActivePaper and insert the dependency information in the JSON code, or > in a supplementary HDF5 dataset. > Our JSON format does have a concept of cell metadata, where this kind of information could be stored. But it would also be interesting to see notebook cells with contents stored in HDF5 datasets. We have considered the possibility of having cells stored externally to the notebook, so you could have a cell whose contents were stored in a plain Python file, which could then be shared between multiple notebooks. It shouldn't be too hard to work with notebooks stored in a different format - you'd need to write a custom notebook server which translates to a JSON structure to load a notebook, and translates back to your own format when it's saved. That's handled in the FileNotebookManager class: https://github.com/ipython/ipython/blob/master/IPython/html/services/notebooks/filenbmanager.py#L37 Best wishes, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehmatthes at gmail.com Wed Dec 4 11:35:20 2013 From: ehmatthes at gmail.com (Eric Matthes) Date: Wed, 4 Dec 2013 07:35:20 -0900 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: <529D5CA5.5030604@ucsf.edu> References: <529D0394.6030004@ucsf.edu> <529D1569.2070206@third-bit.com> <529D1DA4.3050302@ucsf.edu> <529D2443.3020004@ucsf.edu> <529D5CA5.5030604@ucsf.edu> Message-ID: For the record, here's what I ended up implementing. I realized that adding comments before and after every line/s to be highlighted was ridiculous, so I implemented the functionality I saw last time I used wordpress to write a tutorial. At the beginning of a code block, I write a comment in the following format, specifying which lines will be highlighted: ###highlight=[1,2,7,8,9] The raw notebook files do not end up with any highlighting, that's something I can't implement right now. I run nbconvert, and then I wrote a script [0]that parses these html files. Whenever it finds this ###highlight directive, it removes the directive itself and then adds
...
tags to the appropriate lines in that code block. I think the final result [1]looks much better than it did without highlighting, for the purposes of a tutorial. The raw notebook [2]suffers slightly in that it has these ###highlight= comments on the first lines of many cells, but that's worthwhile to me in the short term, and it would be easy to write a quick script to remove them if needed. (This notebook might take a long time to render, because it's a bunch of Basemap examples.) Happy to hear feedback if there is a way to make my approach a little more in line with how IPython does things. Eric [0] - https://github.com/ehmatthes/intro_programming/blob/master/scripts/highlight_code.py [1] - http://introtopython.org/visualization_earthquakes.html [2] - http://nbviewer.ipython.org/urls/raw.github.com/ehmatthes/intro_programming/master/notebooks/visualization_earthquakes.ipynb On Mon, Dec 2, 2013 at 7:23 PM, Mark Voorhies wrote: > On 12/02/2013 05:41 PM, Gabriel Becker wrote: > > Unless I completely miss my guess, shell.history_manager is going to look > > at the code that has been executed. This is related to what is in the > > cells, of course, but it isn't the same. > > > > Also, the concept of "cell number" if I am correct in assuming you are > > referring to the numbers that appear next code cells after the run, is > that > > they are not persistently connected to current cell content and don't > have > > a 1-to-1 correspondence to the actual cells during non-headless > execution. > > > > i.e. if a code cell is executed, then edited and executed again, but the > > cell number targeted for diffing isn't changed, a magic using > > shell.history_manager will grab the contents of the cell *when it was run > > the first time* > > All of this is correct. > > > as opposed to what I think is conceptually intended, which > > is what is in the cell now. > > It is true that my approach won't work for anything that the kernel hasn't > seen, > so it can't do live updates of the diff as cells are edited (and JS > _would_ be > the right approach for that sort of thing). Outside of that, I think most > behaviors can be achieved given the right interface. > > Here's what the current iteration ( > https://github.com/markvoorhies/ipython/commit/216ea233413a20061e6c77863798d449be6200c9 > ) > can handle: > > %diff 6 7 > > --> gives diff of cells [6] and [7]. If [7] is edited (e.g., becoming > [9]), the %diff output does not change, > even if re-run. %diff 6 9 _does_ diff the new content. > > %diff 6 > > --> gives diff of the last cell (e.g., [7]) with cell [6]. If cell [7] is > edited and re-run (e.g., becoming [9]), > re-running the %diff _does_ diff [6] and [9]. > > %diff > > --> gives diff of the last two cells (e.g. [6] and [7]). This is not > useful for re-running (because the %diff cell > itself becomes part of the history), but is a nice shorthand for the > most common use case. > > %diff A B > > --> diffs the most recent cells with first lines "##label:A" and > "##label:B" respectively. If either cell is > edited and re-run, then re-running the %diff _does_ diff the latest > content. > (This is a place-holder, pending an IPythonic mechanism for labeling > cells) > > The last example allows this work flow: > > 1) Create cell "##label:A" > 2) Create cell "##label:B" (via M-C,M-V in cell A and editing) > 3) %diff A B > 4) Fix up A and B with little edits as needed, re-running the edited cell > and "%diff A B" with each edit. > 5) Create cell "##label:C" (via M-C,M-V in cell B and editing) > 6) %diff B C > 7) Fix up C with little edits as needed, re-running the edited cell and > "%diff B C" with each edit. > 8) etc. > > For teaching, I would tend to skip the "little edits" steps, in which case > (3) can become "%diff A" > and (6) can become "%diff B". > > --Mark > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmbecker at ucdavis.edu Wed Dec 4 13:27:32 2013 From: gmbecker at ucdavis.edu (Gabriel Becker) Date: Wed, 4 Dec 2013 10:27:32 -0800 Subject: [IPython-dev] Highlight lines of code in code cells? In-Reply-To: <529D5CA5.5030604@ucsf.edu> References: <529D0394.6030004@ucsf.edu> <529D1569.2070206@third-bit.com> <529D1DA4.3050302@ucsf.edu> <529D2443.3020004@ucsf.edu> <529D5CA5.5030604@ucsf.edu> Message-ID: Mark, This seems reasonable, and like a useful piece of functionality to have. TBH I'd prefer that computations *in* documents and computations *on* documents be more separate conceptually, but that may just be my bias stemming from what my research is about (dynamic data analysis documents that can be operated on and what that buys us). Nice work. ~G On Mon, Dec 2, 2013 at 8:23 PM, Mark Voorhies wrote: > On 12/02/2013 05:41 PM, Gabriel Becker wrote: > > Unless I completely miss my guess, shell.history_manager is going to look > > at the code that has been executed. This is related to what is in the > > cells, of course, but it isn't the same. > > > > Also, the concept of "cell number" if I am correct in assuming you are > > referring to the numbers that appear next code cells after the run, is > that > > they are not persistently connected to current cell content and don't > have > > a 1-to-1 correspondence to the actual cells during non-headless > execution. > > > > i.e. if a code cell is executed, then edited and executed again, but the > > cell number targeted for diffing isn't changed, a magic using > > shell.history_manager will grab the contents of the cell *when it was run > > the first time* > > All of this is correct. > > > as opposed to what I think is conceptually intended, which > > is what is in the cell now. > > It is true that my approach won't work for anything that the kernel hasn't > seen, > so it can't do live updates of the diff as cells are edited (and JS > _would_ be > the right approach for that sort of thing). Outside of that, I think most > behaviors can be achieved given the right interface. > > Here's what the current iteration ( > https://github.com/markvoorhies/ipython/commit/216ea233413a20061e6c77863798d449be6200c9 > ) > can handle: > > %diff 6 7 > > --> gives diff of cells [6] and [7]. If [7] is edited (e.g., becoming > [9]), the %diff output does not change, > even if re-run. %diff 6 9 _does_ diff the new content. > > %diff 6 > > --> gives diff of the last cell (e.g., [7]) with cell [6]. If cell [7] is > edited and re-run (e.g., becoming [9]), > re-running the %diff _does_ diff [6] and [9]. > > %diff > > --> gives diff of the last two cells (e.g. [6] and [7]). This is not > useful for re-running (because the %diff cell > itself becomes part of the history), but is a nice shorthand for the > most common use case. > > %diff A B > > --> diffs the most recent cells with first lines "##label:A" and > "##label:B" respectively. If either cell is > edited and re-run, then re-running the %diff _does_ diff the latest > content. > (This is a place-holder, pending an IPythonic mechanism for labeling > cells) > > The last example allows this work flow: > > 1) Create cell "##label:A" > 2) Create cell "##label:B" (via M-C,M-V in cell A and editing) > 3) %diff A B > 4) Fix up A and B with little edits as needed, re-running the edited cell > and "%diff A B" with each edit. > 5) Create cell "##label:C" (via M-C,M-V in cell B and editing) > 6) %diff B C > 7) Fix up C with little edits as needed, re-running the edited cell and > "%diff B C" with each edit. > 8) etc. > > For teaching, I would tend to skip the "little edits" steps, in which case > (3) can become "%diff A" > and (6) can become "%diff B". > > --Mark > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Gabriel Becker Graduate Student Statistics Department University of California, Davis -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Wed Dec 4 15:23:33 2013 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 4 Dec 2013 12:23:33 -0800 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> Message-ID: Yes, it shouldn't be difficult at all to create an ActivePapers backend for storing notebooks. But in some ways it would also be nice to have this type of conversion in nbconvert. Cheers, Brian On Tue, Dec 3, 2013 at 4:10 PM, Thomas Kluyver wrote: > Hi Konrad, > > On 3 December 2013 08:34, Konrad Hinsen wrote: >> >> My idea for integrating IPython notebooks into ActivePapers is to >> store notebooks inside the ActivePapers HDF5 files and treat each cell >> in a notebook like a calclet. Apart from writing all the interface >> code, the main challenge is how and where to store provenance tracking >> information. In ActivePapers, a calclet is an HDF5 dataset (containing >> a string) whose metadata contains the dependency information. One >> approach would eb to keep this representation and thus turn a notebook >> into a sequence of HDF5 datasets. At the other extreme, one could >> store notebooks in their native JSON representation inside an >> ActivePaper and insert the dependency information in the JSON code, or >> in a supplementary HDF5 dataset. > > > Our JSON format does have a concept of cell metadata, where this kind of > information could be stored. But it would also be interesting to see > notebook cells with contents stored in HDF5 datasets. We have considered the > possibility of having cells stored externally to the notebook, so you could > have a cell whose contents were stored in a plain Python file, which could > then be shared between multiple notebooks. > > It shouldn't be too hard to work with notebooks stored in a different format > - you'd need to write a custom notebook server which translates to a JSON > structure to load a notebook, and translates back to your own format when > it's saved. That's handled in the FileNotebookManager class: > https://github.com/ipython/ipython/blob/master/IPython/html/services/notebooks/filenbmanager.py#L37 > > Best wishes, > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From fperez.net at gmail.com Thu Dec 5 01:22:43 2013 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 4 Dec 2013 22:22:43 -0800 Subject: [IPython-dev] nbviewer.ipython.org In-Reply-To: <82B5B8A3-9FAC-485E-89FF-A498F2609FE3@gmail.com> References: <6F528905-8B77-4663-B8C4-1C285E173B95@gmail.com> <529CAAFF.1090800@sifflez.org> <82B5B8A3-9FAC-485E-89FF-A498F2609FE3@gmail.com> Message-ID: Hi all we just moved it today over to a Rackspace setup (thanks Rackspace and Kyle Kelly for all the support and work). We made a small mistake with DNS that kept it offline for a couple of hours, but the right records should have propagated by now. The situation should be significantly improved, please do let us know if that's not the case. Cheers, f On Mon, Dec 2, 2013 at 8:02 AM, Matthias BUSSONNIER < bussonniermatthias at gmail.com> wrote: > > Le 2 d?c. 2013 ? 16:45, Pablo Oliveira a ?crit : > > On 12/02/2013 02:34 PM, Matthias BUSSONNIER wrote: > > Yes, if you can send us which notebook is throwing error > > and what kind of error it would be useful. > > > I'm observing the same behavior with the following notebook (one of the > example notebooks on the front page), > > http://nbviewer.ipython.org/github/fperez/nipy-notebooks/blob/master/exploring_r_formula.ipynb > > Did you get the nice 404 or 500 error that look like nbviewer > > or the ugly purple one. > > > It throws a purple "Application Error" error page. > > Also you can alway try alternate/dev versions of nbviewer if you need it : > > > http://nbtest.herokuapp.com/ > > http://gistpynb.herokuapp.com/ > > > On nbtest.herokuapp.com, it works. > On gistpynb.herokuapp.com, it throws an "Error 400: Bad Request". > > > Indeed, each of thoses red line represent a period of 2 min where nbviewer > is down. > Strange thing is, new-relic did not alert us. > > Will try to investigate. > > Thanks. > -- > Matthias > > > > > Regards, > > Pablo > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Capture d??cran 2013-12-02 ? 17.00.29.png Type: image/png Size: 48738 bytes Desc: not available URL: From konrad.hinsen at fastmail.net Thu Dec 5 12:13:08 2013 From: konrad.hinsen at fastmail.net (Konrad Hinsen) Date: Thu, 5 Dec 2013 18:13:08 +0100 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> Message-ID: <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> Hi Thomas, > Our JSON format does have a concept of cell metadata, where this > kind of information could be stored. But it would also be > interesting to see notebook cells with contents stored in HDF5 > datasets. One advantage of storing notebooks cells as ActivePapers calclets (and thus as HDF5 datasets) is the possibility to open any ActivePapers file as a notebook, even if it was generated in a different way. But in the end such decisions can't be taken in theory. I'd better shut up until I actually start trying out something. > We have considered the possibility of having cells stored > externally to the notebook, so you could have a cell whose contents > were stored in a plain Python file, which could then be shared > between multiple notebooks. An interesting idea, but potentially a data management headache. What do you do if an external file disappears? > It shouldn't be too hard to work with notebooks stored in a different format - > you'd need to write a custom notebook server which translates to a JSON structure > to load a notebook, and translates back to your own format when it's saved. That's > handled in the FileNotebookManager class: > https://github.com/ipython/ipython/blob/master/IPython/html/services/notebooks/ > filenbmanager.py#L37 Thanks for the pointer! Konrad From konrad.hinsen at fastmail.net Thu Dec 5 12:16:47 2013 From: konrad.hinsen at fastmail.net (Konrad Hinsen) Date: Thu, 5 Dec 2013 18:16:47 +0100 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> Message-ID: <21152.46335.759072.463264@Konrad-Hinsens-MacBook-Pro.local> Hi Brian, > Yes, it shouldn't be difficult at all to create an ActivePapers > backend for storing notebooks. But in some ways it would also be nice > to have this type of conversion in nbconvert. Could you explain what you mean by "this type of conversion"? Conversion from what to what? Konrad From takowl at gmail.com Thu Dec 5 12:55:22 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 5 Dec 2013 09:55:22 -0800 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> Message-ID: On 5 December 2013 09:13, Konrad Hinsen wrote: > One advantage of storing notebooks cells as ActivePapers calclets (and > thus as HDF5 datasets) is the possibility to open any ActivePapers > file as a notebook, even if it was generated in a different way. > Do ActivePapers calclets have a notion of order? i.e. this should be run before that? > > We have considered the possibility of having cells stored > > externally to the notebook, so you could have a cell whose contents > > were stored in a plain Python file, which could then be shared > > between multiple notebooks. > > An interesting idea, but potentially a data management headache. > What do you do if an external file disappears? Presumably it would be an empty cell, or show an error. This was only a concept, we've not implemented it at all. But the paradigm is familiar enough from any file that can reference external resources, like images in an HTML document. Best wishes, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Thu Dec 5 15:37:07 2013 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 5 Dec 2013 12:37:07 -0800 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: <21152.46335.759072.463264@Konrad-Hinsens-MacBook-Pro.local> References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46335.759072.463264@Konrad-Hinsens-MacBook-Pro.local> Message-ID: Konrad, By conversion, I meant between ActivePapers hdf5 files and IPython notebooks. Cheers, Brian On Thu, Dec 5, 2013 at 9:16 AM, Konrad Hinsen wrote: > Hi Brian, > > > Yes, it shouldn't be difficult at all to create an ActivePapers > > backend for storing notebooks. But in some ways it would also be nice > > to have this type of conversion in nbconvert. > > Could you explain what you mean by "this type of conversion"? > Conversion from what to what? > > Konrad > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From gaitan at phasety.com Thu Dec 5 16:00:04 2013 From: gaitan at phasety.com (=?UTF-8?B?TWFydMOtbiBHYWl0w6Fu?=) Date: Thu, 5 Dec 2013 19:00:04 -0200 Subject: [IPython-dev] IPy notebook as simple test system? Message-ID: Hi everybody I need to add some tests to fortran-magic [1] and I realized that, basically, I should test each example in the documentation which is already done as a notebook[2] . So, I got this idea: make a a way to run the code cells of a notebook and compare the calculated output against the stored output in the notebook. Looking for something like this I found [3] and [4], which facilitate the use of doctests, but it's not exactly the same. Conceptually, this are also doctests, but the main difference with what I imagine is that we don't bother writing fake code inside the actual code: No docstrings at all, just use the cells we already have. Of course the comparison should be flexible. Doctest has directives to allow ellipsis or normalizations, for example [5]. I think this could be simplified with an UI to select the portions of the output that must be compared and store this selection as cell metadata. What do you think? [1] https://github.com/mgaitan/fortran_magic [2] http://nbviewer.ipython.org/github/mgaitan/fortran_magic/blob/master/documentation.ipynb [3] http://catherinedevlin.blogspot.co.uk/2012/10/im-increasingly-amazed-at-ipython.html [4] https://github.com/tkf/ipython-doctestmagic [5] http://docs.python.org/2/library/doctest.html#directives -------------- next part -------------- An HTML attachment was scrubbed... URL: From aron at ahmadia.net Thu Dec 5 16:56:59 2013 From: aron at ahmadia.net (Aron Ahmadia) Date: Thu, 5 Dec 2013 16:56:59 -0500 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46335.759072.463264@Konrad-Hinsens-MacBook-Pro.local> Message-ID: Taking this thread into a slightly more general nature, I'm interested in learning more about how the IPython front-end Javascript obtains its information for populating cells. This opens the door for connecting to other data back-ends without necessarily creating a heavy-weight notebook. Would the developers be up for perhaps hosting a G+ hangout at some point to chat about this? Cheers, A On Thu, Dec 5, 2013 at 3:37 PM, Brian Granger wrote: > Konrad, > > By conversion, I meant between ActivePapers hdf5 files and IPython > notebooks. > > Cheers, > > Brian > > On Thu, Dec 5, 2013 at 9:16 AM, Konrad Hinsen > wrote: > > Hi Brian, > > > > > Yes, it shouldn't be difficult at all to create an ActivePapers > > > backend for storing notebooks. But in some ways it would also be nice > > > to have this type of conversion in nbconvert. > > > > Could you explain what you mean by "this type of conversion"? > > Conversion from what to what? > > > > Konrad > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jakevdp at cs.washington.edu Thu Dec 5 17:16:26 2013 From: jakevdp at cs.washington.edu (Jacob Vanderplas) Date: Thu, 5 Dec 2013 14:16:26 -0800 Subject: [IPython-dev] Notebook: grabbing HTML for an arbitrary object Message-ID: Hi, I'm hacking on the notebook a bit and ran into something I don't know how to do. I know that for user-defined objects, one can define a ``_repr_html_()`` method, and then calling ``display()`` in the notebook will use this function to create an html representation. But how can I get the HTML representation for a built-in object which does not define ``_repr_html_``? To be more concrete, I want a function ``get_html_rep()`` so that the following two cells show the same result in the notebook, for an arbitrary object ``myobj``: In [1]: from IPython.display import HTML html = get_html_rep(myobj) HTML(html) In [2]: from IPython.display import display display(myobj) Does a function like this exist? Can it be easily created? Any help would be appreciated. Thanks! Jake -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Thu Dec 5 17:34:50 2013 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 5 Dec 2013 14:34:50 -0800 Subject: [IPython-dev] Notebook: grabbing HTML for an arbitrary object In-Reply-To: References: Message-ID: Example is in this notebook: http://nbviewer.ipython.org/github/ipython/ipython/blob/master/examples/notebooks/Custom%20Display%20Logic.ipynb On Thu, Dec 5, 2013 at 2:16 PM, Jacob Vanderplas wrote: > Hi, > I'm hacking on the notebook a bit and ran into something I don't know how to > do. > > I know that for user-defined objects, one can define a ``_repr_html_()`` > method, and then calling ``display()`` in the notebook will use this > function to create an html representation. > > But how can I get the HTML representation for a built-in object which does > not define ``_repr_html_``? > > To be more concrete, I want a function ``get_html_rep()`` so that the > following two cells show the same result in the notebook, for an arbitrary > object ``myobj``: > > In [1]: > from IPython.display import HTML > html = get_html_rep(myobj) > HTML(html) > > In [2]: > from IPython.display import display > display(myobj) > > Does a function like this exist? Can it be easily created? Any help would > be appreciated. Thanks! > Jake > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From benjaminrk at gmail.com Thu Dec 5 17:42:35 2013 From: benjaminrk at gmail.com (MinRK) Date: Thu, 5 Dec 2013 14:42:35 -0800 Subject: [IPython-dev] IPy notebook as simple test system? In-Reply-To: References: Message-ID: I wrote a toy script a while ago, to show that this kind of thing is feasible: https://gist.github.com/minrk/2620735 -MinRK On Thu, Dec 5, 2013 at 1:00 PM, Mart?n Gait?n wrote: > Hi everybody > > I need to add some tests to fortran-magic [1] and I realized that, > basically, I should test each example in the documentation which is already > done as a notebook[2] . > > So, I got this idea: make a a way to run the code cells of a notebook and > compare the calculated output against the stored output in the notebook. > > Looking for something like this I found [3] and [4], which facilitate the > use of doctests, but it's not exactly the same. Conceptually, this are also > doctests, but the main difference with what I imagine is that we don't > bother writing fake code inside the actual code: No docstrings at all, just > use the cells we already have. > > Of course the comparison should be flexible. Doctest has directives to > allow ellipsis or normalizations, for example [5]. I think this could be > simplified with an UI to select the portions of the output that must be > compared and store this selection as cell metadata. > > What do you think? > > [1] https://github.com/mgaitan/fortran_magic > [2] > http://nbviewer.ipython.org/github/mgaitan/fortran_magic/blob/master/documentation.ipynb > [3] > http://catherinedevlin.blogspot.co.uk/2012/10/im-increasingly-amazed-at-ipython.html > [4] https://github.com/tkf/ipython-doctestmagic > [5] http://docs.python.org/2/library/doctest.html#directives > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan at sun.ac.za Thu Dec 5 17:44:13 2013 From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=) Date: Fri, 6 Dec 2013 00:44:13 +0200 Subject: [IPython-dev] Notebook: grabbing HTML for an arbitrary object In-Reply-To: References: Message-ID: On Fri, Dec 6, 2013 at 12:16 AM, Jacob Vanderplas wrote: > To be more concrete, I want a function ``get_html_rep()`` so that the > following two cells show the same result in the notebook, for an arbitrary > object ``myobj``: > > In [1]: > from IPython.display import HTML > html = get_html_rep(myobj) > HTML(html) Perhaps something like this will work: from IPython import InteractiveShell def get_html_rep(obj): shell = InteractiveShell.instance() format = shell.display_formatter.format rendered, _ = format(obj, ('text/plain', 'text/html')) return rendered['text/html'] if 'text/html' in rendered else rendered['text/plain'] St?fan From asmeurer at gmail.com Thu Dec 5 17:44:43 2013 From: asmeurer at gmail.com (Aaron Meurer) Date: Thu, 5 Dec 2013 15:44:43 -0700 Subject: [IPython-dev] IPy notebook as simple test system? In-Reply-To: References: Message-ID: Here's the relevant issue: https://github.com/ipython/ipython/issues/1195. I would love to see something like this integrated into IPython. Aaron Meurer On Thu, Dec 5, 2013 at 3:42 PM, MinRK wrote: > I wrote a toy script a while ago, to show that this kind of thing is > feasible: > > https://gist.github.com/minrk/2620735 > > -MinRK > > > On Thu, Dec 5, 2013 at 1:00 PM, Mart?n Gait?n wrote: >> >> Hi everybody >> >> I need to add some tests to fortran-magic [1] and I realized that, >> basically, I should test each example in the documentation which is already >> done as a notebook[2] . >> >> So, I got this idea: make a a way to run the code cells of a notebook and >> compare the calculated output against the stored output in the notebook. >> >> Looking for something like this I found [3] and [4], which facilitate the >> use of doctests, but it's not exactly the same. Conceptually, this are also >> doctests, but the main difference with what I imagine is that we don't >> bother writing fake code inside the actual code: No docstrings at all, just >> use the cells we already have. >> >> Of course the comparison should be flexible. Doctest has directives to >> allow ellipsis or normalizations, for example [5]. I think this could be >> simplified with an UI to select the portions of the output that must be >> compared and store this selection as cell metadata. >> >> What do you think? >> >> [1] https://github.com/mgaitan/fortran_magic >> [2] >> http://nbviewer.ipython.org/github/mgaitan/fortran_magic/blob/master/documentation.ipynb >> [3] >> http://catherinedevlin.blogspot.co.uk/2012/10/im-increasingly-amazed-at-ipython.html >> [4] https://github.com/tkf/ipython-doctestmagic >> [5] http://docs.python.org/2/library/doctest.html#directives >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From gvwilson at third-bit.com Thu Dec 5 18:13:16 2013 From: gvwilson at third-bit.com (Greg Wilson) Date: Thu, 05 Dec 2013 18:13:16 -0500 Subject: [IPython-dev] IPy notebook as simple test system? In-Reply-To: References: Message-ID: <52A1088C.3060308@third-bit.com> Is this another case where named cells would be useful? %%test name_of_cell_to_compare_code_against ...code... and then further down, a Markdown cell with a name ? Thanks, Greg On 2013-12-05 5:44 PM, Aaron Meurer wrote: > Here's the relevant issue: > https://github.com/ipython/ipython/issues/1195. I would love to see > something like this integrated into IPython. > > Aaron Meurer > > On Thu, Dec 5, 2013 at 3:42 PM, MinRK wrote: >> I wrote a toy script a while ago, to show that this kind of thing is >> feasible: >> >> https://gist.github.com/minrk/2620735 >> >> -MinRK >> >> >> On Thu, Dec 5, 2013 at 1:00 PM, Mart?n Gait?n wrote: >>> Hi everybody >>> >>> I need to add some tests to fortran-magic [1] and I realized that, >>> basically, I should test each example in the documentation which is already >>> done as a notebook[2] . >>> >>> So, I got this idea: make a a way to run the code cells of a notebook and >>> compare the calculated output against the stored output in the notebook. >>> >>> Looking for something like this I found [3] and [4], which facilitate the >>> use of doctests, but it's not exactly the same. Conceptually, this are also >>> doctests, but the main difference with what I imagine is that we don't >>> bother writing fake code inside the actual code: No docstrings at all, just >>> use the cells we already have. >>> >>> Of course the comparison should be flexible. Doctest has directives to >>> allow ellipsis or normalizations, for example [5]. I think this could be >>> simplified with an UI to select the portions of the output that must be >>> compared and store this selection as cell metadata. >>> >>> What do you think? >>> >>> [1] https://github.com/mgaitan/fortran_magic >>> [2] >>> http://nbviewer.ipython.org/github/mgaitan/fortran_magic/blob/master/documentation.ipynb >>> [3] >>> http://catherinedevlin.blogspot.co.uk/2012/10/im-increasingly-amazed-at-ipython.html >>> [4] https://github.com/tkf/ipython-doctestmagic >>> [5] http://docs.python.org/2/library/doctest.html#directives >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From asmeurer at gmail.com Thu Dec 5 18:57:02 2013 From: asmeurer at gmail.com (Aaron Meurer) Date: Thu, 5 Dec 2013 16:57:02 -0700 Subject: [IPython-dev] IPy notebook as simple test system? In-Reply-To: <52A1088C.3060308@third-bit.com> References: <52A1088C.3060308@third-bit.com> Message-ID: I'd like to be able to do this without any metadata, and especially without any magic. My use-case is that we have several example notebooks for SymPy, which we'd like to ship with our tarball, but we don't want to ship notebooks that don't work because they weren't updated with the code. Ideally, it should "just work". Aaron Meurer On Thu, Dec 5, 2013 at 4:13 PM, Greg Wilson wrote: > Is this another case where named cells would be useful? > > %%test name_of_cell_to_compare_code_against > ...code... > > and then further down, a Markdown cell with a name ? > > Thanks, > Greg > > On 2013-12-05 5:44 PM, Aaron Meurer wrote: >> Here's the relevant issue: >> https://github.com/ipython/ipython/issues/1195. I would love to see >> something like this integrated into IPython. >> >> Aaron Meurer >> >> On Thu, Dec 5, 2013 at 3:42 PM, MinRK wrote: >>> I wrote a toy script a while ago, to show that this kind of thing is >>> feasible: >>> >>> https://gist.github.com/minrk/2620735 >>> >>> -MinRK >>> >>> >>> On Thu, Dec 5, 2013 at 1:00 PM, Mart?n Gait?n wrote: >>>> Hi everybody >>>> >>>> I need to add some tests to fortran-magic [1] and I realized that, >>>> basically, I should test each example in the documentation which is already >>>> done as a notebook[2] . >>>> >>>> So, I got this idea: make a a way to run the code cells of a notebook and >>>> compare the calculated output against the stored output in the notebook. >>>> >>>> Looking for something like this I found [3] and [4], which facilitate the >>>> use of doctests, but it's not exactly the same. Conceptually, this are also >>>> doctests, but the main difference with what I imagine is that we don't >>>> bother writing fake code inside the actual code: No docstrings at all, just >>>> use the cells we already have. >>>> >>>> Of course the comparison should be flexible. Doctest has directives to >>>> allow ellipsis or normalizations, for example [5]. I think this could be >>>> simplified with an UI to select the portions of the output that must be >>>> compared and store this selection as cell metadata. >>>> >>>> What do you think? >>>> >>>> [1] https://github.com/mgaitan/fortran_magic >>>> [2] >>>> http://nbviewer.ipython.org/github/mgaitan/fortran_magic/blob/master/documentation.ipynb >>>> [3] >>>> http://catherinedevlin.blogspot.co.uk/2012/10/im-increasingly-amazed-at-ipython.html >>>> [4] https://github.com/tkf/ipython-doctestmagic >>>> [5] http://docs.python.org/2/library/doctest.html#directives >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From fperez.net at gmail.com Thu Dec 5 20:36:45 2013 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 5 Dec 2013 17:36:45 -0800 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46335.759072.463264@Konrad-Hinsens-MacBook-Pro.local> Message-ID: Hi Aaron, On Thu, Dec 5, 2013 at 1:56 PM, Aron Ahmadia wrote: > Taking this thread into a slightly more general nature, I'm interested in > learning more about how the IPython front-end Javascript obtains its > information for populating cells. This opens the door for connecting to > other data back-ends without necessarily creating a heavy-weight notebook. > Would the developers be up for perhaps hosting a G+ hangout at some point > to chat about this? > We're obviously always open to setting up discussions as needed, but this is the kind of conversation that can perfectly fit in our weekly scheduled public dev meeting. Those meetings are not exclusive to the core team, they are also a channel to interact with the rest of the community closely connected to our development. If a topic requires a long separate discussion, we may for practical reasons divert it, but a 10-20 minute slot in the regular dev meeting is a good starting point. Cheers, f -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Dec 5 23:09:09 2013 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 5 Dec 2013 20:09:09 -0800 Subject: [IPython-dev] old latex templates for nbconvert? In-Reply-To: <529E2768.8060800@gmail.com> References: <529DE0CA.8090806@gmail.com> <529E2768.8060800@gmail.com> Message-ID: By all means do add it up, if you found it useful, there's a chance others will too. We're just trying to keep the shipped templates to a minimum, because the use cases and user preferences are endless. But we also want it to be easy for folks to find the ones they like. Cheers, f On Tue, Dec 3, 2013 at 10:48 AM, Zolt?n V?r?s wrote: > OK, thanks! Should I just try to write it up in the wiki, or you think > that it's useless? > > > > On 03/12/13 19:12, MinRK wrote: > > Yup, it?s here > > > On Tue, Dec 3, 2013 at 5:46 AM, Zolt?n V?r?s wrote: > >> Hi all, >> >> Is there some sort of repository for old nbconverter templates? I used >> to use the "basic" template that produced very nice output with coloured >> frames, but the new article/report templates are really stripped down >> and their output is rather flat. I think it would actually make sense to >> have a collection of various templates (in the wiki, perhaps?) for the >> benefit of everyone. >> >> Cheers, >> Zolt?n >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > _______________________________________________ > IPython-dev mailing listIPython-dev at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Fri Dec 6 00:43:16 2013 From: benjaminrk at gmail.com (MinRK) Date: Thu, 5 Dec 2013 21:43:16 -0800 Subject: [IPython-dev] old latex templates for nbconvert? In-Reply-To: References: <529DE0CA.8090806@gmail.com> <529E2768.8060800@gmail.com> Message-ID: Paul just created a place for exactly that: https://github.com/ipython/ipython/wiki/Cookbook:-nbconvert-templates On Thu, Dec 5, 2013 at 8:09 PM, Fernando Perez wrote: > By all means do add it up, if you found it useful, there's a chance others > will too. We're just trying to keep the shipped templates to a minimum, > because the use cases and user preferences are endless. But we also want > it to be easy for folks to find the ones they like. > > Cheers, > > f > > > On Tue, Dec 3, 2013 at 10:48 AM, Zolt?n V?r?s wrote: > >> OK, thanks! Should I just try to write it up in the wiki, or you think >> that it's useless? >> >> >> >> On 03/12/13 19:12, MinRK wrote: >> >> Yup, it?s here >> >> >> On Tue, Dec 3, 2013 at 5:46 AM, Zolt?n V?r?s wrote: >> >>> Hi all, >>> >>> Is there some sort of repository for old nbconverter templates? I used >>> to use the "basic" template that produced very nice output with coloured >>> frames, but the new article/report templates are really stripped down >>> and their output is rather flat. I think it would actually make sense to >>> have a collection of various templates (in the wiki, perhaps?) for the >>> benefit of everyone. >>> >>> Cheers, >>> Zolt?n >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >> >> >> >> _______________________________________________ >> IPython-dev mailing listIPython-dev at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From konrad.hinsen at fastmail.net Fri Dec 6 02:29:19 2013 From: konrad.hinsen at fastmail.net (Konrad Hinsen) Date: Fri, 6 Dec 2013 08:29:19 +0100 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> Message-ID: <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> Thomas Kluyver writes: > On 5 December 2013 09:13, Konrad Hinsen wrote: > > One advantage of storing notebooks cells as ActivePapers calclets (and > thus as HDF5 datasets) is the possibility to open any ActivePapers > file as a notebook, even if it was generated in a different way. > > Do ActivePapers calclets have a notion of order? i.e. this should be run before > that? Not directly. There is a dependency graph between calclets and datasets, which implies a partial order. But there is no strict linear order as for cells in a notebook. BTW, that's related to one of the open questions I have with integrating notebooks into ActivePapers: how to handle the case of a notebook whose cells have been executed interactively in a different order? This may well result in different datasets than a complete linear execution, so the cell execution order needs to be stored for reproduction. Is this information available in a notebook? Aron Ahmadia writes: > Taking this thread into a slightly more general nature, I'm > interested in learning more about how the IPython front-end > Javascript obtains its information for populating cells. This > opens the door for connecting to other data back-ends without > necessarily creating a heavy-weight notebook. Right, this could be the occasion to think about more general approaches of using notebooks as interfaces for back-ends of various kinds. Konrad From zvoros at gmail.com Fri Dec 6 02:29:44 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Fri, 06 Dec 2013 08:29:44 +0100 Subject: [IPython-dev] old latex templates for nbconvert? In-Reply-To: References: <529DE0CA.8090806@gmail.com> <529E2768.8060800@gmail.com> Message-ID: <52A17CE8.9040407@gmail.com> Thanks, Min! I will then add the stuff there. Cheers, Zolt?n On 06/12/13 06:43, MinRK wrote: > Paul just created a place for exactly that: > > https://github.com/ipython/ipython/wiki/Cookbook:-nbconvert-templates > > > On Thu, Dec 5, 2013 at 8:09 PM, Fernando Perez > wrote: > > By all means do add it up, if you found it useful, there's a > chance others will too. We're just trying to keep the shipped > templates to a minimum, because the use cases and user preferences > are endless. But we also want it to be easy for folks to find the > ones they like. > > Cheers, > > f > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zvoros at gmail.com Fri Dec 6 02:50:56 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Fri, 06 Dec 2013 08:50:56 +0100 Subject: [IPython-dev] listing subfolders in the tree Message-ID: <52A181E0.3010905@gmail.com> Hi all, This issue has come up a couple of times, but since it hasn't been settled, I wonder, whether I could raise it once more. At the moment, the notebook tree lists notebook files that are in the directory, in which the server was started, but the server doesn't descend into subfolders. What I would like to do is search for all .ipynb files in all subfolders of the root, wrap the links in a dynatree structure, and serve that, so that the directory structure would be preserved in the front-end, and directories could be collapsed etc. Alternatively, folders could be added as a new tab, next to clusters. However, I can't really locate the entry point in the code. (Apart from this, I believe, it could be beneficial, if users could serve custom content.) I know that I could just point the address to the subdirectories, but the problem still remains: only one folder is listed at a given time. Oh, and perhaps most importantly, is this a reasonable approach, or there are some design considerations that I don't know of? In any case, comments on why I should or should not do this are welcome. Cheers, Zolt?n From bussonniermatthias at gmail.com Fri Dec 6 04:31:52 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Fri, 6 Dec 2013 10:31:52 +0100 Subject: [IPython-dev] listing subfolders in the tree In-Reply-To: <52A181E0.3010905@gmail.com> References: <52A181E0.3010905@gmail.com> Message-ID: <53B55F30-19B0-4B38-BEE7-A3801E87F633@gmail.com> Le 6 d?c. 2013 ? 08:50, Zolt?n V?r?s a ?crit : > Hi all, > > This issue has come up a couple of times, but since it hasn't been > settled, I wonder, whether I could raise it once more. At the moment, > the notebook tree lists notebook files that are in the directory, in > which the server was started, but the server doesn't descend into > subfolders. > > What I would like to do is search for all .ipynb files in all subfolders > of the root, wrap the links in a dynatree structure, and serve that, so > that the directory structure would be preserved in the front-end, and > directories could be collapsed etc. Alternatively, folders could be > added as a new tab, next to clusters. Knowing that the dashboard is refreshed often that some people have notebook on network storage traversing hard-drive recursively is not an alternative that will be shipped in core. as you are aware master can see content in subfolder go to /tree// and open notebook with /notebook/// The only missing thing is UI , but the full REST URL scheme is ready. -- M > > However, I can't really locate the entry point in the code. (Apart from > this, I believe, it could be beneficial, if users could serve custom > content.) I know that I could just point the address to the > subdirectories, but the problem still remains: only one folder is listed > at a given time. > > Oh, and perhaps most importantly, is this a reasonable approach, or > there are some design considerations that I don't know of? In any case, > comments on why I should or should not do this are welcome. > > Cheers, > Zolt?n > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From zvoros at gmail.com Fri Dec 6 07:36:18 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Fri, 06 Dec 2013 13:36:18 +0100 Subject: [IPython-dev] listing subfolders in the tree In-Reply-To: <53B55F30-19B0-4B38-BEE7-A3801E87F633@gmail.com> References: <52A181E0.3010905@gmail.com> <53B55F30-19B0-4B38-BEE7-A3801E87F633@gmail.com> Message-ID: <52A1C4C2.2090105@gmail.com> Hi Matthias, > Le 6 d?c. 2013 ? 08:50, Zolt?n V?r?s a ?crit : > >> Hi all, >> >> This issue has come up a couple of times, but since it hasn't been >> settled, I wonder, whether I could raise it once more. At the moment, >> the notebook tree lists notebook files that are in the directory, in >> which the server was started, but the server doesn't descend into >> subfolders. >> >> What I would like to do is search for all .ipynb files in all subfolders >> of the root, wrap the links in a dynatree structure, and serve that, so >> that the directory structure would be preserved in the front-end, and >> directories could be collapsed etc. Alternatively, folders could be >> added as a new tab, next to clusters. > Knowing that the dashboard is refreshed often that some people have > notebook on network storage traversing hard-drive recursively is not an > alternative that will be shipped in core. This is a good point, but I don't think that making this configurable would upset anything. > as you are aware master can see content in subfolder > go to > > /tree// > > and open notebook with > > /notebook/// > > The only missing thing is UI , but the full REST URL scheme is ready. Yes, that's true, but then one still has to know the name of the folders. Wouldn't life be much easier, if the folders were served right away? And I think, my related suggestion is still valid: if users could tamper with what exactly is served on the dashboard, that would make the front-end quite flexible. One might want to order notebooks by creation/modification date, or size, or whether a notebook has a particular keyword, or I have no idea. The point is that some processing has to be done. I think it would be OK, if these extra options could be passed to the server on the command line. All I am asking for is a means of executing code and inserting the results into the dashboard when the server examines the folder in question. Is that totally unreasonable? Cheers, Zolt?n From bussonniermatthias at gmail.com Fri Dec 6 10:07:38 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Fri, 6 Dec 2013 16:07:38 +0100 Subject: [IPython-dev] listing subfolders in the tree In-Reply-To: <52A1C4C2.2090105@gmail.com> References: <52A181E0.3010905@gmail.com> <53B55F30-19B0-4B38-BEE7-A3801E87F633@gmail.com> <52A1C4C2.2090105@gmail.com> Message-ID: Hi Zolt?n, Le 6 d?c. 2013 ? 13:36, Zolt?n V?r?s a ?crit : >> Knowing that the dashboard is refreshed often that some people have >> notebook on network storage traversing hard-drive recursively is not an >> alternative that will be shipped in core. > This is a good point, but I don't think that making this configurable > would upset anything. That's configurable. It is probably a DottedObjectName Bookstore (https://pypi.python.org/pypi/bookstore) is an alternative backend that store notebook on RackSpace (and allow versioning) > >> as you are aware master can see content in subfolder >> go to >> >> /tree// >> >> and open notebook with >> >> /notebook/// >> >> The only missing thing is UI , but the full REST URL scheme is ready. > Yes, that's true, but then one still has to know the name of the > folders. Wouldn't life be much easier, if the folders were served right > away? If by right away you mean list all ipynb in subfolder, it can be problematic. Still nothing prevent you to do that. > And I think, my related suggestion is still valid: if users could tamper > with what exactly is served on the dashboard, that would make the > front-end quite flexible. One might want to order notebooks by > creation/modification date, or size, or whether a notebook has a > particular keyword, or I have no idea. The point is that some processing > has to be done. I think it would be OK, if these extra options could be > passed to the server on the command line. All I am asking for is a means > of executing code and inserting the results into the dashboard when the > server examines the folder in question. Is that totally unreasonable? Yes it is reasonable, and we have plan to have a more advanced dashboard. The current handler should be TreeHandler in IPython/html/tree/handler.py Session Api already exposes modified dates act, you can read more here : https://github.com/ipython/ipython/wiki/IPEP-16%3A-Notebook-multi-directory-dashboard-and-URL-mapping The problem is that we have no way (yet) to send configuration to the Javascript Side, even if we would like to. Also we would like to avoid reinventing the wheel and use a pre-existing javascript library to handle the browsing client-side. We in particular also want the ability to chose between different kernel when you start a notebook. So we are moving slowly in this direction. If you start anything, I would suggest for you to contact Zach Sailer that wrote most part of the REST api and have somewhere a Draft Branch of UI where you can open folders, and order things IIRC. -- M -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Fri Dec 6 12:58:42 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 6 Dec 2013 09:58:42 -0800 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> Message-ID: On 5 December 2013 23:29, Konrad Hinsen wrote: > BTW, that's related to one of the open questions I have with > integrating notebooks into ActivePapers: how to handle the case of a > notebook whose cells have been executed interactively in a different > order? This may well result in different datasets than a complete > linear execution, so the cell execution order needs to be stored for > reproduction. Is this information available in a notebook? > Yes - the prompt numbers by the cells record the order of execution. However, I think that the reproducible case should normally be the one where the cells are run in order from top to bottom. Running them out of order is convenient, but to check that a notebook all works together, it's common to clear all output, reset the kernel, and run all cells in order. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From zvoros at gmail.com Fri Dec 6 13:47:37 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Fri, 06 Dec 2013 19:47:37 +0100 Subject: [IPython-dev] listing subfolders in the tree In-Reply-To: References: <52A181E0.3010905@gmail.com> <53B55F30-19B0-4B38-BEE7-A3801E87F633@gmail.com> <52A1C4C2.2090105@gmail.com> Message-ID: <52A21BC9.5030202@gmail.com> Hi Matthias, Thanks for the links, pointers, and suggestions. I will try to sniff around. Cheers, Zolt?n On 06/12/13 16:07, Matthias BUSSONNIER wrote: > Hi Zolt?n, > > Le 6 d?c. 2013 ? 13:36, Zolt?n V?r?s a ?crit : > >>> Knowing that the dashboard is refreshed often that some people have >>> notebook on network storage traversing hard-drive recursively is not an >>> alternative that will be shipped in core. >> This is a good point, but I don't think that making this configurable >> would upset anything. > > That's configurable. It is probably a DottedObjectName > Bookstore (https://pypi.python.org/pypi/bookstore) is an alternative > backend that > store notebook on RackSpace (and allow versioning) > >> >>> as you are aware master can see content in subfolder >>> go to >>> >>> /tree// >>> >>> and open notebook with >>> >>> /notebook/// >>> >>> The only missing thing is UI , but the full REST URL scheme is ready. >> Yes, that's true, but then one still has to know the name of the >> folders. Wouldn't life be much easier, if the folders were served right >> away? > > If by right away you mean list all ipynb in subfolder, it can be > problematic. > Still nothing prevent you to do that. > >> And I think, my related suggestion is still valid: if users could tamper >> with what exactly is served on the dashboard, that would make the >> front-end quite flexible. One might want to order notebooks by >> creation/modification date, or size, or whether a notebook has a >> particular keyword, or I have no idea. The point is that some processing >> has to be done. I think it would be OK, if these extra options could be >> passed to the server on the command line. All I am asking for is a means >> of executing code and inserting the results into the dashboard when the >> server examines the folder in question. Is that totally unreasonable? > > Yes it is reasonable, and we have plan to have a more advanced dashboard. > The current handler should be TreeHandler in IPython/html/tree/handler.py > > Session Api already exposes modified dates act, you can read more here : > https://github.com/ipython/ipython/wiki/IPEP-16%3A-Notebook-multi-directory-dashboard-and-URL-mapping > > > The problem is that we have no way (yet) to send configuration to the > Javascript Side, even if we would like to. > Also we would like to avoid reinventing the wheel and use a > pre-existing javascript library to handle the browsing client-side. > > We in particular also want the ability to chose between different > kernel when you start a notebook. > So we are moving slowly in this direction. > > If you start anything, I would suggest for you to contact Zach Sailer > that wrote most part of the REST api > and have somewhere a Draft Branch of UI where you can open folders, > and order things IIRC. > > -- > M > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From konrad.hinsen at fastmail.net Sun Dec 8 06:16:04 2013 From: konrad.hinsen at fastmail.net (Konrad Hinsen) Date: Sun, 8 Dec 2013 12:16:04 +0100 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> Message-ID: <21156.21748.117583.339638@Ordinateur-de-Catherine-Konrad.local> Thomas Kluyver writes: > On 5 December 2013 23:29, Konrad Hinsen wrote: > > BTW, that's related to one of the open questions I have with > integrating notebooks into ActivePapers: how to handle the case of a > notebook whose cells have been executed interactively in a different > order? This may well result in different datasets than a complete > linear execution, so the cell execution order needs to be stored for > reproduction. Is this information available in a notebook? > > Yes - the prompt numbers by the cells record the order of execution. If that's the same information shown in the browser, then it's only the number of the last execution that remains available. And that means that I cannot reconstruct the history of execution. To make it worse, a cell could have been modified and re-executed, so the original code is no longer available. > However, I think that the reproducible case should normally be the > one where the cells are run in order from top to bottom. Running > them out of order is convenient, but to check that a notebook all > works together, it's common to clear all output, reset the kernel, > and run all cells in order. That's what people should do, but I still need to decide what to do when a user runs cells randomly and then wants to save the notebook to an ActivePapers backend. Since any result datasets are in a non-reproducible state, the options are 1) Refuse with an error message. 2) Re-run the notebook in order before saving. 3) Save the notebook, but mark it and all its results as stale. Stale datasets are marked as such and require an update before they can be used in some other computation. I'll probably go for 3) and see how this works out. Konrad. From takowl at gmail.com Sun Dec 8 14:40:28 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 8 Dec 2013 11:40:28 -0800 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: <21156.21748.117583.339638@Ordinateur-de-Catherine-Konrad.local> References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> <21156.21748.117583.339638@Ordinateur-de-Catherine-Konrad.local> Message-ID: On 8 December 2013 03:16, Konrad Hinsen wrote: > > Yes - the prompt numbers by the cells record the order of execution. > > If that's the same information shown in the browser, then it's only > the number of the last execution that remains available. And that > means that I cannot reconstruct the history of execution. To make it > worse, a cell could have been modified and re-executed, so the > original code is no longer available. > That's all the information available in the notebook frontend, but the kernel records every cell executed, so there's a record of all the code that has run in the present session. The kernel knows nothing about the notebook document, so if for instance a cell has been edited since it was executed, there's no easy way to know which cell that execution represents. However, you can easily enough tell whether the current notebook has been run through once with no editing, by comparing the list of code cells to the history. > > > However, I think that the reproducible case should normally be the > > one where the cells are run in order from top to bottom. Running > > them out of order is convenient, but to check that a notebook all > > works together, it's common to clear all output, reset the kernel, > > and run all cells in order. > > That's what people should do, but I still need to decide what to do > when a user runs cells randomly and then wants to save the notebook to > an ActivePapers backend. Since any result datasets are in a > non-reproducible state, the options are > > 1) Refuse with an error message. > 2) Re-run the notebook in order before saving. > 3) Save the notebook, but mark it and all its results as stale. > Stale datasets are marked as such and require an update > before they can be used in some other computation. > > I'll probably go for 3) and see how this works out. 3 sounds like the best option to me: you can save the information, but it shouldn't be considered as the fixed form for later calculations. Presumably ActivePapers will be able to re-run the cells in order if it needs to freshen the results, without the user having to manually open the notebook and execute the cells. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Sun Dec 8 18:06:16 2013 From: chris.barker at noaa.gov (Chris Barker) Date: Sun, 8 Dec 2013 15:06:16 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X Message-ID: Hi folks, If you are not on the distutils-sig list, and you probably aren't, we've been moving forward with trying to get the new binary wheel format better used and tested. In this spirit, I've built some binary wheels for iPython and its dependencies. Ralf Gommers was nice enough to put them up on source forge for all to see and test. Here's his note to the numpy list: At http://sourceforge.net/projects/numpy/files/wheels_to_test/ you can find > a set of wheels, for numpy, scipy and ipython plus its dependencies. These > are for the following configuration only: > - OSX >= 10.6 > - Python 2.7 32/64-bit from python.org > > It would be great if OS X users could test these. We're interested in > whether these wheels work, and also if you run into any issues installing > wheels. Since (a) I'm not really sure myself what the recommended and most > robust way is to install wheels, and (b) it's interesting to see if the > docs and usability of pip/wheel are ready for prime time, I'm not going to > give the commands to execute but instead link to the relevant docs: > > http://www.pip-installer.org/en/latest/index.html > http://python-packaging-user-guide.readthedocs.org/en/latest/ > http://wheel.readthedocs.org/en/latest/index.html > > Please try this out and share your experience. > You probably want to use ipython[all] to get all the stuff that the notebook needs... One thing I noticed is that readline is NOT listed as a dependency -- strictly speaking, it's not, but you really do want it, particualry on OS-X. Maybe we should add it. Also, the warnign about readline in ipython says that "pip install" does not work right, but it's working fine for me -- outdated note? or am I just lucky? A tiny bit of background: I'm hoping for wheel to get better supported in general, it would be really nice if folks could "pip install" more stuff. And my focus in on OS-X users that aren't unix geeks at the moment. But in particular, at the moment I am teaching a general intro to python class: no numpy,scipy, etc.... But iPython, as we all know, is awesome, so I've been using it in demos, and want my students to be able to use it to. I've told Windows users to go to the Gohlke repo to get it. Linux users can either use their system package maager, or probably pip intstall it. Mac users that aren't using HomeBrew or MacPorts are a bit stuck. The iPython web page suggests that they should go get Anocanda or Canopy -- which may be the best way to go if you want the whole scipy stack, but if you want iPython, and also want a system compatible with virtualenv, all the web development packages, and desktop stuff like wxPython, etc, it may not be the way to go. In particular, I had a couple students install Anaconda, and then find themselves stuck trying to use wxPython. I've had a bunch of emails on the distuitls list, and with Travis O, and maybe the future will be better compatibility with Anocanda and other systems, but nevertheless, I think it's a good idea to have and easy way for folks to "just get ipython" : pip --use_wheel ipython[all] and presto! have it ready to go. By the way, the wheels themselves are trivially easy to build, once setup.py build works.... -Chris PS: we should do Windows,too.....but I only have an old copy of XP on an old netbook, so that probably won't be me! PPS: and py3.... -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Dec 9 00:24:40 2013 From: benjaminrk at gmail.com (MinRK) Date: Sun, 8 Dec 2013 21:24:40 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: Message-ID: On Sun, Dec 8, 2013 at 3:06 PM, Chris Barker wrote: > > Hi folks, > > If you are not on the distutils-sig list, and you probably aren't, we've been moving forward with trying to get the new binary wheel format better used and tested. > > In this spirit, I've built some binary wheels for iPython and its dependencies. Ralf Gommers was nice enough to put them up on source forge for all to see and test. Here's his note to the numpy list: > >> At http://sourceforge.net/projects/numpy/files/wheels_to_test/ you can find a set of wheels, for numpy, scipy and ipython plus its dependencies. These are for the following configuration only: >> - OSX >= 10.6 >> - Python 2.7 32/64-bit from python.org >> >> It would be great if OS X users could test these. We're interested in whether these wheels work, and also if you run into any issues installing wheels. Since (a) I'm not really sure myself what the recommended and most robust way is to install wheels, and (b) it's interesting to see if the docs and usability of pip/wheel are ready for prime time, I'm not going to give the commands to execute but instead link to the relevant docs: >> >> http://www.pip-installer.org/en/latest/index.html >> http://python-packaging-user-guide.readthedocs.org/en/latest/ >> http://wheel.readthedocs.org/en/latest/index.html >> >> Please try this out and share your experience. > > > > You probably want to use ipython[all] to get all the stuff that the notebook needs... > > One thing I noticed is that readline is NOT listed as a dependency -- strictly speaking, it's not, but you really do want it, particualry on OS-X. Maybe we should add it. Also, the warnign about readline in ipython says that "pip install" does not work right, but it's working fine for me -- outdated note? or am I just lucky? readline should be a dependency on OS X, I know it used to be. The logic may have gotten confused a bit. The warning is still correct, and I still recommend that all OS X users easy_install (not pip) readline. The issue has been discussed in various places (here is one), but it's a sys.path priority issue. Nobody should ever use pip to install readline. But since people still do, incorrectly believing that pip is actually a replacement for easy_install, IPython has an unadvertised hack to make the normally never importable pip installed readline available, as long as IPython is the first to try to import it. > > A tiny bit of background: > > I'm hoping for wheel to get better supported in general, it would be really nice if folks could "pip install" more stuff. And my focus in on OS-X users that aren't unix geeks at the moment. > > But in particular, at the moment I am teaching a general intro to python class: no numpy,scipy, etc.... > > But iPython, as we all know, is awesome, so I've been using it in demos, and want my students to be able to use it to. > > I've told Windows users to go to the Gohlke repo to get it. > > Linux users can either use their system package maager, or probably pip intstall it. > > Mac users that aren't using HomeBrew or MacPorts are a bit stuck. > > The iPython web page suggests that they should go get Anocanda or Canopy -- which may be the best way to go if you want the whole scipy stack, but if you want iPython, and also want a system compatible with virtualenv, all the web development packages, and desktop stuff like wxPython, etc, it may not be the way to go. In particular, I had a couple students install Anaconda, and then find themselves stuck trying to use wxPython. I've had a bunch of emails on the distuitls list, and with Travis O, and maybe the future will be better compatibility with Anocanda and other systems, but nevertheless, I think it's a good idea to have and easy way for folks to "just get ipython" : > > pip --use_wheel ipython[all] > > and presto! have it ready to go. pyzmq, the only binary dependency of IPython, has had wheels on OS X and Windows for a few releases now, so `pip install --use-wheel ipython[all]` should already work on a fresh OS X system without any help (after installing pip and wheel, of course). > > By the way, the wheels themselves are trivially easy to build, once setup.py build works.... Yes, they are easy to build for yourself, and are typically reusable for machines sufficiently similar to yours. But it's a lot more delicate to be sure that the eggs actually self-identify correctly, and will be properly ignored on systems where they won't work. This is why IPython has decided against building (official) wheel bdists for at least another release. I am 100% behind a semi-not-so-official-wheels-for-osx effort a la Gohlke, that's actually why I started publishing my own wheels. But honestly, in a student environment, I would recommend anaconda (or just conda). -MinRK > > -Chris > > PS: we should do Windows,too.....but I only have an old copy of XP on an old netbook, so that probably won't be me! > > PPS: and py3.... > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Dec 9 01:02:31 2013 From: chris.barker at noaa.gov (Chris Barker) Date: Sun, 8 Dec 2013 22:02:31 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: Message-ID: On Sun, Dec 8, 2013 at 9:24 PM, MinRK wrote: > readline should be a dependency on OS X, I know it used to be. The logic > may have gotten confused a bit. > Does pip support a way to do platform-dependent dependencies? Anyway, pip isn't lookeing for it for me. Frankly, no biggie, auto-dependencies are nice, but I don't mind hand-installing something if ti's easy to install. But wouldn't it be a dependency o Windows, too? > The warning is still correct, and I still recommend that all OS X users > easy_install (not pip) readline. The issue has been discussed in various > places (here is one), > but it's a sys.path priority issue. Nobody should ever use pip to install > readline. But since people still do, incorrectly believing that pip is > actually a replacement for easy_install, > well, I'm pretty sure that is the goal -- really we don't want to almost the same package installer for Python.... > IPython has an unadvertised hack to make the normally never importable > pip installed readline available, as long as IPython is the first to try to > import it. > > hmm -- pretty cool, and it sure works for me, but a bit fragile ;-) > pyzmq, the only binary dependency of IPython, has had wheels on OS X and > Windows for a few releases now, so `pip install --use-wheel ipython[all]` > should already work on a fresh OS X system without any help (after > installing pip and wheel, of course). > > It sure would be nice if that were advertised a bit.....I really did loo, but of course I never thought just try it! > > > By the way, the wheels themselves are trivially easy to build, once > setup.py build works.... > > Yes, they are easy to build for yourself, and are typically reusable for > machines sufficiently similar to yours. But it's a lot more delicate to be > sure that the eggs actually self-identify correctly, and will be properly > ignored on systems where they won't work. This is why IPython has decided > against building (official) wheel bdists for at least another release. > There has been a lot of discussion about this on the distutils list, and I _think_ there is more or less a consensus that we should put binary wheels up on PyPi for the python.org builds for Windows and OS-X. Those should work at this point (sse issues asside for numpy/scipy) Doing it for Macports or HomeBrew, or what have you is pretty much alost cause, but that's not a big deal -- those are systems designed for people to compile there own stuff anyway. similar for Linux. So what's missing? And, as you say, zmq is really the only one we need anyway, what exactly is iPython not doing? > I am 100% behind a semi-not-so-official-wheels-for-osx effort a la > Gohlke, that's actually why I started publishing my own wheels. > > nice! but not that useful if we don't get the word out. > But honestly, > in a student environment, I would recommend anaconda (or just conda). > > non starter for now -- Again, these are NOT scipy stack classes -- and I do want wx, and the guy teaching the next class on web development really wants virtualenv. Anocanda does not work (for now) on those. Anyway -- I feel very strongly that we should play well with the rest of the Python word, and that mean pip and wheels ... and we really can do it now. I guess I need to do a pull request for the "installing" page of the iPython web sire next... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Dec 9 02:03:23 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Mon, 9 Dec 2013 08:03:23 +0100 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: Message-ID: <14AC83A8-2B48-4808-A238-744330AAB9FF@gmail.com> Hi, Also of interest, previous discussion about that: https://github.com/ipython/ipython/pull/4547 I'm sure there are other threads somewhere. -- M Le 9 d?c. 2013 ? 07:02, Chris Barker a ?crit : > On Sun, Dec 8, 2013 at 9:24 PM, MinRK wrote: > > readline should be a dependency on OS X, I know it used to be. The logic may have gotten confused a bit. > > Does pip support a way to do platform-dependent dependencies? Anyway, pip isn't lookeing for it for me. Frankly, no biggie, auto-dependencies are nice, but I don't mind hand-installing something if ti's easy to install. > > But wouldn't it be a dependency o Windows, too? > > The warning is still correct, and I still recommend that all OS X users easy_install (not pip) readline. The issue has been discussed in various places (here is one), but it's a sys.path priority issue. Nobody should ever use pip to install readline. But since people still do, incorrectly believing that pip is actually a replacement for easy_install, > > well, I'm pretty sure that is the goal -- really we don't want to almost the same package installer for Python.... > > > > IPython has an unadvertised hack to make the normally never importable pip installed readline available, as long as IPython is the first to try to import it. > > hmm -- pretty cool, and it sure works for me, but a bit fragile ;-) > > pyzmq, the only binary dependency of IPython, has had wheels on OS X and Windows for a few releases now, so `pip install --use-wheel ipython[all]` should already work on a fresh OS X system without any help (after installing pip and wheel, of course). > > > It sure would be nice if that were advertised a bit.....I really did loo, but of course I never thought just try it! > > > > > By the way, the wheels themselves are trivially easy to build, once setup.py build works.... > > Yes, they are easy to build for yourself, and are typically reusable for machines sufficiently similar to yours. But it's a lot more delicate to be sure that the eggs actually self-identify correctly, and will be properly ignored on systems where they won't work. This is why IPython has decided against building (official) wheel bdists for at least another release. > > There has been a lot of discussion about this on the distutils list, and I _think_ there is more or less a consensus that we should put binary wheels up on PyPi for the python.org builds for Windows and OS-X. Those should work at this point (sse issues asside for numpy/scipy) > > Doing it for Macports or HomeBrew, or what have you is pretty much alost cause, but that's not a big deal -- those are systems designed for people to compile there own stuff anyway. similar for Linux. > > So what's missing? > > And, as you say, zmq is really the only one we need anyway, what exactly is iPython not doing? > > > I am 100% behind a semi-not-so-official-wheels-for-osx effort a la Gohlke, that's actually why I started publishing my own wheels. > > nice! but not that useful if we don't get the word out. > > But honestly, > in a student environment, I would recommend anaconda (or just conda). > > non starter for now -- Again, these are NOT scipy stack classes -- and I do want wx, and the guy teaching the next class on web development really wants virtualenv. > > Anocanda does not work (for now) on those. > > Anyway -- I feel very strongly that we should play well with the rest of the Python word, and that mean pip and wheels ... and we really can do it now. > > I guess I need to do a pull request for the "installing" page of the iPython web sire next... > > -Chris > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Mon Dec 9 03:01:28 2013 From: asmeurer at gmail.com (Aaron Meurer) Date: Mon, 9 Dec 2013 01:01:28 -0700 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: Message-ID: On Sun, Dec 8, 2013 at 11:02 PM, Chris Barker wrote: > On Sun, Dec 8, 2013 at 9:24 PM, MinRK wrote: > >> >> readline should be a dependency on OS X, I know it used to be. The logic >> may have gotten confused a bit. > > > Does pip support a way to do platform-dependent dependencies? Anyway, pip > isn't lookeing for it for me. Frankly, no biggie, auto-dependencies are > nice, but I don't mind hand-installing something if ti's easy to install. > > But wouldn't it be a dependency o Windows, too? > >> >> The warning is still correct, and I still recommend that all OS X users >> easy_install (not pip) readline. The issue has been discussed in various >> places (here is one), but it's a sys.path priority issue. Nobody should ever >> use pip to install readline. But since people still do, incorrectly >> believing that pip is actually a replacement for easy_install, > > > well, I'm pretty sure that is the goal -- really we don't want to almost the > same package installer for Python.... What's the reason you can't pip install readline? Is it because pip insists on compiling it? If so, wouldn't wheels solve this issue? > > > >> >> IPython has an unadvertised hack to make the normally never importable pip >> installed readline available, as long as IPython is the first to try to >> import it. >> > hmm -- pretty cool, and it sure works for me, but a bit fragile ;-) > >> >> pyzmq, the only binary dependency of IPython, has had wheels on OS X and >> Windows for a few releases now, so `pip install --use-wheel ipython[all]` >> should already work on a fresh OS X system without any help (after >> installing pip and wheel, of course). >> > > It sure would be nice if that were advertised a bit.....I really did loo, > but of course I never thought just try it! > >> >> >> > By the way, the wheels themselves are trivially easy to build, once >> > setup.py build works.... >> >> Yes, they are easy to build for yourself, and are typically reusable for >> machines sufficiently similar to yours. But it's a lot more delicate to be >> sure that the eggs actually self-identify correctly, and will be properly >> ignored on systems where they won't work. This is why IPython has decided >> against building (official) wheel bdists for at least another release. > > > There has been a lot of discussion about this on the distutils list, and I > _think_ there is more or less a consensus that we should put binary wheels > up on PyPi for the python.org builds for Windows and OS-X. Those should work > at this point (sse issues asside for numpy/scipy) > > Doing it for Macports or HomeBrew, or what have you is pretty much alost > cause, but that's not a big deal -- those are systems designed for people to > compile there own stuff anyway. similar for Linux. > > So what's missing? > > And, as you say, zmq is really the only one we need anyway, what exactly is > iPython not doing? > > >> >> I am 100% behind a semi-not-so-official-wheels-for-osx effort a la Gohlke, >> that's actually why I started publishing my own wheels. > > > nice! but not that useful if we don't get the word out. > >> >> But honestly, >> in a student environment, I would recommend anaconda (or just conda). >> > non starter for now -- Again, these are NOT scipy stack classes -- and I do > want wx, and the guy teaching the next class on web development really wants > virtualenv. For wx, if you can figure out how to build a conda recipe for it, that would help a lot. Basically, you could just host your own recipe on binstar, and if it works, it's pretty likely that Continuum would pick it up and ship it with Anaconda. I tried it a while back but I didn't get very far. But if you're actually familiar with it you might get further. As for virtualenv, I really recommend trying conda environments. You can alias virtualenv="conda create -p" if you want. If you have to have virtualenv, I was able to get it working inside Anaconda (see https://github.com/ContinuumIO/conda-recipes/tree/master/virtualenv and https://binstar.org/asmeurer/virtualenv), but only in Python 2. Python 3 segfaults for some reason (but in Python 3, you really should use pyvenv instead of virtualenv). Aaron Meurer > > Anocanda does not work (for now) on those. > > Anyway -- I feel very strongly that we should play well with the rest of the > Python word, and that mean pip and wheels ... and we really can do it now. > > I guess I need to do a pull request for the "installing" page of the iPython > web sire next... > > -Chris > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From konrad.hinsen at fastmail.net Mon Dec 9 08:28:43 2013 From: konrad.hinsen at fastmail.net (Konrad Hinsen) Date: Mon, 9 Dec 2013 14:28:43 +0100 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> <21156.21748.117583.339638@Ordinateur-de-Catherine-Konrad.local> Message-ID: <21157.50571.922623.163499@Konrad-Hinsens-MacBook-Pro-2.local> Thomas Kluyver writes: > However, you can easily enough tell whether the current notebook > has been run through once with no editing, by comparing the list of > code cells to the history. That looks good enough for my purposes. Could you give me a pointer to how I can access the kernel's history information for doing such a comparison? > I'll probably go for 3) and see how this works out. > > 3 sounds like the best option to me: you can save the information, > but it shouldn't be considered as the fixed form for later > calculations. Presumably ActivePapers will be able to re-run the > cells in order if it needs to freshen the results, without the user > having to manually open the notebook and execute the cells. Yes. Well, at least that's my intention, it isn't implemented yet for notebooks. With today's ActivePapers implementation, the command "aptool update" will re-run all calclets whose code or output is marked as "stale" (no longer up to date) or "dummy" (intentionally removed to reduce the file size). Konrad From bussonniermatthias at gmail.com Mon Dec 9 09:59:33 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Mon, 9 Dec 2013 15:59:33 +0100 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: <21157.50571.922623.163499@Konrad-Hinsens-MacBook-Pro-2.local> References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> <21156.21748.117583.339638@Ordinateur-de-Catherine-Konrad.local> <21157.50571.922623.163499@Konrad-Hinsens-MacBook-Pro-2.local> Message-ID: <82980319-7C1F-4418-ADC3-4732A10EF08B@gmail.com> Le 9 d?c. 2013 ? 14:28, Konrad Hinsen a ?crit : > Thomas Kluyver writes: > >> However, you can easily enough tell whether the current notebook >> has been run through once with no editing, by comparing the list of >> code cells to the history. > > That looks good enough for my purposes. Could you give me a pointer > to how I can access the kernel's history information for doing > such a comparison? the history sqlite file (.ipython/profile_default/history.sqlite) contains lots of info like raw input entered by user. you are probably interessed in session and history and join the two: sqlite> .schema history CREATE TABLE history (session integer, line integer, source text, source_raw text, PRIMARY KEY (session, line)); sqlite> .schema sessions CREATE TABLE sessions (session integer primary key autoincrement, start timestamp, end timestamp, num_cmds integer, remark text); I see here that only sessions have timestamp, should we add timestamp to history ? -- M > >> I'll probably go for 3) and see how this works out. >> >> 3 sounds like the best option to me: you can save the information, >> but it shouldn't be considered as the fixed form for later >> calculations. Presumably ActivePapers will be able to re-run the >> cells in order if it needs to freshen the results, without the user >> having to manually open the notebook and execute the cells. > > Yes. Well, at least that's my intention, it isn't implemented yet for > notebooks. With today's ActivePapers implementation, the command > "aptool update" will re-run all calclets whose code or output is > marked as "stale" (no longer up to date) or "dummy" (intentionally > removed to reduce the file size). > > Konrad > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From zvoros at gmail.com Mon Dec 9 10:03:45 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Mon, 09 Dec 2013 16:03:45 +0100 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: <82980319-7C1F-4418-ADC3-4732A10EF08B@gmail.com> References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> <21156.21748.117583.339638@Ordinateur-de-Catherine-Konrad.local> <21157.50571.922623.163499@Konrad-Hinsens-MacBook-Pro-2.local> <82980319-7C1F-4418-ADC3-4732A10EF08B@gmail.com> Message-ID: <52A5DBD1.9000803@gmail.com> You could also look at the python interface ipython-get-history.py at https://github.com/ipython/ipython/blob/master/examples/core/ipython-get-history.py Cheers, Zolt?n On 09/12/13 15:59, Matthias BUSSONNIER wrote: > Le 9 d?c. 2013 ? 14:28, Konrad Hinsen a ?crit : > >> Thomas Kluyver writes: >> >>> However, you can easily enough tell whether the current notebook >>> has been run through once with no editing, by comparing the list of >>> code cells to the history. >> That looks good enough for my purposes. Could you give me a pointer >> to how I can access the kernel's history information for doing >> such a comparison? > the history sqlite file (.ipython/profile_default/history.sqlite) > contains lots of info like raw input entered by user. > > you are probably interessed in session and history and join the two: > > sqlite> .schema history > CREATE TABLE history > (session integer, line integer, source text, source_raw text, > PRIMARY KEY (session, line)); > > sqlite> .schema sessions > CREATE TABLE sessions (session integer > primary key autoincrement, start timestamp, > end timestamp, num_cmds integer, remark text); > > I see here that only sessions have timestamp, should we add timestamp to history ? > -- > M > > From chris.barker at noaa.gov Mon Dec 9 10:45:56 2013 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Mon, 9 Dec 2013 07:45:56 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: <14AC83A8-2B48-4808-A238-744330AAB9FF@gmail.com> References: <14AC83A8-2B48-4808-A238-744330AAB9FF@gmail.com> Message-ID: <-991323323292894709@unknownmsgid> On Dec 8, 2013, at 10:59 PM, Matthias BUSSONNIER < bussonniermatthias at gmail.com> wrote: previous discussion about that: https://github.com/ipython/ipython/pull/4547 Thanks for the link. Helpful to understand the issues. It's interesting to me that I've been following the distutils-sig for a good while, and hadn't seen any of this. I suspect one of hold-ups for wheel is the scattered conversation. Though I see Nick on that thread, so that's good. Chris I'm sure there are other threads somewhere. -- M Le 9 d?c. 2013 ? 07:02, Chris Barker a ?crit : On Sun, Dec 8, 2013 at 9:24 PM, MinRK wrote: > readline should be a dependency on OS X, I know it used to be. The logic > may have gotten confused a bit. > Does pip support a way to do platform-dependent dependencies? Anyway, pip isn't lookeing for it for me. Frankly, no biggie, auto-dependencies are nice, but I don't mind hand-installing something if ti's easy to install. But wouldn't it be a dependency o Windows, too? > The warning is still correct, and I still recommend that all OS X users > easy_install (not pip) readline. The issue has been discussed in various > places (here is one), > but it's a sys.path priority issue. Nobody should ever use pip to install > readline. But since people still do, incorrectly believing that pip is > actually a replacement for easy_install, > well, I'm pretty sure that is the goal -- really we don't want to almost the same package installer for Python.... > IPython has an unadvertised hack to make the normally never importable > pip installed readline available, as long as IPython is the first to try to > import it. > > hmm -- pretty cool, and it sure works for me, but a bit fragile ;-) > pyzmq, the only binary dependency of IPython, has had wheels on OS X and > Windows for a few releases now, so `pip install --use-wheel ipython[all]` > should already work on a fresh OS X system without any help (after > installing pip and wheel, of course). > > It sure would be nice if that were advertised a bit.....I really did loo, but of course I never thought just try it! > > > By the way, the wheels themselves are trivially easy to build, once > setup.py build works.... > > Yes, they are easy to build for yourself, and are typically reusable for > machines sufficiently similar to yours. But it's a lot more delicate to be > sure that the eggs actually self-identify correctly, and will be properly > ignored on systems where they won't work. This is why IPython has decided > against building (official) wheel bdists for at least another release. > There has been a lot of discussion about this on the distutils list, and I _think_ there is more or less a consensus that we should put binary wheels up on PyPi for the python.org builds for Windows and OS-X. Those should work at this point (sse issues asside for numpy/scipy) Doing it for Macports or HomeBrew, or what have you is pretty much alost cause, but that's not a big deal -- those are systems designed for people to compile there own stuff anyway. similar for Linux. So what's missing? And, as you say, zmq is really the only one we need anyway, what exactly is iPython not doing? > I am 100% behind a semi-not-so-official-wheels-for-osx effort a la > Gohlke, that's actually why I started publishing my own wheels. > > nice! but not that useful if we don't get the word out. > But honestly, > in a student environment, I would recommend anaconda (or just conda). > > non starter for now -- Again, these are NOT scipy stack classes -- and I do want wx, and the guy teaching the next class on web development really wants virtualenv. Anocanda does not work (for now) on those. Anyway -- I feel very strongly that we should play well with the rest of the Python word, and that mean pip and wheels ... and we really can do it now. I guess I need to do a pull request for the "installing" page of the iPython web sire next... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov _______________________________________________ IPython-dev mailing list IPython-dev at scipy.org http://mail.scipy.org/mailman/listinfo/ipython-dev _______________________________________________ IPython-dev mailing list IPython-dev at scipy.org http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Dec 9 10:55:05 2013 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Mon, 9 Dec 2013 07:55:05 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: Message-ID: <-5064963750921577331@unknownmsgid> On Dec 9, 2013, at 12:01 AM, Aaron Meurer wrote: What's the reason you can't pip install readline? Is it because pip insists on compiling it? No--it needs to be compiled either way--but I _think_ the issue is that pip and easy_install put packages on sys.path differently. The pop way puts it after the existing readline. If so, wouldn't wheels solve this issue? Nope-because it's an install-time thing. But it is a barrier, as it DOES need to be complied. And it DOES need to be installed on the Mac (at least). -Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Mon Dec 9 11:26:05 2013 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 9 Dec 2013 16:26:05 +0000 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: <-5064963750921577331@unknownmsgid> References: <-5064963750921577331@unknownmsgid> Message-ID: On 9 December 2013 15:55, Chris Barker - NOAA Federal wrote: >> What's the reason you can't pip install readline? Is it because pip >> insists on compiling it? > > No--it needs to be compiled either way--but I _think_ the issue is that pip > and easy_install put packages on sys.path differently. The pop way puts it > after the existing readline. easy_install hacks sys.path to put installed packages before the standard library. That was controversial behaviour at the time it was implemented, IIRC - Guido specifically did not want to have user installed packages able to overwrite stdlib behaviour. I don't know if that position has changed, it's so rarely relevant that I don't think people care that much (they tend to care more about the nasty pth file hacks just for their own sake, rather than for the implications :-)) Pip has never implemented path hacks like this, and uses --single-version-externally-managed which avoids a lot of setuptools' controversial behaviour. So, by policy, it's hard to override stdlib modules - even when they don't work right (which appears to be the issue with readline on OSX, if I understand the comments in this thread properly). IMO, we'd be a lot better off readline was split out of the stdlib so that things like improved OSX support and pyreadline for Windows could be added. But I doubt that will ever happen :-( Paul. From chris.barker at noaa.gov Mon Dec 9 11:40:32 2013 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 9 Dec 2013 08:40:32 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 8:26 AM, Paul Moore wrote: > Guido specifically did not want to have user > installed packages able to overwrite stdlib behaviour. makes some sense... So, by policy, it's hard to override stdlib modules - even when they > don't work right (which appears to be the issue with readline on OSX, > if I understand the comments in this thread properly). IMO, we'd be a > lot better off readline was split out of the stdlib so that things > like improved OSX support and pyreadline for Windows could be added. > But I doubt that will ever happen :-( > Maybe rather than overriding an stsdlib package, it could be given a new name: "readline2", or somethign more creative. This wouldn't allow you to install and "fixed" one that would then automagically work everywhere, but would allow 3rd party packages to depend on it. And, of course, it oculd be imported conditionally: try: import readline2 as readline except: import readline On the other hand, the current third party readline really is a replacement for the (sometimes broken) built-in-readline, so maybe overwriting does make sense in this case -- even though it should be discouraged in teh general case. Anyone know if the readline authors would be amenable to putting the path-hack that's in iPyton in readline instead? That's clearly where it belongs...(or is that impossible?, after all it would have to get imported to run, and it wouldn't get imported......) -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmbecker at ucdavis.edu Mon Dec 9 12:14:34 2013 From: gmbecker at ucdavis.edu (Gabriel Becker) Date: Mon, 9 Dec 2013 09:14:34 -0800 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> <21156.21748.117583.339638@Ordinateur-de-Catherine-Konrad.local> Message-ID: Hey all, I'm actually drafting a thesis chapter about this very subject right now in the context of so-called "dynamic documents" in the R realm such as those processed by Sweave[1] and knitr[2]. The issue is with how we determine and track dependency. The current practice is to think of dependencies in terms of cells; as in Cell 2 depends on Cell 1, so if Cell 1 is changed (or has been run more recently than Cell 2) then Cell 2 needs to be rerun. This is a proxy for what we actually care about, which is whether our cached result for Cell 2 is safe to use. For code implementing deterministic algorithms, and assuming any external data and software versions remain unchanged, the only way for the result of running a piece of code to change is for the inputs (variables it assumes already exist and uses without assigning values to them) passed to/used by it to change. Under the assumption that a dynamic document (notebook) will be run in order from start to finish, checking for changes in cells previous to a particular cell (either all of them or via more sophisticated dependency detection) is sufficient to determine *whether it is possible that the inputs to the cell have changed* . This is what allows the proxy of tracking cell contents to predict result changes to be quite successful (and theoretically sound). In the case of interactive execution (or non-linear documents), however, this proxy breaks down. It is not sufficient to check Cell 1 to determine whether to rerun Cell 2 if we have no guarantee that Cell 1 was the last cell executed. The solution to this that I am pursuing is essentially memoization of code cells. That is, keying dependency detection and caching off of the actual values of the inputs used by a cell, rather than either the names of the input variables (as in the weaver[3], and knitr[2]) or simple position in the document. This means that if you have a certain input state, it doesn't matter whether you arrived there by executing Cell 1, or by some more convoluted method. The answer will be the same, so the same cache will be used (think equivalence partitioning for code cells). The input detection necessary to do this can be difficult in a general language like Python, but with some assumptions about the type of code being put in notebooks (and/or allowing authors to explicitly declare input variables) it can be manageable. I have implemented this concept for R code in the RCacheSuite R package[4] (currently on github, soon to be on CRAN), as well as in a custom %%Rcaching magic that lives in my experimental fork of the IPython project. The magic simply uses the RCacheSuite function though, so not much new there beyond what is in [4]. Happy to chat more about this (who doesn't want to talk about their thesis), but this is already getting a bit long, so I'll sign off for now. ~G P.S. I'm sure this goes without saying, but please remember to cite me if you end up pursuing this strategy [1] http://www.stat.uni-muenchen.de/~leisch/Sweave/ [2] http://yihui.name/knitr/ [3] http://www.bioconductor.org/packages/release/bioc/html/weaver.html [4] https://github.com/gmbecker/RCacheSuite [5] https://github.com/gmbecker/ipython On Sun, Dec 8, 2013 at 11:40 AM, Thomas Kluyver wrote: > > On 8 December 2013 03:16, Konrad Hinsen wrote: > >> > Yes - the prompt numbers by the cells record the order of execution. >> >> If that's the same information shown in the browser, then it's only >> the number of the last execution that remains available. And that >> means that I cannot reconstruct the history of execution. To make it >> worse, a cell could have been modified and re-executed, so the >> original code is no longer available. >> > > That's all the information available in the notebook frontend, but the > kernel records every cell executed, so there's a record of all the code > that has run in the present session. The kernel knows nothing about the > notebook document, so if for instance a cell has been edited since it was > executed, there's no easy way to know which cell that execution represents. > However, you can easily enough tell whether the current notebook has been > run through once with no editing, by comparing the list of code cells to > the history. > > >> >> > However, I think that the reproducible case should normally be the >> > one where the cells are run in order from top to bottom. Running >> > them out of order is convenient, but to check that a notebook all >> > works together, it's common to clear all output, reset the kernel, >> > and run all cells in order. >> >> That's what people should do, but I still need to decide what to do >> when a user runs cells randomly and then wants to save the notebook to >> an ActivePapers backend. Since any result datasets are in a >> non-reproducible state, the options are >> >> 1) Refuse with an error message. >> 2) Re-run the notebook in order before saving. >> 3) Save the notebook, but mark it and all its results as stale. >> Stale datasets are marked as such and require an update >> before they can be used in some other computation. >> >> I'll probably go for 3) and see how this works out. > > > 3 sounds like the best option to me: you can save the information, but it > shouldn't be considered as the fixed form for later calculations. > Presumably ActivePapers will be able to re-run the cells in order if it > needs to freshen the results, without the user having to manually open the > notebook and execute the cells. > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Gabriel Becker Graduate Student Statistics Department University of California, Davis -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Dec 9 12:30:04 2013 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 9 Dec 2013 09:30:04 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: Message-ID: On Mon, Dec 9, 2013 at 12:01 AM, Aaron Meurer wrote: > For wx, if you can figure out how to build a conda recipe for it, that > would help a lot. Sure. And I think Continuum is working on it now -- so that particular problem may be solved. But that was just an example -- until/unless we have most arbitrary third party package maintainers building for Anocanda, I"m not going to recommend it to the general user. There are two routes to a solution, though: 1) conda has some key advantages over pip+wheel -- so maybe we should all dump wheel and start using conda -- but this is going to be a hard sell, and Travis, for one, doesn't have the bandwidth to try. 2) make Anaconda / conda / binstar compatible with the python.org pythons and virtualenv. - It's actually pretty close now, and I think Continuum is working on closing teh gap, so that may be the way of the future. Not quite the "one way to do it" we'd like, but interoperability is a nice way to get close. What I really don't want is people having to drop their entire python install and use an entirely different one to get one new package -- or even worse there being no one install that supports all the packages a user needs. But in the meantime, wheels aren't so bad! (note -- not wheel for wxPython either, and pip-installing it from source is a non-starter. But that's my point. If these various systems are compatible, and one could install a binary intended for the python.orgpython into an Anocada distribution, users wouldn't find dead ends... Basically, you could just host your own recipe on > binstar, and if it works, it's pretty likely that Continuum would pick > it up and ship it with Anaconda. I tried it a while back but I didn't > get very far. But if you're actually familiar with it you might get > further. > I might -- but it really is a pain to build, and I haven't done it for years -- and no bandwidth at the moment. Maybe at some point. As for virtualenv, I really recommend trying conda environments. Well, this is about standards, not technology -- maybe conda environments are better, but the web-dev folks are all pretty committed to pip and virtualenv. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Dec 9 12:35:04 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 9 Dec 2013 09:35:04 -0800 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: <21157.50571.922623.163499@Konrad-Hinsens-MacBook-Pro-2.local> References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> <21156.21748.117583.339638@Ordinateur-de-Catherine-Konrad.local> <21157.50571.922623.163499@Konrad-Hinsens-MacBook-Pro-2.local> Message-ID: On 9 December 2013 05:28, Konrad Hinsen wrote: > That looks good enough for my purposes. Could you give me a pointer > to how I can access the kernel's history information for doing > such a comparison? > Matthias and Zoltan have pointed you to the SQLite history database. However, while the kernel's running, you can make a history_request to retrieve history: http://ipython.org/ipython-doc/dev/development/messaging.html#history I'm assuming that your code will be running in the notebook server process, which handles saving and loading files. It may be that the current implementation of history_request doesn't support selecting the history of an entire session, but the underlying APIs it uses do, so it would be easy to add that to the messages. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Dec 9 12:41:19 2013 From: benjaminrk at gmail.com (MinRK) Date: Mon, 9 Dec 2013 09:41:19 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 8:40 AM, Chris Barker wrote: > On Mon, Dec 9, 2013 at 8:26 AM, Paul Moore wrote: > >> Guido specifically did not want to have user >> installed packages able to overwrite stdlib behaviour. > > > makes some sense... > > So, by policy, it's hard to override stdlib modules - even when they >> don't work right (which appears to be the issue with readline on OSX, >> if I understand the comments in this thread properly). IMO, we'd be a >> lot better off readline was split out of the stdlib so that things >> like improved OSX support and pyreadline for Windows could be added. >> But I doubt that will ever happen :-( >> > > Maybe rather than overriding an stsdlib package, it could be given a new > name: > > "readline2", or somethign more creative. This wouldn't allow you to > install and "fixed" one that would then automagically work everywhere, but > would allow 3rd party packages to depend on it. And, of course, it oculd be > imported conditionally: > > try: > import readline2 as readline > except: > import readline > I actually made just this PR yesterday, it would solve the issue for IPython, if not in general. > > On the other hand, the current third party readline really is a > replacement for the (sometimes broken) built-in-readline, so maybe > overwriting does make sense in this case -- even though it should be > discouraged in teh general case. > > Anyone know if the readline authors would be amenable to putting the > path-hack that's in iPyton in readline instead? That's clearly where it > belongs...(or is that impossible?, after all it would have to get imported > to run, and it wouldn't get imported......) > The path hack must be in the importer of readline, not readline itself. > > -Chris > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Dec 9 12:59:04 2013 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 9 Dec 2013 09:59:04 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 9:41 AM, MinRK wrote: > I actually made just this PR yesterday, > it would solve the issue for IPython, if not in general. > thanks -- good idea. Anyone know if the readline authors would be amenable to putting the >> path-hack that's in iPyton in readline instead? That's clearly where it >> belongs...(or is that impossible?, after all it would have to get imported >> to run, and it wouldn't get imported......) >> > > The path hack must be in the importer of readline, not readline itself. > > I was hoping that someone smarter than me could figure out how to monkey-patch it on the fly -- but it probably really is impossible. Tiny note on the PR: it's not just that people _are_ installing readline with pip -- it's that it's that they _should_ be able to expect that to work. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Mon Dec 9 17:40:31 2013 From: asmeurer at gmail.com (Aaron Meurer) Date: Mon, 9 Dec 2013 15:40:31 -0700 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 9:26 AM, Paul Moore wrote: > On 9 December 2013 15:55, Chris Barker - NOAA Federal > wrote: >>> What's the reason you can't pip install readline? Is it because pip >>> insists on compiling it? >> >> No--it needs to be compiled either way--but I _think_ the issue is that pip >> and easy_install put packages on sys.path differently. The pop way puts it >> after the existing readline. > > easy_install hacks sys.path to put installed packages before the > standard library. That was controversial behaviour at the time it was > implemented, IIRC - Guido specifically did not want to have user > installed packages able to overwrite stdlib behaviour. I don't know if > that position has changed, it's so rarely relevant that I don't think > people care that much (they tend to care more about the nasty pth file > hacks just for their own sake, rather than for the implications :-)) Does Python 3.3 help the situation? They refactored the import system quite a bit there. Aaron Meurer > > Pip has never implemented path hacks like this, and uses > --single-version-externally-managed which avoids a lot of setuptools' > controversial behaviour. > > So, by policy, it's hard to override stdlib modules - even when they > don't work right (which appears to be the issue with readline on OSX, > if I understand the comments in this thread properly). IMO, we'd be a > lot better off readline was split out of the stdlib so that things > like improved OSX support and pyreadline for Windows could be added. > But I doubt that will ever happen :-( > > Paul. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From asmeurer at gmail.com Mon Dec 9 17:42:00 2013 From: asmeurer at gmail.com (Aaron Meurer) Date: Mon, 9 Dec 2013 15:42:00 -0700 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: There's the rl library. https://pypi.python.org/pypi/rl. Aaron Meurer On Mon, Dec 9, 2013 at 9:40 AM, Chris Barker wrote: > On Mon, Dec 9, 2013 at 8:26 AM, Paul Moore wrote: >> >> Guido specifically did not want to have user >> installed packages able to overwrite stdlib behaviour. > > > makes some sense... > >> So, by policy, it's hard to override stdlib modules - even when they >> don't work right (which appears to be the issue with readline on OSX, >> if I understand the comments in this thread properly). IMO, we'd be a >> lot better off readline was split out of the stdlib so that things >> like improved OSX support and pyreadline for Windows could be added. >> But I doubt that will ever happen :-( > > > Maybe rather than overriding an stsdlib package, it could be given a new > name: > > "readline2", or somethign more creative. This wouldn't allow you to install > and "fixed" one that would then automagically work everywhere, but would > allow 3rd party packages to depend on it. And, of course, it oculd be > imported conditionally: > > try: > import readline2 as readline > except: > import readline > > On the other hand, the current third party readline really is a replacement > for the (sometimes broken) built-in-readline, so maybe overwriting does make > sense in this case -- even though it should be discouraged in teh general > case. > > Anyone know if the readline authors would be amenable to putting the > path-hack that's in iPyton in readline instead? That's clearly where it > belongs...(or is that impossible?, after all it would have to get imported > to run, and it wouldn't get imported......) > > -Chris > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From chris.barker at noaa.gov Mon Dec 9 17:44:54 2013 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 9 Dec 2013 14:44:54 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 2:42 PM, Aaron Meurer wrote: > There's the rl library. https://pypi.python.org/pypi/rl. > > Any idea if it supports (a superset of) the same API as the built-in readline? -Chris > Aaron Meurer > > On Mon, Dec 9, 2013 at 9:40 AM, Chris Barker > wrote: > > On Mon, Dec 9, 2013 at 8:26 AM, Paul Moore wrote: > >> > >> Guido specifically did not want to have user > >> installed packages able to overwrite stdlib behaviour. > > > > > > makes some sense... > > > >> So, by policy, it's hard to override stdlib modules - even when they > >> don't work right (which appears to be the issue with readline on OSX, > >> if I understand the comments in this thread properly). IMO, we'd be a > >> lot better off readline was split out of the stdlib so that things > >> like improved OSX support and pyreadline for Windows could be added. > >> But I doubt that will ever happen :-( > > > > > > Maybe rather than overriding an stsdlib package, it could be given a new > > name: > > > > "readline2", or somethign more creative. This wouldn't allow you to > install > > and "fixed" one that would then automagically work everywhere, but would > > allow 3rd party packages to depend on it. And, of course, it oculd be > > imported conditionally: > > > > try: > > import readline2 as readline > > except: > > import readline > > > > On the other hand, the current third party readline really is a > replacement > > for the (sometimes broken) built-in-readline, so maybe overwriting does > make > > sense in this case -- even though it should be discouraged in teh general > > case. > > > > Anyone know if the readline authors would be amenable to putting the > > path-hack that's in iPyton in readline instead? That's clearly where it > > belongs...(or is that impossible?, after all it would have to get > imported > > to run, and it wouldn't get imported......) > > > > -Chris > > > > > > -- > > > > Christopher Barker, Ph.D. > > Oceanographer > > > > Emergency Response Division > > NOAA/NOS/OR&R (206) 526-6959 voice > > 7600 Sand Point Way NE (206) 526-6329 fax > > Seattle, WA 98115 (206) 526-6317 main reception > > > > Chris.Barker at noaa.gov > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Mon Dec 9 17:46:32 2013 From: asmeurer at gmail.com (Aaron Meurer) Date: Mon, 9 Dec 2013 15:46:32 -0700 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: Message-ID: On Mon, Dec 9, 2013 at 10:30 AM, Chris Barker wrote: > On Mon, Dec 9, 2013 at 12:01 AM, Aaron Meurer wrote: >> >> For wx, if you can figure out how to build a conda recipe for it, that >> would help a lot. > > > Sure. And I think Continuum is working on it now -- so that particular > problem may be solved. > > But that was just an example -- until/unless we have most arbitrary third > party package maintainers building for Anocanda, I"m not going to recommend > it to the general user. There are two routes to a solution, though: > > 1) conda has some key advantages over pip+wheel -- so maybe we should all > dump wheel and start using conda -- but this is going to be a hard sell, and > Travis, for one, doesn't have the bandwidth to try. Also read http://technicaldiscovery.blogspot.com/2013/12/why-i-promote-conda.html. To quote: "several of us asked Guido what we can do to fix Python packaging for the NumPy stack. Guido's answer was to 'solve the problem ourselves'. " > > 2) make Anaconda / conda / binstar compatible with the python.org pythons > and virtualenv. > - It's actually pretty close now, and I think Continuum is working on > closing teh gap, so that may be the way of the future. Not quite the "one > way to do it" we'd like, but interoperability is a nice way to get close. > What I really don't want is people having to drop their entire python > install and use an entirely different one to get one new package -- or even > worse there being no one install that supports all the packages a user > needs. > > But in the meantime, wheels aren't so bad! > > (note -- not wheel for wxPython either, and pip-installing it from source is > a non-starter. But that's my point. If these various systems are compatible, > and one could install a binary intended for the python.org python into an > Anocada distribution, users wouldn't find dead ends... > >> Basically, you could just host your own recipe on >> binstar, and if it works, it's pretty likely that Continuum would pick >> it up and ship it with Anaconda. I tried it a while back but I didn't >> get very far. But if you're actually familiar with it you might get >> further. > > > I might -- but it really is a pain to build, and I haven't done it for years > -- and no bandwidth at the moment. Maybe at some point. > >> As for virtualenv, I really recommend trying conda environments. > > > Well, this is about standards, not technology -- maybe conda environments > are better, but the web-dev folks are all pretty committed to pip and > virtualenv. I figured as much. But as I noted, in Python 2 at least, in my minimal testing, virtualenv actually seems to work just fine, once you have a conda recipe for it. I've built a handful of conda recipes that have virtualenv as a hard dependency and they all seemed to work as well. Aaron Meurer > > -Chris > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From asmeurer at gmail.com Mon Dec 9 17:47:49 2013 From: asmeurer at gmail.com (Aaron Meurer) Date: Mon, 9 Dec 2013 15:47:49 -0700 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 3:44 PM, Chris Barker wrote: > On Mon, Dec 9, 2013 at 2:42 PM, Aaron Meurer wrote: >> >> There's the rl library. https://pypi.python.org/pypi/rl. >> > Any idea if it supports (a superset of) the same API as the built-in > readline? > > -Chris I'm not 100%, but I think it does. It definitely supports way more things than built-in readline. But I'm not 100% sure if everything from readline is also in rl. Anyway, if it doesn't, it's all open source :) Aaron Meurer > > > > > > >> >> Aaron Meurer >> >> On Mon, Dec 9, 2013 at 9:40 AM, Chris Barker >> wrote: >> > On Mon, Dec 9, 2013 at 8:26 AM, Paul Moore wrote: >> >> >> >> Guido specifically did not want to have user >> >> installed packages able to overwrite stdlib behaviour. >> > >> > >> > makes some sense... >> > >> >> So, by policy, it's hard to override stdlib modules - even when they >> >> don't work right (which appears to be the issue with readline on OSX, >> >> if I understand the comments in this thread properly). IMO, we'd be a >> >> lot better off readline was split out of the stdlib so that things >> >> like improved OSX support and pyreadline for Windows could be added. >> >> But I doubt that will ever happen :-( >> > >> > >> > Maybe rather than overriding an stsdlib package, it could be given a new >> > name: >> > >> > "readline2", or somethign more creative. This wouldn't allow you to >> > install >> > and "fixed" one that would then automagically work everywhere, but would >> > allow 3rd party packages to depend on it. And, of course, it oculd be >> > imported conditionally: >> > >> > try: >> > import readline2 as readline >> > except: >> > import readline >> > >> > On the other hand, the current third party readline really is a >> > replacement >> > for the (sometimes broken) built-in-readline, so maybe overwriting does >> > make >> > sense in this case -- even though it should be discouraged in teh >> > general >> > case. >> > >> > Anyone know if the readline authors would be amenable to putting the >> > path-hack that's in iPyton in readline instead? That's clearly where it >> > belongs...(or is that impossible?, after all it would have to get >> > imported >> > to run, and it wouldn't get imported......) >> > >> > -Chris >> > >> > >> > -- >> > >> > Christopher Barker, Ph.D. >> > Oceanographer >> > >> > Emergency Response Division >> > NOAA/NOS/OR&R (206) 526-6959 voice >> > 7600 Sand Point Way NE (206) 526-6329 fax >> > Seattle, WA 98115 (206) 526-6317 main reception >> > >> > Chris.Barker at noaa.gov >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From benjaminrk at gmail.com Mon Dec 9 18:03:34 2013 From: benjaminrk at gmail.com (MinRK) Date: Mon, 9 Dec 2013 15:03:34 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 2:47 PM, Aaron Meurer wrote: > On Mon, Dec 9, 2013 at 3:44 PM, Chris Barker > wrote: > > On Mon, Dec 9, 2013 at 2:42 PM, Aaron Meurer wrote: > >> > >> There's the rl library. https://pypi.python.org/pypi/rl. > >> > > Any idea if it supports (a superset of) the same API as the built-in > > readline? > > > > -Chris > > I'm not 100%, but I think it does. It definitely supports way more > things than built-in readline. But I'm not 100% sure if everything > from readline is also in rl. Anyway, if it doesn't, it's all open > source :) > Looks like it's pretty much compatible. This might the the way to go, though Ludwig just merged my PR on python-readline, which we know already works. One of the sticky things with wheels (and easy with setup.py) is that a readline package should only be a dependency if stdlib readline is actually libedit. That's true of System Python and Python.org, but it is not true of conda Python or macports (both of which regularly pip install ipython, so it matters, even though they both have IPython packages). So if IPython is going to start building wheels, it needs to be able to express dependencies like that, and the platform is simply insufficient information. -MinRK > > Aaron Meurer > > > > > > > > > > > > > > >> > >> Aaron Meurer > >> > >> On Mon, Dec 9, 2013 at 9:40 AM, Chris Barker > >> wrote: > >> > On Mon, Dec 9, 2013 at 8:26 AM, Paul Moore > wrote: > >> >> > >> >> Guido specifically did not want to have user > >> >> installed packages able to overwrite stdlib behaviour. > >> > > >> > > >> > makes some sense... > >> > > >> >> So, by policy, it's hard to override stdlib modules - even when they > >> >> don't work right (which appears to be the issue with readline on OSX, > >> >> if I understand the comments in this thread properly). IMO, we'd be a > >> >> lot better off readline was split out of the stdlib so that things > >> >> like improved OSX support and pyreadline for Windows could be added. > >> >> But I doubt that will ever happen :-( > >> > > >> > > >> > Maybe rather than overriding an stsdlib package, it could be given a > new > >> > name: > >> > > >> > "readline2", or somethign more creative. This wouldn't allow you to > >> > install > >> > and "fixed" one that would then automagically work everywhere, but > would > >> > allow 3rd party packages to depend on it. And, of course, it oculd be > >> > imported conditionally: > >> > > >> > try: > >> > import readline2 as readline > >> > except: > >> > import readline > >> > > >> > On the other hand, the current third party readline really is a > >> > replacement > >> > for the (sometimes broken) built-in-readline, so maybe overwriting > does > >> > make > >> > sense in this case -- even though it should be discouraged in teh > >> > general > >> > case. > >> > > >> > Anyone know if the readline authors would be amenable to putting the > >> > path-hack that's in iPyton in readline instead? That's clearly where > it > >> > belongs...(or is that impossible?, after all it would have to get > >> > imported > >> > to run, and it wouldn't get imported......) > >> > > >> > -Chris > >> > > >> > > >> > -- > >> > > >> > Christopher Barker, Ph.D. > >> > Oceanographer > >> > > >> > Emergency Response Division > >> > NOAA/NOS/OR&R (206) 526-6959 voice > >> > 7600 Sand Point Way NE (206) 526-6329 fax > >> > Seattle, WA 98115 (206) 526-6317 main reception > >> > > >> > Chris.Barker at noaa.gov > >> > > >> > _______________________________________________ > >> > IPython-dev mailing list > >> > IPython-dev at scipy.org > >> > http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > > > -- > > > > Christopher Barker, Ph.D. > > Oceanographer > > > > Emergency Response Division > > NOAA/NOS/OR&R (206) 526-6959 voice > > 7600 Sand Point Way NE (206) 526-6329 fax > > Seattle, WA 98115 (206) 526-6317 main reception > > > > Chris.Barker at noaa.gov > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Mon Dec 9 18:28:25 2013 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 9 Dec 2013 23:28:25 +0000 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On 9 December 2013 22:40, Aaron Meurer wrote: >> easy_install hacks sys.path to put installed packages before the >> standard library. That was controversial behaviour at the time it was >> implemented, IIRC - Guido specifically did not want to have user >> installed packages able to overwrite stdlib behaviour. I don't know if >> that position has changed, it's so rarely relevant that I don't think >> people care that much (they tend to care more about the nasty pth file >> hacks just for their own sake, rather than for the implications :-)) > > Does Python 3.3 help the situation? They refactored the import system > quite a bit there. Not particularly - ultimately, the key point here is that you're not *supposed* to override the standard library modules (as I understand it - this is only my recollection of the debates at the time!) To me, the real fix would be for IPython to import its *own* readline, which wraps the stdlib readline when appropriate, and other modules as needed. Then there's no overriding and no name clashes to worry about. (If that was done, pyreadline could also stop making itself available as "readline" so it too becomes available only when requested). AIUI, that's what MinRK's patch does. Ideally, the core Python interactive prompt would *also* allow users to plug in their own readline functionality, but that's somewhat orthogonal to how external projects like IPython deal with the issue. Paul From benjaminrk at gmail.com Mon Dec 9 18:38:39 2013 From: benjaminrk at gmail.com (MinRK) Date: Mon, 9 Dec 2013 15:38:39 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 3:28 PM, Paul Moore wrote: > On 9 December 2013 22:40, Aaron Meurer wrote: > >> easy_install hacks sys.path to put installed packages before the > >> standard library. That was controversial behaviour at the time it was > >> implemented, IIRC - Guido specifically did not want to have user > >> installed packages able to overwrite stdlib behaviour. I don't know if > >> that position has changed, it's so rarely relevant that I don't think > >> people care that much (they tend to care more about the nasty pth file > >> hacks just for their own sake, rather than for the implications :-)) > > > > Does Python 3.3 help the situation? They refactored the import system > > quite a bit there. > > Not particularly - ultimately, the key point here is that you're not > *supposed* to override the standard library modules (as I understand > it - this is only my recollection of the debates at the time!) > > To me, the real fix would be for IPython to import its *own* readline, > which wraps the stdlib readline when appropriate, and other modules as > needed. Then there's no overriding and no name clashes to worry about. > (If that was done, pyreadline could also stop making itself available > as "readline" so it too becomes available only when requested). AIUI, > that's what MinRK's patch does. > This is what IPython already does (in utils.rlineimpl). The only missing piece was that the readline package used the name 'readline'. I should have proposed giving it a non-colliding name as soon as we started seeing the problems caused by pip. I don't know why I didn't do it sooner, it totally solves the issue. > > Ideally, the core Python interactive prompt would *also* allow users > to plug in their own readline functionality, but that's somewhat > orthogonal to how external projects like IPython deal with the issue. > an import statement in PYTHONSTARTUP might be sufficient. > Paul > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Mon Dec 9 18:43:49 2013 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 9 Dec 2013 23:43:49 +0000 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On 9 December 2013 23:03, MinRK wrote: > One of the sticky things with wheels (and easy with setup.py) is that a > readline package should only be a dependency if stdlib readline is actually > libedit. That's true of System Python and Python.org, but it is not true of > conda Python or macports (both of which regularly pip install ipython, so it > matters, even though they both have IPython packages). So if IPython is > going to start building wheels, it needs to be able to express dependencies > like that, and the platform is simply insufficient information. This is similar to some of the current debates on distutils-sig around compatibility checking for wheels, where the same issue arises. On the one hand, we want to avoid executing arbitrary code at install time (because doing so makes static introspection impossible, among other things) and yet we need to be able to express constraints people need. So we need to make sure we understand those constraints. So on that basis, can you tell me (or point me at the relevant code) how IPython determines whether readline is needed? In the setup.py I see if not setupext.check_for_readline(): if sys.platform == 'darwin': requires.append('readline') elif sys.platform.startswith('win'): # Pyreadline 64 bit windows issue solved in versions >=1.7.1 # Also solves issues with some older versions of pyreadline that # satisfy the unconstrained depdendency. requires.append('pyreadline>=1.7.1') else: pass which is basically a platform check if check_for_readline fails. The key thing in check_for_readline appears to me to be whether readline.__doc__ contains "libedit". Is that what you need? Obviously pip/wheel would never implement something this specific, but I *could* see a conditional something like "check the value of a variable imported from a module on the installation target machine" as being worth discussion. I'd quite happily argue, though, that if the core readline library is sufficiently broken when built with libedit on OSX to cause all of these problems, then that should surely be flagged as a bug with core Python. Maybe Python should just not build the builtin readline if all it has available is libedit - has any OSX user had this discussion with the core team? Paul From p.f.moore at gmail.com Mon Dec 9 18:47:52 2013 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 9 Dec 2013 23:47:52 +0000 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On 9 December 2013 23:38, MinRK wrote: >> Ideally, the core Python interactive prompt would *also* allow users >> to plug in their own readline functionality, but that's somewhat >> orthogonal to how external projects like IPython deal with the issue. > > an import statement in PYTHONSTARTUP might be sufficient. Good point. If that works, I may submit a PR to pyreadline and IPython to *not* have pyreadline use the colliding "readline" name by default - users who want to have the functionality in the core interpreter can configure it to do so, and users who have problems with that (as I did - see https://github.com/pyreadline/pyreadline/issues/18) can avoid doing so. I'll try it out. Paul From benjaminrk at gmail.com Mon Dec 9 18:48:49 2013 From: benjaminrk at gmail.com (MinRK) Date: Mon, 9 Dec 2013 15:48:49 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: > which is basically a platform check if check_for_readline fails check_for_readline looks for libedit in readline.__doc__ (now, it always should have). On Mon, Dec 9, 2013 at 3:43 PM, Paul Moore wrote: > On 9 December 2013 23:03, MinRK wrote: > > One of the sticky things with wheels (and easy with setup.py) is that a > > readline package should only be a dependency if stdlib readline is > actually > > libedit. That's true of System Python and Python.org, but it is not true > of > > conda Python or macports (both of which regularly pip install ipython, > so it > > matters, even though they both have IPython packages). So if IPython is > > going to start building wheels, it needs to be able to express > dependencies > > like that, and the platform is simply insufficient information. > > This is similar to some of the current debates on distutils-sig around > compatibility checking for wheels, where the same issue arises. On the > one hand, we want to avoid executing arbitrary code at install time > (because doing so makes static introspection impossible, among other > things) and yet we need to be able to express constraints people need. > So we need to make sure we understand those constraints. > > So on that basis, can you tell me (or point me at the relevant code) > how IPython determines whether readline is needed? In the setup.py I > see > > if not setupext.check_for_readline(): > if sys.platform == 'darwin': > requires.append('readline') > elif sys.platform.startswith('win'): > # Pyreadline 64 bit windows issue solved in versions >=1.7.1 > # Also solves issues with some older versions of pyreadline > that > # satisfy the unconstrained depdendency. > requires.append('pyreadline>=1.7.1') > else: > pass > > which is basically a platform check if check_for_readline fails. The > key thing in check_for_readline appears to me to be whether > readline.__doc__ contains "libedit". Is that what you need? Obviously > pip/wheel would never implement something this specific, but I *could* > see a conditional something like "check the value of a variable > imported from a module on the installation target machine" as being > worth discussion. > > I'd quite happily argue, though, that if the core readline library is > sufficiently broken when built with libedit on OSX to cause all of > these problems, then that should surely be flagged as a bug with core > Python. Maybe Python should just not build the builtin readline if all > it has available is libedit - has any OSX user had this discussion > with the core team? > > Paul > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Dec 9 20:18:19 2013 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 9 Dec 2013 17:18:19 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 3:43 PM, Paul Moore wrote: > we need to be able to express constraints people need. > So we need to make sure we understand those constraints. > > Obviously > pip/wheel would never implement something this specific, but I *could* > see a conditional something like "check the value of a variable > imported from a module on the installation target machine" as being > worth discussion. as you pointed out -- this is essentially a platform check. So we could do OK if there was a way to express paltform-specific dependencies to pip. so that, for instance, "pip install ipython", would get readline as a dependency on OS-X, but not on linux. That would be pretty simple, solve this problem and probably be useful elsewhere. However, in this case, a I tend to agree with what we think Guido thinks ( ;-) ), that overriding a build-in module is a bad idea, pyreadline really should use a different name, and then ipython could simply depend on that. Granted, that would mean duplicate functionality, but if readline is somewhat broken somewhere, maybe not so bad to be consistent. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Dec 9 20:23:40 2013 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 9 Dec 2013 17:23:40 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 3:43 PM, Paul Moore wrote: > I'd quite happily argue, though, that if the core readline library is > sufficiently broken when built with libedit on OSX to cause all of > these problems, then that should surely be flagged as a bug with core > Python. It's actually VERY broken with OS-X 10.9 (Mavericks) https://mail.python.org/pipermail/python-dev/2013-October/129832.html That's being fixed, but it kind of points out how depending on apple's libedit is kind of fragile, and depending on whatever the system happens to have installed is maybe not such a good idea. Too bad about the licensing of readline, but maybe it would actually be better if core python simple used an appropriately licenced library and bundles that -- i.e. libedit, though I guess it really isn't as good. -Chris > Maybe Python should just not build the builtin readline if all > it has available is libedit - has any OSX user had this discussion > with the core team? > > Paul > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Mon Dec 9 21:37:19 2013 From: asmeurer at gmail.com (Aaron Meurer) Date: Mon, 9 Dec 2013 19:37:19 -0700 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: <5251128916867056296@unknownmsgid> On Dec 9, 2013, at 4:28 PM, Paul Moore wrote: On 9 December 2013 22:40, Aaron Meurer wrote: easy_install hacks sys.path to put installed packages before the standard library. That was controversial behaviour at the time it was implemented, IIRC - Guido specifically did not want to have user installed packages able to overwrite stdlib behaviour. I don't know if that position has changed, it's so rarely relevant that I don't think people care that much (they tend to care more about the nasty pth file hacks just for their own sake, rather than for the implications :-)) Does Python 3.3 help the situation? They refactored the import system quite a bit there. Not particularly - ultimately, the key point here is that you're not *supposed* to override the standard library modules (as I understand it - this is only my recollection of the debates at the time!) To me, the real fix would be for IPython to import its *own* readline, which wraps the stdlib readline when appropriate, and other modules as needed. Then there's no overriding and no name clashes to worry about. (If that was done, pyreadline could also stop making itself available as "readline" so it too becomes available only when requested). AIUI, that's what MinRK's patch does. Ideally, the core Python interactive prompt would *also* allow users to plug in their own readline functionality, but that's somewhat orthogonal to how external projects like IPython deal with the issue This is related to the discussion at https://github.com/ipython/ipython/pull/4353. My opinion is that IPython should not try to build an API on top of readline. Rather, just use readline's own API and make sure it works with IPython. Aaron Meurer Paul _______________________________________________ IPython-dev mailing list IPython-dev at scipy.org http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Mon Dec 9 22:02:01 2013 From: asmeurer at gmail.com (Aaron Meurer) Date: Mon, 9 Dec 2013 20:02:01 -0700 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: <-3725129708364469899@unknownmsgid> On Dec 9, 2013, at 6:24 PM, Chris Barker wrote: On Mon, Dec 9, 2013 at 3:43 PM, Paul Moore wrote: > I'd quite happily argue, though, that if the core readline library is > sufficiently broken when built with libedit on OSX to cause all of > these problems, then that should surely be flagged as a bug with core > Python. It's actually VERY broken with OS-X 10.9 (Mavericks) https://mail.python.org/pipermail/python-dev/2013-October/129832.html That's being fixed, but it kind of points out how depending on apple's libedit is kind of fragile, and depending on whatever the system happens to have installed is maybe not such a good idea. Too bad about the licensing of readline, but maybe it would actually be better if core python simple used an appropriately licenced library and bundles that -- i.e. libedit, though I guess it really isn't as good. I had thought or was liberally licensed, but now I'm not so sure. PyPI lists it as both GPL and PSF. I guess we should ask the author. Aaron Meurer -Chris > Maybe Python should just not build the builtin readline if all > it has available is libedit - has any OSX user had this discussion > with the core team? > > Paul > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov _______________________________________________ IPython-dev mailing list IPython-dev at scipy.org http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Dec 9 23:48:14 2013 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 9 Dec 2013 20:48:14 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: <-3725129708364469899@unknownmsgid> References: <-5064963750921577331@unknownmsgid> <-3725129708364469899@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 7:02 PM, Aaron Meurer wrote: > Too bad about the licensing of readline, but maybe it would actually be > better if core python simple used an appropriately licenced library and > bundles that -- i.e. libedit, though I guess it really isn't as good. > > > I had thought or was liberally licensed, but now I'm not so sure. PyPI > lists it as both GPL and PSF. I guess we should ask the author. > > the readline lib itself is GPL -- in fact it's used an example by teh FSF for why you should license things that way -- even libraries. I suspect that readline itself is GPL. and the Python wrappers are PSF -- which makes the whole thing GPL once you actually use it....Which readline itself is not in the python source tree, and why it's not on the Mac. -Chris > Aaron Meurer > > > -Chris > > > > > >> Maybe Python should just not build the builtin readline if all >> it has available is libedit - has any OSX user had this discussion >> with the core team? >> >> Paul >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Tue Dec 10 01:19:57 2013 From: asmeurer at gmail.com (Aaron Meurer) Date: Mon, 9 Dec 2013 23:19:57 -0700 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> <-3725129708364469899@unknownmsgid> Message-ID: > pyreadline really should use a different name, and then ipython could simply depend on that. +1. There are several instances of third party modules "rewriting" standard library modules for whatever reason (urllib3, regex, tulip). In all cases, they just use different names, and leave it up to the libraries that use them to choose them over the stdlib versions. On Mon, Dec 9, 2013 at 9:48 PM, Chris Barker wrote: > On Mon, Dec 9, 2013 at 7:02 PM, Aaron Meurer wrote: >> >> Too bad about the licensing of readline, but maybe it would actually be >> better if core python simple used an appropriately licenced library and >> bundles that -- i.e. libedit, though I guess it really isn't as good. >> >> >> I had thought or was liberally licensed, but now I'm not so sure. PyPI >> lists it as both GPL and PSF. I guess we should ask the author. >> > the readline lib itself is GPL -- in fact it's used an example by teh FSF > for why you should license things that way -- even libraries. > > I suspect that readline itself is GPL. and the Python wrappers are PSF -- > which makes the whole thing GPL once you actually use it....Which readline > itself is not in the python source tree, and why it's not on the Mac. > > -Chris I asked at https://github.com/stefanholek/rl/issues/9 so we can find out for sure. Aaron Meurer > > > > >> >> Aaron Meurer >> >> >> -Chris >> >> >> >> >>> >>> Maybe Python should just not build the builtin readline if all >>> it has available is libedit - has any OSX user had this discussion >>> with the core team? >>> >>> Paul >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> >> >> -- >> >> Christopher Barker, Ph.D. >> Oceanographer >> >> Emergency Response Division >> NOAA/NOS/OR&R (206) 526-6959 voice >> 7600 Sand Point Way NE (206) 526-6329 fax >> Seattle, WA 98115 (206) 526-6317 main reception >> >> Chris.Barker at noaa.gov >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From benjaminrk at gmail.com Tue Dec 10 01:51:49 2013 From: benjaminrk at gmail.com (MinRK) Date: Mon, 9 Dec 2013 22:51:49 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On Mon, Dec 9, 2013 at 5:18 PM, Chris Barker wrote: > On Mon, Dec 9, 2013 at 3:43 PM, Paul Moore wrote: > >> we need to be able to express constraints people need. >> So we need to make sure we understand those constraints. >> >> Obviously >> pip/wheel would never implement something this specific, but I *could* >> see a conditional something like "check the value of a variable >> imported from a module on the installation target machine" as being >> worth discussion. > > > as you pointed out -- this is essentially a platform check. > > So we could do OK if there was a way to express paltform-specific > dependencies to pip. so that, for instance, "pip install ipython", would > get readline as a dependency on OS-X, but not on linux. That would be > pretty simple, solve this problem and probably be useful elsewhere. > It is not jut a platform check - depending on an external readline package is the right thing to do on OS X System Python and Python.org, but not on macports, conda, or manually compiled Python linked against readline. A simple platform check is insufficient, libedit must be specifically checked for before expressing a dependency on the readline package. In the very brief period that it was an unconditional dependency on OS X, we already had reports of failed installs due to failed compilation of the readline module on a Python where it wasn't needed, so that's just not acceptable. > > However, in this case, a I tend to agree with what we think Guido thinks ( > ;-) ), that overriding a build-in module is a bad idea, pyreadline really > should use a different name, and then ipython could simply depend on that. > Granted, that would mean duplicate functionality, but if readline is > somewhat broken somewhere, maybe not so bad to be consistent. > Regardless of where we land on the rest, this is definitely true - renaming the installed module is the easiest way to solve the runtime issue, so that's what we are doing now, and should have done as soon as we realized that we were depending on gross easy_install behavior that pip "fixes". But the condition of the dependency is still more specific than simply `sys.platform=='darwin'`. > overriding a build-in module is a bad idea By the way on this one, Apple has made this extra complicated: 10.9 ships with numpy, scipy, and matplotlib (among other things - setuptools, twisted, etc.). It ships these in an Extras directory, which is, as far as `sys.path` priority is concerned, part of the stdlib. That means that you will not be able to `pip install` any of those packages with System Python on OS X 10.9. Or, more precisely, pip will install them, but after the successful install, they will never be importable. If you want to install or upgrade numpy, scipy, or setuptools on OS X, you must do one (or more) of: 1. use virtualenv 2. use non-System Python 3. use easy_install 4. use manual setup.py install (with setuptools) 5. use `pip install -e`, which *does* appear ahead of stdlib, unlike regular `pip install`. Note that this includes setuptools, so you need to make sure that pip is never in charge of updating setuptools, because it will result in *removing* importable setuptools, and you will have to start from scratch bootstrapping setuptools from ez_setup.py. -MinRK > -Chris > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ludwig.schwardt at gmail.com Tue Dec 10 02:55:56 2013 From: ludwig.schwardt at gmail.com (Ludwig Schwardt) Date: Tue, 10 Dec 2013 09:55:56 +0200 Subject: [IPython-dev] iPython binary wheels for OS-X Message-ID: Hi, > I had thought [rl] was liberally licensed, but now I'm not so sure. PyPI > lists it as both GPL and PSF. I guess we should ask the author. Chris is right on this one. The GNU readline library itself is GPL, which is the real source of all these issues :-) > By the way on this one, Apple has made this extra complicated: 10.9 ships > with numpy, scipy, and matplotlib (among other things - setuptools, > twisted, etc.). I can add a sixth option: "brew install" from samueljohn's tap. I've been running system Python for many years until I decided to try out brew Python in this OS iteration, so I guess I sidestepped the problem... There is also a positive side to Apple's latest incarnation - if a newbie simply wants to try out numpy/scipy/matplotlib to see what it is about, he/she does not have to install *anything*! OK, maybe IPython ;-) Having numpy 1.6.2, scipy 0.11.0 and matplotlib 1.1.1 out of the box is not too shabby for most purposes. For the power users it is still a source of irritation, though. Renaming the module is definitely a good avenue to explore. I'll check with the other regular users such as ActivePython whether it works for them as well. Regards, Ludwig -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Tue Dec 10 02:58:23 2013 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 10 Dec 2013 07:58:23 +0000 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: On 10 December 2013 06:51, MinRK wrote: > By the way on this one, Apple has made this extra complicated: 10.9 ships > with numpy, scipy, and matplotlib (among other things - setuptools, twisted, > etc.). It ships these in an Extras directory, which is, as far as `sys.path` > priority is concerned, part of the stdlib. That means that you will not be > able to `pip install` any of those packages with System Python on OS X 10.9. > Or, more precisely, pip will install them, but after the successful install, > they will never be importable. If you want to install or upgrade numpy, > scipy, or setuptools on OS X, you must do one (or more) of: > > 1. use virtualenv > 2. use non-System Python > 3. use easy_install > 4. use manual setup.py install (with setuptools) > 5. use `pip install -e`, which *does* appear ahead of stdlib, unlike regular > `pip install`. > > Note that this includes setuptools, so you need to make sure that pip is > never in charge of updating setuptools, because it will result in *removing* > importable setuptools, and you will have to start from scratch bootstrapping > setuptools from ez_setup.py. Whoops, this sounds pretty significant. Would it be OK if I reposted it on distutils-sig? I don't think it's something we can fix particularly easily but it would be worth people being aware of it (and Nick Coghlan is on distutils-sig, so he might feel it's worth raising with python-dev for their thoughts) Paul From chris.barker at noaa.gov Tue Dec 10 10:58:53 2013 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Tue, 10 Dec 2013 07:58:53 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: <8829639482250130422@unknownmsgid> On Dec 9, 2013, at 10:52 PM, MinRK wrote: > By the way on this one, Apple has made this extra complicated: 10.9 ships with numpy, scipy, and matplotlib (among other things - setuptools, twisted, etc.). It ships these in an Extras directory, which is, as far as `sys.path` priority is concerned, part of the stdlib. That means that you will not be able to `pip install` any of those packages with System Python on OS X 10.9. Or, more precisely, pip will install them, but after the successful install, they will never be importable. If FWIW, it has always been the case that Apple treats its python install like a system lib, and we all should treat it as such, too. I.e, never upgrade anything in it. And Apple never upgrades it either, even bug fix releases. I've always advised people to install a new python for their own development--generally the python.org one. It just get more important the more stuff Apple ships. Ironically, Apple's python gets more useful out of the box, too. > you want to install or upgrade numpy, scipy, or setuptools on OS X, Or anything else... Even if you _could_ upgrade stuff, you really don't want to mess with system links--you never know what's using them. -Chris From chris.barker at noaa.gov Tue Dec 10 11:03:32 2013 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Tue, 10 Dec 2013 08:03:32 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: References: <-5064963750921577331@unknownmsgid> Message-ID: <-4721714428654618378@unknownmsgid> On Dec 9, 2013, at 11:58 PM, Paul Moore wrote: > Whoops, this sounds pretty significant. Would it be OK if I reposted > it on distutils-sig? I don't think it's something we can fix > particularly easily Or should fix -- people shouldn't be messing with the system python anyway, > but it would be worth people being aware of it Yes, awareness is always good -- The more people that know how to answer the inevitable questions that come up, the better. - Chris > (and Nick Coghlan is on distutils-sig, so he might feel it's worth > raising with python-dev for their thoughts) > > Paul > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From benjaminrk at gmail.com Tue Dec 10 13:36:28 2013 From: benjaminrk at gmail.com (MinRK) Date: Tue, 10 Dec 2013 10:36:28 -0800 Subject: [IPython-dev] iPython binary wheels for OS-X In-Reply-To: <-4721714428654618378@unknownmsgid> References: <-5064963750921577331@unknownmsgid> <-4721714428654618378@unknownmsgid> Message-ID: On Tue, Dec 10, 2013 at 8:03 AM, Chris Barker - NOAA Federal < chris.barker at noaa.gov> wrote: > On Dec 9, 2013, at 11:58 PM, Paul Moore wrote: > > > Whoops, this sounds pretty significant. Would it be OK if I reposted > > it on distutils-sig? I don't think it's something we can fix > > particularly easily > > Or should fix -- people shouldn't be messing with the system python anyway, > This is what's so frustrating - the System Python isn't touched at all, nothing is "messed with", unless by "messed with", you just mean "used at all". No problems are caused by a `--user` install, nor, for that matter, are any caused by a system-wide install in site-packages, though I could imagine it is feasible there. It seems absurd that Python's packaging and import system is so bad that installing a duplicate Python is the only answer to *completely nondestructive installation of a package*. Would you say that nobody should be messing with system Python on Linux by doing a `--user` install of a package, or even a regular system-wide pip install? On linux, system `dist-packages` are lower priority than `site-packages`. I don't think there is anything different about Extras on OS X. > > > but it would be worth people being aware of it > > Yes, awareness is always good -- The more people that know how to > answer the inevitable questions that come up, the better. > > - Chris > > > > > (and Nick Coghlan is on distutils-sig, so he might feel it's worth > > raising with python-dev for their thoughts) > > > > Paul > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave.hirschfeld at gmail.com Wed Dec 11 05:13:38 2013 From: dave.hirschfeld at gmail.com (Dave Hirschfeld) Date: Wed, 11 Dec 2013 10:13:38 +0000 (UTC) Subject: [IPython-dev] Missing toolbar button? Message-ID: I updated IPython recently and am now running: {'codename': 'Work in Progress', 'commit_hash': '4a8c65b', 'commit_source': 'repository', 'default_encoding': 'cp1252', 'ipython_path': 'c:\\dev\\code\\ipython\\IPython', 'ipython_version': '2.0.0-dev', 'os_name': 'nt', 'platform': 'Windows-2008ServerR2-6.1.7601-SP1', 'sys_executable': 'C:\\dev\\bin\\Python27\\python.exe', 'sys_platform': 'win32', 'sys_version': '2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)]'} When working in the notebook I've now realised that the buttons have changed - specifically the Insert Cell buttons have moved to the left which I'm agnostic about but there is now no Insert Cell Above button! I've found this button to be very useful in practice and now find it very painful to have to click two buttons (Insert Cell Below / Move Cell Up) where previously I could do this with a single click! Any chance the Insert Cell Above button could be restored? NB: This problem would be mitigated if I would just use the keyboard shortcuts, but I can never remember what they are. The Keyboard shortcuts help is a modal dialog and I tend to forget the correct keys as soon as I've closed it. It would be a great help if this could be a floating/new window - similar to the help pager. Thanks, Dave From zvoros at gmail.com Wed Dec 11 05:27:38 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Wed, 11 Dec 2013 11:27:38 +0100 Subject: [IPython-dev] Missing toolbar button? In-Reply-To: References: Message-ID: <52A83E1A.8030507@gmail.com> Hi Dave, The shortkeys for adding cells are quite straightforward: B is for below, A is for above. As for the missing button, you can customise your toolbar by adding an extra js file to your profile directory. You can find the details here: http://elacave.lmdb.eu/~carreau/yui/classes/IPython.customjs.html Cheers, Zolt?n On 11/12/13 11:13, Dave Hirschfeld wrote: > I updated IPython recently and am now running: > > {'codename': 'Work in Progress', > 'commit_hash': '4a8c65b', > 'commit_source': 'repository', > 'default_encoding': 'cp1252', > 'ipython_path': 'c:\\dev\\code\\ipython\\IPython', > 'ipython_version': '2.0.0-dev', > 'os_name': 'nt', > 'platform': 'Windows-2008ServerR2-6.1.7601-SP1', > 'sys_executable': 'C:\\dev\\bin\\Python27\\python.exe', > 'sys_platform': 'win32', > 'sys_version': '2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit > (AMD64)]'} > > When working in the notebook I've now realised that the buttons have changed > - specifically the Insert Cell buttons have moved to the left which I'm > agnostic about but there is now no Insert Cell Above button! > > I've found this button to be very useful in practice and now find it very > painful to have to click two buttons (Insert Cell Below / Move Cell Up) > where previously I could do this with a single click! > > Any chance the Insert Cell Above button could be restored? > > NB: This problem would be mitigated if I would just use the keyboard > shortcuts, but I can never remember what they are. The Keyboard shortcuts > help is a modal dialog and I tend to forget the correct keys as soon as I've > closed it. It would be a great help if this could be a floating/new window - > similar to the help pager. > > Thanks, > Dave > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From ellisonbg at gmail.com Wed Dec 11 11:05:39 2013 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 11 Dec 2013 08:05:39 -0800 Subject: [IPython-dev] Missing toolbar button? In-Reply-To: <52A83E1A.8030507@gmail.com> References: <52A83E1A.8030507@gmail.com> Message-ID: The new dual editor mode we are working on allows users to define their own keyboard shortcuts. On Wed, Dec 11, 2013 at 2:27 AM, Zolt?n V?r?s wrote: > Hi Dave, > > The shortkeys for adding cells are quite straightforward: B is for > below, A is for above. > > As for the missing button, you can customise your toolbar by adding an > extra js file to your profile directory. You can find the details here: > > http://elacave.lmdb.eu/~carreau/yui/classes/IPython.customjs.html > > Cheers, > Zolt?n > > On 11/12/13 11:13, Dave Hirschfeld wrote: >> I updated IPython recently and am now running: >> >> {'codename': 'Work in Progress', >> 'commit_hash': '4a8c65b', >> 'commit_source': 'repository', >> 'default_encoding': 'cp1252', >> 'ipython_path': 'c:\\dev\\code\\ipython\\IPython', >> 'ipython_version': '2.0.0-dev', >> 'os_name': 'nt', >> 'platform': 'Windows-2008ServerR2-6.1.7601-SP1', >> 'sys_executable': 'C:\\dev\\bin\\Python27\\python.exe', >> 'sys_platform': 'win32', >> 'sys_version': '2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit >> (AMD64)]'} >> >> When working in the notebook I've now realised that the buttons have changed >> - specifically the Insert Cell buttons have moved to the left which I'm >> agnostic about but there is now no Insert Cell Above button! >> >> I've found this button to be very useful in practice and now find it very >> painful to have to click two buttons (Insert Cell Below / Move Cell Up) >> where previously I could do this with a single click! >> >> Any chance the Insert Cell Above button could be restored? >> >> NB: This problem would be mitigated if I would just use the keyboard >> shortcuts, but I can never remember what they are. The Keyboard shortcuts >> help is a modal dialog and I tend to forget the correct keys as soon as I've >> closed it. It would be a great help if this could be a floating/new window - >> similar to the help pager. >> >> Thanks, >> Dave >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From takowl at gmail.com Wed Dec 11 13:49:11 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 11 Dec 2013 10:49:11 -0800 Subject: [IPython-dev] Missing toolbar button? In-Reply-To: References: Message-ID: On 11 December 2013 02:13, Dave Hirschfeld wrote: > When working in the notebook I've now realised that the buttons have > changed > - specifically the Insert Cell buttons have moved to the left which I'm > agnostic about but there is now no Insert Cell Above button! > Sorry about that. The rationale was that having two sets of up/down arrows on the toolbar was confusing. We've considered adding something like a long click on that button to bring up a drop-down menu for different places to insert a cell. I don't know how easy that is with the UI system we're using, though. For now, Ctrl-m-a is the shortcut for insert cell above. This will be much easier with the dual mode editing Brian mentioned: by default it will be 'a' for insert cell above. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Wed Dec 11 13:51:54 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Wed, 11 Dec 2013 19:51:54 +0100 Subject: [IPython-dev] Missing toolbar button? In-Reply-To: References: Message-ID: Le 11 d?c. 2013 ? 19:49, Thomas Kluyver a ?crit : > On 11 December 2013 02:13, Dave Hirschfeld wrote: > When working in the notebook I've now realised that the buttons have changed > - specifically the Insert Cell buttons have moved to the left which I'm > agnostic about but there is now no Insert Cell Above button! > > Sorry about that. The rationale was that having two sets of up/down arrows on the toolbar was confusing. We've considered adding something like a long click on that button to bring up a drop-down menu for different places to insert a cell. I don't know how easy that is with the UI system we're using, though. Long click are hard but one can have split-button drop down: http://getbootstrap.com/components/#btn-dropdowns-split -- M From ellisonbg at gmail.com Wed Dec 11 18:15:06 2013 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 11 Dec 2013 15:15:06 -0800 Subject: [IPython-dev] Missing toolbar button? In-Reply-To: References: Message-ID: I think the new dual mode editor will improve the situation a lot. On Wed, Dec 11, 2013 at 10:51 AM, Matthias BUSSONNIER wrote: > > Le 11 d?c. 2013 ? 19:49, Thomas Kluyver a ?crit : > >> On 11 December 2013 02:13, Dave Hirschfeld wrote: >> When working in the notebook I've now realised that the buttons have changed >> - specifically the Insert Cell buttons have moved to the left which I'm >> agnostic about but there is now no Insert Cell Above button! >> >> Sorry about that. The rationale was that having two sets of up/down arrows on the toolbar was confusing. We've considered adding something like a long click on that button to bring up a drop-down menu for different places to insert a cell. I don't know how easy that is with the UI system we're using, though. > > Long click are hard but one can have split-button drop down: > > http://getbootstrap.com/components/#btn-dropdowns-split > > -- > M > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From vasco+python at tenner.nl Thu Dec 12 09:12:46 2013 From: vasco+python at tenner.nl (Vasco Tenner) Date: Thu, 12 Dec 2013 15:12:46 +0100 Subject: [IPython-dev] output from script is delayed Message-ID: <52A9C45E.8050600@tenner.nl> Dear all, I encountered that the output from a script run by %run -i is not directly displayed in the notebook. When investigating I tested the following example: import time for i in range(10): print i time.sleep(2) In this example 0 and 1 are displayed at the same time, after 4 seconds! In the script I run the first sys.stdout.write call does not output anything, but after the second call both are displayed (the same for 3 and 4 etc). This can partialy overcome by adding writing an empty string after each sys.stdout.write call: sys.stdout.write('foo') sys.stdout.write('') However, still the first one is not displayed immediately. Is this a flaw in the ipython notebook kernel/frontend communication or do I miss something? Vasco Tenner From konrad.hinsen at fastmail.net Thu Dec 12 09:54:42 2013 From: konrad.hinsen at fastmail.net (Konrad Hinsen) Date: Thu, 12 Dec 2013 15:54:42 +0100 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: <82980319-7C1F-4418-ADC3-4732A10EF08B@gmail.com> References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> <21156.21748.117583.339638@Ordinateur-de-Catherine-Konrad.local> <21157.50571.922623.163499@Konrad-Hinsens-MacBook-Pro-2.local> <82980319-7C1F-4418-ADC3-4732A10EF08B@gmail.com> Message-ID: <21161.52786.221802.221816@Konrad-Hinsens-MacBook-Pro-2.local> Matthias BUSSONNIER writes: > the history sqlite file (.ipython/profile_default/history.sqlite) > contains lots of info like raw input entered by user. > > you are probably interessed in session and history and join the two: ... Thanks, that looks like what I need, though I can't say I am looking forward to have to deal with SQL. Zolt?n V?r?s writes: > You could also look at the python interface ipython-get-history.py at > https://github.com/ipython/ipython/blob/master/examples/core/ipython-get-history.py Python interface sounds much better :-) I'll explore this first. Thomas Kluyver writes: > Matthias and Zoltan have pointed you to the SQLite history database. However, while the > kernel's running, you can make a history_request to retrieve history: > http://ipython.org/ipython-doc/dev/development/messaging.html#history That raises an important point: what if the user restarts the kernel? I should at least try to recognize that situation, because it can completely disconnect the state of the date (in the ActivePaper) from the state of the notebook. > I'm assuming that your code will be running in the notebook server > process, which handles saving and loading files. It may be that the Yes, I want to keep the user interface as close to standard IPython notebook usage as possible. It's the surrounding environment that changes, from "the user's computer" to "the contents of an ActivePaper". Thanks to everyone, Konrad. From konrad.hinsen at fastmail.net Thu Dec 12 10:11:47 2013 From: konrad.hinsen at fastmail.net (Konrad Hinsen) Date: Thu, 12 Dec 2013 16:11:47 +0100 Subject: [IPython-dev] Fwd: [SciTech] ActivePapers Python edition In-Reply-To: References: <21150.2041.52709.320967@Konrad-Hinsens-MacBook-Pro.local> <21152.46116.717846.556000@Konrad-Hinsens-MacBook-Pro.local> <21153.31951.575254.729992@Konrad-Hinsens-MacBook-Pro.local> <21156.21748.117583.339638@Ordinateur-de-Catherine-Konrad.local> Message-ID: <21161.53811.537098.362745@Konrad-Hinsens-MacBook-Pro-2.local> Gabriel Becker writes: > I'm actually drafting a thesis chapter about this very subject right now in the context > of so-called "dynamic documents" in the R realm such as those processed by Sweave[1] and > knitr[2]. The issue is with how we determine and track dependency. Thanks for your comments. What you describe is indeed very close to the issues at the core of both ActivePapers in its current for, and of an integration with IPython. > The solution to this that I am pursuing is essentially memoization > of code cells. That is, keying dependency detection and caching off > of the actual values of the inputs used by a cell, rather than > either the names of the input variables (as in the weaver[3], and > knitr[2]) or simple position in the document. This means that if That sounds like a good approach. > The input detection necessary to do this can be difficult in a > general language like Python, but with some assumptions about the I'll probably go for a simple approach that prefers to stay on the safe side and re-run too much rather than not enough: unless the code that was run is exactly equal to the current contents of the cells concatenated in linear order, I assume the whole notebook needs to be re-run. There's always room for optimization later. Konrad. From gregor.thalhammer at gmail.com Thu Dec 12 13:07:02 2013 From: gregor.thalhammer at gmail.com (Gregor Thalhammer) Date: Thu, 12 Dec 2013 19:07:02 +0100 Subject: [IPython-dev] output from script is delayed In-Reply-To: <52A9C45E.8050600@tenner.nl> References: <52A9C45E.8050600@tenner.nl> Message-ID: <306F6B19-1814-4934-8BE1-43D76A120DE6@gmail.com> Am 12.12.2013 um 15:12 schrieb Vasco Tenner : > Dear all, > > I encountered that the output from a script run by %run -i is not > directly displayed in the notebook. > > When investigating I tested the following example: > > import time > for i in range(10): > print i > time.sleep(2) > > In this example 0 and 1 are displayed at the same time, after 4 seconds! > > In the script I run the first sys.stdout.write call does not output > anything, but after the second call both are displayed (the same for 3 > and 4 etc). This can partialy overcome by adding writing an empty string > after each sys.stdout.write call: > sys.stdout.write('foo') > sys.stdout.write('') > > However, still the first one is not displayed immediately. > > Is this a flaw in the ipython notebook kernel/frontend communication or > do I miss something? > In Python output to stdout is buffered, so you have to add a sys.stdout.flush() after the print to immediately show the output. Gregor > > Vasco Tenner > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From clyde.fare at gmail.com Fri Dec 13 08:42:24 2013 From: clyde.fare at gmail.com (Clyde) Date: Fri, 13 Dec 2013 05:42:24 -0800 (PST) Subject: [IPython-dev] Tab completion for composed objects Message-ID: <1386942144241-5041635.post@n6.nabble.com> Hi, Are there any plans to extend tab completion to classes that use composition rather than inheritance to extend a baseclass? (Or is it possible already with some suitable trick?) Cheers Clyde -- View this message in context: http://python.6.x6.nabble.com/Tab-completion-for-composed-objects-tp5041635.html Sent from the IPython - Development mailing list archive at Nabble.com. From takowl at gmail.com Fri Dec 13 12:19:18 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 13 Dec 2013 09:19:18 -0800 Subject: [IPython-dev] Tab completion for composed objects In-Reply-To: <1386942144241-5041635.post@n6.nabble.com> References: <1386942144241-5041635.post@n6.nabble.com> Message-ID: On 13 December 2013 05:42, Clyde wrote: > Are there any plans to extend tab completion to classes that use > composition > rather than inheritance to extend a baseclass? (Or is it possible already > with some suitable trick?) Do you mean using something like __getattr__() to redirect attribute access to another object? If so, the relevant trick is to override __dir__() to list the attributes you want tab completion to see: http://docs.python.org/3/reference/datamodel.html#object.__dir__ Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From kiranbkrishna at gmail.com Sat Dec 14 07:53:23 2013 From: kiranbkrishna at gmail.com (Kiran Balakrishnan) Date: Sat, 14 Dec 2013 12:53:23 +0000 (UTC) Subject: [IPython-dev] Invitation to connect on LinkedIn Message-ID: <1163789818.40643751.1387025603007.JavaMail.app@ela4-app0211.prod> LinkedIn ------------ I'd like to add you to my professional network on LinkedIn. - Kiran Kiran Balakrishnan software engineer at Claylabs India Confirm that you know Kiran Balakrishnan: https://www.linkedin.com/e/l8a04l-hp6v8crh-5h/isd/18683386382/Z3zZ8wUs/?hs=false&tok=0ivpmj4iABLm01 -- You are receiving Invitation to Connect emails. Click to unsubscribe: http://www.linkedin.com/e/l8a04l-hp6v8crh-5h/qsHV3tjwE0wiU1PxqRuES2Mdi3ZZfiFtw_/goo/IPython-dev%40scipy%2Eorg/20061/I6093399454_1/?hs=false&tok=3Bkk_AazIBLm01 (c) 2012 LinkedIn Corporation. 2029 Stierlin Ct, Mountain View, CA 94043, USA. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max_linke at gmx.de Mon Dec 16 04:40:09 2013 From: max_linke at gmx.de (Max Linke) Date: Mon, 16 Dec 2013 10:40:09 +0100 Subject: [IPython-dev] Table of content for notebook Message-ID: <1387186809.26450.12.camel@x200.kel.wh.lokal> Hi I was wondering if it is possible to have a special cell that dynamically creates a table of content with links to `Heading 1` and `Heading 2` cells? This would help navigating in bigger notebook. best Max From zvoros at gmail.com Mon Dec 16 04:52:29 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Mon, 16 Dec 2013 10:52:29 +0100 Subject: [IPython-dev] Table of content for notebook In-Reply-To: <1387186809.26450.12.camel@x200.kel.wh.lokal> References: <1387186809.26450.12.camel@x200.kel.wh.lokal> Message-ID: <52AECD5D.7040507@gmail.com> Hi Max, Have you looked at Min's extension here: https://github.com/minrk/ipython_extensions ? Cheers, Zolt?n On 16/12/13 10:40, Max Linke wrote: > Hi > > I was wondering if it is possible to have a special cell that > dynamically creates a table of content with links to `Heading 1` and > `Heading 2` cells? This would help navigating in bigger notebook. > > best Max > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From maidos93 at laposte.net Mon Dec 16 06:53:53 2013 From: maidos93 at laposte.net (thwiouz) Date: Mon, 16 Dec 2013 03:53:53 -0800 (PST) Subject: [IPython-dev] Bootstrap 3 Message-ID: <1387194833693-5041817.post@n6.nabble.com> Hi, I just wanted to know if the upgrade of Bootstrap v3 was planed in a future release? Cheers, -- View this message in context: http://python.6.x6.nabble.com/Bootstrap-3-tp5041817.html Sent from the IPython - Development mailing list archive at Nabble.com. From bussonniermatthias at gmail.com Mon Dec 16 07:18:34 2013 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Mon, 16 Dec 2013 13:18:34 +0100 Subject: [IPython-dev] Bootstrap 3 In-Reply-To: <1387194833693-5041817.post@n6.nabble.com> References: <1387194833693-5041817.post@n6.nabble.com> Message-ID: <99CCEE7A-3635-42E6-A660-2615FE62A2E7@gmail.com> Yes we will probably do it. Probably not for 2.0. We have no roadmap as to when. It will though be difficult because bootstrap drop support for submenu. And also now, all font awesome icons are prefixed with fa-. Any reason you wish we update/we do not update? Envoy? de mon iPhone > Le 16 d?c. 2013 ? 12:53, thwiouz a ?crit : > > Hi, > I just wanted to know if the upgrade of Bootstrap v3 was planed in a future > release? > > Cheers, > > > > -- > View this message in context: http://python.6.x6.nabble.com/Bootstrap-3-tp5041817.html > Sent from the IPython - Development mailing list archive at Nabble.com. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From damianavila at gmail.com Mon Dec 16 07:50:32 2013 From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=) Date: Mon, 16 Dec 2013 10:50:32 -0200 Subject: [IPython-dev] Bootstrap 3 In-Reply-To: <99CCEE7A-3635-42E6-A660-2615FE62A2E7@gmail.com> References: <1387194833693-5041817.post@n6.nabble.com> <99CCEE7A-3635-42E6-A660-2615FE62A2E7@gmail.com> Message-ID: >Probably not for 2.0. We have no roadmap as to when. I was planned to deal with b3 after the release of 2.0, probably beginning the task after the merging of the big PRs. About the submenus... I agree there is a problem there... but with the font awesome problem, probably it can be easily fixed. 2013/12/16 Matthias Bussonnier > Yes we will probably do it. Probably not for 2.0. We have no roadmap as to > when. > It will though be difficult because bootstrap drop support for submenu. > And also now, all font awesome icons are prefixed with fa-. > > Any reason you wish we update/we do not update? > > > Envoy? de mon iPhone > > > Le 16 d?c. 2013 ? 12:53, thwiouz a ?crit : > > > > Hi, > > I just wanted to know if the upgrade of Bootstrap v3 was planed in a > future > > release? > > > > Cheers, > > > > > > > > -- > > View this message in context: > http://python.6.x6.nabble.com/Bootstrap-3-tp5041817.html > > Sent from the IPython - Development mailing list archive at Nabble.com. > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Dami?n Avila Scientific Python Developer Quantitative Finance Analyst Statistics, Biostatistics and Econometrics Consultant Biochemist -------------- next part -------------- An HTML attachment was scrubbed... URL: From zvoros at gmail.com Mon Dec 16 15:28:27 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Mon, 16 Dec 2013 21:28:27 +0100 Subject: [IPython-dev] comment toggling in notebook Message-ID: <52AF626B.8070900@gmail.com> Hi all, In the notebook, it is possible to comment out lines by pressing Cntr-/ . However, I don't think that it produces the intended result in the following case: print 'test' # this is just a test When I press Cntr-/, I get print 'test' this is just a test and when I press it the second time, I get # print 'test' this is just a test Is there a reason for not leaving the line alone, and not removing only the leading hash marks? If the developers want me to, I can try to fix the code. Cheers, Zolt?n From takowl at gmail.com Mon Dec 16 15:53:05 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 16 Dec 2013 12:53:05 -0800 Subject: [IPython-dev] comment toggling in notebook In-Reply-To: <52AF626B.8070900@gmail.com> References: <52AF626B.8070900@gmail.com> Message-ID: On 16 December 2013 12:28, Zolt?n V?r?s wrote: > print 'test' # this is just a test > > When I press Cntr-/, I get > > print 'test' this is just a test > > and when I press it the second time, I get > > # print 'test' this is just a test > > Is there a reason for not leaving the line alone, and not removing only > the leading hash marks? If the developers want me to, I can try to fix > the code. > No, that seems like a bug. If it's something in our code, by all means make a PR, but I'd guess that something like that is actually part of CodeMirror, the editor component we use for code cells. If it is, I imagine the CodeMirror devs would be happy to get a fix, unless it's already been fixed there, and we just need to update. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From zvoros at gmail.com Mon Dec 16 15:55:18 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Mon, 16 Dec 2013 21:55:18 +0100 Subject: [IPython-dev] comment toggling in notebook In-Reply-To: References: <52AF626B.8070900@gmail.com> Message-ID: <52AF68B6.7080300@gmail.com> Thomas, Thanks for the prompt feedback! OK, then I will try to figure out first where the glitch is. Cheers, Zolt?n On 16/12/13 21:53, Thomas Kluyver wrote: > On 16 December 2013 12:28, Zolt?n V?r?s > wrote: > > print 'test' # this is just a test > > When I press Cntr-/, I get > > print 'test' this is just a test > > and when I press it the second time, I get > > # print 'test' this is just a test > > Is there a reason for not leaving the line alone, and not removing > only > the leading hash marks? If the developers want me to, I can try to fix > the code. > > > No, that seems like a bug. If it's something in our code, by all means > make a PR, but I'd guess that something like that is actually part of > CodeMirror, the editor component we use for code cells. If it is, I > imagine the CodeMirror devs would be happy to get a fix, unless it's > already been fixed there, and we just need to update. > > Thomas > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at elbonia.de Mon Dec 16 16:28:25 2013 From: python at elbonia.de (Juergen Hasch) Date: Mon, 16 Dec 2013 22:28:25 +0100 Subject: [IPython-dev] comment toggling in notebook In-Reply-To: <52AF68B6.7080300@gmail.com> References: <52AF626B.8070900@gmail.com> <52AF68B6.7080300@gmail.com> Message-ID: <52AF7079.8010000@elbonia.de> I ran into the same issue some time ago. The comment hotkeys are handled by CodeMirror using the comment addon (\addon\comment\comment.js). It doesn't occur for me in the latest Github version of CodeMirror anymore. Am 16.12.2013 21:55, schrieb Zolt?n V?r?s: > Thomas, > > Thanks for the prompt feedback! OK, then I will try to figure out first where the glitch is. > > Cheers, > Zolt?n > > On 16/12/13 21:53, Thomas Kluyver wrote: >> On 16 December 2013 12:28, Zolt?n V?r?s > wrote: >> >> print 'test' # this is just a test >> >> When I press Cntr-/, I get >> >> print 'test' this is just a test >> >> and when I press it the second time, I get >> >> # print 'test' this is just a test >> >> Is there a reason for not leaving the line alone, and not removing only >> the leading hash marks? If the developers want me to, I can try to fix >> the code. >> >> >> No, that seems like a bug. If it's something in our code, by all means make a PR, but I'd guess that something like >> that is actually part of CodeMirror, the editor component we use for code cells. If it is, I imagine the CodeMirror >> devs would be happy to get a fix, unless it's already been fixed there, and we just need to update. >> >> Thomas >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From benjaminrk at gmail.com Mon Dec 16 17:55:24 2013 From: benjaminrk at gmail.com (MinRK) Date: Mon, 16 Dec 2013 14:55:24 -0800 Subject: [IPython-dev] comment toggling in notebook In-Reply-To: <52AF7079.8010000@elbonia.de> References: <52AF626B.8070900@gmail.com> <52AF68B6.7080300@gmail.com> <52AF7079.8010000@elbonia.de> Message-ID: Thanks for looking into it. Then it should be fixed the next time we update codemirror. On Mon, Dec 16, 2013 at 1:28 PM, Juergen Hasch wrote: > I ran into the same issue some time ago. The comment hotkeys are handled > by CodeMirror using the comment addon > (\addon\comment\comment.js). It doesn't occur for me in the latest Github > version of CodeMirror anymore. > > > > Am 16.12.2013 21:55, schrieb Zolt?n V?r?s: > > Thomas, > > > > Thanks for the prompt feedback! OK, then I will try to figure out first > where the glitch is. > > > > Cheers, > > Zolt?n > > > > On 16/12/13 21:53, Thomas Kluyver wrote: > >> On 16 December 2013 12:28, Zolt?n V?r?s zvoros at gmail.com>> wrote: > >> > >> print 'test' # this is just a test > >> > >> When I press Cntr-/, I get > >> > >> print 'test' this is just a test > >> > >> and when I press it the second time, I get > >> > >> # print 'test' this is just a test > >> > >> Is there a reason for not leaving the line alone, and not removing > only > >> the leading hash marks? If the developers want me to, I can try to > fix > >> the code. > >> > >> > >> No, that seems like a bug. If it's something in our code, by all means > make a PR, but I'd guess that something like > >> that is actually part of CodeMirror, the editor component we use for > code cells. If it is, I imagine the CodeMirror > >> devs would be happy to get a fix, unless it's already been fixed there, > and we just need to update. > >> > >> Thomas > >> > >> > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max_linke at gmx.de Tue Dec 17 16:25:42 2013 From: max_linke at gmx.de (Max Linke) Date: Tue, 17 Dec 2013 22:25:42 +0100 Subject: [IPython-dev] Table of content for notebook In-Reply-To: <52AECD5D.7040507@gmail.com> References: <1387186809.26450.12.camel@x200.kel.wh.lokal> <52AECD5D.7040507@gmail.com> Message-ID: <1387315542.12049.1.camel@x200.kel.wh.lokal> On Mon, 2013-12-16 at 10:52 +0100, Zolt?n V?r?s wrote: > Hi Max, > > Have you looked at Min's extension here: > https://github.com/minrk/ipython_extensions ? No I did knew it before. It is exactly what I was looking for. Thanks best Max > Cheers, > Zolt?n > > On 16/12/13 10:40, Max Linke wrote: > > Hi > > > > I was wondering if it is possible to have a special cell that > > dynamically creates a table of content with links to `Heading 1` and > > `Heading 2` cells? This would help navigating in bigger notebook. > > > > best Max > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From jakevdp at cs.washington.edu Thu Dec 19 16:21:25 2013 From: jakevdp at cs.washington.edu (Jacob Vanderplas) Date: Thu, 19 Dec 2013 13:21:25 -0800 Subject: [IPython-dev] A notebook-based D3 viewer for matplotlib Message-ID: Hi folks, I thought this would be of interest here. This week I hacked together an interactive D3 viewer for matplotlib graphics, which is compatible with the notebook. It's still extremely limited, but I think it has potential. If anybody enjoys D3 and wants to start hacking on this and adding features, I think it could end up being useful to a lot of people! Let me know if you're interested, Jake Blog post: http://jakevdp.github.io/blog/2013/12/19/a-d3-viewer-for-matplotlib/ Code repository: http://github.com/jakevdp/mpld3 -------------- next part -------------- An HTML attachment was scrubbed... URL: From zvoros at gmail.com Thu Dec 19 16:47:59 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Thu, 19 Dec 2013 22:47:59 +0100 Subject: [IPython-dev] A notebook-based D3 viewer for matplotlib In-Reply-To: References: Message-ID: <52B3698F.8050906@gmail.com> Jacob, I have just quickly tried out a couple of things, and this library rocks. Thanks for sharing! Cheers, Zolt?n On 19/12/13 22:21, Jacob Vanderplas wrote: > Hi folks, > I thought this would be of interest here. This week I hacked together > an interactive D3 viewer for matplotlib graphics, which is compatible > with the notebook. It's still extremely limited, but I think it has > potential. If anybody enjoys D3 and wants to start hacking on this > and adding features, I think it could end up being useful to a lot of > people! Let me know if you're interested, > Jake > > Blog post: > http://jakevdp.github.io/blog/2013/12/19/a-d3-viewer-for-matplotlib/ > Code repository: http://github.com/jakevdp/mpld3 > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Thu Dec 19 17:33:36 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 19 Dec 2013 14:33:36 -0800 Subject: [IPython-dev] A notebook-based D3 viewer for matplotlib In-Reply-To: References: Message-ID: On 19 December 2013 13:21, Jacob Vanderplas wrote: > I thought this would be of interest here. This week I hacked together an > interactive D3 viewer for matplotlib graphics, which is compatible with the > notebook. It's still extremely limited, but I think it has potential. If > anybody enjoys D3 and wants to start hacking on this and adding features, I > think it could end up being useful to a lot of people! Let me know if > you're interested, Nice! I look forward to seeing this develop. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From patrick.surry at gmail.com Thu Dec 19 18:39:41 2013 From: patrick.surry at gmail.com (Patrick Surry) Date: Thu, 19 Dec 2013 18:39:41 -0500 Subject: [IPython-dev] Integrating a JS grid control with ipython notebook? Message-ID: One of the things that would make pandas even more awesome(!) for me in the ipython notebook would be if I could display wide/long tables in a more flexible way than static html with horizontal and vertical scrollbars. (Then maybe a pivot table/chart widget :-) It seems like there are tons of JS grid controls out there - has anyone experimenting with rendering a pandas dataframe into a bit of interactive JS? I've had a little experience with the google charts table visualization tho it isn't super flexible. I'd be willing to hack on it if someone could give me some "getting started" pointers. Cheers, Patrick -------------- next part -------------- An HTML attachment was scrubbed... URL: From paddy at paddymullen.com Thu Dec 19 19:22:19 2013 From: paddy at paddymullen.com (paddy mullen) Date: Thu, 19 Dec 2013 19:22:19 -0500 Subject: [IPython-dev] A notebook-based D3 viewer for matplotlib In-Reply-To: References: Message-ID: <61D3C22F-7F1B-4754-8457-DCCCA94B0256@paddymullen.com> Nice work Jake, Just a note, the bokeh library also supports the ipython notebook. Here is an example notebook I made. http://nbviewer.ipython.org/github/paddymul/bokeh_tutorial/blob/master/multiplot.ipynb You can find bokeh at http://bokeh.pydata.org Paddy On Dec 19, 2013, at 5:33 PM, Thomas Kluyver wrote: > On 19 December 2013 13:21, Jacob Vanderplas wrote: > I thought this would be of interest here. This week I hacked together an interactive D3 viewer for matplotlib graphics, which is compatible with the notebook. It's still extremely limited, but I think it has potential. If anybody enjoys D3 and wants to start hacking on this and adding features, I think it could end up being useful to a lot of people! Let me know if you're interested, > > Nice! I look forward to seeing this develop. > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Thu Dec 19 19:22:03 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 19 Dec 2013 16:22:03 -0800 Subject: [IPython-dev] Integrating a JS grid control with ipython notebook? In-Reply-To: References: Message-ID: On 19 December 2013 15:39, Patrick Surry wrote: > One of the things that would make pandas even more awesome(!) for me in > the ipython notebook would be if I could display wide/long tables in a more > flexible way than static html with horizontal and vertical scrollbars. > (Then maybe a pivot table/chart widget :-) > > It seems like there are tons of JS grid controls out there - has anyone > experimenting with rendering a pandas dataframe into a bit of interactive > JS? I've had a little experience with the google charts table > visualization tho it isn't super flexible. > > I'd be willing to hack on it if someone could give me some "getting > started" pointers. > I don't know of anyone who's looked into that, but I agree it would be neat. There are two possible approaches, I think: First, you could spice up the HTML repr (e.g. to allow sorting/filtering columns, hiding parts of the table, etc.), but still keep all the data in the display. This would be relatively easy - you just need to copy a DataFrame's to_html() method and add a reference to some JS. Second, you could use the new widget stuff ( https://github.com/ipython/ipython/pull/4374 ) to communicate between the table view and the DataFrame. This would be considerably trickier, but it would allow you to work with very large data frames, because you don't have to send all the data to the frontend at once. It could also enable two-way communications, so you could edit the DataFrame from the table view. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From sychan at lbl.gov Thu Dec 19 19:35:43 2013 From: sychan at lbl.gov (Stephen Chan) Date: Thu, 19 Dec 2013 16:35:43 -0800 Subject: [IPython-dev] Reaping idle ipython server instances Message-ID: Hi, I've been working with docker and specifically Jiffylab for sandboxed, dynamically provisioned IPython notebook instances and wondered if there was any suggestions about how to reap idle IPython server instances. Ideally the ipython server would simply notice that there has been no activity, and no active client sessions after some period of time and then politely exit. But that may be wishful thinking. Any suggestions for how to go about it? The generic docker approach seems to be to monitor network activity periodically and if there isn't any, shutdown the container using the docker API. I can do this, though I am worried about long running tasks that may result in the network traffic being idle while its running. An 'external' option that might work seems be to examine the output from 'netstat' to see if there are established TCP connections to the listening port on the notebook instance. If a certain amount of time has passed without seeing an established connection, shutdown the container. It would be pretty straightforward to implement, but I am wondering if there is something else that others have used. This would be for the current 1.x branch of IPython. Thanks! Steve From takowl at gmail.com Thu Dec 19 20:23:36 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 19 Dec 2013 17:23:36 -0800 Subject: [IPython-dev] Reaping idle ipython server instances In-Reply-To: References: Message-ID: On 19 December 2013 16:35, Stephen Chan wrote: > Ideally the ipython server would simply notice that there has been > no activity, and no active client sessions after some period of time > and then politely exit. But that may be wishful thinking. > An idle server may still have open kernels which are storing state, but users often don't stop the kernel when they close a notebook. So the tricky question is when to kill off an idle kernel, as that destroys some information. The notebook server is easier, because AFAIK, all its information is stored on disk, so you can always bring it back up later. > The generic docker approach seems to be to monitor network > activity periodically and if there isn't any, shutdown the container > using the docker API. I can do this, though I am worried about long > running tasks that may result in the network traffic being idle while > its running. > The kernels do send busy and idle messages, so if you monitor these, you should be able to avoid killing long running jobs. That probably requires a custom solution, not a generic docker monitor, though. http://ipython.org/ipython-doc/dev/development/messaging.html#kernel-status Best wishes, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From zvoros at gmail.com Fri Dec 20 03:07:05 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Fri, 20 Dec 2013 09:07:05 +0100 Subject: [IPython-dev] A notebook-based D3 viewer for matplotlib In-Reply-To: <61D3C22F-7F1B-4754-8457-DCCCA94B0256@paddymullen.com> References: <61D3C22F-7F1B-4754-8457-DCCCA94B0256@paddymullen.com> Message-ID: <52B3FAA9.6000208@gmail.com> But I think Jacob's point was that the syntax in his library is the same as in matplotlib, while all other libraries define their own nomenclature. I think, keeping the syntax the same is a huge benefit that should not be overlooked. Zolt?n On 20/12/13 01:22, paddy mullen wrote: > Nice work Jake, > > Just a note, the bokeh library also supports the ipython notebook. > > Here is an example notebook I made. > http://nbviewer.ipython.org/github/paddymul/bokeh_tutorial/blob/master/multiplot.ipynb > > You can find bokeh at http://bokeh.pydata.org > > Paddy > > On Dec 19, 2013, at 5:33 PM, Thomas Kluyver > wrote: > >> On 19 December 2013 13:21, Jacob Vanderplas >> > wrote: >> >> I thought this would be of interest here. This week I hacked >> together an interactive D3 viewer for matplotlib graphics, which >> is compatible with the notebook. It's still extremely limited, >> but I think it has potential. If anybody enjoys D3 and wants to >> start hacking on this and adding features, I think it could end >> up being useful to a lot of people! Let me know if you're interested, >> >> >> Nice! I look forward to seeing this develop. >> >> Thomas >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason-sage at creativetrax.com Fri Dec 20 03:13:06 2013 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 20 Dec 2013 03:13:06 -0500 Subject: [IPython-dev] A notebook-based D3 viewer for matplotlib In-Reply-To: <61D3C22F-7F1B-4754-8457-DCCCA94B0256@paddymullen.com> References: <61D3C22F-7F1B-4754-8457-DCCCA94B0256@paddymullen.com> Message-ID: <52B3FC12.9050203@creativetrax.com> On 12/19/13, 7:22 PM, paddy mullen wrote: > Nice work Jake, > > Just a note, the bokeh library also supports the ipython notebook. > > Here is an example notebook I made. > http://nbviewer.ipython.org/github/paddymul/bokeh_tutorial/blob/master/multiplot.ipynb > > You can find bokeh at http://bokeh.pydata.org > Does bokeh now support the new comm communication infrastructure, or do you do something like run a separate bokeh web service to serve up plots? Thanks, Jason From pelson.pub at gmail.com Fri Dec 20 05:04:36 2013 From: pelson.pub at gmail.com (Phil Elson) Date: Fri, 20 Dec 2013 10:04:36 +0000 Subject: [IPython-dev] A notebook-based D3 viewer for matplotlib In-Reply-To: <52B3FC12.9050203@creativetrax.com> References: <61D3C22F-7F1B-4754-8457-DCCCA94B0256@paddymullen.com> <52B3FC12.9050203@creativetrax.com> Message-ID: Very nice. I can see this being massively valuable for the presentation of graphics for kernel-less notebook usage (i.e. nbviewer), the only problem IMHO is that there is a disconnect with some of the functionality (such as Axis locators and formatters) which has to happen at the Python level and cannot be computed in JS (unless, in the common cases, it is possible to compile the Python using skulpt, or some other tool). Did you play with exporting 2D visualisations such as QuadMesh and Images? Jason (Grout) has been working on an extension to the already released WebAgg backend (http://matplotlib.org/users/whats_new.html#webagg-backend) which uses IPython's comm interface for truly interactive figures in a live IPython environment - I'm hoping to take that in the near future and turn it into a backend (with very little work on my part) so that you can use the usual pyplot interface rather than the OO one, and so that the end-to-end experience is seamless (right through to a static rendering of the figure in the ipynb file itself). Anyway, thanks for sharing Jake! Cheers, Phil On 20 December 2013 08:13, Jason Grout wrote: > On 12/19/13, 7:22 PM, paddy mullen wrote: > > Nice work Jake, > > > > Just a note, the bokeh library also supports the ipython notebook. > > > > Here is an example notebook I made. > > > http://nbviewer.ipython.org/github/paddymul/bokeh_tutorial/blob/master/multiplot.ipynb > > > > You can find bokeh at http://bokeh.pydata.org > > > > Does bokeh now support the new comm communication infrastructure, or do > you do something like run a separate bokeh web service to serve up plots? > > Thanks, > > Jason > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jorisvandenbossche at gmail.com Fri Dec 20 05:35:24 2013 From: jorisvandenbossche at gmail.com (Joris Van den Bossche) Date: Fri, 20 Dec 2013 11:35:24 +0100 Subject: [IPython-dev] Integrating a JS grid control with ipython notebook? In-Reply-To: References: Message-ID: 2013/12/20 Thomas Kluyver > > On 19 December 2013 15:39, Patrick Surry wrote: > >> One of the things that would make pandas even more awesome(!) for me in >> the ipython notebook would be if I could display wide/long tables in a more >> flexible way than static html with horizontal and vertical scrollbars. >> (Then maybe a pivot table/chart widget :-) >> >> It seems like there are tons of JS grid controls out there - has anyone >> experimenting with rendering a pandas dataframe into a bit of interactive >> JS? I've had a little experience with the google charts table >> visualization tho it isn't super flexible. >> >> I'd be willing to hack on it if someone could give me some "getting >> started" pointers. >> > > I don't know of anyone who's looked into that, but I agree it would be > neat. > > There are two possible approaches, I think: > > First, you could spice up the HTML repr (e.g. to allow sorting/filtering > columns, hiding parts of the table, etc.), but still keep all the data in > the display. This would be relatively easy - you just need to copy a > DataFrame's to_html() method and add a reference to some JS. > > Second, you could use the new widget stuff ( > https://github.com/ipython/ipython/pull/4374 ) to communicate between the > table view and the DataFrame. This would be considerably trickier, but it > would allow you to work with very large data frames, because you don't have > to send all the data to the frontend at once. It could also enable two-way > communications, so you could edit the DataFrame from the table view. > This could maybe of interest: Javascript grid UI for pandas Dataframes: https://github.com/pydata/pandas/issues/2974 -------------- next part -------------- An HTML attachment was scrubbed... URL: From antseabra at gmail.com Thu Dec 26 10:46:24 2013 From: antseabra at gmail.com (=?ISO-8859-1?Q?Ant=F3nio_Seabra?=) Date: Thu, 26 Dec 2013 15:46:24 +0000 Subject: [IPython-dev] Internationalisation and creating domain specific Help items Message-ID: Hi everybody, I'm planning to use IPython (under the clothes of IJulia actually) in an undergraduate class where some of the students don't know much English. Would like to be able to do two things: - Translate IPython Notebook Menu items and Dashboard. - Change items in Help Menu by: (i) deleting some items and (ii) creating new domain specific Help items (eg. course related topics would be more relevant than some Python packages that would no be used). I understand that at the moment there's no machinery in place for easily achiving this. Which would be the best way to proceed? Many Thanks! Antonio -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Thu Dec 26 12:40:33 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Thu, 26 Dec 2013 18:40:33 +0100 Subject: [IPython-dev] Internationalisation and creating domain specific Help items In-Reply-To: References: Message-ID: Hi Antonio, I suppose you come here after the following question on stack overflow [1], which redirected you here. as you've pointed out, there is currently no good mechanism to translate IPython. One of the things that might change the final answer for you is wether student will run IPython themselves on their machines, or wether it will be a installation that will be provided on a cluster they can connect to. If you are in the case of the second option, then I suppose you could have a custom patched version of IPython installed with the key-part translated. The part you are interested in will mostly be in IPython/html/templates/*.html that is to say the menu, part of the dashboard, and changing the items in the help menu. Editting those file and reinstalling should give you a mostly translated UI. I have to check wether or not the path where IPython search for templates is configurable, if so new template could allow some internationalization to be done without modifying IPython source and just specifying where theses new templates are. (from a quick look probably c.NotebookApp.webapp_settings = {'template_path':} but not sure. cf notebookapp.py) Personally, as a non native english speaker, I think more and more that no translation at all, is better that a partially translated software, even in a context of classes. Having to deal with software that sadly are partially translated into french make finding information on the internet quite hard as almost every piece of help you will find will have the menu-name in english then you need to find what the translation might be, and for the few things you find in your language, you will see that the translation have changed between 2 versions of the software. So before absolutely translating for your undergrads, try to consider the above fact and wether or not having a manual that explain the key element of the UI could be more beneficial. If we are interested in a longer term i18n, - Jinja support extension for internationalization [2], that will allow to translate most of the part of the templates. - I don't know of any solution to i18n with javascript. - Some other things are hard-code in python itself. (doctoring and so on) - You have the extra constraint that most of the library you will use are in english (and in julia in your case) The question is how much effort we should put into translating IPython, and also that most languages are more verbose than english so some menu "won't fit" in screen anymore. We have to take into account that for correct translation, of a full application, we need to put extra constraint on the developers to think the application for translation. This mean wrapping each strings that could be translated into _() most of the time avoid string concatenation... etc. This is a process that for IPython itself would take quite some time if we don't get external help. I think we would be ok to go into such a direction if we have insurance that we will actually have people that will use that to provide IPython translation. Seeing how difficult it can be for popular project and even OS, I'm doubtful we can have a correct internationalization of IPython. Also, if there is a translation effort, it should also target IPython documentation, which already need some love in english. If I was in your position I would : - Search/Ask who is interested would participate in such a translation project. - Consider that only 5% of people who said will be interested in participating will participate - depending on the result decide what is worth to do. In any case A document that describe in your language IPython and the significance of each term will be the easiest to update and take the less time. It will be useful even in a translated software to search for english documentation on the internet. If having a way to plug external translation is an minimal patch for IPython itself, we'll be happy to integrate it. As soon as you add something into IPython itself it will considerably slow you down as every change will have to go through the core team. If a deep change is required, go through an IPython Ehencement Proposal [3]. Hope that respond to some of your question, I'm sure other dev will have things to add on that. Cheers, -- Matthias [1] : http://stackoverflow.com/questions/20777151/ipython-notebook-internationalization [2] : http://jinja.pocoo.org/docs/extensions/ [3] : https://github.com/ipython/ipython/wiki/IPEPs:-IPython-Enhancement-Proposals Le 26 d?c. 2013 ? 16:46, Ant?nio Seabra a ?crit : > > Hi everybody, > I'm planning to use IPython (under the clothes of IJulia actually) in an undergraduate class where some of the students don't know much English. > Would like to be able to do two things: > ? Translate IPython Notebook Menu items and Dashboard. > ? Change items in Help Menu by: (i) deleting some items and (ii) creating new domain specific Help items (eg. course related topics would be more relevant than some Python packages that would no be used). > I understand that at the moment there's no machinery in place for easily achiving this. Which would be the best way to proceed? > Many Thanks! > Antonio > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From antseabra at gmail.com Fri Dec 27 09:30:52 2013 From: antseabra at gmail.com (=?ISO-8859-1?Q?Ant=F3nio_Seabra?=) Date: Fri, 27 Dec 2013 14:30:52 +0000 Subject: [IPython-dev] Internationalisation and creating domain specific Help items In-Reply-To: References: Message-ID: Hi Matthias, Thank you very much for your reply. Agree with you that, instead of incompletley internationalising the UI, a short manual explaining the key element of the UI will be a better solution. Followed your suggestion concerning patching IPython/html/templates/*.html and it worked like a charm! Changed some Help items concerning Python packages to the Julia equivalents and added a few domain specific items. Couln't find something similar to NotebookApp.webapp_settings = {'template_path':} in notebookapp.py, but perhaps this was the line of code (obviously replacing ) I should add? It would be nice if IPython could search at startup for alternative templates. In my opinion being able to easily create domain specific IPython notebook UIs (even if only changing the menu structure), would be a great feature. Many thanks again for your help Antonio. On Thu, Dec 26, 2013 at 5:40 PM, Matthias BUSSONNIER < bussonniermatthias at gmail.com> wrote: > Hi Antonio, > > I suppose you come here after the following question on stack overflow > [1], which > redirected you here. as you've pointed out, there is currently no good > mechanism to > translate IPython. One of the things that might change the final answer > for you is wether > student will run IPython themselves on their machines, or wether it will > be a installation > that will be provided on a cluster they can connect to. If you are in the > case of the second > option, then I suppose you could have a custom patched version of IPython > installed with the > key-part translated. The part you are interested in will mostly be in > IPython/html/templates/*.html > that is to say the menu, part of the dashboard, and changing the items in > the help menu. > Editting those file and reinstalling should give you a mostly translated > UI. > > I have to check wether or not the path where IPython search for templates > is configurable, > if so new template could allow some internationalization to be done > without modifying IPython > source and just specifying where theses new templates are. > > (from a quick look probably c.NotebookApp.webapp_settings = > {'template_path':} but not sure. cf notebookapp.py) > > > Personally, as a non native english speaker, I think more and more that no > translation at all, > is better that a partially translated software, even in a context of > classes. Having to deal with software > that sadly are partially translated into french make finding information > on the internet quite hard as almost > every piece of help you will find will have the menu-name in english then > you need to find what the translation > might be, and for the few things you find in your language, you will see > that the translation have changed > between 2 versions of the software. So before absolutely translating for > your undergrads, try to consider the > above fact and wether or not having a manual that explain the key element > of the UI could be more beneficial. > > If we are interested in a longer term i18n, > > - Jinja support extension for internationalization [2], that will allow to > translate most of the part of the templates. > - I don't know of any solution to i18n with javascript. > - Some other things are hard-code in python itself. (doctoring and so on) > - You have the extra constraint that most of the library you will use are > in english (and in julia in your case) > > The question is how much effort we should put into translating IPython, > and also that most languages > are more verbose than english so some menu "won't fit" in screen anymore. > > We have to take into account that for correct translation, of a full > application, we need to put extra constraint > on the developers to think the application for translation. This mean > wrapping each strings that could be translated > into _() most of the time avoid string concatenation... etc. This is a > process that for IPython itself would take quite > some time if we don't get external help. I think we would be ok to go into > such a direction if we have > insurance that we will actually have people that will use that to provide > IPython translation. Seeing how difficult it > can be for popular project and even OS, I'm doubtful we can have a correct > internationalization of IPython. > Also, if there is a translation effort, it should also target IPython > documentation, which already need some love in english. > > If I was in your position I would : > > - Search/Ask who is interested would participate in such a translation > project. > - Consider that only 5% of people who said will be interested in > participating will participate > - depending on the result decide what is worth to do. > > In any case A document that describe in your language IPython and the > significance of each term > will be the easiest to update and take the less time. It will be useful > even in a translated software to > search for english documentation on the internet. > > If having a way to plug external translation is an minimal patch for > IPython itself, we'll be > happy to integrate it. As soon as you add something into IPython itself it > will considerably > slow you down as every change will have to go through the core team. > > If a deep change is required, go through an IPython Ehencement Proposal > [3]. > > Hope that respond to some of your question, I'm sure other dev will have > things to add on that. > > Cheers, > -- > Matthias > > [1] : > http://stackoverflow.com/questions/20777151/ipython-notebook-internationalization > [2] : http://jinja.pocoo.org/docs/extensions/ > [3] : > https://github.com/ipython/ipython/wiki/IPEPs:-IPython-Enhancement-Proposals > > Le 26 d?c. 2013 ? 16:46, Ant?nio Seabra a ?crit : > > > > > Hi everybody, > > I'm planning to use IPython (under the clothes of IJulia actually) in an > undergraduate class where some of the students don't know much English. > > Would like to be able to do two things: > > ? Translate IPython Notebook Menu items and Dashboard. > > ? Change items in Help Menu by: (i) deleting some items and (ii) > creating new domain specific Help items (eg. course related topics would > be more relevant than some Python packages that would no be used). > > I understand that at the moment there's no machinery in place for easily > achiving this. Which would be the best way to proceed? > > Many Thanks! > > Antonio > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Fri Dec 27 09:46:02 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Fri, 27 Dec 2013 15:46:02 +0100 Subject: [IPython-dev] Internationalisation and creating domain specific Help items In-Reply-To: References: Message-ID: Le 27 d?c. 2013 ? 15:30, Ant?nio Seabra a ?crit : > Couln't find something similar to NotebookApp.webapp_settings = {'template_path':} in notebookapp.py, but perhaps this was the line of code (obviously replacing ) I should add? It would be nice if IPython could search at startup for alternative templates. $ mkdir eraseme $ cp ~/ipython/IPython/html/templates/*.html ./ modify templates $ ipython notebook --NotebookApp.webapp_settings="{'template_path':'/Users/bussonniermatthias/eraseme'}" Works for me, but it is try that we could try to make it an extra search path. You could of course set c.NotebookApp.webapp_settings="{'template_path':'/Users/bussonniermatthias/eraseme'}" In your config files. -- M From drrt at cs.utexas.edu Fri Dec 27 13:12:49 2013 From: drrt at cs.utexas.edu (DrrT) Date: Fri, 27 Dec 2013 10:12:49 -0800 (PST) Subject: [IPython-dev] Multi-directory support landed in master In-Reply-To: References: Message-ID: <1388167969608-5042662.post@n6.nabble.com> I need a stable IPython version + Multi-directory dashboard support. Is there something like this already available? I am thinking about grabbing the source code from version 1.1.0 + the files from the multi directory pull request https://github.com/ipython/ipython/pull/4303/files Would this work well? Is there an easy way to do this? Any help would be appreciated, thanks. -- View this message in context: http://python.6.x6.nabble.com/Multi-directory-support-landed-in-master-tp5036774p5042662.html Sent from the IPython - Development mailing list archive at Nabble.com. From benjaminrk at gmail.com Fri Dec 27 14:32:10 2013 From: benjaminrk at gmail.com (MinRK) Date: Fri, 27 Dec 2013 11:32:10 -0800 Subject: [IPython-dev] Multi-directory support landed in master In-Reply-To: <1388167969608-5042662.post@n6.nabble.com> References: <1388167969608-5042662.post@n6.nabble.com> Message-ID: The multidir PR changes almost everything in the notebook server and client. Plus, that PR introduced several bugs that were fixed in smaller subsequent PRs. I would definitely not recommend cherry picking these changes. Can you be more specific about your needs, as to why you need 'stable' IPython, but with new features not in any stable release? -MinRK On Fri, Dec 27, 2013 at 10:12 AM, DrrT wrote: > I need a stable IPython version + Multi-directory dashboard support. Is > there > something like this already available? > > I am thinking about grabbing the source code from version 1.1.0 + the files > from the multi directory pull request > https://github.com/ipython/ipython/pull/4303/files > Would this work well? Is there an easy way to do this? > > Any help would be appreciated, thanks. > > > > -- > View this message in context: > http://python.6.x6.nabble.com/Multi-directory-support-landed-in-master-tp5036774p5042662.html > Sent from the IPython - Development mailing list archive at Nabble.com. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From drrt at cs.utexas.edu Fri Dec 27 15:09:46 2013 From: drrt at cs.utexas.edu (DrrT) Date: Fri, 27 Dec 2013 12:09:46 -0800 (PST) Subject: [IPython-dev] Multi-directory support landed in master In-Reply-To: References: <1388167969608-5042662.post@n6.nabble.com> Message-ID: <1388174986556-5042672.post@n6.nabble.com> Well, I am setting up an environment for many users, and the multi-directory feature would be quite useful. I would use the latest dev version, but I am guessing it still has many bugs. Now I see my initial idea is not going to work so what other options do I have? -- View this message in context: http://python.6.x6.nabble.com/Multi-directory-support-landed-in-master-tp5036774p5042672.html Sent from the IPython - Development mailing list archive at Nabble.com. From zvoros at gmail.com Fri Dec 27 15:23:44 2013 From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=) Date: Fri, 27 Dec 2013 21:23:44 +0100 Subject: [IPython-dev] Multi-directory support landed in master In-Reply-To: <1388174986556-5042672.post@n6.nabble.com> References: <1388167969608-5042662.post@n6.nabble.com> <1388174986556-5042672.post@n6.nabble.com> Message-ID: <52BDE1D0.8050807@gmail.com> On 27/12/13 21:09, DrrT wrote: > Well, I am setting up an environment for many users, and the multi-directory > feature would be quite useful. > I would use the latest dev version, but I am guessing it still has many > bugs. I use the development version all the time, and honestly, I have never had a rough ride with it. It occurred once or twice that there was some small glitch, but it was fixed very quickly. If I were you, and needed the multidirectory support, then I would just go with the latest version from master. > > Now I see my initial idea is not going to work so what other options do I > have? > > > > -- > View this message in context: http://python.6.x6.nabble.com/Multi-directory-support-landed-in-master-tp5036774p5042672.html > Sent from the IPython - Development mailing list archive at Nabble.com. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From andrew.gibiansky at gmail.com Sat Dec 28 22:25:04 2013 From: andrew.gibiansky at gmail.com (Andrew Gibiansky) Date: Sat, 28 Dec 2013 22:25:04 -0500 Subject: [IPython-dev] Passing Arguments to Kernels Message-ID: Hello All, I need to invoke `ipython console` and `ipython notebook` in such a way so that they pass some of their arguments down to the kernel that they eventually spawn. Is there a way to do this? I see that in start_kernel in KernelManager the keyword arguments and connection_file are substituted into the values defined by KernelManager.kernel_cmd, but I am not sure how these keyword arguments get populated. Thanks! -- Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Sun Dec 29 05:55:42 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Sun, 29 Dec 2013 11:55:42 +0100 Subject: [IPython-dev] Passing Arguments to Kernels In-Reply-To: References: Message-ID: <55099A34-22EA-4135-801A-2AE385542504@gmail.com> Le 29 d?c. 2013 ? 04:25, Andrew Gibiansky a ?crit : > Hello All, > > I need to invoke `ipython console` and `ipython notebook` in such a way so that they pass some of their arguments down to the kernel that they eventually spawn. Is there a way to do this? 1) This is one of the things we are thinking about for future version of IPython Notebook, in particular we want with the notebook sever to be able to choose which kernel you would like to launch with a notebook. (to avoid spawning 1 server/kernel type) but this is challenging as javascript depend on kernel type. > I see that in start_kernel in KernelManager the keyword arguments and connection_file are substituted into the values defined by KernelManager.kernel_cmd, but I am not sure how these keyword arguments get populated. 2) The only place where I see this get through is IPython/consoleapp.py:331 IPythonConsoleApp seem to have `kernel_argv = List(Unicode)` but is not configurable, and use a specific Python-only logic in some places (same file) So for now I would say the only way is to edit the kernel command in profile, but we are working on that, you are welcomed to join the discussion, (I'm not sure there is already a place for that). -- M From andrew.gibiansky at gmail.com Sun Dec 29 11:55:37 2013 From: andrew.gibiansky at gmail.com (Andrew Gibiansky) Date: Sun, 29 Dec 2013 11:55:37 -0500 Subject: [IPython-dev] Passing Arguments to Kernels In-Reply-To: <55099A34-22EA-4135-801A-2AE385542504@gmail.com> References: <55099A34-22EA-4135-801A-2AE385542504@gmail.com> Message-ID: Thanks! I can't really fix this by editing the kernel command in the profile. For the time being I'm hacking around this by writing arguments to a file and having my kernel read them from that file - which is sort of terrible, but works. For (1), I have no idea how you'd go about doing this. I want to be able to start a notebook with something like "IHaskell notebook --extension DataKinds --rcfile ~/.rc.hs", or the same for console. These eventually just call `ipython console` and `ipython notebook`, of course, which then calls `IHaskell kernel`. If there is just a single `ipython notebook` which can open any kernel, you can't necessarily do that... -- Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From moorepants at gmail.com Sun Dec 29 12:36:21 2013 From: moorepants at gmail.com (Jason Moore) Date: Sun, 29 Dec 2013 12:36:21 -0500 Subject: [IPython-dev] mimetypes from format() Message-ID: I'm having trouble figuring out how to get the DisplayFormatter to return the latex representation, in particular the format function seems to be able to return many mimetypes: http://ipython.org/ipython-doc/dev/api/generated/IPython.core.formatters.html#IPython.core.formatters.DisplayFormatter.format I'm writing this test: https://github.com/moorepants/sympy/blob/b748fae55c2cc3e973340b5e85cadd83a80f3405/sympy/interactive/tests/test_ipythonprinting.py#L59 And want to verify that the 'text/plain' and 'text/latex' function as expected after changing settings in SymPy. I can only get the 'text/plain' to output from 'format()'. How do I output other mimetypes? Jason moorepants.info +01 530-601-9791 -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Sun Dec 29 13:02:52 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Sun, 29 Dec 2013 19:02:52 +0100 Subject: [IPython-dev] Passing Arguments to Kernels In-Reply-To: References: <55099A34-22EA-4135-801A-2AE385542504@gmail.com> Message-ID: <34F12697-9548-41F2-BA38-E3CEFE06C2A0@gmail.com> Le 29 d?c. 2013 ? 17:55, Andrew Gibiansky a ?crit : > Thanks! > > I can't really fix this by editing the kernel command in the profile. For the time being I'm hacking around this by writing arguments to a file and having my kernel read them from that file - which is sort of terrible, but works. Ok, happy you got a workaround :-) > For (1), I have no idea how you'd go about doing this. I want to be able to start a notebook with something like "IHaskell notebook --extension DataKinds --rcfile ~/.rc.hs", or the same for console. > These eventually just call `ipython console` and `ipython notebook`, of course, which then calls `IHaskell kernel`. If there is just a single `ipython notebook` which can open any kernel, you can't necessarily do that... I suppose we will have to separate the server profiles from kernel profiles, and you can register a kernel "profile" as to one of the options to start a kernel. I suppose we will try to discuss that during mid-january meeting in berkley. Of course I think we are adventuring in an erea which hasn't been explored and where it is difficult to imagine the future especially with multiple kernel. Thats typically the area where the things you need to pass to your kernel are valuable input on what we need to design. -- M > > -- Andrew > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From benjaminrk at gmail.com Sun Dec 29 13:39:39 2013 From: benjaminrk at gmail.com (MinRK) Date: Sun, 29 Dec 2013 10:39:39 -0800 Subject: [IPython-dev] mimetypes from format() In-Reply-To: References: Message-ID: If it's a plain terminal session, only the plaintext formatter is enabled by default, since no other formats are renderable. You probably just want to set `display_formatter.formatters['text/latex'].enabled = True`. On Sun, Dec 29, 2013 at 9:36 AM, Jason Moore wrote: > I'm having trouble figuring out how to get the DisplayFormatter to return > the latex representation, in particular the format function seems to be > able to return many mimetypes: > http://ipython.org/ipython-doc/dev/api/generated/IPython.core.formatters.html#IPython.core.formatters.DisplayFormatter.format > > I'm writing this test: > > > https://github.com/moorepants/sympy/blob/b748fae55c2cc3e973340b5e85cadd83a80f3405/sympy/interactive/tests/test_ipythonprinting.py#L59 > > And want to verify that the 'text/plain' and 'text/latex' function as > expected after changing settings in SymPy. I can only get the 'text/plain' > to output from 'format()'. > > How do I output other mimetypes? > Jason > moorepants.info > +01 530-601-9791 > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From moorepants at gmail.com Sun Dec 29 14:04:59 2013 From: moorepants at gmail.com (Jason Moore) Date: Sun, 29 Dec 2013 14:04:59 -0500 Subject: [IPython-dev] mimetypes from format() In-Reply-To: References: Message-ID: Thanks. I just tried this: In [16]: ip = get_ipython() In [17]: inst = ip.instance() In [18]: inst.display_formatter.formatters['text/latex'].enabled = True In [19]: inst.display_formatter.format('1.0') Out[19]: ({u'text/plain': "'1.0'"}, {}) And the plaint text representation is the only that that was returned. Am I missing something else? Jason moorepants.info +01 530-601-9791 On Sun, Dec 29, 2013 at 1:39 PM, MinRK wrote: > If it's a plain terminal session, only the plaintext formatter is enabled > by default, since no other formats are renderable. You probably just want > to set `display_formatter.formatters['text/latex'].enabled = True`. > > > On Sun, Dec 29, 2013 at 9:36 AM, Jason Moore wrote: > >> I'm having trouble figuring out how to get the DisplayFormatter to return >> the latex representation, in particular the format function seems to be >> able to return many mimetypes: >> http://ipython.org/ipython-doc/dev/api/generated/IPython.core.formatters.html#IPython.core.formatters.DisplayFormatter.format >> >> I'm writing this test: >> >> >> https://github.com/moorepants/sympy/blob/b748fae55c2cc3e973340b5e85cadd83a80f3405/sympy/interactive/tests/test_ipythonprinting.py#L59 >> >> And want to verify that the 'text/plain' and 'text/latex' function as >> expected after changing settings in SymPy. I can only get the 'text/plain' >> to output from 'format()'. >> >> How do I output other mimetypes? >> Jason >> moorepants.info >> +01 530-601-9791 >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From moorepants at gmail.com Sun Dec 29 14:14:20 2013 From: moorepants at gmail.com (Jason Moore) Date: Sun, 29 Dec 2013 14:14:20 -0500 Subject: [IPython-dev] mimetypes from format() In-Reply-To: References: Message-ID: Ok this works: In [32]: ip = get_ipython() In [33]: inst = ip.instance() In [34]: from sympy import Symbol, init_printing In [35]: init_printing(use_latex=True) In [36]: inst.display_formatter.formatters['text/latex'].enabled = True In [37]: inst.display_formatter.format({Symbol("a"): 3.14}) Out[37]: ({u'text/latex': '$$\\begin{Bmatrix}a : 3.14\\end{Bmatrix}$$', u'text/plain': u'{a: 3.14}'}, {}) Thanks, for the help! Jason moorepants.info +01 530-601-9791 On Sun, Dec 29, 2013 at 2:04 PM, Jason Moore wrote: > Thanks. I just tried this: > > In [16]: ip = get_ipython() > > In [17]: inst = ip.instance() > > In [18]: inst.display_formatter.formatters['text/latex'].enabled = True > > In [19]: inst.display_formatter.format('1.0') > Out[19]: ({u'text/plain': "'1.0'"}, {}) > > > And the plaint text representation is the only that that was returned. Am > I missing something else? > > > Jason > moorepants.info > +01 530-601-9791 > > > On Sun, Dec 29, 2013 at 1:39 PM, MinRK wrote: > >> If it's a plain terminal session, only the plaintext formatter is enabled >> by default, since no other formats are renderable. You probably just want >> to set `display_formatter.formatters['text/latex'].enabled = True`. >> >> >> On Sun, Dec 29, 2013 at 9:36 AM, Jason Moore wrote: >> >>> I'm having trouble figuring out how to get the DisplayFormatter to >>> return the latex representation, in particular the format function seems to >>> be able to return many mimetypes: >>> http://ipython.org/ipython-doc/dev/api/generated/IPython.core.formatters.html#IPython.core.formatters.DisplayFormatter.format >>> >>> I'm writing this test: >>> >>> >>> https://github.com/moorepants/sympy/blob/b748fae55c2cc3e973340b5e85cadd83a80f3405/sympy/interactive/tests/test_ipythonprinting.py#L59 >>> >>> And want to verify that the 'text/plain' and 'text/latex' function as >>> expected after changing settings in SymPy. I can only get the 'text/plain' >>> to output from 'format()'. >>> >>> How do I output other mimetypes? >>> Jason >>> moorepants.info >>> +01 530-601-9791 >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From damianavila at gmail.com Mon Dec 30 14:55:20 2013 From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=) Date: Mon, 30 Dec 2013 17:55:20 -0200 Subject: [IPython-dev] IPython-powered semi-automatic git workflow Message-ID: I just want to share this post: http://www.damian.oquanta.info/posts/my-ipython-powered-semi-automatic-git-workflow.html that I think it is conceptually interesting because present the ipynbs as a sort a templates to do and document some jobs... Have a nice New Year! -- Dami?n Avila Scientific Python Developer Quantitative Finance Analyst Statistics, Biostatistics and Econometrics Consultant Biochemist -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmwoodman at gmail.com Mon Dec 30 18:25:43 2013 From: mmwoodman at gmail.com (Marmaduke Woodman) Date: Tue, 31 Dec 2013 00:25:43 +0100 Subject: [IPython-dev] NDArray for traitlets Message-ID: Hi, I was taking IPython's traitlets module for a spin to do manifest typing in a project I work on, and the only thing I found missing was an NDArray class which was fairly easy to put together, to traitlets' credit, and is posted at https://gist.github.com/mmwoodman/8187958 Basically, it is designed mainly to support shape and dtype specifications, which are enforced during validation, and change notification, configured per trait using `id()`, `all()` or `allclose()`. I'm writing to share this is case it is useful for other users of the traitlets module, and if anyone is willing to provide some feedback on how I've used the traitlets module, I'd love to hear it. I've also heard that traitlets might also be spun-off into a standalone module, in which case I'd find useful to have something like this be included. Thanks, Marmaduke -------------- next part -------------- An HTML attachment was scrubbed... URL: From tburnett at myuw.net Tue Dec 31 20:27:52 2013 From: tburnett at myuw.net (Toby Burnett) Date: Wed, 1 Jan 2014 01:27:52 +0000 Subject: [IPython-dev] Scorpion saver hijacked my Ipython notebook.. Message-ID: <5a4390aad98d4668bf96d5d7d2588ca2@BLUPR01MB306.prod.exchangelabs.com> For the second time. When It got it several weeks ago, I thought that I had eliminated it after half a day of struggles. The immediate symptom is that the IPython notebook, as served from my Linux compute server, is suddenly is a blank screen. The reason is that the HTML is modified by the browser: this is the first few lines: IPython Notebook The script src property is modified in a way that is very hard to fix. (This is Chrome, but IE has the same issue.) It should be I'll try to get rid of it again, but might have to make a clean install, ugh. When looking into how this can happen, and what safeguards there might me, I came across the idea of a "content security policy" that a web site could implement, and would, I think, at least give a warning about this: https://developer.chrome.com/extensions/contentSecurityPolicy.html Any thoughts/suggestions welcomed. --Toby Burnett (And Happy New Year to the team!) -------------- next part -------------- An HTML attachment was scrubbed... URL: