From rjtskolasinski at gmail.com Fri Jan 2 12:43:48 2015 From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=) Date: Fri, 2 Jan 2015 18:43:48 +0100 Subject: [IPython-dev] Making custom converter/template with nbconvert In-Reply-To: <CAAiurtdW4k1Ne0CUvz54TkLpdi+iiJYJR=Yrp7ADemXfnEDzFQ@mail.gmail.com> References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com> <CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com> <CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com> <17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com> <CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com> <CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com> <CAHAreOpPmg2bVUmeRMEYPOFdRMvkU_A1nfvgTxz=kcQ=T9ghjg@mail.gmail.com> <CAAiurtc0hT5muWuupzym20nNDrZ3TswH2t7d0YAuXUeSN54ipw@mail.gmail.com> <CAAiurtcXF5FRcXd7GiO5Jsd3+mHEGHB81cKfME1ypezEYcT-7w@mail.gmail.com> <CAAiurtdW4k1Ne0CUvz54TkLpdi+iiJYJR=Yrp7ADemXfnEDzFQ@mail.gmail.com> Message-ID: <CAAiurtfQicdQJYXtAGLF-sjNTYHxqDnwS1k7=9hMV3KTgCh+3w@mail.gmail.com> Hi guys, My converter starts to take the final form. I am now working on handling used figures in some clever way. All of them that are generated in notebooks and are in output fields go in smooth way. I have some problem with figures that are put inside text in markdowns, i.ex. as  which goes in html into <img src="dot.png" alt="some_caption" /><p class="caption">caption</p> (as a template very similar to basic one). How can force converter to not skip 'files/' to have src="files/dot.png"? Probably it lies somewhere in template but I couldn't localize it. Cheers, Rafa? 2014-12-19 17:57 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>: > Ok. I found it. > Now the corresponding line should be > exportHtml = > HTMLExporter(config=Config({'HTMLExporter':{'template_file':'basic'}})) > instead of > exportHtml = > HTMLExporter(config=Config({'HTMLExporter':{'default_template':'basic'}})) > > > 2014-12-19 17:12 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>: >> >> Hi guys, >> >> I got some kind of working version. It almost do what I want from it (at >> least for first working version). >> http://pastebin.com/bi0003LS >> >> Of course later there will come more functionality and polishing. >> >> Although I got strange problem. It looks like my converter ignores fact I >> ask it for using 'basic' template. >> >> Does the syntax for it changed? I was basing on this old tutorial: >> >> http://nbviewer.ipython.org/github/ipython/ipython-in-depth/blob/master/examples/Notebook/Using%20nbconvert%20as%20a%20Library.ipynb >> >> Best, >> Rafal >> >> 2014-12-18 22:38 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>: >>> >>> I will ; - ) >>> >>> 2014-12-18 22:36 GMT+01:00 Fernando Perez <fperez.net at gmail.com>: >>> >>>> >>>> On Thu, Dec 18, 2014 at 9:06 AM, Rafa? Skolasi?ski < >>>> rjtskolasinski at gmail.com> wrote: >>>>> >>>>> I will let you know if I will encounter any problems. >>>> >>>> >>>> And please post here on the list once you have something up and >>>> running. It's very important for the project to show how others use it not >>>> only as an interactive environment, but also as infrastructure to build >>>> upon. That's the kind of clear value that helps us secure resources to >>>> sustain its development. >>>> >>>> Best >>>> >>>> f >>>> >>>> >>>> -- >>>> Fernando Perez (@fperez_org; http://fperez.org) >>>> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) >>>> fernando.perez-at-berkeley: contact me here for any direct mail >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150102/a8acb956/attachment.html> From dsdale24 at gmail.com Sat Jan 3 10:45:41 2015 From: dsdale24 at gmail.com (Darren Dale) Date: Sat, 03 Jan 2015 15:45:41 +0000 Subject: [IPython-dev] embedding ipython, namespace question References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com> <AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov> <CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com> <FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com> <CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com> <DD2429D4-4BE1-49FC-A891-BF153FBA942F@anl.gov> Message-ID: <CAK6O52ns=WiaMSGRista9wE_VHdgFap1Wt65h1yDwEECawAULA@mail.gmail.com> Hi Ray, On Sat Dec 27 2014 at 8:48:20 PM Osborn, Raymond <rosborn at anl.gov> wrote: > I don?t really understand what you are trying to achieve, but the > ?user_ns? dictionary isn?t an isolated namespace - it?s the namespace that > is used by the console, and I would have thought you would have to do some > kind of injection to add other objects from within the application. > What I am trying to achieve is explicitly documented at http://ipython.org/ipython-doc/dev/interactive/reference.html#embedding-ipython : --- It is also possible to embed an IPython shell in a namespace in your Python code. This allows you to evaluate dynamically the state of your code, operate with your variables, analyze them, etc. Note however that any changes you make to values while in the shell do not propagate back to the running code, so it is safe to modify your values because you won?t break your code in bizarre ways by doing so. Note At present, embedding IPython cannot be done from inside IPython. Run the code samples below outside IPython. [DD: I am not attempting to embed ipython from inside ipython] This feature allows you to easily have a fully functional python environment for doing object introspection anywhere in your code with a simple function call. In some cases a simple print statement is enough, but if you need to do more detailed analysis of a code fragment this feature can be very valuable. It can also be useful in scientific computing situations where it is common to need to do some automatic, computationally intensive part and then stop to look at data, plots, etc. Opening an IPython instance will give you full access to your data and functions, and you can resume program execution once you are done with the interactive part (perhaps to stop again later, as many times as needed). The following code snippet is the bare minimum you need to include in your Python programs for this to work (detailed examples follow later): from IPython import embed embed() # this call anywhere in your program will start IPython You can also embed an IPython kernel, for use with qtconsole, etc. via IPython.embed_kernel(). This should function work the same way, but you can connect an external frontend (ipython qtconsole or ipython console), rather than interacting with it in the terminal. --- This is impressively simple for the embed function: --- C:\Users\darren> python Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul 2 2014, 15:12:11) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://binstar.org >>> from IPython import embed >>> a=1 >>> embed() Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul 2 2014, 15:12:11) [MSC v.1500 64 bit (AMD64)] Type "copyright", "credits" or "license" for more information. IPython 2.3.1 -- An enhanced Interactive Python. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://binstar.org ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. In [1]: a Out[1]: 1 --- But I have not been able to achieve the same behavior with the qt in-process console. Darren -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150103/9e2d2fd7/attachment.html> From ra092767 at ime.unicamp.br Sat Jan 3 21:27:02 2015 From: ra092767 at ime.unicamp.br (Raniere Silva) Date: Sun, 4 Jan 2015 00:27:02 -0200 Subject: [IPython-dev] Making custom converter/template with nbconvert In-Reply-To: <CAAiurtfQicdQJYXtAGLF-sjNTYHxqDnwS1k7=9hMV3KTgCh+3w@mail.gmail.com> References: <CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com> <CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com> <17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com> <CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com> <CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com> <CAHAreOpPmg2bVUmeRMEYPOFdRMvkU_A1nfvgTxz=kcQ=T9ghjg@mail.gmail.com> <CAAiurtc0hT5muWuupzym20nNDrZ3TswH2t7d0YAuXUeSN54ipw@mail.gmail.com> <CAAiurtcXF5FRcXd7GiO5Jsd3+mHEGHB81cKfME1ypezEYcT-7w@mail.gmail.com> <CAAiurtdW4k1Ne0CUvz54TkLpdi+iiJYJR=Yrp7ADemXfnEDzFQ@mail.gmail.com> <CAAiurtfQicdQJYXtAGLF-sjNTYHxqDnwS1k7=9hMV3KTgCh+3w@mail.gmail.com> Message-ID: <20150104022702.GM1209@buriti.rgaiacs.com> Hi Rafa?, > I have some problem with figures that are put inside text in markdowns, > i.ex. as >  > which goes in html into <img src="dot.png" alt="some_caption" /><p > class="caption">caption</p> > (as a template very similar to basic one). > > How can force converter to not skip 'files/' to have src="files/dot.png"? > Probably it lies somewhere in template but I couldn't localize it. I couldn't reproduce your problem. $ cat sample.ipynb { "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "IPython (Python 3)", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3" }, "signature": "sha256:5c8cb31d4083cbf815d3e1dff7a3cc229fa54b9c02cff82af4548cef72581573" }, "nbformat": 4, "nbformat_minor": 0 } $ ipython nbconvert --to html sample.ipynb [NbConvertApp] Using existing profile dir: '/home/raniere/.ipython/profile_default' [NbConvertApp] Converting notebook sample.ipynb to html [NbConvertApp] Support files will be in sample_files/ [NbConvertApp] Loaded template full.tpl [NbConvertApp] Writing 223380 bytes to sample.html $ grep '<img' sample.html <p><img src="folder/bar.jpg" alt="foo"></p> What version of IPython/Jupyter are you using? I tested with $ ipython --version 3.0.0-dev Cheers, Raniere -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150104/78c6ae6b/attachment.sig> From fperez.net at gmail.com Sat Jan 3 22:29:50 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 3 Jan 2015 19:29:50 -0800 Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the core Jupyter/IPython team Message-ID: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com> We'd like to welcome Kyle Kelley and Nick Bollweg to the core development team. In Kyle's case this is a long-overdue announcement, as Kyle has been a core dev for quite a while now, we just hadn't been very good about communicating that on-list. Kyle, thanks to Racksspace's generous support for the project, Trying to redress that lack of communication, in Nick's case we're not waiting :) Nick has been making great contributions to nbviewer, widgets, and overall discussions in the project, and accepted to be a core developer moving forward, as his other obligations allow. Welcome (if belatedly) to both! Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150103/56f0ded7/attachment.html> From rjtskolasinski at gmail.com Sun Jan 4 08:19:01 2015 From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=) Date: Sun, 4 Jan 2015 14:19:01 +0100 Subject: [IPython-dev] Making custom converter/template with nbconvert In-Reply-To: <20150104022702.GM1209@buriti.rgaiacs.com> References: <CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com> <CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com> <17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com> <CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com> <CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com> <CAHAreOpPmg2bVUmeRMEYPOFdRMvkU_A1nfvgTxz=kcQ=T9ghjg@mail.gmail.com> <CAAiurtc0hT5muWuupzym20nNDrZ3TswH2t7d0YAuXUeSN54ipw@mail.gmail.com> <CAAiurtcXF5FRcXd7GiO5Jsd3+mHEGHB81cKfME1ypezEYcT-7w@mail.gmail.com> <CAAiurtdW4k1Ne0CUvz54TkLpdi+iiJYJR=Yrp7ADemXfnEDzFQ@mail.gmail.com> <CAAiurtfQicdQJYXtAGLF-sjNTYHxqDnwS1k7=9hMV3KTgCh+3w@mail.gmail.com> <20150104022702.GM1209@buriti.rgaiacs.com> Message-ID: <CAAiurtdmBLcDa6ygAZmxP61q=z1F=Eo=uif+0sSFxyqsPfUtSw@mail.gmail.com> Hi Raniere, The problem occurred when folder was named 'files'. When I renamed it to 'figures' it behaves as in your example. I must say that this very weird behavior but there must be some reason for that probably. Cheers, Rafa? 2015-01-04 3:27 GMT+01:00 Raniere Silva <ra092767 at ime.unicamp.br>: > Hi Rafa?, > > > I have some problem with figures that are put inside text in markdowns, > > i.ex. as > >  > > which goes in html into <img src="dot.png" alt="some_caption" /><p > > class="caption">caption</p> > > (as a template very similar to basic one). > > > > How can force converter to not skip 'files/' to have src="files/dot.png"? > > Probably it lies somewhere in template but I couldn't localize it. > > I couldn't reproduce your problem. > > $ cat sample.ipynb > { > "cells": [ > { > "cell_type": "markdown", > "metadata": {}, > "source": [ > "" > ] > }, > { > "cell_type": "code", > "execution_count": null, > "metadata": { > "collapsed": true > }, > "outputs": [], > "source": [] > } > ], > "metadata": { > "kernelspec": { > "display_name": "IPython (Python 3)", > "name": "python3" > }, > "language_info": { > "codemirror_mode": { > "name": "ipython", > "version": 3 > }, > "file_extension": ".py", > "mimetype": "text/x-python", > "name": "python", > "nbconvert_exporter": "python", > "pygments_lexer": "ipython3" > }, > "signature": > "sha256:5c8cb31d4083cbf815d3e1dff7a3cc229fa54b9c02cff82af4548cef72581573" > }, > "nbformat": 4, > "nbformat_minor": 0 > } > $ ipython nbconvert --to html sample.ipynb > [NbConvertApp] Using existing profile dir: > '/home/raniere/.ipython/profile_default' > [NbConvertApp] Converting notebook sample.ipynb to html > [NbConvertApp] Support files will be in sample_files/ > [NbConvertApp] Loaded template full.tpl > [NbConvertApp] Writing 223380 bytes to sample.html > $ grep '<img' sample.html > <p><img src="folder/bar.jpg" alt="foo"></p> > > What version of IPython/Jupyter are you using? I tested with > > $ ipython --version > 3.0.0-dev > > Cheers, > Raniere > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150104/94f8eab6/attachment.html> From schueller at phimeca.com Sun Jan 4 12:31:00 2015 From: schueller at phimeca.com (Julien Schueller) Date: Sun, 4 Jan 2015 18:31:00 +0100 (CET) Subject: [IPython-dev] graph displayed twice while overloading _repr_svg_ for notebook In-Reply-To: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com> References: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com> Message-ID: <1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com> Hello, I'm having trouble overloading _repr_svg_ for graph inlining in the notebook; my object is displayed twice if I call plt.plot just before. I assumed a figure was left open or something, but even if I call plt.close('all') between the matplotlib calls, the result remains the same. I did some imports first: import numpy as np import matplotlib.pyplot as plt import matplotlib from scipy.stats import norm import sys import io print('matplotlib version %s' % matplotlib.__version__) import IPython print('IPython version %s' % IPython.__version__) Here'my object: class NormGraph(object): def __init__(self): super(NormGraph, self).__init__() def _repr_svg_(self): if sys.version_info[0] >= 3: output = io.StringIO() else: output = io.BytesIO() self._fig = plt.figure() axes_kwargs = {} plot_kwargs = {} self._ax = [self._fig.add_subplot(111, **axes_kwargs)] x = np.arange(-10, 10, 0.001) y = norm.pdf(x,0,2) self._ax[0].plot(x, y, **plot_kwargs) self._fig.savefig(output, format='svg') return output.getvalue() Then a basic call to pyplot: %matplotlib inline plt.plot(0, 1) Then if I try to inline my object it's displayed twice: NormGraph() I atached the complete notebook. -- Julien Schueller Phimeca Engineering www.phimeca.com -------------- next part -------------- A non-text attachment was scrubbed... Name: Untitled0.ipynb Type: application/octet-stream Size: 40341 bytes Desc: not available URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150104/27afca35/attachment.obj> From ssanderson at quantopian.com Sun Jan 4 13:01:24 2015 From: ssanderson at quantopian.com (ssanderson) Date: Sun, 4 Jan 2015 10:01:24 -0800 (PST) Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the core Jupyter/IPython team In-Reply-To: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com> References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com> Message-ID: <1420394484530-5082155.post@n6.nabble.com> Congrats! -- View this message in context: http://python.6.x6.nabble.com/Welcoming-Kyle-Kelley-and-Nick-Bollweg-to-the-core-Jupyter-IPython-team-tp5082131p5082155.html Sent from the IPython - Development mailing list archive at Nabble.com. From takowl at gmail.com Mon Jan 5 12:43:57 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 5 Jan 2015 17:43:57 +0000 Subject: [IPython-dev] graph displayed twice while overloading _repr_svg_ for notebook In-Reply-To: <1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com> References: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com> <1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com> Message-ID: <CAOvn4qgmon9DXa-CcWd7ZRv3wy=gJUsZPFAAB_p9OXZ_HW7dUQ@mail.gmail.com> If you've done '%matplotlib inline' previously, when you create a figure inside a cell, it will automatically be displayed at the end of that cell, separately from the regular repr system. So your _repr_svg_ method is getting called once, and the figure you create in that is being displayed again. I'm not sure what the best way to deal with that is. Thomas On 4 January 2015 at 17:31, Julien Schueller <schueller at phimeca.com> wrote: > > > > Hello, > > I'm having trouble overloading _repr_svg_ for graph inlining in the > notebook; my object is displayed twice if I call plt.plot just before. > I assumed a figure was left open or something, but even if I call > plt.close('all') between the matplotlib calls, the result remains the same. > > I did some imports first: > import numpy as np > import matplotlib.pyplot as plt > import matplotlib > from scipy.stats import norm > import sys > import io > print('matplotlib version %s' % matplotlib.__version__) > import IPython > print('IPython version %s' % IPython.__version__) > > > Here'my object: > class NormGraph(object): > def __init__(self): > super(NormGraph, self).__init__() > def _repr_svg_(self): > if sys.version_info[0] >= 3: > output = io.StringIO() > else: > output = io.BytesIO() > self._fig = plt.figure() > axes_kwargs = {} > plot_kwargs = {} > self._ax = [self._fig.add_subplot(111, **axes_kwargs)] > x = np.arange(-10, 10, 0.001) > y = norm.pdf(x,0,2) > self._ax[0].plot(x, y, **plot_kwargs) > self._fig.savefig(output, format='svg') > > return output.getvalue() > > > Then a basic call to pyplot: > %matplotlib inline > plt.plot(0, 1) > > Then if I try to inline my object it's displayed twice: > NormGraph() > > I atached the complete notebook. > > -- > Julien Schueller > Phimeca Engineering > www.phimeca.com > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150105/8238f256/attachment.html> From tritemio at gmail.com Mon Jan 5 14:06:50 2015 From: tritemio at gmail.com (Antonino Ingargiola) Date: Mon, 5 Jan 2015 11:06:50 -0800 Subject: [IPython-dev] graph displayed twice while overloading _repr_svg_ for notebook In-Reply-To: <CAOvn4qgmon9DXa-CcWd7ZRv3wy=gJUsZPFAAB_p9OXZ_HW7dUQ@mail.gmail.com> References: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com> <1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com> <CAOvn4qgmon9DXa-CcWd7ZRv3wy=gJUsZPFAAB_p9OXZ_HW7dUQ@mail.gmail.com> Message-ID: <CANn2QUyL99k-z6at_NssGFVgNr33RbReOH6jKQdDX5U91A0uqw@mail.gmail.com> Hi, I don't know if it helps but with plain matplotlib you can close the figure and call display(fig) on the closed figure. I use this trick to plot several figures in a loop (closing them at each iteration) or for updating plots via widgets. Antonio On Mon, Jan 5, 2015 at 9:43 AM, Thomas Kluyver <takowl at gmail.com> wrote: > If you've done '%matplotlib inline' previously, when you create a figure > inside a cell, it will automatically be displayed at the end of that cell, > separately from the regular repr system. So your _repr_svg_ method is > getting called once, and the figure you create in that is being displayed > again. I'm not sure what the best way to deal with that is. > > Thomas > > On 4 January 2015 at 17:31, Julien Schueller <schueller at phimeca.com> > wrote: > >> >> >> >> Hello, >> >> I'm having trouble overloading _repr_svg_ for graph inlining in the >> notebook; my object is displayed twice if I call plt.plot just before. >> I assumed a figure was left open or something, but even if I call >> plt.close('all') between the matplotlib calls, the result remains the same. >> >> I did some imports first: >> import numpy as np >> import matplotlib.pyplot as plt >> import matplotlib >> from scipy.stats import norm >> import sys >> import io >> print('matplotlib version %s' % matplotlib.__version__) >> import IPython >> print('IPython version %s' % IPython.__version__) >> >> >> Here'my object: >> class NormGraph(object): >> def __init__(self): >> super(NormGraph, self).__init__() >> def _repr_svg_(self): >> if sys.version_info[0] >= 3: >> output = io.StringIO() >> else: >> output = io.BytesIO() >> self._fig = plt.figure() >> axes_kwargs = {} >> plot_kwargs = {} >> self._ax = [self._fig.add_subplot(111, **axes_kwargs)] >> x = np.arange(-10, 10, 0.001) >> y = norm.pdf(x,0,2) >> self._ax[0].plot(x, y, **plot_kwargs) >> self._fig.savefig(output, format='svg') >> >> return output.getvalue() >> >> >> Then a basic call to pyplot: >> %matplotlib inline >> plt.plot(0, 1) >> >> Then if I try to inline my object it's displayed twice: >> NormGraph() >> >> I atached the complete notebook. >> >> -- >> Julien Schueller >> Phimeca Engineering >> www.phimeca.com >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150105/1d446bfa/attachment.html> From darcamo at gmail.com Mon Jan 5 14:14:56 2015 From: darcamo at gmail.com (darcamo at gmail.com) Date: Mon, 05 Jan 2015 19:14:56 +0000 Subject: [IPython-dev] graph displayed twice while overloading _repr_svg_ for notebook References: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com> <1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com> <CAOvn4qgmon9DXa-CcWd7ZRv3wy=gJUsZPFAAB_p9OXZ_HW7dUQ@mail.gmail.com> Message-ID: <CAGjSdXqi-Cv_GzOthhkRt_to-56p5hQTTE5F0XQ=HoQcX3igkQ@mail.gmail.com> Call plt.close(self._fig) after savefig and the inline plot should not be shown. Since you are using the figure only to convert to svg with savefig and you need to close it to avoid the inline plot, then there is no reason to keep it as an instance attribute. It might also be a good idea to disable matplotlib interactive mode by calling plt.ioff() before creating the figure object and reenabling it after the figure is closed by calling plt.ion(). -- Darlan -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150105/a74c1c65/attachment.html> From ellisonbg at gmail.com Mon Jan 5 17:01:53 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 5 Jan 2015 14:01:53 -0800 Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the core Jupyter/IPython team In-Reply-To: <1420394484530-5082155.post@n6.nabble.com> References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com> <1420394484530-5082155.post@n6.nabble.com> Message-ID: <CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com> Thanks so much for your work Kyle and Nick! Cheers, Brian On Sun, Jan 4, 2015 at 10:01 AM, ssanderson <ssanderson at quantopian.com> wrote: > Congrats! > > > > -- > View this message in context: > http://python.6.x6.nabble.com/Welcoming-Kyle-Kelley-and-Nick-Bollweg-to-the-core-Jupyter-IPython-team-tp5082131p5082155.html > Sent from the IPython - Development mailing list archive at Nabble.com. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150105/dec7cbb7/attachment.html> From schueller at phimeca.com Tue Jan 6 04:20:52 2015 From: schueller at phimeca.com (Julien Schueller) Date: Tue, 6 Jan 2015 10:20:52 +0100 (CET) Subject: [IPython-dev] graph displayed twice while overloading _repr_svg_ for notebook In-Reply-To: <CAGjSdXqi-Cv_GzOthhkRt_to-56p5hQTTE5F0XQ=HoQcX3igkQ@mail.gmail.com> References: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com> <1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com> <CAOvn4qgmon9DXa-CcWd7ZRv3wy=gJUsZPFAAB_p9OXZ_HW7dUQ@mail.gmail.com> <CAGjSdXqi-Cv_GzOthhkRt_to-56p5hQTTE5F0XQ=HoQcX3igkQ@mail.gmail.com> Message-ID: <1463575469.219037.1420536052858.JavaMail.zimbra@phimeca.com> ----- Mail original ----- > De: darcamo at gmail.com > ?: "IPython developers list" <ipython-dev at scipy.org> > Envoy?: Lundi 5 Janvier 2015 20:14:56 > Objet: Re: [IPython-dev] graph displayed twice while overloading _repr_svg_ for notebook > > Call plt.close(self._fig) after savefig and the inline plot should not be > shown. Since you are using the figure only to convert to svg with savefig > and you need to close it to avoid the inline plot, then there is no reason > to keep it as an instance attribute. > > It might also be a good idea to disable matplotlib interactive mode by > calling plt.ioff() before creating the figure object and reenabling it after > the figure is closed by calling plt.ion(). > > -- > Darlan > That was it, thanks! From ellisonbg at gmail.com Tue Jan 6 13:53:06 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 6 Jan 2015 10:53:06 -0800 Subject: [IPython-dev] Change in weekly dev meeting times Message-ID: <CAH4pYpRht0rCGse2wDx6qd+wBvDUUozw91naG2-oxn4vE0LZTw@mail.gmail.com> Hi all! In the past we have had two weekly dev meetings: * Tuesday 9:30-11am PST = special topics as needed * Thursday 9:30-11am PST = general topics, always happens Because of some scheduling conflicts, we are going to swap those starting this week: * Tuesday 9:30-11am PST = general topics, always happens * Thursday 9:30-11am PST = special topics as needed Thus, this Th, we will not have a regular, general meeting. We may have a special topics meeting if needed, but we aren't sure yet. Out next general meeting will be next Tuesday 9:30am PST. Se you all soon! Cheers, Brian -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150106/b09877c5/attachment.html> From benjaminrk at gmail.com Thu Jan 8 13:04:10 2015 From: benjaminrk at gmail.com (MinRK) Date: Thu, 8 Jan 2015 10:04:10 -0800 Subject: [IPython-dev] Notebook websocket change Message-ID: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> Alternative notebook frontend authors: There is an upcoming change <https://github.com/ipython/ipython/pull/7389> to websocket connections to the notebook server. Instead of three websockets per notebook, one for each zmq channel, there is a single websocket connection between the server and client. The channel associated with each message is identified by a channel key in the message dict itself. Relevant changes: - url is /channels instead of /shell, etc. - when sending a message, add a channel key with the channel name ('shell', 'stdin') - when receiving a message, check the channel key for routing -MinRK ? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/3e25d78e/attachment.html> From wstein at gmail.com Thu Jan 8 13:10:10 2015 From: wstein at gmail.com (William Stein) Date: Thu, 8 Jan 2015 10:10:10 -0800 Subject: [IPython-dev] Notebook websocket change In-Reply-To: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> Message-ID: <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote: > Alternative notebook frontend authors: > > There is an upcoming change to websocket connections to the notebook server. > Instead of three websockets per notebook, one for each zmq channel, there is > a single websocket connection between the server and client. The channel > associated with each message is identified by a channel key in the message > dict itself. > > Relevant changes: > > url is /channels instead of /shell, etc. > when sending a message, add a channel key with the channel name ('shell', > 'stdin') > when receiving a message, check the channel key for routing > > -MinRK Thanks. Of related interest, what the situation involving providing a fallback to websockets? I tried to only support websockets connections with SageMathCloud for a few weeks, and immediately ran into many complaints from users who couldn't connect as a result. This is even with https and so on -- it's just a sad fact that in some corporate or otherwise constrained environments that websockets don't work. By far the best approach I found was to use Primus + Engine.io, which uses websockets if possible, but will fallback to polling. So this is what SMC does now, and everything works, even for people that don't have websockets as an option... except for IPython-in-SMC of course, which doesn't work. Another unrelated issue is that I'm now hardcoding making this change around line 171 of IPython/html/notebookapp.py, since I want to serve the purely static html of IPython from a completely different web server: #static_url_prefix = url_path_join(base_url,'/static/'), static_url_prefix = '/static/ipython/', (Sorry for derailing the thread.) -- William > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- William Stein Professor of Mathematics University of Washington http://wstein.org From ellisonbg at gmail.com Thu Jan 8 13:19:19 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 8 Jan 2015 10:19:19 -0800 Subject: [IPython-dev] Notebook websocket change In-Reply-To: <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> Message-ID: <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> So far we have felt that it isn't worth the complexity cost for us to support people that deliberately choose to break the internet. It is not like WebSockets is some crazy, insecure, unsupported hack over a weird port. It is just different bytes going over the same HTTP/HTTPS socket connection and is essentially a formal standard (I know you know this - this is our messaging to companies who complain). Our situation is a bit different though than SMC because most of our current users run the notebook on their own computers. This may be something that we eventually want to rethink as more people run this in the cloud. However, I would like to push back on the companies choosing to break the internet really hard before giving in. I am guessing that in most cases, WebSockets are broken in enterprise setting not because of some important deliberate choice, but rather because people don't understand it and are using default settings that disable them (I could be wrong though).... Cheers, Brian On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> wrote: > On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote: > > Alternative notebook frontend authors: > > > > There is an upcoming change to websocket connections to the notebook > server. > > Instead of three websockets per notebook, one for each zmq channel, > there is > > a single websocket connection between the server and client. The channel > > associated with each message is identified by a channel key in the > message > > dict itself. > > > > Relevant changes: > > > > url is /channels instead of /shell, etc. > > when sending a message, add a channel key with the channel name ('shell', > > 'stdin') > > when receiving a message, check the channel key for routing > > > > -MinRK > > Thanks. > > Of related interest, what the situation involving providing a fallback > to websockets? > > I tried to only support websockets connections with SageMathCloud for > a few weeks, and immediately ran into many complaints from users who > couldn't connect as a result. This is even with https and so on -- > it's just a sad fact that in some corporate or otherwise constrained > environments that websockets don't work. By far the best approach I > found was to use Primus + Engine.io, which uses websockets if > possible, but will fallback to polling. So this is what SMC does > now, and everything works, even for people that don't have websockets > as an option... except for IPython-in-SMC of course, which doesn't > work. > > Another unrelated issue is that I'm now hardcoding making this change > around line 171 of IPython/html/notebookapp.py, since I want to serve > the purely static html of IPython from a completely different web > server: > > #static_url_prefix = url_path_join(base_url,'/static/'), > static_url_prefix = '/static/ipython/', > > (Sorry for derailing the thread.) > > -- William > > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > William Stein > Professor of Mathematics > University of Washington > http://wstein.org > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/4eb68b6f/attachment.html> From wstein at gmail.com Thu Jan 8 13:55:53 2015 From: wstein at gmail.com (William Stein) Date: Thu, 8 Jan 2015 10:55:53 -0800 Subject: [IPython-dev] Notebook websocket change In-Reply-To: <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> Message-ID: <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com> wrote: > So far we have felt that it isn't worth the complexity cost for us to > support people that deliberately choose to break the internet. It is not > like WebSockets is some crazy, insecure, unsupported hack over a weird port. > It is just different bytes going over the same HTTP/HTTPS socket connection > and is essentially a formal standard (I know you know this - this is our > messaging to companies who complain). I know -- that's why I switched SMC to pure websockets for a while. Then I received complaints at a rate of about 2 very desperate users per week, and these were the people who cared enough to complain repeatedly. > Our situation is a bit different though than SMC because most of our current > users run the notebook on their own computers. This may be something that we > eventually want to rethink as more people run this in the cloud. However, I > would like to push back on the companies choosing to break the internet > really hard before giving in. I am guessing that in most cases, WebSockets > are broken in enterprise setting not because of some important deliberate > choice, but rather because people don't understand it and are using default > settings that disable them (I could be wrong though).... In my experience, for some people that's exactly the problem. For one person I helped it turned out their specific problem was the Windows install they had access to had websockets explicitly disabled -- perhaps that was an IT policy. For other people the problem is further along the stack (e.g., possibly involving caching). Anyway, this change so that "there is a single websocket connection between the server and client. " makes implementing an alternative to WebSocket as a fallback dramatically easier, which I greatly appreciate. > Our situation is a bit different though than SMC because most of our current > users run the notebook on their own computers. This may be something that we Just curious -- how do you know that most people using IPython aren't already using it through SMC? William > > Cheers, > > Brian > > > > On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> wrote: >> >> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote: >> > Alternative notebook frontend authors: >> > >> > There is an upcoming change to websocket connections to the notebook >> > server. >> > Instead of three websockets per notebook, one for each zmq channel, >> > there is >> > a single websocket connection between the server and client. The channel >> > associated with each message is identified by a channel key in the >> > message >> > dict itself. >> > >> > Relevant changes: >> > >> > url is /channels instead of /shell, etc. >> > when sending a message, add a channel key with the channel name >> > ('shell', >> > 'stdin') >> > when receiving a message, check the channel key for routing >> > >> > -MinRK >> >> Thanks. >> >> Of related interest, what the situation involving providing a fallback >> to websockets? >> >> I tried to only support websockets connections with SageMathCloud for >> a few weeks, and immediately ran into many complaints from users who >> couldn't connect as a result. This is even with https and so on -- >> it's just a sad fact that in some corporate or otherwise constrained >> environments that websockets don't work. By far the best approach I >> found was to use Primus + Engine.io, which uses websockets if >> possible, but will fallback to polling. So this is what SMC does >> now, and everything works, even for people that don't have websockets >> as an option... except for IPython-in-SMC of course, which doesn't >> work. >> >> Another unrelated issue is that I'm now hardcoding making this change >> around line 171 of IPython/html/notebookapp.py, since I want to serve >> the purely static html of IPython from a completely different web >> server: >> >> #static_url_prefix = url_path_join(base_url,'/static/'), >> static_url_prefix = '/static/ipython/', >> >> (Sorry for derailing the thread.) >> >> -- William >> >> > >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> >> >> >> -- >> William Stein >> Professor of Mathematics >> University of Washington >> http://wstein.org >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- William Stein Professor of Mathematics University of Washington http://wstein.org From benjaminrk at gmail.com Thu Jan 8 14:18:43 2015 From: benjaminrk at gmail.com (MinRK) Date: Thu, 8 Jan 2015 11:18:43 -0800 Subject: [IPython-dev] Notebook websocket change In-Reply-To: <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> Message-ID: <CAHNn8BUSrDb8kVO94-dv9FsNJtZu=u=dW2QMdLgEn5_B3v4CRQ@mail.gmail.com> Some time ago, I made a PR to IPython switching to socket.io, which required the same basic changes since it only allowed a single connection per page, so I know it's not too hard. We may start to see more pressure as hosted instances, and reconsider the fallback in the future, but at this point, we don't feel that pressure. Wind proper websockets to be sufficiently preferable that we will wait as long as we can before appeasing old, broken, or misconfigured network environments that prevent websockets from working. The majority of notebook users seem to be running as a desktop app via localhost, where websockets are rarely a problem (The nightmare of Sophos AV/firewall on Windows excluded). -MinRK On Thu, Jan 8, 2015 at 10:55 AM, William Stein <wstein at gmail.com> wrote: > On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com> > wrote: > > So far we have felt that it isn't worth the complexity cost for us to > > support people that deliberately choose to break the internet. It is not > > like WebSockets is some crazy, insecure, unsupported hack over a weird > port. > > It is just different bytes going over the same HTTP/HTTPS socket > connection > > and is essentially a formal standard (I know you know this - this is our > > messaging to companies who complain). > > I know -- that's why I switched SMC to pure websockets for a while. > Then I received complaints at a rate of about 2 very desperate users > per week, and these were the people who cared enough to complain > repeatedly. > > > Our situation is a bit different though than SMC because most of our > current > > users run the notebook on their own computers. This may be something > that we > > eventually want to rethink as more people run this in the cloud. > However, I > > would like to push back on the companies choosing to break the internet > > really hard before giving in. I am guessing that in most cases, > WebSockets > > are broken in enterprise setting not because of some important deliberate > > choice, but rather because people don't understand it and are using > default > > settings that disable them (I could be wrong though).... > > In my experience, for some people that's exactly the problem. For one > person I helped it turned out their specific problem was the Windows > install they had access to had websockets explicitly disabled -- > perhaps that was an IT policy. For other people the problem is > further along the stack (e.g., possibly involving caching). > > Anyway, this change so that "there is a single websocket connection > between the server and client. " makes implementing an alternative to > WebSocket as a fallback dramatically easier, which I greatly > appreciate. > > > Our situation is a bit different though than SMC because most of our > current > > users run the notebook on their own computers. This may be something > that we > > Just curious -- how do you know that most people using IPython aren't > already using it through SMC? > > William > > > > > Cheers, > > > > Brian > > > > > > > > On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> wrote: > >> > >> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote: > >> > Alternative notebook frontend authors: > >> > > >> > There is an upcoming change to websocket connections to the notebook > >> > server. > >> > Instead of three websockets per notebook, one for each zmq channel, > >> > there is > >> > a single websocket connection between the server and client. The > channel > >> > associated with each message is identified by a channel key in the > >> > message > >> > dict itself. > >> > > >> > Relevant changes: > >> > > >> > url is /channels instead of /shell, etc. > >> > when sending a message, add a channel key with the channel name > >> > ('shell', > >> > 'stdin') > >> > when receiving a message, check the channel key for routing > >> > > >> > -MinRK > >> > >> Thanks. > >> > >> Of related interest, what the situation involving providing a fallback > >> to websockets? > >> > >> I tried to only support websockets connections with SageMathCloud for > >> a few weeks, and immediately ran into many complaints from users who > >> couldn't connect as a result. This is even with https and so on -- > >> it's just a sad fact that in some corporate or otherwise constrained > >> environments that websockets don't work. By far the best approach I > >> found was to use Primus + Engine.io, which uses websockets if > >> possible, but will fallback to polling. So this is what SMC does > >> now, and everything works, even for people that don't have websockets > >> as an option... except for IPython-in-SMC of course, which doesn't > >> work. > >> > >> Another unrelated issue is that I'm now hardcoding making this change > >> around line 171 of IPython/html/notebookapp.py, since I want to serve > >> the purely static html of IPython from a completely different web > >> server: > >> > >> #static_url_prefix = url_path_join(base_url,'/static/'), > >> static_url_prefix = '/static/ipython/', > >> > >> (Sorry for derailing the thread.) > >> > >> -- William > >> > >> > > >> > > >> > _______________________________________________ > >> > IPython-dev mailing list > >> > IPython-dev at scipy.org > >> > http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > >> > >> > >> > >> -- > >> William Stein > >> Professor of Mathematics > >> University of Washington > >> http://wstein.org > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > > > -- > > Brian E. Granger > > Cal Poly State University, San Luis Obispo > > @ellisonbg on Twitter and GitHub > > bgranger at calpoly.edu and ellisonbg at gmail.com > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > William Stein > Professor of Mathematics > University of Washington > http://wstein.org > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/c688ec6e/attachment.html> From rgbkrk at gmail.com Thu Jan 8 14:33:52 2015 From: rgbkrk at gmail.com (Kyle Kelley) Date: Thu, 8 Jan 2015 13:33:52 -0600 Subject: [IPython-dev] Notebook websocket change In-Reply-To: <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> Message-ID: <CA+tbMaV2tUVnr5HOiXUPddoRmanb1YbqBO99eHJRUrtwtaqC4Q@mail.gmail.com> You're right Brian, it's about them buying some $FIREWALL_IPS_SUPER_PRICEY_DEVICE that doesn't proxy websockets appropriately and the person purchasing having no clue other than the fact that $VENDOR gave them a neat trinket at a conference. Firewalls and proxies that do handle websockets appropriately do exist. I'd be tempted to make a list of ones that do it well and display them prominently, if only for the benefit of our own users. I have witnessed these flavors of IPython in the wild (not including our own recent stunts): * Corporate entities using an Anaconda server (websockets not a problem internally but were externally) * People of all backgrounds using Anaconda's IPython notebook bundling * Vanilla installations from package managers or pip * Hosted instance of their own flavor * DIT4C Most users that we hear from are using a local installation and there's probably a large amount of confirmation bias. It would be great if we had actual shared metrics between us but we only know: * Unique viewers/users of the Nature demo (> 14,000 total, according to GA) * Viewers on the notebook viewer (> 200k per month, also according to GA) * Downloads on PyPI (skewed, not unique) * Clones from github (also skewed) We could actually be tracking who has websocket support of those hitting the notebook viewer for a bigger sense of potential users (since viewers are not necessarily IPython/Jupyter users). On Thu, Jan 8, 2015 at 12:55 PM, William Stein <wstein at gmail.com> wrote: > On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com> > wrote: > > So far we have felt that it isn't worth the complexity cost for us to > > support people that deliberately choose to break the internet. It is not > > like WebSockets is some crazy, insecure, unsupported hack over a weird > port. > > It is just different bytes going over the same HTTP/HTTPS socket > connection > > and is essentially a formal standard (I know you know this - this is our > > messaging to companies who complain). > > I know -- that's why I switched SMC to pure websockets for a while. > Then I received complaints at a rate of about 2 very desperate users > per week, and these were the people who cared enough to complain > repeatedly. > > > Our situation is a bit different though than SMC because most of our > current > > users run the notebook on their own computers. This may be something > that we > > eventually want to rethink as more people run this in the cloud. > However, I > > would like to push back on the companies choosing to break the internet > > really hard before giving in. I am guessing that in most cases, > WebSockets > > are broken in enterprise setting not because of some important deliberate > > choice, but rather because people don't understand it and are using > default > > settings that disable them (I could be wrong though).... > > In my experience, for some people that's exactly the problem. For one > person I helped it turned out their specific problem was the Windows > install they had access to had websockets explicitly disabled -- > perhaps that was an IT policy. For other people the problem is > further along the stack (e.g., possibly involving caching). > > Anyway, this change so that "there is a single websocket connection > between the server and client. " makes implementing an alternative to > WebSocket as a fallback dramatically easier, which I greatly > appreciate. > > > Our situation is a bit different though than SMC because most of our > current > > users run the notebook on their own computers. This may be something > that we > > Just curious -- how do you know that most people using IPython aren't > already using it through SMC? > > William > > > > > Cheers, > > > > Brian > > > > > > > > On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> wrote: > >> > >> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote: > >> > Alternative notebook frontend authors: > >> > > >> > There is an upcoming change to websocket connections to the notebook > >> > server. > >> > Instead of three websockets per notebook, one for each zmq channel, > >> > there is > >> > a single websocket connection between the server and client. The > channel > >> > associated with each message is identified by a channel key in the > >> > message > >> > dict itself. > >> > > >> > Relevant changes: > >> > > >> > url is /channels instead of /shell, etc. > >> > when sending a message, add a channel key with the channel name > >> > ('shell', > >> > 'stdin') > >> > when receiving a message, check the channel key for routing > >> > > >> > -MinRK > >> > >> Thanks. > >> > >> Of related interest, what the situation involving providing a fallback > >> to websockets? > >> > >> I tried to only support websockets connections with SageMathCloud for > >> a few weeks, and immediately ran into many complaints from users who > >> couldn't connect as a result. This is even with https and so on -- > >> it's just a sad fact that in some corporate or otherwise constrained > >> environments that websockets don't work. By far the best approach I > >> found was to use Primus + Engine.io, which uses websockets if > >> possible, but will fallback to polling. So this is what SMC does > >> now, and everything works, even for people that don't have websockets > >> as an option... except for IPython-in-SMC of course, which doesn't > >> work. > >> > >> Another unrelated issue is that I'm now hardcoding making this change > >> around line 171 of IPython/html/notebookapp.py, since I want to serve > >> the purely static html of IPython from a completely different web > >> server: > >> > >> #static_url_prefix = url_path_join(base_url,'/static/'), > >> static_url_prefix = '/static/ipython/', > >> > >> (Sorry for derailing the thread.) > >> > >> -- William > >> > >> > > >> > > >> > _______________________________________________ > >> > IPython-dev mailing list > >> > IPython-dev at scipy.org > >> > http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > >> > >> > >> > >> -- > >> William Stein > >> Professor of Mathematics > >> University of Washington > >> http://wstein.org > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > > > -- > > Brian E. Granger > > Cal Poly State University, San Luis Obispo > > @ellisonbg on Twitter and GitHub > > bgranger at calpoly.edu and ellisonbg at gmail.com > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > William Stein > Professor of Mathematics > University of Washington > http://wstein.org > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/4320a011/attachment.html> From ellisonbg at gmail.com Thu Jan 8 17:08:54 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 8 Jan 2015 14:08:54 -0800 Subject: [IPython-dev] Notebook websocket change In-Reply-To: <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> Message-ID: <CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com> > > > Just curious -- how do you know that most people using IPython aren't > already using it through SMC? > Great question! As with all such things, if you are asking for a precise data driven answers, then we don't know. But here is the constellation of evidence that would lead me to this conclusion (that SMC represents a minority fraction of the overall number of ipython users): * We estimate that the total ipython user base is in the 0.5-1.5 million range. However, this is likely a very conservative estimate. Fernando has the best summary of where we get these estimates (I don't remember where the most recent information is - probably in a private email to someone). If SMC has this number of users, then my assertion is clearly to be questioned. * We spend a significant amount of time out meeting with ipython users an conferences, universities, research labs, companies. Over this sample, I know of very few people using SMC (less than 1%). I will be the first to admit that this is both anecdotal and subject my the sampling bias of my particular travel schedule... * Users get and use ipython through a large number of avenues: anaconda, pip, github, SMC, Canopy, apt-get, yum, etc. Our overall usage numbers come from trying to estimate the install base of each of these things and adding them up - we then double check with things like nbviewer usage, etc. Anecdotally, it appears that anaconda is the most common way that people get and use ipython, but the most reliable numbers we have come from apt-get and yum. * Commercial usage of the notebook (in private with private data) is quickly overtaking the more open academic and educational uses of the notebook. The difficulty here is that companies tend to keep their usage private. However, each time we get a glimpse at commercial usage (which is happening more and more) we are blown away by the scale of it. My assumption is that industry usage with private code+data is not the main audience for SMC. I want to emphasize that I don't in any way mean to minimize the impact or importance of SMC. We all think it is fantastic what you are doing and I personally recommend it to people all the time. ps - are you willing to share SMC usage stats with us? It would really help us to reduce the error bars on our usage numbers, which is very helpful in talking to funding agencies... Cheers, Brian -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/0fc1855d/attachment.html> From wstein at gmail.com Thu Jan 8 17:33:50 2015 From: wstein at gmail.com (William Stein) Date: Thu, 8 Jan 2015 14:33:50 -0800 Subject: [IPython-dev] Notebook websocket change In-Reply-To: <CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> <CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com> Message-ID: <CACLE5GC1w3+anLwAFK6dwg_7OmzR6ULdrpH=rZFVcY6Ee4vqdw@mail.gmail.com> On Thu, Jan 8, 2015 at 2:08 PM, Brian Granger <ellisonbg at gmail.com> wrote: >> >> Just curious -- how do you know that most people using IPython aren't >> already using it through SMC? > > > Great question! > > As with all such things, if you are asking for a precise data driven > answers, then we don't know. > > But here is the constellation of evidence that would lead me to this > conclusion (that SMC represents a minority fraction of the overall number of > ipython users): > > * We estimate that the total ipython user base is in the 0.5-1.5 million > range. However, this is likely a very conservative estimate. Fernando has > the best summary of where we get these estimates (I don't remember where the > most recent information is - probably in a private email to someone). If SMC > has this number of users, then my assertion is clearly to be questioned. > * We spend a significant amount of time out meeting with ipython users an > conferences, universities, research labs, companies. Over this sample, I > know of very few people using SMC (less than 1%). I will be the first to > admit that this is both anecdotal and subject my the sampling bias of my > particular travel schedule... > * Users get and use ipython through a large number of avenues: anaconda, > pip, github, SMC, Canopy, apt-get, yum, etc. Our overall usage numbers come > from trying to estimate the install base of each of these things and adding > them up - we then double check with things like nbviewer usage, etc. > Anecdotally, it appears that anaconda is the most common way that people get > and use ipython, but the most reliable numbers we have come from apt-get and > yum. > * Commercial usage of the notebook (in private with private data) is quickly > overtaking the more open academic and educational uses of the notebook. The > difficulty here is that companies tend to keep their usage private. However, > each time we get a glimpse at commercial usage (which is happening more and > more) we are blown away by the scale of it. My assumption is that industry > usage with private code+data is not the main audience for SMC. > > I want to emphasize that I don't in any way mean to minimize the impact or > importance of SMC. We all think it is fantastic what you are doing and I > personally recommend it to people all the time. > > ps - are you willing to share SMC usage stats with us? It would really help > us to reduce the error bars on our usage numbers, which is very helpful in > talking to funding agencies... Thanks for explaining how you measure usage, which I knew nothing about. Regarding my SMC usage stats, it's all here: http://boxen.math.washington.edu/home/schilly/salvus/stats/stats.html There is in theory more precise data, but it's not been mined yet. Usage of SMC appears to be miniscule compared to any of the numbers you list above. William > > Cheers, Brian > > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- William Stein Professor of Mathematics University of Washington http://wstein.org From ellisonbg at gmail.com Thu Jan 8 17:42:52 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 8 Jan 2015 14:42:52 -0800 Subject: [IPython-dev] Notebook websocket change In-Reply-To: <CACLE5GC1w3+anLwAFK6dwg_7OmzR6ULdrpH=rZFVcY6Ee4vqdw@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> <CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com> <CACLE5GC1w3+anLwAFK6dwg_7OmzR6ULdrpH=rZFVcY6Ee4vqdw@mail.gmail.com> Message-ID: <CAH4pYpSetKLCzT7--uHEns7mbDonvBc3L+nyT7LoXrmu11XiMQ@mail.gmail.com> Thanks for sharing this data! This will be really helpful for us! Cheers, Brian On Thu, Jan 8, 2015 at 2:33 PM, William Stein <wstein at gmail.com> wrote: > On Thu, Jan 8, 2015 at 2:08 PM, Brian Granger <ellisonbg at gmail.com> wrote: > >> > >> Just curious -- how do you know that most people using IPython aren't > >> already using it through SMC? > > > > > > Great question! > > > > As with all such things, if you are asking for a precise data driven > > answers, then we don't know. > > > > But here is the constellation of evidence that would lead me to this > > conclusion (that SMC represents a minority fraction of the overall > number of > > ipython users): > > > > * We estimate that the total ipython user base is in the 0.5-1.5 million > > range. However, this is likely a very conservative estimate. Fernando has > > the best summary of where we get these estimates (I don't remember where > the > > most recent information is - probably in a private email to someone). If > SMC > > has this number of users, then my assertion is clearly to be questioned. > > * We spend a significant amount of time out meeting with ipython users an > > conferences, universities, research labs, companies. Over this sample, I > > know of very few people using SMC (less than 1%). I will be the first to > > admit that this is both anecdotal and subject my the sampling bias of my > > particular travel schedule... > > * Users get and use ipython through a large number of avenues: anaconda, > > pip, github, SMC, Canopy, apt-get, yum, etc. Our overall usage numbers > come > > from trying to estimate the install base of each of these things and > adding > > them up - we then double check with things like nbviewer usage, etc. > > Anecdotally, it appears that anaconda is the most common way that people > get > > and use ipython, but the most reliable numbers we have come from apt-get > and > > yum. > > * Commercial usage of the notebook (in private with private data) is > quickly > > overtaking the more open academic and educational uses of the notebook. > The > > difficulty here is that companies tend to keep their usage private. > However, > > each time we get a glimpse at commercial usage (which is happening more > and > > more) we are blown away by the scale of it. My assumption is that > industry > > usage with private code+data is not the main audience for SMC. > > > > I want to emphasize that I don't in any way mean to minimize the impact > or > > importance of SMC. We all think it is fantastic what you are doing and I > > personally recommend it to people all the time. > > > > ps - are you willing to share SMC usage stats with us? It would really > help > > us to reduce the error bars on our usage numbers, which is very helpful > in > > talking to funding agencies... > > Thanks for explaining how you measure usage, which I knew nothing > about. Regarding my SMC usage stats, it's all here: > > http://boxen.math.washington.edu/home/schilly/salvus/stats/stats.html > > There is in theory more precise data, but it's not been mined yet. > > Usage of SMC appears to be miniscule compared to any of the numbers > you list above. > > William > > > > > Cheers, Brian > > > > > > > > > > -- > > Brian E. Granger > > Cal Poly State University, San Luis Obispo > > @ellisonbg on Twitter and GitHub > > bgranger at calpoly.edu and ellisonbg at gmail.com > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > William Stein > Professor of Mathematics > University of Washington > http://wstein.org > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/b91b47dc/attachment.html> From fperez.net at gmail.com Thu Jan 8 18:20:03 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 8 Jan 2015 15:20:03 -0800 Subject: [IPython-dev] [jupyter] Re: Notebook websocket change In-Reply-To: <CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> <CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com> Message-ID: <CAHAreOrQ+WvU2V2uxKYrho=XGYqSS6F1BrGmZth88kKU0s4Uaw@mail.gmail.com> On Thu, Jan 8, 2015 at 2:08 PM, Brian Granger <ellisonbg at gmail.com> wrote: > * We estimate that the total ipython user base is in the 0.5-1.5 million > range. However, this is likely a very conservative estimate. Fernando has > the best summary of where we get these estimates (I don't remember where > the most recent information is - probably in a private email to someone). This is how I most recently answered the question "what is the impact and user base of IPython", indeed in the context of a private email as part of the EU Sage-math grant proposal: Getting metrics for usage is very tough, but we have some information. You can read the report we wrote early this year for Sloan, that provides a bunch of info (notebook and PDF format): http://nbviewer.ipython.org/github/ipython/sloan-2013-reports/blob/master/IPython%202013%20Progress%20Report%20-%20Sloan%20Foundation.ipynb http://archive.ipython.org/sloan-2013-reports/IPython%202013%20Progress%20Report%20-%20Sloan%20Foundation.pdf As you can see, rather than a single blurb of download stats, we paint a more complex (but I think richer) picture of broad usage and impact. A few more things since: - Recently highlighted in the cover of Nature: http://www.nature.com/news/interactive-notebooks-sharing-the-code-1.16261 and that article links to a live, interactive demo here: http://www.nature.com/news/ipython-interactive-demo-7.21492 This demo broke readership records for Nature, to the best of our understanding (we blew past the estimates the editors gave us in terms of highest numbers of simultaneous readers to expect). - The NBviewer site has very significant usage, these are the Google Analytics numbers for the period Nov 8, 2014-Dec 8, 2014: Sessions 423,334 Users 215,546 Pageviews 844,523 That's almost 1 million page views for people sharing content as IPython notebooks (that's the only purpose of that site). - Our Gallery of notebooks is a good showcase of the breadth of usage and impact of the notebook across domains: https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks - This list shows the adoption of our kernel architecture across languages: 23 independent implementations and counting, meaning this system is becoming the lingua franca of interactive computing: https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages - Here is a list of other projects (in academia and industry) using IPython: https://github.com/ipython/ipython/wiki/Projects-using-IPython - It's very hard to get accurate user counts, but we *estimate* at least ~2 Million users. This is a rough estimate, but I think realistic (and if anything, a conservative undercount). The estimate goes as follows: a) Estimates of linux users (not datacenter servers) range from 20M for Ubuntu to ~70M across distros: http://en.wikipedia.org/wiki/Ubuntu_(operating_system)#Installed_base http://en.wikipedia.org/wiki/Linux_adoption#Measuring_desktop_adoption b) The Debian 'popcon' statistics: https://qa.debian.org/popcon.php?package=ipython show IPython to be installed on ~ 5% of Debian installs. If we use that as a baseline, and estimate total linux user counts at say ~50M (rough average of the above two numbers), we get about 2.5M installs of IPython on linux. This doesn't count source installs, pip installs, nor any of the Windows and Mac OSX users who get it via Anaconda, Enthought Canopy, etc. Nor does it count the increasing number of server-side hosted deployments we see more and more of. But at least it gives a credible way to justify that IPython probably has *at least* 2M users. -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/f319b925/attachment.html> From ellisonbg at gmail.com Thu Jan 8 18:23:32 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 8 Jan 2015 15:23:32 -0800 Subject: [IPython-dev] [jupyter] Re: Notebook websocket change In-Reply-To: <CAHAreOrQ+WvU2V2uxKYrho=XGYqSS6F1BrGmZth88kKU0s4Uaw@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> <CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com> <CAHAreOrQ+WvU2V2uxKYrho=XGYqSS6F1BrGmZth88kKU0s4Uaw@mail.gmail.com> Message-ID: <CAH4pYpSGxeXE3RiHog-mWi6KZhrfixrNLZQ997+QK=syj9TkGA@mail.gmail.com> Thanks Fernando, that was it exactly! On Thu, Jan 8, 2015 at 3:20 PM, Fernando Perez <fperez.net at gmail.com> wrote: > On Thu, Jan 8, 2015 at 2:08 PM, Brian Granger <ellisonbg at gmail.com> wrote: > >> * We estimate that the total ipython user base is in the 0.5-1.5 million >> range. However, this is likely a very conservative estimate. Fernando has >> the best summary of where we get these estimates (I don't remember where >> the most recent information is - probably in a private email to someone). > > > This is how I most recently answered the question "what is the impact and > user base of IPython", indeed in the context of a private email as part of > the EU Sage-math grant proposal: > > > Getting metrics for usage is very tough, but we have some information. You > can read the report we wrote early this year for Sloan, that provides a > bunch of info (notebook and PDF format): > > > http://nbviewer.ipython.org/github/ipython/sloan-2013-reports/blob/master/IPython%202013%20Progress%20Report%20-%20Sloan%20Foundation.ipynb > > > http://archive.ipython.org/sloan-2013-reports/IPython%202013%20Progress%20Report%20-%20Sloan%20Foundation.pdf > > As you can see, rather than a single blurb of download stats, we paint a > more complex (but I think richer) picture of broad usage and impact. A few > more things since: > > - Recently highlighted in the cover of Nature: > > http://www.nature.com/news/interactive-notebooks-sharing-the-code-1.16261 > > and that article links to a live, interactive demo here: > > http://www.nature.com/news/ipython-interactive-demo-7.21492 > > This demo broke readership records for Nature, to the best of our > understanding (we blew past the estimates the editors gave us in terms of > highest numbers of simultaneous readers to expect). > > - The NBviewer site has very significant usage, these are the Google > Analytics numbers for the period Nov 8, 2014-Dec 8, 2014: > > > Sessions > 423,334 > Users > 215,546 > Pageviews > 844,523 > That's almost 1 million page views for people sharing content as IPython > notebooks (that's the only purpose of that site). > > - Our Gallery of notebooks is a good showcase of the breadth of usage and > impact of the notebook across domains: > > > https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks > > - This list shows the adoption of our kernel architecture across > languages: 23 independent implementations and counting, meaning this system > is becoming the lingua franca of interactive computing: > > https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages > > - Here is a list of other projects (in academia and industry) using > IPython: > > https://github.com/ipython/ipython/wiki/Projects-using-IPython > > - It's very hard to get accurate user counts, but we *estimate* at least > ~2 Million users. This is a rough estimate, but I think realistic (and if > anything, a conservative undercount). The estimate goes as follows: > > a) Estimates of linux users (not datacenter servers) range from 20M for > Ubuntu to ~70M across distros: > > http://en.wikipedia.org/wiki/Ubuntu_(operating_system)#Installed_base > http://en.wikipedia.org/wiki/Linux_adoption#Measuring_desktop_adoption > > b) The Debian 'popcon' statistics: > > https://qa.debian.org/popcon.php?package=ipython > > show IPython to be installed on ~ 5% of Debian installs. > > If we use that as a baseline, and estimate total linux user counts at say > ~50M (rough average of the above two numbers), we get about 2.5M installs > of IPython on linux. This doesn't count source installs, pip installs, nor > any of the Windows and Mac OSX users who get it via Anaconda, Enthought > Canopy, etc. Nor does it count the increasing number of server-side hosted > deployments we see more and more of. > > But at least it gives a credible way to justify that IPython probably has > *at least* 2M users. > > > > > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > -- > You received this message because you are subscribed to the Google Groups > "Project Jupyter" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jupyter+unsubscribe at googlegroups.com. > To post to this group, send email to jupyter at googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jupyter/CAHAreOrQ%2BWvU2V2uxKYrho%3DXGYqSS6F1BrGmZth88kKU0s4Uaw%40mail.gmail.com > <https://groups.google.com/d/msgid/jupyter/CAHAreOrQ%2BWvU2V2uxKYrho%3DXGYqSS6F1BrGmZth88kKU0s4Uaw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/6144b025/attachment.html> From jackson_loper at brown.edu Thu Jan 8 22:28:03 2015 From: jackson_loper at brown.edu (Loper, Jackson) Date: Thu, 8 Jan 2015 22:28:03 -0500 Subject: [IPython-dev] threadsafe widgets Message-ID: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com> It is my understanding that the traitlets in most ipython notebook widgets aren't threadsafe. If you're not sure what I mean, I attached a small example at the bottom of this note. What's the best way for me to make a threadsafe widget? I could inherit from DOMWidget and override a whole bunch of methods with a wrapper that puts things inside a threading.Lock, but it seems kinda like overkill. I suppose theoretically you could lock all the traitlets & comms calls, but that also seems like overkill. Or maybe not...I have no sense of proportion for these things. The application is straightforward, maybe there's an easier way to do it. I'd like to be able to edit files from jupyterhub's host directly inside the notebook; CodeMirror is much more responsive than vim over ssh. It was easy to make a text-editing widget based on codemirror, but I'd like to pop up a CodeMirror.dialog if somebody has changed the file being edited. Can't figure out how to do that in a threadsafe way. Happy Thursday! Jackson Loper Pattern Theory Group Division of Applied Math Brown University ====== threadsome danger ===== Running the latest github clone of jupyterhub, code like this is a very bad idea: import IPython.display from IPython.html import widgets, nbextensions import threading from numpy.random import randint isw=widgets.IntSlider() IPython.display.display(isw) def randoset(N): for i in range(N): isw.value=randint(0,50) t1=threading.Thread(target=randoset,args=[100]) t2=threading.Thread(target=randoset,args=[100]) t1.start() t2.start() The jupyterhub logs errors such as Malformed message: [b'comm-....', b'<IDS|MSG>', b'comm-...', b'<IDS|MSG>', ... more stuff.. ] Presumably two attempts at sending data over comms got mixeduptogether here, which is why we ended up with two <IDS_MSG> in one message! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/e7edef5e/attachment.html> From fperez.net at gmail.com Thu Jan 8 23:25:51 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 8 Jan 2015 20:25:51 -0800 Subject: [IPython-dev] threadsafe widgets In-Reply-To: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com> References: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com> Message-ID: <CAHAreOoLp3tqbbOcC76Xt79vdS+2k9DBuns=HyX_FQ3Eb=ESew@mail.gmail.com> On Thu, Jan 8, 2015 at 7:28 PM, Loper, Jackson <jackson_loper at brown.edu> wrote: > The application is straightforward, maybe there's an easier way to do it. > > I'd like to be able to edit files from jupyterhub's host directly inside > the notebook; > Gotta run, so skipping thinking about threads right now :) But indeed, there's an easier way to do the above: git pull. In master of IPython, you can create/edit full files, just try it out at tmpnb.org and let us know how it goes. Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/059efb28/attachment.html> From dsdale24 at gmail.com Fri Jan 9 11:38:29 2015 From: dsdale24 at gmail.com (Darren Dale) Date: Fri, 09 Jan 2015 16:38:29 +0000 Subject: [IPython-dev] embedding ipython, namespace question References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com> <AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov> <CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com> <FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com> <CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com> <DD2429D4-4BE1-49FC-A891-BF153FBA942F@anl.gov> <CAK6O52ns=WiaMSGRista9wE_VHdgFap1Wt65h1yDwEECawAULA@mail.gmail.com> Message-ID: <CAK6O52kd_uAKQ4U8Hs5FS03b9VJ4iFv-8NRPM23pjgYOPtoZvg@mail.gmail.com> On Sat Jan 03 2015 at 10:45:42 AM Darren Dale <dsdale24 at gmail.com> wrote: > Hi Ray, > > On Sat Dec 27 2014 at 8:48:20 PM Osborn, Raymond <rosborn at anl.gov> wrote: > >> I don?t really understand what you are trying to achieve, but the >> ?user_ns? dictionary isn?t an isolated namespace - it?s the namespace that >> is used by the console, and I would have thought you would have to do some >> kind of injection to add other objects from within the application. >> > > What I am trying to achieve is explicitly documented at > http://ipython.org/ipython-doc/dev/interactive/reference.html#embedding-ipython > : > > --- > It is also possible to embed an IPython shell in a namespace in your > Python code. This allows you to evaluate dynamically the state of your > code, operate with your variables, analyze them, etc. Note however that any > changes you make to values while in the shell do not propagate back to the > running code, so it is safe to modify your values because you won?t break > your code in bizarre ways by doing so. > > Note > At present, embedding IPython cannot be done from inside IPython. Run the > code samples below outside IPython. > [DD: I am not attempting to embed ipython from inside ipython] > > This feature allows you to easily have a fully functional python > environment for doing object introspection anywhere in your code with a > simple function call. In some cases a simple print statement is enough, but > if you need to do more detailed analysis of a code fragment this feature > can be very valuable. > > It can also be useful in scientific computing situations where it is > common to need to do some automatic, computationally intensive part and > then stop to look at data, plots, etc. Opening an IPython instance will > give you full access to your data and functions, and you can resume program > execution once you are done with the interactive part (perhaps to stop > again later, as many times as needed). > The following code snippet is the bare minimum you need to include in your > Python programs for this to work (detailed examples follow later): > > from IPython import embed > embed() # this call anywhere in your program will start IPython > > You can also embed an IPython kernel, for use with qtconsole, etc. via > IPython.embed_kernel(). This should function work the same way, but you can > connect an external frontend (ipython qtconsole or ipython console), rather > than interacting with it in the terminal. > --- > > This is impressively simple for the embed function: > > --- > C:\Users\darren> python > Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul 2 2014, 15:12:11) > [MSC v.1500 64 bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > Anaconda is brought to you by Continuum Analytics. > Please check out: http://continuum.io/thanks and https://binstar.org > >>> from IPython import embed > >>> a=1 > >>> embed() > Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul 2 2014, 15:12:11) > [MSC v.1500 64 bit (AMD64)] > Type "copyright", "credits" or "license" for more information. > > IPython 2.3.1 -- An enhanced Interactive Python. > Anaconda is brought to you by Continuum Analytics. > Please check out: http://continuum.io/thanks and https://binstar.org > ? -> Introduction and overview of IPython's features. > %quickref -> Quick reference. > help -> Python's own help system. > object? -> Details about 'object', use 'object??' for extra details. > > In [1]: a > Out[1]: 1 > --- > > But I have not been able to achieve the same behavior with the qt > in-process console. > If this is not possible, perhaps the documentation should be changed? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150109/6b5208e5/attachment.html> From jackson_loper at brown.edu Fri Jan 9 17:59:39 2015 From: jackson_loper at brown.edu (Loper, Jackson) Date: Fri, 9 Jan 2015 17:59:39 -0500 Subject: [IPython-dev] threadsafe widgets In-Reply-To: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com> References: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com> Message-ID: <CAP8T2VKxJ0S7bMMJ8au+343FCYF7s__nVZOph5BZows2y86u5g@mail.gmail.com> Ah, the threading situation seems to be more diresome than I thought. I believe the following code is unsafe to run. === Problem code === import IPython.display from IPython.html import widgets import threading from numpy.random import randint isw1=widgets.IntSlider() IPython.display.display(isw1) isw2=widgets.IntSlider() IPython.display.display(isw2) def randoset(wid,N): for i in range(N): wid.value=randint(0,50) t1=threading.Thread(target=randoset,args=[isw1,100]) t2=threading.Thread(target=randoset,args=[isw2,100]) t1.start() t2.start() This code produces malformed message errors for me in the jupyterhub terminal output. I believe this is because all widgets seem to share the same iopub_socket and write to it whenever the user changes a traitlet. And of course zmq sockets aren't threadsafe. === A few potential solutions === 1) "Quick" -- Wrap iopub_socket in a threadsafe wrapper. Surround every important function call in a mutex. 2) "Hintjens-approved" -- Give every single widget its own inproc PUB socket. We'd have to spin up an extra thread (or a greenlet in some pre-extant thread? I don't really understand what the different IPython threads do...but I could find out...) with a SUB socket to collect them all together and forward them all out on iopub_socket. 3) "Dirty" -- Wrap IPython.kernel.zmq.session.Session.send inside a mutex. Are there use-cases where anybody besides Session actually does anything with iopub_socket? === Motivation === I think there could be valid use-cases for widgets which display results from polling external resources, e.g. widgets which keep track of EC2 instances, widgets which monitor IPython.parallel workerbees, etc. === Conclusion === I'd be happy to attack this problem, if you guys think it would be useful. If so, let me know what would be a good general direction, and I'll put together a pull request. Keep warm! Jackson Loper Pattern Theory Group Division of Applied Math Brown University PS. I love the new text editor feature in the ipython master branch! My days of vim-over-ssh aren't over yet (NERDTree and push-notification of filechange detection are still too useful to me when logging on from multiple machines) but I can see the end in sight! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150109/1084d54e/attachment.html> From ellisonbg at gmail.com Fri Jan 9 18:34:43 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 9 Jan 2015 15:34:43 -0800 Subject: [IPython-dev] threadsafe widgets In-Reply-To: <CAP8T2VKxJ0S7bMMJ8au+343FCYF7s__nVZOph5BZows2y86u5g@mail.gmail.com> References: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com> <CAP8T2VKxJ0S7bMMJ8au+343FCYF7s__nVZOph5BZows2y86u5g@mail.gmail.com> Message-ID: <CAH4pYpSxUOxKrzwWw1ptMAFKL_gapFN4XGVHFECoosBsuZTrGA@mail.gmail.com> Yuck :( but I am not surprised at this... > > > I believe this is because all widgets seem to share the same iopub_socket > and write to it whenever the user changes a traitlet. And of course zmq > sockets aren't threadsafe. > > Yep, they definitely share a single iopub socket. > === A few potential solutions === > > 1) "Quick" -- Wrap iopub_socket in a threadsafe wrapper. Surround every > important function call in a mutex. > Probably my second choice. > 2) "Hintjens-approved" -- Give every single widget its own inproc PUB > socket. We'd have to spin up an extra thread (or a greenlet in some > pre-extant thread? I don't really understand what the different IPython > threads do...but I could find out...) with a SUB socket to collect them all > together and forward them all out on iopub_socket. > Definitely not going to happen. One of the issues we are running into, especially in multiuser deployments of the notebook is that file descriptors get consumed. If anything we are wanting to *reduce* the number of sockets used by ipython. > 3) "Dirty" -- Wrap IPython.kernel.zmq.session.Session.send inside a > mutex. Are there use-cases where anybody besides Session actually does > anything with iopub_socket? > > There shouldn't be any other way that send gets called, so this is probably my top choice. > === Motivation === > > I think there could be valid use-cases for widgets which display results > from polling external resources, e.g. widgets which keep track of EC2 > instances, widgets which monitor IPython.parallel workerbees, etc. > > I have already started to use widgets with threads, but in my case I didn't quite run into issue you are seeing - almost though. This is definitely going to show up more. > === Conclusion === > > I'd be happy to attack this problem, if you guys think it would be > useful. If so, let me know what would be a good general direction, and > I'll put together a pull request. > @minrk is our expert on the zeromq stuff. Can you open an issue on the ipython/ipython repo and mention @minrk and @ellisonbg and @jdfreder Thanks! Brian > > Keep warm! > > Jackson Loper > Pattern Theory Group > Division of Applied Math > Brown University > > PS. I love the new text editor feature in the ipython master branch! My > days of vim-over-ssh aren't over yet (NERDTree and push-notification of > filechange detection are still too useful to me when logging on from > multiple machines) but I can see the end in sight! > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150109/23b886e3/attachment.html> From fperez.net at gmail.com Fri Jan 9 18:57:17 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 9 Jan 2015 15:57:17 -0800 Subject: [IPython-dev] threadsafe widgets In-Reply-To: <CAH4pYpSxUOxKrzwWw1ptMAFKL_gapFN4XGVHFECoosBsuZTrGA@mail.gmail.com> References: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com> <CAP8T2VKxJ0S7bMMJ8au+343FCYF7s__nVZOph5BZows2y86u5g@mail.gmail.com> <CAH4pYpSxUOxKrzwWw1ptMAFKL_gapFN4XGVHFECoosBsuZTrGA@mail.gmail.com> Message-ID: <CAHAreOqFtmaT=sj0obNPBQWVz3wZzcbyjmsUPTnjH+T=0VsJXg@mail.gmail.com> On Fri, Jan 9, 2015 at 3:34 PM, Brian Granger <ellisonbg at gmail.com> wrote: > @minrk is our expert on the zeromq stuff. Can you open an issue on the > ipython/ipython repo and mention @minrk and @ellisonbg and @jdfreder Add me @fperez as well please, I need to keep an eye on where this goes too... -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150109/423951a6/attachment.html> From benjaminrk at gmail.com Fri Jan 9 19:25:15 2015 From: benjaminrk at gmail.com (MinRK) Date: Fri, 9 Jan 2015 16:25:15 -0800 Subject: [IPython-dev] threadsafe widgets In-Reply-To: <CAHAreOqFtmaT=sj0obNPBQWVz3wZzcbyjmsUPTnjH+T=0VsJXg@mail.gmail.com> References: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com> <CAP8T2VKxJ0S7bMMJ8au+343FCYF7s__nVZOph5BZows2y86u5g@mail.gmail.com> <CAH4pYpSxUOxKrzwWw1ptMAFKL_gapFN4XGVHFECoosBsuZTrGA@mail.gmail.com> <CAHAreOqFtmaT=sj0obNPBQWVz3wZzcbyjmsUPTnjH+T=0VsJXg@mail.gmail.com> Message-ID: <CAHNn8BXJNFEsL40EZtF8GvNxGfQ2GOPB4sH2HqiwNeLxXfUVVg@mail.gmail.com> Since there?s already a single-threaded eventloop running (tornado), the easiest way to ensure that actions in threads are properly serialized is to hand them off via IOLoop.add_callback. This has the advantage over the mutexes-everywhere approach that there isn?t anything to do in the vanilla single-threaded case. Where to put this serialization is a bit of an open question. The simplest place to do it is on Session itself, since it would be always correct, but Session isn?t always used in the context of an IOLoop, so to do it at that level would require the most general, and thus most costly, approach (mutex on every send/recv). We could add a send method on Kernel that does any threading checks outside of Session, assuming that tornado is involved, which is always True when there?s a Kernel, unlike for Session. A mockup of threadsafe send in a tornado context: def threadsafe_send(*args, **kwargs): if not in_main_thread: # can't touch zmq sockets except from main thread # call me again ASAP from the main thread io_loop.add_callback(threadsafe_send, *args, **kwargs) return actually_send() The advantage of this over a mutex is that there?s no lock to grab in the 99% case where it is actually called from the main thread. The issue of mixed-up messages is actually a slightly separate from libzmq sockets themselves not being threadsafe. This is additional thread-unsafety provided by pyzmq?s send/recv_multipart, which was allowed because the underlying C socket is also not threadsafe. Making pyzmq itself threadsafe was considered (implemented, even), but it?s easy enough to do this at the application level that pyzmq doesn?t provided it for performance reasons. -MinRK ? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150109/ea36ec3a/attachment.html> From dsdale24 at gmail.com Sat Jan 10 09:03:45 2015 From: dsdale24 at gmail.com (Darren Dale) Date: Sat, 10 Jan 2015 14:03:45 +0000 Subject: [IPython-dev] embedding ipython, namespace question References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com> <AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov> <CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com> <FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com> <CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com> <DD2429D4-4BE1-49FC-A891-BF153FBA942F@anl.gov> <CAK6O52ns=WiaMSGRista9wE_VHdgFap1Wt65h1yDwEECawAULA@mail.gmail.com> <CAK6O52kd_uAKQ4U8Hs5FS03b9VJ4iFv-8NRPM23pjgYOPtoZvg@mail.gmail.com> Message-ID: <CAK6O52=JCJFbGcMjTm0bNEPwE_U995KUi04G-QkvRct+G0OF=A@mail.gmail.com> I found the start of a solution by digging into the IPython library, especially https://github.com/ipython/ipython/blob/master/IPython/kernel/zmq/ipkernel.py#L33 . After creating the in-process kernel, it is possible to reinitialize the console namespace with, for example, `kernel.user_ns = globals()`. Now its just a matter of wiring up the gui so the user_ns gets updated whenever the console widget receives focus. On Fri Jan 09 2015 at 11:38:33 AM Darren Dale <dsdale24 at gmail.com> wrote: > On Sat Jan 03 2015 at 10:45:42 AM Darren Dale <dsdale24 at gmail.com> wrote: > >> Hi Ray, >> >> On Sat Dec 27 2014 at 8:48:20 PM Osborn, Raymond <rosborn at anl.gov> wrote: >> >>> I don?t really understand what you are trying to achieve, but the >>> ?user_ns? dictionary isn?t an isolated namespace - it?s the namespace that >>> is used by the console, and I would have thought you would have to do some >>> kind of injection to add other objects from within the application. >>> >> >> What I am trying to achieve is explicitly documented at >> http://ipython.org/ipython-doc/dev/interactive/reference. >> html#embedding-ipython : >> >> --- >> It is also possible to embed an IPython shell in a namespace in your >> Python code. This allows you to evaluate dynamically the state of your >> code, operate with your variables, analyze them, etc. Note however that any >> changes you make to values while in the shell do not propagate back to the >> running code, so it is safe to modify your values because you won?t break >> your code in bizarre ways by doing so. >> >> Note >> At present, embedding IPython cannot be done from inside IPython. Run the >> code samples below outside IPython. >> [DD: I am not attempting to embed ipython from inside ipython] >> >> This feature allows you to easily have a fully functional python >> environment for doing object introspection anywhere in your code with a >> simple function call. In some cases a simple print statement is enough, but >> if you need to do more detailed analysis of a code fragment this feature >> can be very valuable. >> >> It can also be useful in scientific computing situations where it is >> common to need to do some automatic, computationally intensive part and >> then stop to look at data, plots, etc. Opening an IPython instance will >> give you full access to your data and functions, and you can resume program >> execution once you are done with the interactive part (perhaps to stop >> again later, as many times as needed). >> The following code snippet is the bare minimum you need to include in >> your Python programs for this to work (detailed examples follow later): >> >> from IPython import embed >> embed() # this call anywhere in your program will start IPython >> >> You can also embed an IPython kernel, for use with qtconsole, etc. via >> IPython.embed_kernel(). This should function work the same way, but you can >> connect an external frontend (ipython qtconsole or ipython console), rather >> than interacting with it in the terminal. >> --- >> >> This is impressively simple for the embed function: >> >> --- >> C:\Users\darren> python >> Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul 2 2014, 15:12:11) >> [MSC v.1500 64 bit (AMD64)] on win32 >> Type "help", "copyright", "credits" or "license" for more information. >> Anaconda is brought to you by Continuum Analytics. >> Please check out: http://continuum.io/thanks and https://binstar.org >> >>> from IPython import embed >> >>> a=1 >> >>> embed() >> Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul 2 2014, 15:12:11) >> [MSC v.1500 64 bit (AMD64)] >> Type "copyright", "credits" or "license" for more information. >> >> IPython 2.3.1 -- An enhanced Interactive Python. >> Anaconda is brought to you by Continuum Analytics. >> Please check out: http://continuum.io/thanks and https://binstar.org >> ? -> Introduction and overview of IPython's features. >> %quickref -> Quick reference. >> help -> Python's own help system. >> object? -> Details about 'object', use 'object??' for extra details. >> >> In [1]: a >> Out[1]: 1 >> --- >> >> But I have not been able to achieve the same behavior with the qt >> in-process console. >> > > If this is not possible, perhaps the documentation should be changed? > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150110/48f3a163/attachment.html> From dsdale24 at gmail.com Sat Jan 10 10:30:30 2015 From: dsdale24 at gmail.com (Darren Dale) Date: Sat, 10 Jan 2015 15:30:30 +0000 Subject: [IPython-dev] embedding ipython, namespace question References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com> <AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov> <CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com> <FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com> <CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com> <DD2429D4-4BE1-49FC-A891-BF153FBA942F@anl.gov> <CAK6O52ns=WiaMSGRista9wE_VHdgFap1Wt65h1yDwEECawAULA@mail.gmail.com> <CAK6O52kd_uAKQ4U8Hs5FS03b9VJ4iFv-8NRPM23pjgYOPtoZvg@mail.gmail.com> <CAK6O52=JCJFbGcMjTm0bNEPwE_U995KUi04G-QkvRct+G0OF=A@mail.gmail.com> Message-ID: <CAK6O52ma0DKMSUmgVf5EdnPv6xsGX5gMj1BJTScr3W7767uYhQ@mail.gmail.com> In case anyone else may find it useful, attached is a small modification of the qt in-process example that implements the behavior I was looking for. The namespace gets updated each time the mouse enters the widget, so every time you use the console, the environment appears to be the same as that of the GUI. On Sat Jan 10 2015 at 9:03:48 AM Darren Dale <dsdale24 at gmail.com> wrote: > I found the start of a solution by digging into the IPython library, > especially https://github.com/ipython/ipython/blob/master/IPython/ > kernel/zmq/ipkernel.py#L33 . After creating the in-process kernel, it is > possible to reinitialize the console namespace with, for example, > `kernel.user_ns = globals()`. Now its just a matter of wiring up the gui so > the user_ns gets updated whenever the console widget receives focus. > On Fri Jan 09 2015 at 11:38:33 AM Darren Dale <dsdale24 at gmail.com> wrote: > >> On Sat Jan 03 2015 at 10:45:42 AM Darren Dale <dsdale24 at gmail.com> wrote: >> >>> Hi Ray, >>> >>> On Sat Dec 27 2014 at 8:48:20 PM Osborn, Raymond <rosborn at anl.gov> >>> wrote: >>> >>>> I don?t really understand what you are trying to achieve, but the >>>> ?user_ns? dictionary isn?t an isolated namespace - it?s the namespace that >>>> is used by the console, and I would have thought you would have to do some >>>> kind of injection to add other objects from within the application. >>>> >>> >>> What I am trying to achieve is explicitly documented at >>> http://ipython.org/ipython-doc/dev/interactive/reference.htm >>> l#embedding-ipython : >>> >>> --- >>> It is also possible to embed an IPython shell in a namespace in your >>> Python code. This allows you to evaluate dynamically the state of your >>> code, operate with your variables, analyze them, etc. Note however that any >>> changes you make to values while in the shell do not propagate back to the >>> running code, so it is safe to modify your values because you won?t break >>> your code in bizarre ways by doing so. >>> >>> Note >>> At present, embedding IPython cannot be done from inside IPython. Run >>> the code samples below outside IPython. >>> [DD: I am not attempting to embed ipython from inside ipython] >>> >>> This feature allows you to easily have a fully functional python >>> environment for doing object introspection anywhere in your code with a >>> simple function call. In some cases a simple print statement is enough, but >>> if you need to do more detailed analysis of a code fragment this feature >>> can be very valuable. >>> >>> It can also be useful in scientific computing situations where it is >>> common to need to do some automatic, computationally intensive part and >>> then stop to look at data, plots, etc. Opening an IPython instance will >>> give you full access to your data and functions, and you can resume program >>> execution once you are done with the interactive part (perhaps to stop >>> again later, as many times as needed). >>> The following code snippet is the bare minimum you need to include in >>> your Python programs for this to work (detailed examples follow later): >>> >>> from IPython import embed >>> embed() # this call anywhere in your program will start IPython >>> >>> You can also embed an IPython kernel, for use with qtconsole, etc. via >>> IPython.embed_kernel(). This should function work the same way, but you can >>> connect an external frontend (ipython qtconsole or ipython console), rather >>> than interacting with it in the terminal. >>> --- >>> >>> This is impressively simple for the embed function: >>> >>> --- >>> C:\Users\darren> python >>> Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul 2 2014, >>> 15:12:11) [MSC v.1500 64 bit (AMD64)] on win32 >>> Type "help", "copyright", "credits" or "license" for more information. >>> Anaconda is brought to you by Continuum Analytics. >>> Please check out: http://continuum.io/thanks and https://binstar.org >>> >>> from IPython import embed >>> >>> a=1 >>> >>> embed() >>> Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul 2 2014, >>> 15:12:11) [MSC v.1500 64 bit (AMD64)] >>> Type "copyright", "credits" or "license" for more information. >>> >>> IPython 2.3.1 -- An enhanced Interactive Python. >>> Anaconda is brought to you by Continuum Analytics. >>> Please check out: http://continuum.io/thanks and https://binstar.org >>> ? -> Introduction and overview of IPython's features. >>> %quickref -> Quick reference. >>> help -> Python's own help system. >>> object? -> Details about 'object', use 'object??' for extra details. >>> >>> In [1]: a >>> Out[1]: 1 >>> --- >>> >>> But I have not been able to achieve the same behavior with the qt >>> in-process console. >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150110/699cf809/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: inprocess_qtconsole.py Type: application/octet-stream Size: 1457 bytes Desc: not available URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150110/699cf809/attachment.obj> From dsdale24 at gmail.com Sun Jan 11 10:37:22 2015 From: dsdale24 at gmail.com (Darren Dale) Date: Sun, 11 Jan 2015 15:37:22 +0000 Subject: [IPython-dev] Incompatible subclass instances of InteractiveShellEmbed Message-ID: <CAK6O52=wGAMhamdKToM7WLoxDDjS5hEHV6mL_m_H-KF2uVyu1w@mail.gmail.com> Hello, I'm working on an application with an in-process qt console, following the example at https://github.com/ipython/ipython/blob/master/examples/Embedding/inprocess_qtconsole.py . I would also like to be able to use the `embed` function occassionally for troubleshooting other parts of the application that are unrelated to the in-process qt console. However, calling embed yields an error: "IPython.config.configurable.MultipleInstanceError: Multiple incompatible subclass instances of InteractiveShellEmbed are being created." I can reproduce this with the inprocess example by adding a call to `embed()` just before the call to `guisupport.start_event_loop_qt4(app)`. Any ideas how to make this work? Thanks, Darren -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150111/ca47c719/attachment.html> From rosborn at anl.gov Sun Jan 11 17:49:08 2015 From: rosborn at anl.gov (Osborn, Raymond) Date: Sun, 11 Jan 2015 22:49:08 +0000 Subject: [IPython-dev] Incompatible subclass instances of InteractiveShellEmbed In-Reply-To: <CAK6O52=wGAMhamdKToM7WLoxDDjS5hEHV6mL_m_H-KF2uVyu1w@mail.gmail.com> References: <CAK6O52=wGAMhamdKToM7WLoxDDjS5hEHV6mL_m_H-KF2uVyu1w@mail.gmail.com> Message-ID: <7191368A-9B31-4D73-BB7B-D10ACE44F31D@anl.gov> Until someone more knowledgeable replies, this is presumably because InteractiveShellEmbed ultimately inherits from SingletonConfigurable, which only allows one instance to be created. There is some code in embed.py, which saves an existing instance, clears it, and allows another to be created: saved_shell_instance = InteractiveShell._instance if saved_shell_instance is not None: cls = type(saved_shell_instance) cls.clear_instance() shell = InteractiveShellEmbed.instance(**kwargs) If something like that is not in your current code, perhaps you need to add it. Ray On Jan 11, 2015, at 9:37 AM, Darren Dale <dsdale24 at gmail.com<mailto:dsdale24 at gmail.com>> wrote: Hello, I'm working on an application with an in-process qt console, following the example at https://github.com/ipython/ipython/blob/master/examples/Embedding/inprocess_qtconsole.py . I would also like to be able to use the `embed` function occassionally for troubleshooting other parts of the application that are unrelated to the in-process qt console. However, calling embed yields an error: "IPython.config.configurable.MultipleInstanceError: Multiple incompatible subclass instances of InteractiveShellEmbed are being created." I can reproduce this with the inprocess example by adding a call to `embed()` just before the call to `guisupport.start_event_loop_qt4(app)`. Any ideas how to make this work? Thanks, Darren _______________________________________________ IPython-dev mailing list IPython-dev at scipy.org<mailto:IPython-dev at scipy.org> http://mail.scipy.org/mailman/listinfo/ipython-dev -- Ray Osborn, Senior Scientist Materials Science Division Argonne National Laboratory Argonne, IL 60439, USA Phone: +1 (630) 252-9011 Email: ROsborn at anl.gov<mailto:ROsborn at anl.gov> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150111/803c3fe8/attachment.html> From fperez.net at gmail.com Sun Jan 11 22:47:22 2015 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 11 Jan 2015 19:47:22 -0800 Subject: [IPython-dev] IPython 3.0, the homestretch. We need your help! Message-ID: <CAHAreOppTKvawbCj+4k+qd7-Sg3cLKTLpUFpcGbnbMn7NzuaVA@mail.gmail.com> Hi all, we're behind schedule, but there is light at the end of the 3.0 tunnel. Now is the time where we need the help from our community. This is the list of currently open issues tagged for 3.0: https://github.com/ipython/ipython/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0 If you see anything there for which you can contribute in any way a fix, patch, or even constructive feedback, we'd be very grateful. Eventually some of those will inevitably have to be retargetted for 3.1/4.0, since we won't be able to get to all of them. But there's a reason we'd labeled them 3.0, so it would be great to actually close as many of them as possible (or at least make partial progress on them, enough to address the key problem and perhaps leave an easier but potentially longer tail for 3.1/4). So please jump in! f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150111/54c7a75c/attachment.html> From dsdale24 at gmail.com Mon Jan 12 12:28:34 2015 From: dsdale24 at gmail.com (Darren Dale) Date: Mon, 12 Jan 2015 17:28:34 +0000 Subject: [IPython-dev] Incompatible subclass instances of InteractiveShellEmbed References: <CAK6O52=wGAMhamdKToM7WLoxDDjS5hEHV6mL_m_H-KF2uVyu1w@mail.gmail.com> <7191368A-9B31-4D73-BB7B-D10ACE44F31D@anl.gov> Message-ID: <CAK6O52=uTzgpGgRLDJJxkHrp_eihB9pA934Cz3_zkhwuBd_Pzw@mail.gmail.com> The following seems to work for me: in my main hexrd/__init__.py, I do `from IPython import embed as debug`. Then, in my gui module, I monkeypatch hexrd.debug so it uses the inprocess qtconsole: class Debug(object): def __init__(self, shell): self.shell = shell def __call__(self): import inspect frame = inspect.currentframe() try: # prepare the debug namespace temp = frame.f_back.f_globals temp.update(frame.f_back.f_locals) finally: del frame # clear and repopulate the qtconsole namespace self.shell.run_line_magic('reset', '-f -s') self.shell.push(temp) hexrd.debug = Debug(kernel_manager.kernel.shell) Darren On Sun Jan 11 2015 at 5:49:16 PM Osborn, Raymond <rosborn at anl.gov> wrote: > Until someone more knowledgeable replies, this is presumably because > InteractiveShellEmbed ultimately inherits from SingletonConfigurable, which > only allows one instance to be created. There is some code in embed.py, > which saves an existing instance, clears it, and allows another to be > created: > > saved_shell_instance = InteractiveShell._instance > if saved_shell_instance is not None: > cls = type(saved_shell_instance) > cls.clear_instance() > shell = InteractiveShellEmbed.instance(**kwargs) > > If something like that is not in your current code, perhaps you need to > add it. > > Ray > > On Jan 11, 2015, at 9:37 AM, Darren Dale <dsdale24 at gmail.com> wrote: > > Hello, > > I'm working on an application with an in-process qt console, following > the example at > https://github.com/ipython/ipython/blob/master/examples/Embedding/inprocess_qtconsole.py > . I would also like to be able to use the `embed` function occassionally > for troubleshooting other parts of the application that are unrelated to > the in-process qt console. However, calling embed yields an error: > "IPython.config.configurable.MultipleInstanceError: Multiple incompatible > subclass instances of InteractiveShellEmbed are being created." I can > reproduce this with the inprocess example by adding a call to `embed()` > just before the call to `guisupport.start_event_loop_qt4(app)`. Any ideas > how to make this work? > > Thanks, > Darren > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > -- > Ray Osborn, Senior Scientist > Materials Science Division > Argonne National Laboratory > Argonne, IL 60439, USA > Phone: +1 (630) 252-9011 > Email: ROsborn at anl.gov > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150112/7d46578d/attachment.html> From wstein at gmail.com Wed Jan 14 19:12:50 2015 From: wstein at gmail.com (William Stein) Date: Wed, 14 Jan 2015 16:12:50 -0800 Subject: [IPython-dev] Notebook websocket change In-Reply-To: <CAHNn8BUSrDb8kVO94-dv9FsNJtZu=u=dW2QMdLgEn5_B3v4CRQ@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> <CAHNn8BUSrDb8kVO94-dv9FsNJtZu=u=dW2QMdLgEn5_B3v4CRQ@mail.gmail.com> Message-ID: <CACLE5GA4G74y5V_WCurA6tYS76yWWgVwnMJ+eZTVENHqUNzODg@mail.gmail.com> On Thu, Jan 8, 2015 at 11:18 AM, MinRK <benjaminrk at gmail.com> wrote: > Some time ago, I made a PR to IPython switching to socket.io, which required > the same basic changes since it only allowed a single connection per page, > so I know it's not too hard. We may start to see more pressure as hosted > instances, and reconsider the fallback in the future, but at this point, we > don't feel that pressure. Wind proper websockets to be sufficiently > preferable that we will wait as long as we can before appeasing old, broken, > or misconfigured network environments that prevent websockets from working. > > The majority of notebook users seem to be running as a desktop app via > localhost, where websockets are rarely a problem (The nightmare of Sophos > AV/firewall on Windows excluded). Do you have a pointer to the above pull request? In my experience battling with websockets and the cloud over the last few years, it's not just an issue of websockets versus fallbacks -- there's also issues with the robustness of the websocket connection, how it's maintained, automatically reconnecting if the connection has problems, heartbeats, etc. If you only use IPython locally then none of this is a problem. However, if you use IPython over a complicated network that drops packets sometimes, flakes out, etc., it can be. I get fairly regular emails from people having trouble specifically with the robustness of the iPython connection inside of SMC. Libraries like socket.io and engine.io [1] seem to have pretty good debuged and thought out code for dealing with these situations (and for fallbacks in case websockets doesn't work). Since this use case is only a tiny part of the total IPython notebook usage, it makes sense for me to take this on myself. I will definitely highly prioritize trying to hack on IPython in order to get it to use engine.io+primus, since I have to deal with this websocket robustness situation to improve my user's experience. For example, I'm getting fairly regular complaints from Randy Leveque and his colleagues at UW. So I'd greatly appreciate a link to that pull request, and of course I'll share anything I do here. [1] https://github.com/Automattic/engine.io > > -MinRK > > On Thu, Jan 8, 2015 at 10:55 AM, William Stein <wstein at gmail.com> wrote: >> >> On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com> >> wrote: >> > So far we have felt that it isn't worth the complexity cost for us to >> > support people that deliberately choose to break the internet. It is not >> > like WebSockets is some crazy, insecure, unsupported hack over a weird >> > port. >> > It is just different bytes going over the same HTTP/HTTPS socket >> > connection >> > and is essentially a formal standard (I know you know this - this is our >> > messaging to companies who complain). >> >> I know -- that's why I switched SMC to pure websockets for a while. >> Then I received complaints at a rate of about 2 very desperate users >> per week, and these were the people who cared enough to complain >> repeatedly. >> >> > Our situation is a bit different though than SMC because most of our >> > current >> > users run the notebook on their own computers. This may be something >> > that we >> > eventually want to rethink as more people run this in the cloud. >> > However, I >> > would like to push back on the companies choosing to break the internet >> > really hard before giving in. I am guessing that in most cases, >> > WebSockets >> > are broken in enterprise setting not because of some important >> > deliberate >> > choice, but rather because people don't understand it and are using >> > default >> > settings that disable them (I could be wrong though).... >> >> In my experience, for some people that's exactly the problem. For one >> person I helped it turned out their specific problem was the Windows >> install they had access to had websockets explicitly disabled -- >> perhaps that was an IT policy. For other people the problem is >> further along the stack (e.g., possibly involving caching). >> >> Anyway, this change so that "there is a single websocket connection >> between the server and client. " makes implementing an alternative to >> WebSocket as a fallback dramatically easier, which I greatly >> appreciate. >> >> > Our situation is a bit different though than SMC because most of our >> > current >> > users run the notebook on their own computers. This may be something >> > that we >> >> Just curious -- how do you know that most people using IPython aren't >> already using it through SMC? >> >> William >> >> > >> > Cheers, >> > >> > Brian >> > >> > >> > >> > On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> wrote: >> >> >> >> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote: >> >> > Alternative notebook frontend authors: >> >> > >> >> > There is an upcoming change to websocket connections to the notebook >> >> > server. >> >> > Instead of three websockets per notebook, one for each zmq channel, >> >> > there is >> >> > a single websocket connection between the server and client. The >> >> > channel >> >> > associated with each message is identified by a channel key in the >> >> > message >> >> > dict itself. >> >> > >> >> > Relevant changes: >> >> > >> >> > url is /channels instead of /shell, etc. >> >> > when sending a message, add a channel key with the channel name >> >> > ('shell', >> >> > 'stdin') >> >> > when receiving a message, check the channel key for routing >> >> > >> >> > -MinRK >> >> >> >> Thanks. >> >> >> >> Of related interest, what the situation involving providing a fallback >> >> to websockets? >> >> >> >> I tried to only support websockets connections with SageMathCloud for >> >> a few weeks, and immediately ran into many complaints from users who >> >> couldn't connect as a result. This is even with https and so on -- >> >> it's just a sad fact that in some corporate or otherwise constrained >> >> environments that websockets don't work. By far the best approach I >> >> found was to use Primus + Engine.io, which uses websockets if >> >> possible, but will fallback to polling. So this is what SMC does >> >> now, and everything works, even for people that don't have websockets >> >> as an option... except for IPython-in-SMC of course, which doesn't >> >> work. >> >> >> >> Another unrelated issue is that I'm now hardcoding making this change >> >> around line 171 of IPython/html/notebookapp.py, since I want to serve >> >> the purely static html of IPython from a completely different web >> >> server: >> >> >> >> #static_url_prefix = url_path_join(base_url,'/static/'), >> >> static_url_prefix = '/static/ipython/', >> >> >> >> (Sorry for derailing the thread.) >> >> >> >> -- William >> >> >> >> > >> >> > >> >> > _______________________________________________ >> >> > IPython-dev mailing list >> >> > IPython-dev at scipy.org >> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > >> >> >> >> >> >> >> >> -- >> >> William Stein >> >> Professor of Mathematics >> >> University of Washington >> >> http://wstein.org >> >> _______________________________________________ >> >> IPython-dev mailing list >> >> IPython-dev at scipy.org >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> > >> > >> > >> > -- >> > Brian E. Granger >> > Cal Poly State University, San Luis Obispo >> > @ellisonbg on Twitter and GitHub >> > bgranger at calpoly.edu and ellisonbg at gmail.com >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> >> >> >> -- >> William Stein >> Professor of Mathematics >> University of Washington >> http://wstein.org >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- William Stein Professor of Mathematics University of Washington http://wstein.org From benjaminrk at gmail.com Wed Jan 14 19:50:46 2015 From: benjaminrk at gmail.com (MinRK) Date: Wed, 14 Jan 2015 16:50:46 -0800 Subject: [IPython-dev] Notebook websocket change In-Reply-To: <CACLE5GA4G74y5V_WCurA6tYS76yWWgVwnMJ+eZTVENHqUNzODg@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> <CAHNn8BUSrDb8kVO94-dv9FsNJtZu=u=dW2QMdLgEn5_B3v4CRQ@mail.gmail.com> <CACLE5GA4G74y5V_WCurA6tYS76yWWgVwnMJ+eZTVENHqUNzODg@mail.gmail.com> Message-ID: <CAHNn8BVvt+wFwt5u1OvQ-6D2fiGenscaAdFaQd729gp3WXVQRw@mail.gmail.com> The PR is 2321 <https://github.com/ipython/ipython/pull/2321>, switching IPython to use SockJS instead of WebSockets. I?m not sure how informative it is, since things have moved around *a lot* since then, and it is from a time when we were shipping more things in external, rather than accepting them as dependencies. I think a similar patch today would be a great deal simpler, especially now that it doesn?t need to include merging sockets. I also had to include a hack to workaround the fact that SockJS didn?t allow empty messages, and we were sending empty messages as part of the initial handshake (also not true anymore). -MinRK ? On Wed, Jan 14, 2015 at 4:12 PM, William Stein <wstein at gmail.com> wrote: > On Thu, Jan 8, 2015 at 11:18 AM, MinRK <benjaminrk at gmail.com> wrote: > > Some time ago, I made a PR to IPython switching to socket.io, which > required > > the same basic changes since it only allowed a single connection per > page, > > so I know it's not too hard. We may start to see more pressure as hosted > > instances, and reconsider the fallback in the future, but at this point, > we > > don't feel that pressure. Wind proper websockets to be sufficiently > > preferable that we will wait as long as we can before appeasing old, > broken, > > or misconfigured network environments that prevent websockets from > working. > > > > The majority of notebook users seem to be running as a desktop app via > > localhost, where websockets are rarely a problem (The nightmare of Sophos > > AV/firewall on Windows excluded). > > Do you have a pointer to the above pull request? > > In my experience battling with websockets and the cloud over the last > few years, it's not just an issue of websockets versus fallbacks -- > there's also issues with the robustness of the websocket connection, > how it's maintained, automatically reconnecting if the connection has > problems, heartbeats, etc. If you only use IPython locally then none > of this is a problem. However, if you use IPython over a complicated > network that drops packets sometimes, flakes out, etc., it can be. I > get fairly regular emails from people having trouble specifically with > the robustness of the iPython connection inside of SMC. Libraries > like socket.io and engine.io [1] seem to have pretty good debuged and > thought out code for dealing with these situations (and for fallbacks > in case websockets doesn't work). > > Since this use case is only a tiny part of the total IPython notebook > usage, it makes sense for me to take this on myself. I will > definitely highly prioritize trying to hack on IPython in order to get > it to use engine.io+primus, since I have to deal with this websocket > robustness situation to improve my user's experience. For example, > I'm getting fairly regular complaints from Randy Leveque and his > colleagues at UW. So I'd greatly appreciate a link to that pull > request, and of course I'll share anything I do here. > > [1] https://github.com/Automattic/engine.io > > > > > -MinRK > > > > On Thu, Jan 8, 2015 at 10:55 AM, William Stein <wstein at gmail.com> wrote: > >> > >> On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com> > >> wrote: > >> > So far we have felt that it isn't worth the complexity cost for us to > >> > support people that deliberately choose to break the internet. It is > not > >> > like WebSockets is some crazy, insecure, unsupported hack over a weird > >> > port. > >> > It is just different bytes going over the same HTTP/HTTPS socket > >> > connection > >> > and is essentially a formal standard (I know you know this - this is > our > >> > messaging to companies who complain). > >> > >> I know -- that's why I switched SMC to pure websockets for a while. > >> Then I received complaints at a rate of about 2 very desperate users > >> per week, and these were the people who cared enough to complain > >> repeatedly. > >> > >> > Our situation is a bit different though than SMC because most of our > >> > current > >> > users run the notebook on their own computers. This may be something > >> > that we > >> > eventually want to rethink as more people run this in the cloud. > >> > However, I > >> > would like to push back on the companies choosing to break the > internet > >> > really hard before giving in. I am guessing that in most cases, > >> > WebSockets > >> > are broken in enterprise setting not because of some important > >> > deliberate > >> > choice, but rather because people don't understand it and are using > >> > default > >> > settings that disable them (I could be wrong though).... > >> > >> In my experience, for some people that's exactly the problem. For one > >> person I helped it turned out their specific problem was the Windows > >> install they had access to had websockets explicitly disabled -- > >> perhaps that was an IT policy. For other people the problem is > >> further along the stack (e.g., possibly involving caching). > >> > >> Anyway, this change so that "there is a single websocket connection > >> between the server and client. " makes implementing an alternative to > >> WebSocket as a fallback dramatically easier, which I greatly > >> appreciate. > >> > >> > Our situation is a bit different though than SMC because most of our > >> > current > >> > users run the notebook on their own computers. This may be something > >> > that we > >> > >> Just curious -- how do you know that most people using IPython aren't > >> already using it through SMC? > >> > >> William > >> > >> > > >> > Cheers, > >> > > >> > Brian > >> > > >> > > >> > > >> > On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> > wrote: > >> >> > >> >> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote: > >> >> > Alternative notebook frontend authors: > >> >> > > >> >> > There is an upcoming change to websocket connections to the > notebook > >> >> > server. > >> >> > Instead of three websockets per notebook, one for each zmq channel, > >> >> > there is > >> >> > a single websocket connection between the server and client. The > >> >> > channel > >> >> > associated with each message is identified by a channel key in the > >> >> > message > >> >> > dict itself. > >> >> > > >> >> > Relevant changes: > >> >> > > >> >> > url is /channels instead of /shell, etc. > >> >> > when sending a message, add a channel key with the channel name > >> >> > ('shell', > >> >> > 'stdin') > >> >> > when receiving a message, check the channel key for routing > >> >> > > >> >> > -MinRK > >> >> > >> >> Thanks. > >> >> > >> >> Of related interest, what the situation involving providing a > fallback > >> >> to websockets? > >> >> > >> >> I tried to only support websockets connections with SageMathCloud for > >> >> a few weeks, and immediately ran into many complaints from users who > >> >> couldn't connect as a result. This is even with https and so on -- > >> >> it's just a sad fact that in some corporate or otherwise constrained > >> >> environments that websockets don't work. By far the best approach I > >> >> found was to use Primus + Engine.io, which uses websockets if > >> >> possible, but will fallback to polling. So this is what SMC does > >> >> now, and everything works, even for people that don't have websockets > >> >> as an option... except for IPython-in-SMC of course, which doesn't > >> >> work. > >> >> > >> >> Another unrelated issue is that I'm now hardcoding making this change > >> >> around line 171 of IPython/html/notebookapp.py, since I want to serve > >> >> the purely static html of IPython from a completely different web > >> >> server: > >> >> > >> >> #static_url_prefix = url_path_join(base_url,'/static/'), > >> >> static_url_prefix = '/static/ipython/', > >> >> > >> >> (Sorry for derailing the thread.) > >> >> > >> >> -- William > >> >> > >> >> > > >> >> > > >> >> > _______________________________________________ > >> >> > IPython-dev mailing list > >> >> > IPython-dev at scipy.org > >> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev > >> >> > > >> >> > >> >> > >> >> > >> >> -- > >> >> William Stein > >> >> Professor of Mathematics > >> >> University of Washington > >> >> http://wstein.org > >> >> _______________________________________________ > >> >> IPython-dev mailing list > >> >> IPython-dev at scipy.org > >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > >> > > >> > > >> > > >> > -- > >> > Brian E. Granger > >> > Cal Poly State University, San Luis Obispo > >> > @ellisonbg on Twitter and GitHub > >> > bgranger at calpoly.edu and ellisonbg at gmail.com > >> > > >> > _______________________________________________ > >> > IPython-dev mailing list > >> > IPython-dev at scipy.org > >> > http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > >> > >> > >> > >> -- > >> William Stein > >> Professor of Mathematics > >> University of Washington > >> http://wstein.org > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > William Stein > Professor of Mathematics > University of Washington > http://wstein.org > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150114/4a3f1ac5/attachment.html> From wstein at gmail.com Wed Jan 14 19:55:21 2015 From: wstein at gmail.com (William Stein) Date: Wed, 14 Jan 2015 16:55:21 -0800 Subject: [IPython-dev] Notebook websocket change In-Reply-To: <CAHNn8BVvt+wFwt5u1OvQ-6D2fiGenscaAdFaQd729gp3WXVQRw@mail.gmail.com> References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com> <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com> <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com> <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com> <CAHNn8BUSrDb8kVO94-dv9FsNJtZu=u=dW2QMdLgEn5_B3v4CRQ@mail.gmail.com> <CACLE5GA4G74y5V_WCurA6tYS76yWWgVwnMJ+eZTVENHqUNzODg@mail.gmail.com> <CAHNn8BVvt+wFwt5u1OvQ-6D2fiGenscaAdFaQd729gp3WXVQRw@mail.gmail.com> Message-ID: <CACLE5GAaNC_n9q-Nr+y+CTyz9T8u71iSZNe4LTT-bsjWV=zyHw@mail.gmail.com> On Wed, Jan 14, 2015 at 4:50 PM, MinRK <benjaminrk at gmail.com> wrote: > The PR is 2321, switching IPython to use SockJS instead of WebSockets. I'm > not sure how informative it is, since things have moved around a lot since > then, and it is from a time when we were shipping more things in external, > rather than accepting them as dependencies. I think a similar patch today > would be a great deal simpler, especially now that it doesn't need to > include merging sockets. I also had to include a hack to workaround the fact > that SockJS didn't allow empty messages, and we were sending empty messages > as part of the initial handshake (also not true anymore). > > -MinRK Thanks! I'll carefully think through a range of options. Another might be to just plug into your code to send/receive messages, and just send/receive them over my existing websocket connection, so there is just a single websocket connection, shared across ipython and smc. -- William > > > On Wed, Jan 14, 2015 at 4:12 PM, William Stein <wstein at gmail.com> wrote: >> >> On Thu, Jan 8, 2015 at 11:18 AM, MinRK <benjaminrk at gmail.com> wrote: >> > Some time ago, I made a PR to IPython switching to socket.io, which >> > required >> > the same basic changes since it only allowed a single connection per >> > page, >> > so I know it's not too hard. We may start to see more pressure as hosted >> > instances, and reconsider the fallback in the future, but at this point, >> > we >> > don't feel that pressure. Wind proper websockets to be sufficiently >> > preferable that we will wait as long as we can before appeasing old, >> > broken, >> > or misconfigured network environments that prevent websockets from >> > working. >> > >> > The majority of notebook users seem to be running as a desktop app via >> > localhost, where websockets are rarely a problem (The nightmare of >> > Sophos >> > AV/firewall on Windows excluded). >> >> Do you have a pointer to the above pull request? >> >> In my experience battling with websockets and the cloud over the last >> few years, it's not just an issue of websockets versus fallbacks -- >> there's also issues with the robustness of the websocket connection, >> how it's maintained, automatically reconnecting if the connection has >> problems, heartbeats, etc. If you only use IPython locally then none >> of this is a problem. However, if you use IPython over a complicated >> network that drops packets sometimes, flakes out, etc., it can be. I >> get fairly regular emails from people having trouble specifically with >> the robustness of the iPython connection inside of SMC. Libraries >> like socket.io and engine.io [1] seem to have pretty good debuged and >> thought out code for dealing with these situations (and for fallbacks >> in case websockets doesn't work). >> >> Since this use case is only a tiny part of the total IPython notebook >> usage, it makes sense for me to take this on myself. I will >> definitely highly prioritize trying to hack on IPython in order to get >> it to use engine.io+primus, since I have to deal with this websocket >> robustness situation to improve my user's experience. For example, >> I'm getting fairly regular complaints from Randy Leveque and his >> colleagues at UW. So I'd greatly appreciate a link to that pull >> request, and of course I'll share anything I do here. >> >> [1] https://github.com/Automattic/engine.io >> >> > >> > -MinRK >> > >> > On Thu, Jan 8, 2015 at 10:55 AM, William Stein <wstein at gmail.com> wrote: >> >> >> >> On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com> >> >> wrote: >> >> > So far we have felt that it isn't worth the complexity cost for us to >> >> > support people that deliberately choose to break the internet. It is >> >> > not >> >> > like WebSockets is some crazy, insecure, unsupported hack over a >> >> > weird >> >> > port. >> >> > It is just different bytes going over the same HTTP/HTTPS socket >> >> > connection >> >> > and is essentially a formal standard (I know you know this - this is >> >> > our >> >> > messaging to companies who complain). >> >> >> >> I know -- that's why I switched SMC to pure websockets for a while. >> >> Then I received complaints at a rate of about 2 very desperate users >> >> per week, and these were the people who cared enough to complain >> >> repeatedly. >> >> >> >> > Our situation is a bit different though than SMC because most of our >> >> > current >> >> > users run the notebook on their own computers. This may be something >> >> > that we >> >> > eventually want to rethink as more people run this in the cloud. >> >> > However, I >> >> > would like to push back on the companies choosing to break the >> >> > internet >> >> > really hard before giving in. I am guessing that in most cases, >> >> > WebSockets >> >> > are broken in enterprise setting not because of some important >> >> > deliberate >> >> > choice, but rather because people don't understand it and are using >> >> > default >> >> > settings that disable them (I could be wrong though).... >> >> >> >> In my experience, for some people that's exactly the problem. For one >> >> person I helped it turned out their specific problem was the Windows >> >> install they had access to had websockets explicitly disabled -- >> >> perhaps that was an IT policy. For other people the problem is >> >> further along the stack (e.g., possibly involving caching). >> >> >> >> Anyway, this change so that "there is a single websocket connection >> >> between the server and client. " makes implementing an alternative to >> >> WebSocket as a fallback dramatically easier, which I greatly >> >> appreciate. >> >> >> >> > Our situation is a bit different though than SMC because most of our >> >> > current >> >> > users run the notebook on their own computers. This may be something >> >> > that we >> >> >> >> Just curious -- how do you know that most people using IPython aren't >> >> already using it through SMC? >> >> >> >> William >> >> >> >> > >> >> > Cheers, >> >> > >> >> > Brian >> >> > >> >> > >> >> > >> >> > On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> >> >> > wrote: >> >> >> >> >> >> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote: >> >> >> > Alternative notebook frontend authors: >> >> >> > >> >> >> > There is an upcoming change to websocket connections to the >> >> >> > notebook >> >> >> > server. >> >> >> > Instead of three websockets per notebook, one for each zmq >> >> >> > channel, >> >> >> > there is >> >> >> > a single websocket connection between the server and client. The >> >> >> > channel >> >> >> > associated with each message is identified by a channel key in the >> >> >> > message >> >> >> > dict itself. >> >> >> > >> >> >> > Relevant changes: >> >> >> > >> >> >> > url is /channels instead of /shell, etc. >> >> >> > when sending a message, add a channel key with the channel name >> >> >> > ('shell', >> >> >> > 'stdin') >> >> >> > when receiving a message, check the channel key for routing >> >> >> > >> >> >> > -MinRK >> >> >> >> >> >> Thanks. >> >> >> >> >> >> Of related interest, what the situation involving providing a >> >> >> fallback >> >> >> to websockets? >> >> >> >> >> >> I tried to only support websockets connections with SageMathCloud >> >> >> for >> >> >> a few weeks, and immediately ran into many complaints from users who >> >> >> couldn't connect as a result. This is even with https and so on -- >> >> >> it's just a sad fact that in some corporate or otherwise constrained >> >> >> environments that websockets don't work. By far the best approach >> >> >> I >> >> >> found was to use Primus + Engine.io, which uses websockets if >> >> >> possible, but will fallback to polling. So this is what SMC does >> >> >> now, and everything works, even for people that don't have >> >> >> websockets >> >> >> as an option... except for IPython-in-SMC of course, which doesn't >> >> >> work. >> >> >> >> >> >> Another unrelated issue is that I'm now hardcoding making this >> >> >> change >> >> >> around line 171 of IPython/html/notebookapp.py, since I want to >> >> >> serve >> >> >> the purely static html of IPython from a completely different web >> >> >> server: >> >> >> >> >> >> #static_url_prefix = url_path_join(base_url,'/static/'), >> >> >> static_url_prefix = '/static/ipython/', >> >> >> >> >> >> (Sorry for derailing the thread.) >> >> >> >> >> >> -- William >> >> >> >> >> >> > >> >> >> > >> >> >> > _______________________________________________ >> >> >> > IPython-dev mailing list >> >> >> > IPython-dev at scipy.org >> >> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> William Stein >> >> >> Professor of Mathematics >> >> >> University of Washington >> >> >> http://wstein.org >> >> >> _______________________________________________ >> >> >> IPython-dev mailing list >> >> >> IPython-dev at scipy.org >> >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > Brian E. Granger >> >> > Cal Poly State University, San Luis Obispo >> >> > @ellisonbg on Twitter and GitHub >> >> > bgranger at calpoly.edu and ellisonbg at gmail.com >> >> > >> >> > _______________________________________________ >> >> > IPython-dev mailing list >> >> > IPython-dev at scipy.org >> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > >> >> >> >> >> >> >> >> -- >> >> William Stein >> >> Professor of Mathematics >> >> University of Washington >> >> http://wstein.org >> >> _______________________________________________ >> >> IPython-dev mailing list >> >> IPython-dev at scipy.org >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> > >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> >> >> >> -- >> William Stein >> Professor of Mathematics >> University of Washington >> http://wstein.org >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- William Stein Professor of Mathematics University of Washington http://wstein.org From liam.gretton at leicester.ac.uk Thu Jan 15 04:28:03 2015 From: liam.gretton at leicester.ac.uk (Liam Gretton) Date: Thu, 15 Jan 2015 09:28:03 +0000 Subject: [IPython-dev] %load hangs in qtconsole Message-ID: <54B78823.1050506@leicester.ac.uk> I've just built Python 2.7.9, iPython 2.3.1, PyQT 4.11.3 (against Qt 4.8.6) with matplotlib 1.4.2 etc on a Scientific Linux 6.4 system (64 bit). What I'm finding is that any attempt to %load a Python script from within the qtconsole hangs after displaying the final line of the script. The script neither executes nor returns the command prompt. >From within the command line ipython console there's no such problem. Manually typing or pasting in contents to the qtconsole works and the script executes as expected, it just hangs if %load is used. Any ideas what might be causing this? -- Liam Gretton liam.gretton at le.ac.uk Systems Specialist http://www.le.ac.uk/its/ IT Services Tel: +44 (0)116 2522254 University Of Leicester, University Road Leicestershire LE1 7RH, United Kingdom From benjaminrk at gmail.com Thu Jan 15 13:13:45 2015 From: benjaminrk at gmail.com (MinRK) Date: Thu, 15 Jan 2015 10:13:45 -0800 Subject: [IPython-dev] %load hangs in qtconsole In-Reply-To: <54B78823.1050506@leicester.ac.uk> References: <54B78823.1050506@leicester.ac.uk> Message-ID: <CAHNn8BWYMdROY65PSR94XFAM5OmOYgBHqzxGRJktpNhO2MdQmw@mail.gmail.com> It looks like it's putting the cursor in the wrong place, confusing the UI. If you place the cursor in the input area, it should allow you to execute the input. -MinRK On Thu, Jan 15, 2015 at 1:28 AM, Liam Gretton <liam.gretton at leicester.ac.uk> wrote: > I've just built Python 2.7.9, iPython 2.3.1, PyQT 4.11.3 (against Qt > 4.8.6) with matplotlib 1.4.2 etc on a Scientific Linux 6.4 system (64 bit). > > What I'm finding is that any attempt to %load a Python script from > within the qtconsole hangs after displaying the final line of the > script. The script neither executes nor returns the command prompt. > > >From within the command line ipython console there's no such problem. > Manually typing or pasting in contents to the qtconsole works and the > script executes as expected, it just hangs if %load is used. > > Any ideas what might be causing this? > > -- > Liam Gretton liam.gretton at le.ac.uk > Systems Specialist http://www.le.ac.uk/its/ > IT Services Tel: +44 (0)116 2522254 > University Of Leicester, University Road > Leicestershire LE1 7RH, United Kingdom > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150115/35908ca9/attachment.html> From soumith at fb.com Thu Jan 15 22:26:58 2015 From: soumith at fb.com (Soumith Chintala) Date: Fri, 16 Jan 2015 03:26:58 +0000 Subject: [IPython-dev] iTorch - Weird issues with mkfifo on linux Message-ID: <D0DDC4FF.3884%soumith@fb.com> Hey everyone, As some of you might be aware, we released an ipython kernel for Lua (works only with LuaJIT because we use FFI for non-blocking reads). We also integrated visualization of images/video/audio and plotting using Bokeh. https://github.com/facebook/iTorch We use a two-process kernel, one of them handles stdout from the other process and sends it over to ipython. To pipe the processes, we use mkfifo, which is a posix utility. http://pubs.opengroup.org/onlinepubs/009695399/functions/mkfifo.html One problem we oncovered is that on certain (or possibly all linux distros) ipython thinks our kernel died, even though it is alive and well. This only happen if we use mkfifo to redirect the output stream from one process to another. If we use a standard file like with this commit: https://github.com/soumith/iTorch/commit/96e8a26a91a90d47583761a55b308364d391585c we do not see any issues. I am trying to understand Ipython semantics for thinking that a kernel died, as the heartbeat is still ticking. I initially thought a SIGPIPE might be triggering Ipython to think that the kernel died, but I am not sure of that. I tried to set a trap here: https://github.com/soumith/iTorch/blob/master/itorch#L7 for SIGPIPE so that before the launch of ipython, SIGPIPE is trapped completely. However that did not help the ?kernel died? message. The "kernel died? message appears when the first execute_request is processed, not before that. Do let me know if you have any pointers. Thanks, Soumith -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150116/ca6d322d/attachment.html> From liam.gretton at leicester.ac.uk Fri Jan 16 05:34:53 2015 From: liam.gretton at leicester.ac.uk (Liam Gretton) Date: Fri, 16 Jan 2015 10:34:53 +0000 Subject: [IPython-dev] %load hangs in qtconsole In-Reply-To: <CAHNn8BWYMdROY65PSR94XFAM5OmOYgBHqzxGRJktpNhO2MdQmw@mail.gmail.com> References: <54B78823.1050506@leicester.ac.uk> <CAHNn8BWYMdROY65PSR94XFAM5OmOYgBHqzxGRJktpNhO2MdQmw@mail.gmail.com> Message-ID: <54B8E94D.8080706@leicester.ac.uk> On 15/01/2015 18:13, MinRK wrote: > It looks like it's putting the cursor in the wrong place, confusing the UI. > If you place the cursor in the input area, it should allow you to execute > the input. Yes! That is indeed the problem, thanks. I'll report this in GitHub and I'll take a look at the code myself to see if I can fix it. -- Liam Gretton liam.gretton at le.ac.uk Systems Specialist http://www.le.ac.uk/its/ IT Services Tel: +44 (0)116 2522254 University Of Leicester, University Road Leicestershire LE1 7RH, United Kingdom From takowl at gmail.com Fri Jan 16 13:16:48 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 16 Jan 2015 10:16:48 -0800 Subject: [IPython-dev] iTorch - Weird issues with mkfifo on linux In-Reply-To: <D0DDC4FF.3884%soumith@fb.com> References: <D0DDC4FF.3884%soumith@fb.com> Message-ID: <CAOvn4qiqGhQ1bTiBUKbSs29gv-0G_FHGYtxtemiPY02PVrty2Q@mail.gmail.com> On 15 January 2015 at 19:26, Soumith Chintala <soumith at fb.com> wrote: > One problem we oncovered is that on certain (or possibly all linux > distros) ipython thinks our kernel died, even though it is alive and well. FIFOs have some bizarre properties which might be related - for instance, opening a FIFO for reading will block until something else opens it for writing (the open() call itself blocks, not just attempts to read). Could it be that an unexpectedly blocking system call is stopping the heartbeat and causing IPython to think that your kernel is dead? Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150116/9ac086cb/attachment.html> From soumith at fb.com Fri Jan 16 14:05:44 2015 From: soumith at fb.com (Soumith Chintala) Date: Fri, 16 Jan 2015 19:05:44 +0000 Subject: [IPython-dev] iTorch - Weird issues with mkfifo on linux In-Reply-To: <CAOvn4qiqGhQ1bTiBUKbSs29gv-0G_FHGYtxtemiPY02PVrty2Q@mail.gmail.com> References: <D0DDC4FF.3884%soumith@fb.com> <CAOvn4qiqGhQ1bTiBUKbSs29gv-0G_FHGYtxtemiPY02PVrty2Q@mail.gmail.com> Message-ID: <D0DEA0AE.3CAE%soumith@fb.com> This is not a problem of blocking reads, or the heartbeat stopping, I have verified both those parts. But I?ll try investigating more into if it?s a system-call/signaling weirdness. Thanks, Soumith From: Thomas Kluyver <takowl at gmail.com<mailto:takowl at gmail.com>> Reply-To: IPython developers list <ipython-dev at scipy.org<mailto:ipython-dev at scipy.org>> Date: Friday, January 16, 2015 at 10:16 AM To: IPython developers list <ipython-dev at scipy.org<mailto:ipython-dev at scipy.org>> Subject: Re: [IPython-dev] iTorch - Weird issues with mkfifo on linux On 15 January 2015 at 19:26, Soumith Chintala <soumith at fb.com<mailto:soumith at fb.com>> wrote: One problem we oncovered is that on certain (or possibly all linux distros) ipython thinks our kernel died, even though it is alive and well. FIFOs have some bizarre properties which might be related - for instance, opening a FIFO for reading will block until something else opens it for writing (the open() call itself blocks, not just attempts to read). Could it be that an unexpectedly blocking system call is stopping the heartbeat and causing IPython to think that your kernel is dead? Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150116/37b89aa3/attachment.html> From benjaminrk at gmail.com Fri Jan 16 23:44:38 2015 From: benjaminrk at gmail.com (MinRK) Date: Fri, 16 Jan 2015 20:44:38 -0800 Subject: [IPython-dev] iTorch - Weird issues with mkfifo on linux In-Reply-To: <D0DEA0AE.3CAE%soumith@fb.com> References: <D0DDC4FF.3884%soumith@fb.com> <CAOvn4qiqGhQ1bTiBUKbSs29gv-0G_FHGYtxtemiPY02PVrty2Q@mail.gmail.com> <D0DEA0AE.3CAE%soumith@fb.com> Message-ID: <CAHNn8BVaNsRe2=W6NL09SOpsb2yj-d40c8HUo-4BfwR75d8Jkw@mail.gmail.com> Can you run the server with `--debug`, and post the logs around when the failure occurs? On Fri, Jan 16, 2015 at 11:05 AM, Soumith Chintala <soumith at fb.com> wrote: > This is not a problem of blocking reads, or the heartbeat stopping, I > have verified both those parts. > But I?ll try investigating more into if it?s a system-call/signaling > weirdness. > Thanks, > Soumith > > From: Thomas Kluyver <takowl at gmail.com> > Reply-To: IPython developers list <ipython-dev at scipy.org> > Date: Friday, January 16, 2015 at 10:16 AM > To: IPython developers list <ipython-dev at scipy.org> > Subject: Re: [IPython-dev] iTorch - Weird issues with mkfifo on linux > > On 15 January 2015 at 19:26, Soumith Chintala <soumith at fb.com> wrote: > >> One problem we oncovered is that on certain (or possibly all linux >> distros) ipython thinks our kernel died, even though it is alive and well. > > > FIFOs have some bizarre properties which might be related - for > instance, opening a FIFO for reading will block until something else opens > it for writing (the open() call itself blocks, not just attempts to read). > Could it be that an unexpectedly blocking system call is stopping the > heartbeat and causing IPython to think that your kernel is dead? > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150116/8fd48d15/attachment.html> From rnelsonchem at gmail.com Sat Jan 17 14:14:12 2015 From: rnelsonchem at gmail.com (Ryan Nelson) Date: Sat, 17 Jan 2015 14:14:12 -0500 Subject: [IPython-dev] Run IPython Parallel from custom module? Message-ID: <CAFbt6dcnqqPvQ3aS4-sCLV9r_fS9HfRkxoN-z3LZsHjAMDF4AA@mail.gmail.com> Hello everyone, I'm developing a module for processing some data files. There are several steps to this process, which can be time consuming, so I would like to write a helper function in my module that uses IPython.parallel to spread out the work over several processors. Unfortunately, I'm having trouble getting things to work properly. Note: I can get something similar to work when it is not written as a module... I tried my best to make a simple example that fails with the same error that I'm seeing in the larger case (below); however, it is still a little complicated. After running the test script, I get a "NameError" on the cluster nodes (traceback below). If I check the nodes manually, the objects are defined. I can also define new functions from the IPython terminal that act on those objects, and those work as expected. Any help you can provide is most appreciated. Sorry if this is a duplicate, but I couldn't find it anywhere else. Python 3.4 and 2.7 and IPython 2.3.1 Ryan ________________________________________ Here's the approximate structure of my module: mymod/ __init__.py functs.py : helper functions with Parallel support objects.py : contains my custom processing objects # functs.py # ------------- from IPython.parallel import Client import mymod.objects as objects def process(files): client = Client() dview = client[:] dview.block = True # Do this to make module path known # However, my mod will eventually be installed via pip, so this # isn't necessary to reproduce error in that case with dview.sync_imports(): import sys sys.path.append(['/home/nelson/code/testing/',]) dview['sys.path'] = sys.path File = objects.File1 dview['File'] = File result = dview.map_sync(_do_this, files) return result def _do_this(fname): f = File(fname) f.extra = 'hello' return f ------------- # objects.py # ------------- class File1(object): def __init__(self, fname): self.fname = fname ------------- I also have a test script ("test.py") with the following: ------------- from mymod.functs import process result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g']) [print(i.fname, i.extra) for i in result] ------------- When I run this script, I get the following errors: importing sys on engine(s) Traceback (most recent call last): File "test2.py", line 3, in <module> result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g']) File "/home/nelson/code/testing/mymod/functs.py", line 16, in process result = dview.map_sync(_do_this, files) File "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line 366, in map_sync return self.map(f,*sequences,**kwargs) File "<string>", line 2, in map File "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line 66, in sync_results ret = f(self, *args, **kwargs) File "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line 624, in map return pf.map(*sequences) File "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", line 271, in map ret = self(*sequences) File "<string>", line 2, in __call__ File "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", line 78, in sync_view_results return f(self, *args, **kwargs) File "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", line 254, in __call__ return r.get() File "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py", line 118, in get raise self._exception File "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py", line 153, in wait results = error.collect_exceptions(results, self._fname) File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py", line 233, in collect_exceptions raise e File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py", line 231, in collect_exceptions raise CompositeError(msg, elist) IPython.parallel.error.CompositeError: one or more exceptions from call to method: _do_this [0:apply]: NameError: name 'File' is not defined [1:apply]: NameError: name 'File' is not defined [2:apply]: NameError: name 'File' is not defined [3:apply]: NameError: name 'File' is not defined However, if I jump into an interactive Python terminal, these objects are defined on the cluster nodes. In [1]: from IPython.parallel import Client In [2]: client = Client() In [3]: dview = client[:] In [4]: dview['File'] Out[4]: [mymod.objects.File1, mymod.objects.File1, mymod.objects.File1, mymod.objects.File1] -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150117/e3dc9b1a/attachment.html> From benjaminrk at gmail.com Sat Jan 17 15:15:47 2015 From: benjaminrk at gmail.com (MinRK) Date: Sat, 17 Jan 2015 12:15:47 -0800 Subject: [IPython-dev] Run IPython Parallel from custom module? In-Reply-To: <CAFbt6dcnqqPvQ3aS4-sCLV9r_fS9HfRkxoN-z3LZsHjAMDF4AA@mail.gmail.com> References: <CAFbt6dcnqqPvQ3aS4-sCLV9r_fS9HfRkxoN-z3LZsHjAMDF4AA@mail.gmail.com> Message-ID: <CAHNn8BUcN7KQ4bpnAqSUd75SF_6QtsXS2w4DrkcK88sPui4j7Q@mail.gmail.com> This is one of the most common sources of confusion with IPython.parallel. When you call a function, it?s globals are resolved to its own module. So when you call _do_this, it is looking for File in mymod, not in the interactive namespace. If you had defined _do_this interactively (in __main__), it would have found File. Similarly, if File were defined in the same module as _do_this, it would also be found. IPython provides a decorator that lets you define functions in modules that will be treated as if they were defined in __main__, and will thus have access to the interactive namespace. from IPython.parallel import interactive @interactivedef _do_this(fname): f = File(fname) ... -MinRK ? On Sat, Jan 17, 2015 at 11:14 AM, Ryan Nelson <rnelsonchem at gmail.com> wrote: > Hello everyone, > > I'm developing a module for processing some data files. There are several > steps to this process, which can be time consuming, so I would like to > write a helper function in my module that uses IPython.parallel to spread > out the work over several processors. Unfortunately, I'm having trouble > getting things to work properly. Note: I can get something similar to work > when it is not written as a module... > > I tried my best to make a simple example that fails with the same error > that I'm seeing in the larger case (below); however, it is still a little > complicated. > > After running the test script, I get a "NameError" on the cluster nodes > (traceback below). If I check the nodes manually, the objects are defined. > I can also define new functions from the IPython terminal that act on those > objects, and those work as expected. > > Any help you can provide is most appreciated. Sorry if this is a > duplicate, but I couldn't find it anywhere else. > > Python 3.4 and 2.7 and IPython 2.3.1 > > Ryan > > ________________________________________ > > Here's the approximate structure of my module: > > mymod/ > __init__.py > functs.py : helper functions with Parallel support > objects.py : contains my custom processing objects > > # functs.py # > ------------- > from IPython.parallel import Client > > import mymod.objects as objects > > def process(files): > client = Client() > dview = client[:] > dview.block = True > > # Do this to make module path known > # However, my mod will eventually be installed via pip, so this > # isn't necessary to reproduce error in that case > with dview.sync_imports(): > import sys > sys.path.append(['/home/nelson/code/testing/',]) > > dview['sys.path'] = sys.path > File = objects.File1 > dview['File'] = File > result = dview.map_sync(_do_this, files) > return result > > def _do_this(fname): > f = File(fname) > f.extra = 'hello' > return f > ------------- > > # objects.py # > ------------- > class File1(object): > def __init__(self, fname): > self.fname = fname > ------------- > > I also have a test script ("test.py") with the following: > ------------- > from mymod.functs import process > result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g']) > [print(i.fname, i.extra) for i in result] > ------------- > > When I run this script, I get the following errors: > > importing sys on engine(s) > Traceback (most recent call last): > File "test2.py", line 3, in <module> > result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g']) > File "/home/nelson/code/testing/mymod/functs.py", line 16, in process > result = dview.map_sync(_do_this, files) > File > "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line > 366, in map_sync > return self.map(f,*sequences,**kwargs) > File "<string>", line 2, in map > File > "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line > 66, in sync_results > ret = f(self, *args, **kwargs) > File > "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line > 624, in map > return pf.map(*sequences) > File > "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", > line 271, in map > ret = self(*sequences) > File "<string>", line 2, in __call__ > File > "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", > line 78, in sync_view_results > return f(self, *args, **kwargs) > File > "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", > line 254, in __call__ > return r.get() > File > "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py", > line 118, in get > raise self._exception > File > "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py", > line 153, in wait > results = error.collect_exceptions(results, self._fname) > File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py", > line 233, in collect_exceptions > raise e > File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py", > line 231, in collect_exceptions > raise CompositeError(msg, elist) > IPython.parallel.error.CompositeError: one or more exceptions from call to > method: _do_this > [0:apply]: NameError: name 'File' is not defined > [1:apply]: NameError: name 'File' is not defined > [2:apply]: NameError: name 'File' is not defined > [3:apply]: NameError: name 'File' is not defined > > However, if I jump into an interactive Python terminal, these objects are > defined on the cluster nodes. > In [1]: from IPython.parallel import Client > > In [2]: client = Client() > > In [3]: dview = client[:] > > In [4]: dview['File'] > > > Out[4]: > [mymod.objects.File1, > mymod.objects.File1, > mymod.objects.File1, > mymod.objects.File1] > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150117/23cef824/attachment.html> From rnelsonchem at gmail.com Sat Jan 17 16:17:07 2015 From: rnelsonchem at gmail.com (Ryan Nelson) Date: Sat, 17 Jan 2015 16:17:07 -0500 Subject: [IPython-dev] Run IPython Parallel from custom module? In-Reply-To: <CAHNn8BUcN7KQ4bpnAqSUd75SF_6QtsXS2w4DrkcK88sPui4j7Q@mail.gmail.com> References: <CAFbt6dcnqqPvQ3aS4-sCLV9r_fS9HfRkxoN-z3LZsHjAMDF4AA@mail.gmail.com> <CAHNn8BUcN7KQ4bpnAqSUd75SF_6QtsXS2w4DrkcK88sPui4j7Q@mail.gmail.com> Message-ID: <CAFbt6dcu3u55JO6P2+GqOXS9gMr8rdj-hy4vOddDas56sZBfjA@mail.gmail.com> Thanks! Searching for that decorator brings up a bunch of search hits. However, it looks like it isn't really documented in the main document yet (?). I'll try this out. Ryan On Sat, Jan 17, 2015 at 3:15 PM, MinRK <benjaminrk at gmail.com> wrote: > This is one of the most common sources of confusion with IPython.parallel. > When you call a function, it?s globals are resolved to its own module. So > when you call _do_this, it is looking for File in mymod, not in the > interactive namespace. If you had defined _do_this interactively (in > __main__), it would have found File. Similarly, if File were defined in > the same module as _do_this, it would also be found. > > IPython provides a decorator that lets you define functions in modules > that will be treated as if they were defined in __main__, and will thus > have access to the interactive namespace. > > from IPython.parallel import interactive > @interactivedef _do_this(fname): > f = File(fname) > ... > > -MinRK > ? > > On Sat, Jan 17, 2015 at 11:14 AM, Ryan Nelson <rnelsonchem at gmail.com> > wrote: > >> Hello everyone, >> >> I'm developing a module for processing some data files. There are several >> steps to this process, which can be time consuming, so I would like to >> write a helper function in my module that uses IPython.parallel to spread >> out the work over several processors. Unfortunately, I'm having trouble >> getting things to work properly. Note: I can get something similar to work >> when it is not written as a module... >> >> I tried my best to make a simple example that fails with the same error >> that I'm seeing in the larger case (below); however, it is still a little >> complicated. >> >> After running the test script, I get a "NameError" on the cluster nodes >> (traceback below). If I check the nodes manually, the objects are defined. >> I can also define new functions from the IPython terminal that act on those >> objects, and those work as expected. >> >> Any help you can provide is most appreciated. Sorry if this is a >> duplicate, but I couldn't find it anywhere else. >> >> Python 3.4 and 2.7 and IPython 2.3.1 >> >> Ryan >> >> ________________________________________ >> >> Here's the approximate structure of my module: >> >> mymod/ >> __init__.py >> functs.py : helper functions with Parallel support >> objects.py : contains my custom processing objects >> >> # functs.py # >> ------------- >> from IPython.parallel import Client >> >> import mymod.objects as objects >> >> def process(files): >> client = Client() >> dview = client[:] >> dview.block = True >> >> # Do this to make module path known >> # However, my mod will eventually be installed via pip, so this >> # isn't necessary to reproduce error in that case >> with dview.sync_imports(): >> import sys >> sys.path.append(['/home/nelson/code/testing/',]) >> >> dview['sys.path'] = sys.path >> File = objects.File1 >> dview['File'] = File >> result = dview.map_sync(_do_this, files) >> return result >> >> def _do_this(fname): >> f = File(fname) >> f.extra = 'hello' >> return f >> ------------- >> >> # objects.py # >> ------------- >> class File1(object): >> def __init__(self, fname): >> self.fname = fname >> ------------- >> >> I also have a test script ("test.py") with the following: >> ------------- >> from mymod.functs import process >> result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g']) >> [print(i.fname, i.extra) for i in result] >> ------------- >> >> When I run this script, I get the following errors: >> >> importing sys on engine(s) >> Traceback (most recent call last): >> File "test2.py", line 3, in <module> >> result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g']) >> File "/home/nelson/code/testing/mymod/functs.py", line 16, in process >> result = dview.map_sync(_do_this, files) >> File >> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line >> 366, in map_sync >> return self.map(f,*sequences,**kwargs) >> File "<string>", line 2, in map >> File >> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line >> 66, in sync_results >> ret = f(self, *args, **kwargs) >> File >> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line >> 624, in map >> return pf.map(*sequences) >> File >> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", >> line 271, in map >> ret = self(*sequences) >> File "<string>", line 2, in __call__ >> File >> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", >> line 78, in sync_view_results >> return f(self, *args, **kwargs) >> File >> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", >> line 254, in __call__ >> return r.get() >> File >> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py", >> line 118, in get >> raise self._exception >> File >> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py", >> line 153, in wait >> results = error.collect_exceptions(results, self._fname) >> File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py", >> line 233, in collect_exceptions >> raise e >> File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py", >> line 231, in collect_exceptions >> raise CompositeError(msg, elist) >> IPython.parallel.error.CompositeError: one or more exceptions from call >> to method: _do_this >> [0:apply]: NameError: name 'File' is not defined >> [1:apply]: NameError: name 'File' is not defined >> [2:apply]: NameError: name 'File' is not defined >> [3:apply]: NameError: name 'File' is not defined >> >> However, if I jump into an interactive Python terminal, these objects are >> defined on the cluster nodes. >> In [1]: from IPython.parallel import Client >> >> In [2]: client = Client() >> >> In [3]: dview = client[:] >> >> In [4]: dview['File'] >> >> >> Out[4]: >> [mymod.objects.File1, >> mymod.objects.File1, >> mymod.objects.File1, >> mymod.objects.File1] >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150117/415828c1/attachment.html> From rnelsonchem at gmail.com Sat Jan 17 20:23:07 2015 From: rnelsonchem at gmail.com (Ryan Nelson) Date: Sat, 17 Jan 2015 20:23:07 -0500 Subject: [IPython-dev] Run IPython Parallel from custom module? In-Reply-To: <CAFbt6dcu3u55JO6P2+GqOXS9gMr8rdj-hy4vOddDas56sZBfjA@mail.gmail.com> References: <CAFbt6dcnqqPvQ3aS4-sCLV9r_fS9HfRkxoN-z3LZsHjAMDF4AA@mail.gmail.com> <CAHNn8BUcN7KQ4bpnAqSUd75SF_6QtsXS2w4DrkcK88sPui4j7Q@mail.gmail.com> <CAFbt6dcu3u55JO6P2+GqOXS9gMr8rdj-hy4vOddDas56sZBfjA@mail.gmail.com> Message-ID: <CAFbt6dfE-U-CbfNJZ66cpx1V=Bp6iTo0yjzvDbfDc7pGFaKiqg@mail.gmail.com> This worked well for the example that I posted. Thanks again. However, it brings up another question. What are the restrictions on using this on methods inside a custom object? For example, if I change "functs.py" into an object with the following methods: @interactive def _do_this(self, fname): f = File(fname) f.extra = 'hello' self._do_something_else(f) return f @interactive def _do_something_else(self, f): f.junk = 'World' I don't get any errors, but "_do_something_else" doesn't seem to be called, but "_do_this" works fine. (Actually, in my larger code, I'm getting a pickling error... But that's tough to reproduce.) Ryan On Sat, Jan 17, 2015 at 4:17 PM, Ryan Nelson <rnelsonchem at gmail.com> wrote: > Thanks! > > Searching for that decorator brings up a bunch of search hits. However, it > looks like it isn't really documented in the main document yet (?). > > I'll try this out. > > Ryan > > > > On Sat, Jan 17, 2015 at 3:15 PM, MinRK <benjaminrk at gmail.com> wrote: > >> This is one of the most common sources of confusion with >> IPython.parallel. When you call a function, it?s globals are resolved to >> its own module. So when you call _do_this, it is looking for File in >> mymod, not in the interactive namespace. If you had defined _do_this >> interactively (in __main__), it would have found File. Similarly, if File >> were defined in the same module as _do_this, it would also be found. >> >> IPython provides a decorator that lets you define functions in modules >> that will be treated as if they were defined in __main__, and will thus >> have access to the interactive namespace. >> >> from IPython.parallel import interactive >> @interactivedef _do_this(fname): >> f = File(fname) >> ... >> >> -MinRK >> ? >> >> On Sat, Jan 17, 2015 at 11:14 AM, Ryan Nelson <rnelsonchem at gmail.com> >> wrote: >> >>> Hello everyone, >>> >>> I'm developing a module for processing some data files. There are >>> several steps to this process, which can be time consuming, so I would like >>> to write a helper function in my module that uses IPython.parallel to >>> spread out the work over several processors. Unfortunately, I'm having >>> trouble getting things to work properly. Note: I can get something similar >>> to work when it is not written as a module... >>> >>> I tried my best to make a simple example that fails with the same error >>> that I'm seeing in the larger case (below); however, it is still a little >>> complicated. >>> >>> After running the test script, I get a "NameError" on the cluster nodes >>> (traceback below). If I check the nodes manually, the objects are defined. >>> I can also define new functions from the IPython terminal that act on those >>> objects, and those work as expected. >>> >>> Any help you can provide is most appreciated. Sorry if this is a >>> duplicate, but I couldn't find it anywhere else. >>> >>> Python 3.4 and 2.7 and IPython 2.3.1 >>> >>> Ryan >>> >>> ________________________________________ >>> >>> Here's the approximate structure of my module: >>> >>> mymod/ >>> __init__.py >>> functs.py : helper functions with Parallel support >>> objects.py : contains my custom processing objects >>> >>> # functs.py # >>> ------------- >>> from IPython.parallel import Client >>> >>> import mymod.objects as objects >>> >>> def process(files): >>> client = Client() >>> dview = client[:] >>> dview.block = True >>> >>> # Do this to make module path known >>> # However, my mod will eventually be installed via pip, so this >>> # isn't necessary to reproduce error in that case >>> with dview.sync_imports(): >>> import sys >>> sys.path.append(['/home/nelson/code/testing/',]) >>> >>> dview['sys.path'] = sys.path >>> File = objects.File1 >>> dview['File'] = File >>> result = dview.map_sync(_do_this, files) >>> return result >>> >>> def _do_this(fname): >>> f = File(fname) >>> f.extra = 'hello' >>> return f >>> ------------- >>> >>> # objects.py # >>> ------------- >>> class File1(object): >>> def __init__(self, fname): >>> self.fname = fname >>> ------------- >>> >>> I also have a test script ("test.py") with the following: >>> ------------- >>> from mymod.functs import process >>> result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g']) >>> [print(i.fname, i.extra) for i in result] >>> ------------- >>> >>> When I run this script, I get the following errors: >>> >>> importing sys on engine(s) >>> Traceback (most recent call last): >>> File "test2.py", line 3, in <module> >>> result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g']) >>> File "/home/nelson/code/testing/mymod/functs.py", line 16, in process >>> result = dview.map_sync(_do_this, files) >>> File >>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line >>> 366, in map_sync >>> return self.map(f,*sequences,**kwargs) >>> File "<string>", line 2, in map >>> File >>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line >>> 66, in sync_results >>> ret = f(self, *args, **kwargs) >>> File >>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line >>> 624, in map >>> return pf.map(*sequences) >>> File >>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", >>> line 271, in map >>> ret = self(*sequences) >>> File "<string>", line 2, in __call__ >>> File >>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", >>> line 78, in sync_view_results >>> return f(self, *args, **kwargs) >>> File >>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py", >>> line 254, in __call__ >>> return r.get() >>> File >>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py", >>> line 118, in get >>> raise self._exception >>> File >>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py", >>> line 153, in wait >>> results = error.collect_exceptions(results, self._fname) >>> File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py", >>> line 233, in collect_exceptions >>> raise e >>> File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py", >>> line 231, in collect_exceptions >>> raise CompositeError(msg, elist) >>> IPython.parallel.error.CompositeError: one or more exceptions from call >>> to method: _do_this >>> [0:apply]: NameError: name 'File' is not defined >>> [1:apply]: NameError: name 'File' is not defined >>> [2:apply]: NameError: name 'File' is not defined >>> [3:apply]: NameError: name 'File' is not defined >>> >>> However, if I jump into an interactive Python terminal, these objects >>> are defined on the cluster nodes. >>> In [1]: from IPython.parallel import Client >>> >>> In [2]: client = Client() >>> >>> In [3]: dview = client[:] >>> >>> In [4]: dview['File'] >>> >>> >>> Out[4]: >>> [mymod.objects.File1, >>> mymod.objects.File1, >>> mymod.objects.File1, >>> mymod.objects.File1] >>> >>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150117/7f178703/attachment.html> From lev at columbia.edu Sun Jan 18 08:28:55 2015 From: lev at columbia.edu (Lev Givon) Date: Sun, 18 Jan 2015 08:28:55 -0500 Subject: [IPython-dev] question about KernelClient.is_alive() Message-ID: <20150118132855.GA5685@avicenna> I recently tried using IPython.kernel.KernelClient (in IPython 2.3.1 on Ubuntu 14.04.1) to connect to a locally running kernel (started from the command line by running "ipython kernel") by passing it the connection file associated with the latter. While doing so, I noticed that KernelClient.is_alive() returns True even when KernelClient is instantiated with a nonexistent connection file, i.e., import IPython.kernel kc = IPython.kernel.KernelClient(connection_file='/tmp/nonexistent_file') print kc.is_alive() Is this expected? The online documentation seems to imply that is_alive() should return True only if the kernel to which KernelClient is connected is running. -- Lev Givon Bionet Group | Neurokernel Project http://www.columbia.edu/~lev/ http://lebedov.github.io/ http://neurokernel.github.io/ From damianavila at gmail.com Sun Jan 18 09:19:44 2015 From: damianavila at gmail.com (=?UTF-8?Q?Dami=C3=A1n_Avila?=) Date: Sun, 18 Jan 2015 11:19:44 -0300 Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the core Jupyter/IPython team In-Reply-To: <CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com> References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com> <1420394484530-5082155.post@n6.nabble.com> <CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com> Message-ID: <CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com> (Belatedly) Congrats! You both make an amazing work every day! Cheers. Damian 2015-01-05 19:01 GMT-03:00 Brian Granger <ellisonbg at gmail.com>: > Thanks so much for your work Kyle and Nick! > > Cheers, > > Brian > > On Sun, Jan 4, 2015 at 10:01 AM, ssanderson <ssanderson at quantopian.com> > wrote: > >> Congrats! >> >> >> >> -- >> View this message in context: >> http://python.6.x6.nabble.com/Welcoming-Kyle-Kelley-and-Nick-Bollweg-to-the-core-Jupyter-IPython-team-tp5082131p5082155.html >> Sent from the IPython - Development mailing list archive at Nabble.com. >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- *Dami?n* -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150118/6f92c07d/attachment.html> From lev at columbia.edu Sun Jan 18 11:43:19 2015 From: lev at columbia.edu (Lev Givon) Date: Sun, 18 Jan 2015 11:43:19 -0500 Subject: [IPython-dev] question about KernelClient.is_alive() In-Reply-To: <20150118132855.GA5685@avicenna> References: <20150118132855.GA5685@avicenna> Message-ID: <20150118164319.GB7084@avicenna.ee.columbia.edu> Received from Lev Givon on Sun, Jan 18, 2015 at 08:28:55AM EST: > I recently tried using IPython.kernel.KernelClient (in IPython 2.3.1 on Ubuntu > 14.04.1) to connect to a locally running kernel (started from the command line > by running "ipython kernel") by passing it the connection file associated with > the latter. While doing so, I noticed that KernelClient.is_alive() returns True > even when KernelClient is instantiated with a nonexistent connection file, i.e., > > import IPython.kernel > kc = IPython.kernel.KernelClient(connection_file='/tmp/nonexistent_file') > print kc.is_alive() > > Is this expected? The online documentation seems to imply that is_alive() should > return True only if the kernel to which KernelClient is connected is running. (Answering my own question for the benefit of others.) The value of KernelClient.is_alive() evidently isn't indicative of actual kernel status until the heartbeat channel is activated and unpaused to determine whether the client is interacting with a live kernel. -- Lev Givon Bionet Group | Neurokernel Project http://www.columbia.edu/~lev/ http://lebedov.github.io/ http://neurokernel.github.io/ From nick.bollweg at gmail.com Sun Jan 18 13:30:40 2015 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Sun, 18 Jan 2015 13:30:40 -0500 Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the core Jupyter/IPython team In-Reply-To: <CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com> References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com> <1420394484530-5082155.post@n6.nabble.com> <CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com> <CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com> Message-ID: <CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com> Thanks everybody! Just as an update: we're back from the hospital, and our new little person is doing great! We're all a bit sleepy, so take anything posted at strange hours of the day with a grain of salt :) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150118/45b3b005/attachment.html> From ellisonbg at gmail.com Sun Jan 18 13:45:09 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Sun, 18 Jan 2015 10:45:09 -0800 Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the core Jupyter/IPython team In-Reply-To: <CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com> References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com> <1420394484530-5082155.post@n6.nabble.com> <CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com> <CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com> <CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com> Message-ID: <CAH4pYpTu89Lqwz7oBBB9TBSxFAx+xzmBaA2xCcLcbMoCketd5A@mail.gmail.com> Congrats! On Sun, Jan 18, 2015 at 10:30 AM, Nicholas Bollweg <nick.bollweg at gmail.com> wrote: > Thanks everybody! > > Just as an update: we're back from the hospital, and our new little person > is doing great! We're all a bit sleepy, so take anything posted at strange > hours of the day with a grain of salt :) > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150118/f20594c8/attachment.html> From rgbkrk at gmail.com Sun Jan 18 14:31:07 2015 From: rgbkrk at gmail.com (Kyle Kelley) Date: Sun, 18 Jan 2015 13:31:07 -0600 Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the core Jupyter/IPython team In-Reply-To: <CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com> References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com> <1420394484530-5082155.post@n6.nabble.com> <CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com> <CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com> <CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com> Message-ID: <CA+tbMaVkdfXs3oHS6xWip22CM2c3jmNaT+Qt4OOxEACyULCBaA@mail.gmail.com> Congratulations! We have little ones too, so we understand. The team and community seem to put up with my sleep deprived contributions. ;) Rest well and enjoy the new addition. They're a lot of fun! -- Kyle On Sunday, January 18, 2015, Nicholas Bollweg <nick.bollweg at gmail.com> wrote: > Thanks everybody! > > Just as an update: we're back from the hospital, and our new little person > is doing great! We're all a bit sleepy, so take anything posted at strange > hours of the day with a grain of salt :) > -- Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150118/00c52a09/attachment.html> From DavidAnthonyPowell+python at gmail.com Mon Jan 19 01:47:21 2015 From: DavidAnthonyPowell+python at gmail.com (David Powell) Date: Mon, 19 Jan 2015 17:47:21 +1100 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js Message-ID: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> Hello all, I recently managed to get 3D vector plots working in the IPython notebook, by statically producing three.js output. I thought I'd share it here in case this example is useful for anyone else, or if anyone has any suggestions as to how I could do things better. The advantage of the static approach is that the output survives the nbconvert process, which you can see in this notebook here (which also compares with a couple of other unsatisfactory ways of visualising 3D vector plots). http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%20to%20create%203D%20plots.ipynb Another example can be seen at: http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%20and%20creating%20geometric%20shapes.ipynb If you are interested to see how this works, you can check out the python source: https://github.com/DavidPowell/OpenModes/blob/master/openmodes/ipython.py As well as the javascript: https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js and finally the HTML: https://github.com/DavidPowell/OpenModes/blob/master/openmodes/templates/three_js_plot.html By the way, I am aware of the efforts to make a proper three.js IPython widget (https://github.com/jasongrout/pythreejs/), and will probably switch over to this once it is stable, and when the output can survive ipython's nbconvert process. Is this something we can expect for widgets in IPython 3.0? regards David -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/d65e9f23/attachment.html> From soumith at fb.com Mon Jan 19 01:50:30 2015 From: soumith at fb.com (Soumith Chintala) Date: Mon, 19 Jan 2015 06:50:30 +0000 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js In-Reply-To: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> Message-ID: <D0E1E8D1.3E2F%soumith@fb.com> This is fantastic! I was going to do some of this work myself over the next month for iTorch (https://github.com/facebook/iTorch), and you saved me some trouble. I will take a look at your code over the next couple of days, I am wondering if you would have any issues if I directly adapted your javascript side of things in iTorch rather than rewriting from scratch. ? Soumith From: David Powell <DavidAnthonyPowell+python at gmail.com<mailto:DavidAnthonyPowell+python at gmail.com>> Reply-To: IPython developers list <ipython-dev at scipy.org<mailto:ipython-dev at scipy.org>> Date: Sunday, January 18, 2015 at 10:47 PM To: IPython developers list <ipython-dev at scipy.org<mailto:ipython-dev at scipy.org>> Subject: [IPython-dev] WebGL example in IPython notebook based on three.js Hello all, I recently managed to get 3D vector plots working in the IPython notebook, by statically producing three.js output. I thought I'd share it here in case this example is useful for anyone else, or if anyone has any suggestions as to how I could do things better. The advantage of the static approach is that the output survives the nbconvert process, which you can see in this notebook here (which also compares with a couple of other unsatisfactory ways of visualising 3D vector plots). http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%20to%20create%203D%20plots.ipynb<https://urldefense.proofpoint.com/v1/url?u=http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%2520to%2520create%25203D%2520plots.ipynb&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=5S5uukCkPxLR1dZ0syN0fw%3D%3D%0A&m=70WnYrYAO8zL773kKr3zku5qmdNjduxrZK8pOaiqCaE%3D%0A&s=aad5ead78a1b3391d5b9c2a10e1214ef17fd7df8d223ad6076c6b78f9ff1246c> Another example can be seen at: http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%20and%20creating%20geometric%20shapes.ipynb<https://urldefense.proofpoint.com/v1/url?u=http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%2520and%2520creating%2520geometric%2520shapes.ipynb&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=5S5uukCkPxLR1dZ0syN0fw%3D%3D%0A&m=70WnYrYAO8zL773kKr3zku5qmdNjduxrZK8pOaiqCaE%3D%0A&s=76948f7a6b29f01f0eccbe0899c7d91d427a1c1588d11ecbc4e568f992fddea7> If you are interested to see how this works, you can check out the python source: https://github.com/DavidPowell/OpenModes/blob/master/openmodes/ipython.py As well as the javascript: https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js and finally the HTML: https://github.com/DavidPowell/OpenModes/blob/master/openmodes/templates/three_js_plot.html By the way, I am aware of the efforts to make a proper three.js IPython widget (https://github.com/jasongrout/pythreejs/), and will probably switch over to this once it is stable, and when the output can survive ipython's nbconvert process. Is this something we can expect for widgets in IPython 3.0? regards David -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/1a95b887/attachment.html> From bussonniermatthias at gmail.com Mon Jan 19 06:08:52 2015 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Mon, 19 Jan 2015 12:08:52 +0100 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js In-Reply-To: <D0E1E8D1.3E2F%soumith@fb.com> References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> <D0E1E8D1.3E2F%soumith@fb.com> Message-ID: <AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com> Hi all, Le 19 janv. 2015 ? 07:50, Soumith Chintala a ?crit : > This is fantastic! I was going to do some of this work myself over the next month for iTorch (https://github.com/facebook/iTorch), and you saved me some trouble. > I will take a look at your code over the next couple of days, I am wondering if you would have any issues if I directly adapted your javascript side of things in iTorch rather than rewriting from scratch. We still haven't really figured it out how to share javascript in between packages of different languages. (well technically in 3.0 yo can install into $NBEXTENSION directory, but install process is not perfect) If you can figured that out, and that all kernels agree on a set of relatively common JS "plugins" that are often used we might be able to work on making theses plugin available on nbviewer. Long term planning of course, but I'd like people to keep that in mind. -- M -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/129dbaf2/attachment.html> From bussonniermatthias at gmail.com Mon Jan 19 06:19:02 2015 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Mon, 19 Jan 2015 12:19:02 +0100 Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the core Jupyter/IPython team In-Reply-To: <CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com> References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com> <1420394484530-5082155.post@n6.nabble.com> <CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com> <CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com> <CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com> Message-ID: <9202D5B9-022F-4A95-B9D2-427E5B2A84A0@gmail.com> Congratulation too, Looking forward to your nightly contributions ! Le 18 janv. 2015 ? 19:30, Nicholas Bollweg a ?crit : > Thanks everybody! > > Just as an update: we're back from the hospital, and our new little person is doing great! We're all a bit sleepy, so take anything posted at strange hours of the day with a grain of salt :) > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From zvoros at gmail.com Mon Jan 19 06:26:00 2015 From: zvoros at gmail.com (=?UTF-8?B?Wm9sdMOhbiBWw7Zyw7Zz?=) Date: Mon, 19 Jan 2015 12:26:00 +0100 Subject: [IPython-dev] status of nbconvert Message-ID: <54BCE9C8.6080304@gmail.com> Hi all, I have recently had problems with nbconvert, and I was wondering, whether I do something improper, or the code is really broken. I have pulled the latest from master, and nbconvert fails outright with a simple notebook. (This eventually leads to a 500 server error, whenever I try to download a notebook.) Here is my traceback: ipython nbconvert test.ipynb Traceback (most recent call last): File "/usr/local/bin/ipython", line 5, in <module> start_ipython() File "/usr/local/lib/python2.7/dist-packages/IPython/__init__.py", line 120, in start_ipython return launch_new_instance(argv=argv, **kwargs) File "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", line 573, in launch_instance app.initialize(argv) File "<string>", line 2, in initialize File "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", line 75, in catch_config_error return method(app, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/IPython/terminal/ipapp.py", line 321, in initialize super(TerminalIPythonApp, self).initialize(argv) File "<string>", line 2, in initialize File "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", line 75, in catch_config_error return method(app, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/IPython/core/application.py", line 369, in initialize self.parse_command_line(argv) File "/usr/local/lib/python2.7/dist-packages/IPython/terminal/ipapp.py", line 316, in parse_command_line return super(TerminalIPythonApp, self).parse_command_line(argv) File "<string>", line 2, in parse_command_line File "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", line 75, in catch_config_error return method(app, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", line 471, in parse_command_line return self.initialize_subcommand(subc, subargv) File "<string>", line 2, in initialize_subcommand File "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", line 75, in catch_config_error return method(app, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", line 402, in initialize_subcommand subapp = import_item(subapp) File "/usr/local/lib/python2.7/dist-packages/IPython/utils/importstring.py", line 42, in import_item module = __import__(package, fromlist=[obj]) File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/__init__.py", line 3, in <module> from .exporters import * File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/__init__.py", line 1, in <module> from .export import * File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/export.py", line 13, in <module> from .templateexporter import TemplateExporter File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/templateexporter.py", line 30, in <module> from IPython.nbconvert import filters File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/__init__.py", line 6, in <module> from .markdown import * File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/markdown.py", line 79, in <module> class MathBlockLexer(mistune.BlockLexer): File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/markdown.py", line 80, in MathBlockLexer default_rules = ['block_math', 'latex_environment'] + mistune.BlockLexer.default_rules AttributeError: type object 'BlockLexer' has no attribute 'default_rules' If, in markdown.py, I remove + mistune.BlockLexer.default_rules on line 80, and + mistune.InlineLexer.default_rules on line 108, then everything seems to work fine. Could it be that the mistune version that I have (0.4) doesn't have the default_rules property, but the version with which ipython was tested on travis has? If so, shouldn't nbconvert bail out when the version requirement is not fulfilled, or shouldn't simply the default_rules property be defined, when the class is inherited in markdown.py? If this is the case, I could file a fix on github. Somewhat related: if there is a server error, wouldn't it be more meaningful to return the traceback, instead of "500: internal server error"? Cheers, Zolt?n From bussonniermatthias at gmail.com Mon Jan 19 06:31:17 2015 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Mon, 19 Jan 2015 12:31:17 +0100 Subject: [IPython-dev] status of nbconvert In-Reply-To: <54BCE9C8.6080304@gmail.com> References: <54BCE9C8.6080304@gmail.com> Message-ID: <467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com> There is definitively a mistune api change, we check dependency in setup.py : https://github.com/ipython/ipython/blob/50799e6943174cbd24998c4fb018eff42aac5cfb/setup.py#L257 'mistune>=0.5' But we definitively don't check at runtime. So the bug should be encounters only by people having installed master some time ago and just pulling new versions. Normal install should not suffer of this. You are right about the 500 though. -- M Le 19 janv. 2015 ? 12:26, Zolt?n V?r?s a ?crit : > Hi all, > > I have recently had problems with nbconvert, and I was wondering, > whether I do something improper, or the code is really broken. I have > pulled the latest from master, and nbconvert fails outright with a > simple notebook. (This eventually leads to a 500 server error, whenever > I try to download a notebook.) > > Here is my traceback: > > ipython nbconvert test.ipynb > Traceback (most recent call last): > File "/usr/local/bin/ipython", line 5, in <module> > start_ipython() > File "/usr/local/lib/python2.7/dist-packages/IPython/__init__.py", > line 120, in start_ipython > return launch_new_instance(argv=argv, **kwargs) > File > "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", > line 573, in launch_instance > app.initialize(argv) > File "<string>", line 2, in initialize > File > "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", > line 75, in catch_config_error > return method(app, *args, **kwargs) > File > "/usr/local/lib/python2.7/dist-packages/IPython/terminal/ipapp.py", line > 321, in initialize > super(TerminalIPythonApp, self).initialize(argv) > File "<string>", line 2, in initialize > File > "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", > line 75, in catch_config_error > return method(app, *args, **kwargs) > File > "/usr/local/lib/python2.7/dist-packages/IPython/core/application.py", > line 369, in initialize > self.parse_command_line(argv) > File > "/usr/local/lib/python2.7/dist-packages/IPython/terminal/ipapp.py", line > 316, in parse_command_line > return super(TerminalIPythonApp, self).parse_command_line(argv) > File "<string>", line 2, in parse_command_line > File > "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", > line 75, in catch_config_error > return method(app, *args, **kwargs) > File > "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", > line 471, in parse_command_line > return self.initialize_subcommand(subc, subargv) > File "<string>", line 2, in initialize_subcommand > File > "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", > line 75, in catch_config_error > return method(app, *args, **kwargs) > File > "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", > line 402, in initialize_subcommand > subapp = import_item(subapp) > File > "/usr/local/lib/python2.7/dist-packages/IPython/utils/importstring.py", > line 42, in import_item > module = __import__(package, fromlist=[obj]) > File > "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/__init__.py", > line 3, in <module> > from .exporters import * > File > "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/__init__.py", > line 1, in <module> > from .export import * > File > "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/export.py", > line 13, in <module> > from .templateexporter import TemplateExporter > File > "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/templateexporter.py", > line 30, in <module> > from IPython.nbconvert import filters > File > "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/__init__.py", > line 6, in <module> > from .markdown import * > File > "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/markdown.py", > line 79, in <module> > class MathBlockLexer(mistune.BlockLexer): > File > "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/markdown.py", > line 80, in MathBlockLexer > default_rules = ['block_math', 'latex_environment'] + > mistune.BlockLexer.default_rules > AttributeError: type object 'BlockLexer' has no attribute 'default_rules' > > > > If, in markdown.py, I remove > > + mistune.BlockLexer.default_rules on line 80, and > + mistune.InlineLexer.default_rules on line 108, then everything seems > to work fine. > > Could it be that the mistune version that I have (0.4) doesn't have the > default_rules property, but the version with which ipython was tested on > travis has? If so, shouldn't nbconvert bail out when the version > requirement is not fulfilled, or shouldn't simply the default_rules > property be defined, when the class is inherited in markdown.py? If this > is the case, I could file a fix on github. > > Somewhat related: if there is a server error, wouldn't it be more > meaningful to return the traceback, instead of "500: internal server error"? > > Cheers, > Zolt?n > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/4d0de64f/attachment.html> From zvoros at gmail.com Mon Jan 19 06:40:45 2015 From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=) Date: Mon, 19 Jan 2015 12:40:45 +0100 Subject: [IPython-dev] status of nbconvert In-Reply-To: <467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com> References: <54BCE9C8.6080304@gmail.com> <467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com> Message-ID: <54BCED3D.8040705@gmail.com> Hi Matthias, On 01/19/2015 12:31 PM, Matthias BUSSONNIER wrote: > > > There is definitively a mistune api change, > we check dependency in setup.py : > > https://github.com/ipython/ipython/blob/50799e6943174cbd24998c4fb018eff42aac5cfb/setup.py#L257 > > 'mistune>=0.5' > > But we definitively don't check at runtime. > So the bug should be encounters only by people having installed > master some time ago and just pulling new versions. Yes, I think this is what has happened. What is a bit odd, though, is that both easy_install and pip want to install version 0.4. I guess, you installed it from source. Cheers, Zolt?n From bussonniermatthias at gmail.com Mon Jan 19 07:06:00 2015 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Mon, 19 Jan 2015 13:06:00 +0100 Subject: [IPython-dev] status of nbconvert In-Reply-To: <54BCED3D.8040705@gmail.com> References: <54BCE9C8.6080304@gmail.com> <467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com> <54BCED3D.8040705@gmail.com> Message-ID: <76A22C20-FFC4-4B59-A06D-F144FCF863F2@gmail.com> Le 19 janv. 2015 ? 12:40, Zolt?n V?r?s a ?crit : > Hi Matthias, > > On 01/19/2015 12:31 PM, Matthias BUSSONNIER wrote: >> >> >> There is definitively a mistune api change, >> we check dependency in setup.py : >> >> https://github.com/ipython/ipython/blob/50799e6943174cbd24998c4fb018eff42aac5cfb/setup.py#L257 >> >> 'mistune>=0.5' >> >> But we definitively don't check at runtime. >> So the bug should be encounters only by people having installed >> master some time ago and just pulling new versions. > > Yes, I think this is what has happened. What is a bit odd, though, is > that both easy_install and pip want to install version 0.4. I guess, you > installed it from source. Hum, that's weird. I'll try to have a look. Thanks ! -- M > > Cheers, > Zolt?n > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From zvoros at gmail.com Mon Jan 19 07:33:11 2015 From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=) Date: Mon, 19 Jan 2015 13:33:11 +0100 Subject: [IPython-dev] status of nbconvert In-Reply-To: <76A22C20-FFC4-4B59-A06D-F144FCF863F2@gmail.com> References: <54BCE9C8.6080304@gmail.com> <467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com> <54BCED3D.8040705@gmail.com> <76A22C20-FFC4-4B59-A06D-F144FCF863F2@gmail.com> Message-ID: <54BCF987.3030501@gmail.com> Hi Matthias, On 01/19/2015 01:06 PM, Matthias BUSSONNIER wrote: >> Yes, I think this is what has happened. What is a bit odd, though, is >> that both easy_install and pip want to install version 0.4. I guess, you >> installed it from source. > Hum, that's weird. > > I'll try to have a look. > > Thanks ! I have installed the source, and I no longer have problems with nbconvert. Is there someone whom I could notify to fix pip/easy_install? I have absolutely no idea how that is managed. I have seen that Thomas Kluyver is on the development team. Thomas? Cheers, Zolt?n From cyrille.rossant at gmail.com Mon Jan 19 07:30:15 2015 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Mon, 19 Jan 2015 13:30:15 +0100 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js In-Reply-To: <AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com> References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> <D0E1E8D1.3E2F%soumith@fb.com> <AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com> Message-ID: <CA+-1RQTj_=3qGri9n+HKtvjmFWgWOLtGDjxeQZSXsLVEkrme8g@mail.gmail.com> Thanks for sharing this! I'd like to add that there's also work being done currently for bringing WebGL visualization to the notebook via VisPy. We don't use three.js but custom WebGL code generated by Python. The WebGL backend currently requires a live Python server, but a standalone version is also in the pipes. 2015-01-19 12:08 GMT+01:00 Matthias BUSSONNIER <bussonniermatthias at gmail.com>: > Hi all, > Le 19 janv. 2015 ? 07:50, Soumith Chintala a ?crit : > > This is fantastic! I was going to do some of this work myself over the next > month for iTorch (https://github.com/facebook/iTorch), and you saved me some > trouble. > I will take a look at your code over the next couple of days, I am wondering > if you would have any issues if I directly adapted your javascript side of > things in iTorch rather than rewriting from scratch. > > > We still haven't really figured it out how to share javascript in between > packages of different languages. > (well technically in 3.0 yo can install into $NBEXTENSION directory, but > install process is not perfect) > If you can figured that out, and that all kernels agree on a set of > relatively common JS "plugins" that > are often used we might be able to work on making theses plugin available > on nbviewer. > > Long term planning of course, but I'd like people to keep that in mind. > > -- > M > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From takowl at gmail.com Mon Jan 19 12:54:54 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 19 Jan 2015 09:54:54 -0800 Subject: [IPython-dev] status of nbconvert In-Reply-To: <54BCF987.3030501@gmail.com> References: <54BCE9C8.6080304@gmail.com> <467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com> <54BCED3D.8040705@gmail.com> <76A22C20-FFC4-4B59-A06D-F144FCF863F2@gmail.com> <54BCF987.3030501@gmail.com> Message-ID: <CAOvn4qiyETucNaNiZ9M5hJGMmkax=cur9NaJtuccx-ck5ZE1Dw@mail.gmail.com> On 19 January 2015 at 04:33, Zolt?n V?r?s <zvoros at gmail.com> wrote: > I have installed the source, and I no longer have problems with > nbconvert. Is there someone whom I could notify to fix pip/easy_install? > I have absolutely no idea how that is managed. I have seen that Thomas > Kluyver is on the development team. Thomas? > No ideas, sorry. My pip found mistune 0.5 OK. I wouldn't use easy_install for anything now. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/8d8a4741/attachment.html> From andy at payne.org Mon Jan 19 14:30:30 2015 From: andy at payne.org (Andrew Payne) Date: Mon, 19 Jan 2015 14:30:30 -0500 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js In-Reply-To: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> Message-ID: <CABbuyuVmV3ymZkYFUcbKCdPh_KhpbAKJW5S8_oguhTyPfomUvQ@mail.gmail.com> > I recently managed to get 3D vector plots working in the IPython notebook, > by statically producing three.js output. I thought I'd share it here in > case this example is useful for anyone else, or if anyone has any > suggestions as to how I could do things better. > Great work! Thanks for sharing that. Did you consider or explore using require.js to load the three.js library? You can reference and load the library once, and then not have to inline it within each cell. Aside for folks fiddling with Javscript libraries & the Web notebook: the obvious approach of dropping a <script> reference in an HTML cell or result isn't robust. When the cell is reexecuted, the browser will load the library again, overwriting the original library name space. However, the first instance of the library will likely still be there because of lingering references. So you'll have a variable "foo" of type X in the old library instance, and you'll pass it into a function call (new library instance) and things will fail in strange ways because all the type tests in the new library instance fail. And you will drive yourself crazy because you'll be in the JS console and wonder why foo is clearly of type "X", but "isinstance X" is giving False. That's why David's JS code has this test: https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js#L175 if ( camera instanceof THREE.Camera === false || ! document.body.contains(three_container)) { console.log("Animation loop failed: stopping"); return; } > > > The advantage of the static approach is that the output survives the > nbconvert process, which you can see in this notebook here (which also > compares with a couple of other unsatisfactory ways of visualising 3D > vector plots). > > > http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%20to%20create%203D%20plots.ipynb > > Another example can be seen at: > > > http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%20and%20creating%20geometric%20shapes.ipynb > > If you are interested to see how this works, you can check out the python > source: > > https://github.com/DavidPowell/OpenModes/blob/master/openmodes/ipython.py > > As well as the javascript: > > > https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js > > and finally the HTML: > > > https://github.com/DavidPowell/OpenModes/blob/master/openmodes/templates/three_js_plot.html > > By the way, I am aware of the efforts to make a proper three.js IPython > widget (https://github.com/jasongrout/pythreejs/), and will probably > switch over to this once it is stable, and when the output can survive > ipython's nbconvert process. Is this something we can expect for widgets in > IPython 3.0? > > regards > David > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/09199558/attachment.html> From andy at payne.org Mon Jan 19 14:39:18 2015 From: andy at payne.org (Andrew Payne) Date: Mon, 19 Jan 2015 14:39:18 -0500 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js In-Reply-To: <CABbuyuVmV3ymZkYFUcbKCdPh_KhpbAKJW5S8_oguhTyPfomUvQ@mail.gmail.com> References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> <CABbuyuVmV3ymZkYFUcbKCdPh_KhpbAKJW5S8_oguhTyPfomUvQ@mail.gmail.com> Message-ID: <CABbuyuVNKj57qYahiMa9szNft3TQeTD9=gqs_xU+BKfr+McnsQ@mail.gmail.com> [*clicked send accidentally, please ignore my previous incomplete email*] I recently managed to get 3D vector plots working in the IPython notebook, > by statically producing three.js output. I thought I'd share it here in > case this example is useful for anyone else, or if anyone has any > suggestions as to how I could do things better. > Great work! Thanks for sharing that. Did you consider or explore using require.js to load the three.js library? You can reference and load the library once, and then not have to inline it within each cell. Aside for folks fiddling with Javscript libraries & the Web notebook: the obvious approach of dropping a <script> reference in an HTML cell or result isn't robust. When the cell is reexecuted, the browser will load the library again, overwriting the original library name space. However, the first instance of the library will likely still be there because of lingering references. So you'll have a variable "foo" of type X in the old library instance, and you'll pass it into a function call (new library instance) and things will fail in strange ways because all the type tests in the new library instance fail. And you will drive yourself crazy because you'll be in the JS console and wonder why foo is clearly of type "X", but "isinstanceof X" is giving False. That's why David's JS code has this test: https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js#L175 if ( *camera instanceof THREE.Camera === false* || ! document.body.contains(three_container)) { console.log("Animation loop failed: stopping"); return; } See my example of using require.js for three.js, here: http://nbviewer.ipython.org/github/payne92/notebooks/blob/master/00%20Javascript%20In%20Notebooks.ipynb For the dev team, this raises the issue of: what's the best idiom for JS in a cell to access the output area? (See SO question: http://stackoverflow.com/questions/20019690/how-to-access-current-cell-output-area-with-javascript-in-ipython-web-notebook ) On a "live" notebook, the Javascript *element *variable is fine. But as you can see with my notebook (above) vs David's, the element variable approach doesn't survive nbconvert -- my Javascript examples break because element doesn't exist in an nbconvert(ed) notebook. David made things work by generating (effectively) a UID as part of his HTML cell result that the corresponding Javascript can pick up. As we see Javascript used more with the Web notebook, folks are going to consistently run into this pattern. I think the notebook could provide some "standard" way to reference the output area for the cell, that survives nbconvert. What do you think? -andy -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/a0284f19/attachment.html> From antgonza at gmail.com Mon Jan 19 16:49:31 2015 From: antgonza at gmail.com (=?UTF-8?Q?Antonio_Gonz=C3=A1lez_Pe=C3=B1a?=) Date: Mon, 19 Jan 2015 14:49:31 -0700 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js In-Reply-To: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> Message-ID: <CAH4xwWpfzr14uUteNYmLg9G3ayy9MvkUqUN9ZXat5hfwdOE4CA@mail.gmail.com> Hello, Related to this topic. We maintain Emperor (http://biocore.github.io/emperor/), a package that uses python to create very simple THREE.js code that can be embedded as a web page to visualize and manipulate scatter plots + their metadata. We have successfully been able to include Emperor plots into IPython notebooks using _html_repr_. Note that in this case resources are stored in a server, but we should really be making use of the $NBEXTENSION. Anyway, as a side note we encountered a small issue with nbviewer+Emperor: https://github.com/jupyter/nbviewer/issues/316, which you perhaps might encounter. Cheers, On Sun, Jan 18, 2015 at 11:47 PM, David Powell <DavidAnthonyPowell+python at gmail.com> wrote: > Hello all, > > I recently managed to get 3D vector plots working in the IPython notebook, > by statically producing three.js output. I thought I'd share it here in case > this example is useful for anyone else, or if anyone has any suggestions as > to how I could do things better. > > The advantage of the static approach is that the output survives the > nbconvert process, which you can see in this notebook here (which also > compares with a couple of other unsatisfactory ways of visualising 3D vector > plots). > > http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%20to%20create%203D%20plots.ipynb > > Another example can be seen at: > > http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%20and%20creating%20geometric%20shapes.ipynb > > If you are interested to see how this works, you can check out the python > source: > > https://github.com/DavidPowell/OpenModes/blob/master/openmodes/ipython.py > > As well as the javascript: > > https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js > > and finally the HTML: > > https://github.com/DavidPowell/OpenModes/blob/master/openmodes/templates/three_js_plot.html > > By the way, I am aware of the efforts to make a proper three.js IPython > widget (https://github.com/jasongrout/pythreejs/), and will probably switch > over to this once it is stable, and when the output can survive ipython's > nbconvert process. Is this something we can expect for widgets in IPython > 3.0? > > regards > David > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Antonio From DavidAnthonyPowell+python at gmail.com Mon Jan 19 19:47:36 2015 From: DavidAnthonyPowell+python at gmail.com (David Powell) Date: Tue, 20 Jan 2015 11:47:36 +1100 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js In-Reply-To: <D0E1E8D1.3E2F%soumith@fb.com> References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> <D0E1E8D1.3E2F%soumith@fb.com> Message-ID: <CAGXzvidyYpqu2wYL721jdDCFOnoLz_PDB=yfdJpV_9nph=7jeQ@mail.gmail.com> Hello Soumith, My code is GPL3, but if you tell me which files you want to use, I will release them under a BSD license to be compatible with itorch. Regards, David This is fantastic! I was going to do some of this work myself over the next month for iTorch (https://github.com/facebook/iTorch), and you saved me some trouble. I will take a look at your code over the next couple of days, I am wondering if you would have any issues if I directly adapted your javascript side of things in iTorch rather than rewriting from scratch. ? Soumith From: David Powell <DavidAnthonyPowell+python at gmail.com> Reply-To: IPython developers list <ipython-dev at scipy.org> Date: Sunday, January 18, 2015 at 10:47 PM To: IPython developers list <ipython-dev at scipy.org> Subject: [IPython-dev] WebGL example in IPython notebook based on three.js Hello all, I recently managed to get 3D vector plots working in the IPython notebook, by statically producing three.js output. I thought I'd share it here in case this example is useful for anyone else, or if anyone has any suggestions as to how I could do things better. The advantage of the static approach is that the output survives the nbconvert process, which you can see in this notebook here (which also compares with a couple of other unsatisfactory ways of visualising 3D vector plots). http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%20to%20create%203D%20plots.ipynb <https://urldefense.proofpoint.com/v1/url?u=http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%2520to%2520create%25203D%2520plots.ipynb&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=5S5uukCkPxLR1dZ0syN0fw%3D%3D%0A&m=70WnYrYAO8zL773kKr3zku5qmdNjduxrZK8pOaiqCaE%3D%0A&s=aad5ead78a1b3391d5b9c2a10e1214ef17fd7df8d223ad6076c6b78f9ff1246c> Another example can be seen at: http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%20and%20creating%20geometric%20shapes.ipynb <https://urldefense.proofpoint.com/v1/url?u=http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%2520and%2520creating%2520geometric%2520shapes.ipynb&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=5S5uukCkPxLR1dZ0syN0fw%3D%3D%0A&m=70WnYrYAO8zL773kKr3zku5qmdNjduxrZK8pOaiqCaE%3D%0A&s=76948f7a6b29f01f0eccbe0899c7d91d427a1c1588d11ecbc4e568f992fddea7> If you are interested to see how this works, you can check out the python source: https://github.com/DavidPowell/OpenModes/blob/master/openmodes/ipython.py As well as the javascript: https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js and finally the HTML: https://github.com/DavidPowell/OpenModes/blob/master/openmodes/templates/three_js_plot.html By the way, I am aware of the efforts to make a proper three.js IPython widget (https://github.com/jasongrout/pythreejs/), and will probably switch over to this once it is stable, and when the output can survive ipython's nbconvert process. Is this something we can expect for widgets in IPython 3.0? regards David _______________________________________________ IPython-dev mailing list IPython-dev at scipy.org http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150120/e96afd4a/attachment.html> From DavidAnthonyPowell+python at gmail.com Mon Jan 19 21:17:09 2015 From: DavidAnthonyPowell+python at gmail.com (David Powell) Date: Tue, 20 Jan 2015 13:17:09 +1100 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js In-Reply-To: <CABbuyuVNKj57qYahiMa9szNft3TQeTD9=gqs_xU+BKfr+McnsQ@mail.gmail.com> References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> <CABbuyuVmV3ymZkYFUcbKCdPh_KhpbAKJW5S8_oguhTyPfomUvQ@mail.gmail.com> <CABbuyuVNKj57qYahiMa9szNft3TQeTD9=gqs_xU+BKfr+McnsQ@mail.gmail.com> Message-ID: <CAGXzvievJLrsGPXV33AqwEOF=BrkM9wOV_D_A3CBugMNy=U8pA@mail.gmail.com> > > Did you consider or explore using require.js to load the three.js > library? You can reference and load the library once, and then not have to > inline it within each cell. > > My reason for not using require is because I need parts of three.js (e.g orbit controls) which are not part of the main library, but are instead included as examples, and hence are not hosted on CDNs. Also, my code for injecting javascript libraries is in a separate function which only needs to be called once at the start of the notebook, so it shouldn't need to be included more than once. > Aside for folks fiddling with Javscript libraries & the Web notebook: > the obvious approach of dropping a <script> reference in an HTML cell or > result isn't robust. When the cell is reexecuted, the browser will load > the library again, overwriting the original library name space. However, > the first instance of the library will likely still be there because of > lingering references. So you'll have a variable "foo" of type X in the old > library instance, and you'll pass it into a function call (new library > instance) and things will fail in strange ways because all the type tests > in the new library instance fail. And you will drive yourself crazy > because you'll be in the JS console and wonder why foo is clearly of type > "X", but "isinstanceof X" is giving False. > > That's why David's JS code has this test: > > > https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js#L175 > > if ( *camera instanceof THREE.Camera === false* || ! > document.body.contains(three_container)) { > console.log("Animation loop failed: stopping"); > return; > } > > I agree that it's a horrible hack, I'm definitely looking for a more elegant long-term solution. This hack here isn't strictly needed for the code to work, but if it's missing, then the browser's javascript console will spew out error messages 60 times per second, which makes debugging javascript essentially impossible. I think I minimise some of the problems you describe by injecting the javascript libraries once at the start, rather than with every plot. > See my example of using require.js for three.js, here: > http://nbviewer.ipython.org/github/payne92/notebooks/blob/master/00%20Javascript%20In%20Notebooks.ipynb > > I did find your examples online, but had real trouble getting your approach to work with the additional javascript from the three.js examples. So in the end I just went back to this simpler approach of directly injecting code. > > For the dev team, this raises the issue of: what's the best idiom for JS > in a cell to access the output area? (See SO question: > http://stackoverflow.com/questions/20019690/how-to-access-current-cell-output-area-with-javascript-in-ipython-web-notebook > ) > > On a "live" notebook, the Javascript *element *variable is fine. But as > you can see with my notebook (above) vs David's, the element variable > approach doesn't survive nbconvert -- my Javascript examples break because > element doesn't exist in an nbconvert(ed) notebook. > > David made things work by generating (effectively) a UID as part of his > HTML cell result that the corresponding Javascript can pick up. > > As we see Javascript used more with the Web notebook, folks are going to > consistently run into this pattern. I think the notebook could provide > some "standard" way to reference the output area for the cell, that > survives nbconvert. > I agree, you've exactly described the problems I ran into, and the reason why I generate html with a <script> tag rather than javascript output. > > What do you think? > > -andy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150120/763e1e1e/attachment.html> From DavidAnthonyPowell+python at gmail.com Mon Jan 19 21:44:28 2015 From: DavidAnthonyPowell+python at gmail.com (David Powell) Date: Tue, 20 Jan 2015 13:44:28 +1100 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js In-Reply-To: <CA+-1RQTj_=3qGri9n+HKtvjmFWgWOLtGDjxeQZSXsLVEkrme8g@mail.gmail.com> References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> <D0E1E8D1.3E2F%soumith@fb.com> <AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com> <CA+-1RQTj_=3qGri9n+HKtvjmFWgWOLtGDjxeQZSXsLVEkrme8g@mail.gmail.com> Message-ID: <CAGXzvienQNyrbSENJzKt2N2vzp=iW6SkAPL=8v3WSt7jUC5AOQ@mail.gmail.com> Hello Cyrille, Thanks for the info. In case it helps you, the reasons why I didn't go with vispy are - When I started this coding, the IPython notebook examples for vispy weren't readily available (I vagely remember failing to find them with google even though something was mentioned on the home page) - The idea that you *have* to write low-level GLSL code seems a bit intimidating (with three.js you can, but don't need to) - three.js includes nice controls to pan, zoom etc. out of the box - Because I'm trying to attract users to my software, I think it's absolutely essential to be able to use the nbconverted output to show nice interactive plots to potential users who haven't installed my program yet (and who may not even be familiar with the scipy ecosystem), as a kind of "marketing" tool. However, I'll definitely keep an eye out for progress on vispy, it sounds like some cool features are planned. The one thing I don't like about three.js is that it is very much fixed around the idea of always animating at 60 frames/sec for games etc. This means that it is chewing up significant CPU, even if the figure if not updating. regards David On 19 January 2015 at 23:30, Cyrille Rossant <cyrille.rossant at gmail.com> wrote: > Thanks for sharing this! > > I'd like to add that there's also work being done currently for > bringing WebGL visualization to the notebook via VisPy. We don't use > three.js but custom WebGL code generated by Python. The WebGL backend > currently requires a live Python server, but a standalone version is > also in the pipes. > > 2015-01-19 12:08 GMT+01:00 Matthias BUSSONNIER < > bussonniermatthias at gmail.com>: > > Hi all, > > Le 19 janv. 2015 ? 07:50, Soumith Chintala a ?crit : > > > > This is fantastic! I was going to do some of this work myself over the > next > > month for iTorch (https://github.com/facebook/iTorch), and you saved me > some > > trouble. > > I will take a look at your code over the next couple of days, I am > wondering > > if you would have any issues if I directly adapted your javascript side > of > > things in iTorch rather than rewriting from scratch. > > > > > > We still haven't really figured it out how to share javascript in between > > packages of different languages. > > (well technically in 3.0 yo can install into $NBEXTENSION directory, but > > install process is not perfect) > > If you can figured that out, and that all kernels agree on a set of > > relatively common JS "plugins" that > > are often used we might be able to work on making theses plugin > available > > on nbviewer. > > > > Long term planning of course, but I'd like people to keep that in mind. > > > > -- > > M > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150120/8ee50390/attachment.html> From sylvain.corlay at gmail.com Mon Jan 19 22:05:12 2015 From: sylvain.corlay at gmail.com (Sylvain Corlay) Date: Mon, 19 Jan 2015 22:05:12 -0500 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js In-Reply-To: <CAGXzvienQNyrbSENJzKt2N2vzp=iW6SkAPL=8v3WSt7jUC5AOQ@mail.gmail.com> References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> <D0E1E8D1.3E2F%soumith@fb.com> <AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com> <CA+-1RQTj_=3qGri9n+HKtvjmFWgWOLtGDjxeQZSXsLVEkrme8g@mail.gmail.com> <CAGXzvienQNyrbSENJzKt2N2vzp=iW6SkAPL=8v3WSt7jUC5AOQ@mail.gmail.com> Message-ID: <CAK=Phk7ugWXRC13A4xXGjKVZTF8RsaC-XOCYLezhWo=+-kVPbg@mail.gmail.com> @David, PyThreejs works pretty well with the current dev version of IPython (3.0dev). Older versions should not be supported as it uses the new API with the javascript promises. (Ping Jason Grout). Sylvain On Mon, Jan 19, 2015 at 9:44 PM, David Powell < DavidAnthonyPowell+python at gmail.com> wrote: > Hello Cyrille, > > Thanks for the info. In case it helps you, the reasons why I didn't go > with vispy are > - When I started this coding, the IPython notebook examples for vispy > weren't readily available (I vagely remember failing to find them with > google even though something was mentioned on the home page) > - The idea that you *have* to write low-level GLSL code seems a bit > intimidating (with three.js you can, but don't need to) > - three.js includes nice controls to pan, zoom etc. out of the box > - Because I'm trying to attract users to my software, I think it's > absolutely essential to be able to use the nbconverted output to show nice > interactive plots to potential users who haven't installed my program yet > (and who may not even be familiar with the scipy ecosystem), as a kind of > "marketing" tool. > > However, I'll definitely keep an eye out for progress on vispy, it sounds > like some cool features are planned. The one thing I don't like about > three.js is that it is very much fixed around the idea of always animating > at 60 frames/sec for games etc. This means that it is chewing up > significant CPU, even if the figure if not updating. > > regards > David > > On 19 January 2015 at 23:30, Cyrille Rossant <cyrille.rossant at gmail.com> > wrote: > >> Thanks for sharing this! >> >> I'd like to add that there's also work being done currently for >> bringing WebGL visualization to the notebook via VisPy. We don't use >> three.js but custom WebGL code generated by Python. The WebGL backend >> currently requires a live Python server, but a standalone version is >> also in the pipes. >> >> 2015-01-19 12:08 GMT+01:00 Matthias BUSSONNIER < >> bussonniermatthias at gmail.com>: >> > Hi all, >> > Le 19 janv. 2015 ? 07:50, Soumith Chintala a ?crit : >> > >> > This is fantastic! I was going to do some of this work myself over the >> next >> > month for iTorch (https://github.com/facebook/iTorch), and you saved >> me some >> > trouble. >> > I will take a look at your code over the next couple of days, I am >> wondering >> > if you would have any issues if I directly adapted your javascript side >> of >> > things in iTorch rather than rewriting from scratch. >> > >> > >> > We still haven't really figured it out how to share javascript in >> between >> > packages of different languages. >> > (well technically in 3.0 yo can install into $NBEXTENSION directory, but >> > install process is not perfect) >> > If you can figured that out, and that all kernels agree on a set of >> > relatively common JS "plugins" that >> > are often used we might be able to work on making theses plugin >> available >> > on nbviewer. >> > >> > Long term planning of course, but I'd like people to keep that in mind. >> > >> > -- >> > M >> > >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/35521b2f/attachment.html> From cyrille.rossant at gmail.com Tue Jan 20 04:09:08 2015 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Tue, 20 Jan 2015 10:09:08 +0100 Subject: [IPython-dev] WebGL example in IPython notebook based on three.js In-Reply-To: <CAGXzvienQNyrbSENJzKt2N2vzp=iW6SkAPL=8v3WSt7jUC5AOQ@mail.gmail.com> References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com> <D0E1E8D1.3E2F%soumith@fb.com> <AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com> <CA+-1RQTj_=3qGri9n+HKtvjmFWgWOLtGDjxeQZSXsLVEkrme8g@mail.gmail.com> <CAGXzvienQNyrbSENJzKt2N2vzp=iW6SkAPL=8v3WSt7jUC5AOQ@mail.gmail.com> Message-ID: <CA+-1RQSYZwpnFfe7JYzWue5hPhSkWAC33VHgFQgLDRb0faMCFg@mail.gmail.com> Thank you for your feedback. VisPy/WebGL is definitely not yet ready for production, I was essentially mentioning that there was work in progress in that direction! > - When I started this coding, the IPython notebook examples for vispy > weren't readily available (I vagely remember failing to find them with > google even though something was mentioned on the home page) It's still experimental at this point, but a more robust version will be available in the coming weeks. > - The idea that you *have* to write low-level GLSL code seems a bit > intimidating (with three.js you can, but don't need to) That's actually not the case in the latest version, see for example <http://vispy.org/examples/basics/scene/surface_plot.html>. We do have high-level interfaces that don't require users to write GLSL. Only the most advanced users will be interested in writing custom GLSL. > - three.js includes nice controls to pan, zoom etc. out of the box Idem, in VisPy we do have interactive "cameras" that provide controls to pan, zoom, etc. > - Because I'm trying to attract users to my software, I think it's > absolutely essential to be able to use the nbconverted output to show nice > interactive plots to potential users who haven't installed my program yet > (and who may not even be familiar with the scipy ecosystem), as a kind of > "marketing" tool. That's a good point and that's definitely something on our radar. > However, I'll definitely keep an eye out for progress on vispy, it sounds > like some cool features are planned. The one thing I don't like about > three.js is that it is very much fixed around the idea of always animating > at 60 frames/sec for games etc. This means that it is chewing up significant > CPU, even if the figure if not updating. VisPy is indeed more efficient in that the scene is only updated when needed (pan, zoom, etc.). But you can also have timer-based animations. Cyrille From hughesadam87 at gmail.com Wed Jan 21 17:07:55 2015 From: hughesadam87 at gmail.com (Adam Hughes) Date: Wed, 21 Jan 2015 17:07:55 -0500 Subject: [IPython-dev] Replacing PopUp? Message-ID: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com> Hi, We have a widget GUI that was previously using Popup, and I noticed it's been deleted from dev. We're kind of stuck now in a weird point between 2.x and master, and it seems like everytime we update ipython, a new widget component is changed. So my question is, can we still get Popup behavior in our widgets? IE can I go through the code and make a simple fix to retain the old functionality of the Popups? And secondly, are there a lot more major changes to the widget system planned? If so, we'll probably bundle our working version if IPython rather than try to maintain compatibility until the official 3.x release. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150121/9bdb416d/attachment.html> From ellisonbg at gmail.com Wed Jan 21 17:19:16 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 21 Jan 2015 14:19:16 -0800 Subject: [IPython-dev] Replacing PopUp? In-Reply-To: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com> References: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com> Message-ID: <CAH4pYpShRHR4FTdDe0gQ9mU8w+STCu+K28LOKz85a84N76uSoA@mail.gmail.com> Adam, We apologize for all of the instability. We consider the IPython widgets to be research grade code for now and there will continue to be API and architecture changes for a few releases to come. For 3.0 we have added a warning upon import that clarifies this aspect of widgets. Moving forward, there will eventually be something that provides popup-like capabilities for widgets, but it was causing a lots of problems and the implementation had usability/UI problems for users. But, you should be able to take the popup widget out of ipython 2.0 and get it working with 3.0 when it comes out as a third party widget. Cheers, Brian On Wed, Jan 21, 2015 at 2:07 PM, Adam Hughes <hughesadam87 at gmail.com> wrote: > Hi, > > We have a widget GUI that was previously using Popup, and I noticed it's > been deleted from dev. We're kind of stuck now in a weird point between > 2.x and master, and it seems like everytime we update ipython, a new widget > component is changed. So my question is, can we still get Popup behavior > in our widgets? IE can I go through the code and make a simple fix to > retain the old functionality of the Popups? And secondly, are there a lot > more major changes to the widget system planned? If so, we'll probably > bundle our working version if IPython rather than try to maintain > compatibility until the official 3.x release. > > Thanks > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150121/887e2862/attachment.html> From hughesadam87 at gmail.com Wed Jan 21 20:21:27 2015 From: hughesadam87 at gmail.com (Adam Hughes) Date: Wed, 21 Jan 2015 20:21:27 -0500 Subject: [IPython-dev] Replacing PopUp? In-Reply-To: <CAH4pYpShRHR4FTdDe0gQ9mU8w+STCu+K28LOKz85a84N76uSoA@mail.gmail.com> References: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com> <CAH4pYpShRHR4FTdDe0gQ9mU8w+STCu+K28LOKz85a84N76uSoA@mail.gmail.com> Message-ID: <CAMHV+dCz-0NF-PQBU9KcHRn8k=cxPmnHeykzFWeNKQzH15djaA@mail.gmail.com> Thanks Brian. Turns out the Popup, while imported, was not even used by our program. The new notebook looks awesome by the way. On Wed, Jan 21, 2015 at 5:19 PM, Brian Granger <ellisonbg at gmail.com> wrote: > Adam, > > We apologize for all of the instability. We consider the IPython widgets > to be research grade code for now and there will continue to be API and > architecture changes for a few releases to come. For 3.0 we have added a > warning upon import that clarifies this aspect of widgets. > > Moving forward, there will eventually be something that provides > popup-like capabilities for widgets, but it was causing a lots of problems > and the implementation had usability/UI problems for users. But, you should > be able to take the popup widget out of ipython 2.0 and get it working with > 3.0 when it comes out as a third party widget. > > Cheers, > > Brian > > On Wed, Jan 21, 2015 at 2:07 PM, Adam Hughes <hughesadam87 at gmail.com> > wrote: > >> Hi, >> >> We have a widget GUI that was previously using Popup, and I noticed it's >> been deleted from dev. We're kind of stuck now in a weird point between >> 2.x and master, and it seems like everytime we update ipython, a new widget >> component is changed. So my question is, can we still get Popup behavior >> in our widgets? IE can I go through the code and make a simple fix to >> retain the old functionality of the Popups? And secondly, are there a lot >> more major changes to the widget system planned? If so, we'll probably >> bundle our working version if IPython rather than try to maintain >> compatibility until the official 3.x release. >> >> Thanks >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150121/5ea46b21/attachment.html> From bussonniermatthias at gmail.com Thu Jan 22 04:20:07 2015 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Thu, 22 Jan 2015 10:20:07 +0100 Subject: [IPython-dev] Replacing PopUp? In-Reply-To: <CAMHV+dCz-0NF-PQBU9KcHRn8k=cxPmnHeykzFWeNKQzH15djaA@mail.gmail.com> References: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com> <CAH4pYpShRHR4FTdDe0gQ9mU8w+STCu+K28LOKz85a84N76uSoA@mail.gmail.com> <CAMHV+dCz-0NF-PQBU9KcHRn8k=cxPmnHeykzFWeNKQzH15djaA@mail.gmail.com> Message-ID: <3E532361-42C7-404A-BC2F-3DED7C18FEBD@gmail.com> Le 22 janv. 2015 ? 02:21, Adam Hughes a ?crit : > Thanks Brian. Turns out the Popup, while imported, was not even used by our program. The new notebook looks awesome by the way. Thanks, I want to precise that widgets will become a standalone repo at some point in the future, so could get a release schedule decoupled from IPython. -- M > > On Wed, Jan 21, 2015 at 5:19 PM, Brian Granger <ellisonbg at gmail.com> wrote: > Adam, > > We apologize for all of the instability. We consider the IPython widgets to be research grade code for now and there will continue to be API and architecture changes for a few releases to come. For 3.0 we have added a warning upon import that clarifies this aspect of widgets. > > Moving forward, there will eventually be something that provides popup-like capabilities for widgets, but it was causing a lots of problems and the implementation had usability/UI problems for users. But, you should be able to take the popup widget out of ipython 2.0 and get it working with 3.0 when it comes out as a third party widget. > > Cheers, > > Brian > > On Wed, Jan 21, 2015 at 2:07 PM, Adam Hughes <hughesadam87 at gmail.com> wrote: > Hi, > > We have a widget GUI that was previously using Popup, and I noticed it's been deleted from dev. We're kind of stuck now in a weird point between 2.x and master, and it seems like everytime we update ipython, a new widget component is changed. So my question is, can we still get Popup behavior in our widgets? IE can I go through the code and make a simple fix to retain the old functionality of the Popups? And secondly, are there a lot more major changes to the widget system planned? If so, we'll probably bundle our working version if IPython rather than try to maintain compatibility until the official 3.x release. > > Thanks > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150122/d1f171a4/attachment.html> From hughesadam87 at gmail.com Thu Jan 22 14:06:28 2015 From: hughesadam87 at gmail.com (Adam Hughes) Date: Thu, 22 Jan 2015 14:06:28 -0500 Subject: [IPython-dev] Replacing PopUp? In-Reply-To: <3E532361-42C7-404A-BC2F-3DED7C18FEBD@gmail.com> References: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com> <CAH4pYpShRHR4FTdDe0gQ9mU8w+STCu+K28LOKz85a84N76uSoA@mail.gmail.com> <CAMHV+dCz-0NF-PQBU9KcHRn8k=cxPmnHeykzFWeNKQzH15djaA@mail.gmail.com> <3E532361-42C7-404A-BC2F-3DED7C18FEBD@gmail.com> Message-ID: <CAMHV+dB5EGbRctVXCDfYBf9V=jKhDEPYr3EA-H=43qi_i2KUDA@mail.gmail.com> Cool, thanks for the heads up On Thu, Jan 22, 2015 at 4:20 AM, Matthias BUSSONNIER < bussonniermatthias at gmail.com> wrote: > > Le 22 janv. 2015 ? 02:21, Adam Hughes a ?crit : > > Thanks Brian. Turns out the Popup, while imported, was not even used by > our program. The new notebook looks awesome by the way. > > > Thanks, > > I want to precise that widgets will become a standalone repo at some point > in the future, so could get a release schedule decoupled from IPython. > > -- > M > > > > > > On Wed, Jan 21, 2015 at 5:19 PM, Brian Granger <ellisonbg at gmail.com> > wrote: > >> Adam, >> >> We apologize for all of the instability. We consider the IPython widgets >> to be research grade code for now and there will continue to be API and >> architecture changes for a few releases to come. For 3.0 we have added a >> warning upon import that clarifies this aspect of widgets. >> >> Moving forward, there will eventually be something that provides >> popup-like capabilities for widgets, but it was causing a lots of problems >> and the implementation had usability/UI problems for users. But, you should >> be able to take the popup widget out of ipython 2.0 and get it working with >> 3.0 when it comes out as a third party widget. >> >> Cheers, >> >> Brian >> >> On Wed, Jan 21, 2015 at 2:07 PM, Adam Hughes <hughesadam87 at gmail.com> >> wrote: >> >>> Hi, >>> >>> We have a widget GUI that was previously using Popup, and I noticed it's >>> been deleted from dev. We're kind of stuck now in a weird point between >>> 2.x and master, and it seems like everytime we update ipython, a new widget >>> component is changed. So my question is, can we still get Popup behavior >>> in our widgets? IE can I go through the code and make a simple fix to >>> retain the old functionality of the Popups? And secondly, are there a lot >>> more major changes to the widget system planned? If so, we'll probably >>> bundle our working version if IPython rather than try to maintain >>> compatibility until the official 3.x release. >>> >>> Thanks >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> >> -- >> Brian E. Granger >> Cal Poly State University, San Luis Obispo >> @ellisonbg on Twitter and GitHub >> bgranger at calpoly.edu and ellisonbg at gmail.com >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150122/6db13f3e/attachment.html> From doug.blank at gmail.com Fri Jan 23 09:01:29 2015 From: doug.blank at gmail.com (Doug Blank) Date: Fri, 23 Jan 2015 09:01:29 -0500 Subject: [IPython-dev] Jupyter + Processing for education Message-ID: <CAAusYCjQYpYX5C62fjGrRHuUDaP5Wigt6CWj8=aLnumMyCnMEA@mail.gmail.com> IPython community, Thought you might be interested in seeing how students can use the new Jupyterhub with a new, external kernel. This semester I am using the language "Processing"... originally designed for artists by artists. My colleagues have written a textbook on how to use it for teaching computer science [1]. Here is a 5 minute video on how it looks to the student: https://www.youtube.com/watch?v=V4TzARh-ClY and here is a live notebook: http://jupyter.cs.brynmawr.edu/hub/kohara/public/Untitled.ipynb (mouse over some of the sketches) If you know Processing, this is different from the regular Processing IDE: * students log into the Jupyterhub system over the web * uses the Processing-Javascript system * each Jupyter notebook "cell" is a Processing "sketch" * Shift+Tab shows Processing on-line help * Tab will do completion (completes with Processing variables and functions) * Rendered notebooks (eg, without a live computational engine/kernel behind them) will still run * Slight bit of interpreter feel: can execute draw/setup functions interactively * uses MetaKernel magics [2] I like the setup for education because: * I can control/configure the environment exactly (what buttons are available, etc). I have a few nbextensions in the newly supported system folder, and load them in the notebook.html template. * I can update the system at any time * I can track students' time-on-task * Students will be able to submit their homework (as notebooks) via a button * I can add more stuff as I think of it (like the "Publish" and "Submit Homework" buttons) * All of the benefits of using a notebook: by students, for students ("runs like code; reads like a journal paper", "literate computing", "reproducible research", etc) Things I don't like: * Processing -> Javascript error messages are terrible We'll have all of the code easily installable (via pip) as soon as Jupyter is out. Of course, you can also get the code directly [3]. It could be that a Processing IPython magic could turn Processing code into a really fancy, low-level, dynamic widget system for Python. Because rendered Processing sketches run in rendered notebooks, it also solves some of those issues. There needs to be a way to get data to Processing. Also, for live kernels, it would be handy to have Processing be able to set Python variables. -Doug [1] - http://www.apress.com/9781430244646 [2] - https://github.com/Calysto/metakernel [3] - https://github.com/Calysto/calysto/tree/master/calysto/language/processing -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/add33c91/attachment.html> From python at elbonia.de Fri Jan 23 11:42:43 2015 From: python at elbonia.de (Juergen Hasch) Date: Fri, 23 Jan 2015 17:42:43 +0100 Subject: [IPython-dev] Jupyter + Processing for education In-Reply-To: <CAAusYCjQYpYX5C62fjGrRHuUDaP5Wigt6CWj8=aLnumMyCnMEA@mail.gmail.com> References: <CAAusYCjQYpYX5C62fjGrRHuUDaP5Wigt6CWj8=aLnumMyCnMEA@mail.gmail.com> Message-ID: <54C27A03.6080408@elbonia.de> Hey Doug, this is nice! I always wanted to something like that, but never got around. You can be sure I will play with it. Juergen Am 23.01.2015 um 15:01 schrieb Doug Blank: > IPython community, > > Thought you might be interested in seeing how students can use the new Jupyterhub with a new, external kernel. This > semester I am using the language "Processing"... originally designed for artists by artists. My colleagues have written > a textbook on how to use it for teaching computer science [1]. > > Here is a 5 minute video on how it looks to the student: > > https://www.youtube.com/watch?v=V4TzARh-ClY > > and here is a live notebook: > > http://jupyter.cs.brynmawr.edu/hub/kohara/public/Untitled.ipynb > > (mouse over some of the sketches) > > If you know Processing, this is different from the regular Processing IDE: > > * students log into the Jupyterhub system over the web > * uses the Processing-Javascript system > * each Jupyter notebook "cell" is a Processing "sketch" > * Shift+Tab shows Processing on-line help > * Tab will do completion (completes with Processing variables and functions) > * Rendered notebooks (eg, without a live computational engine/kernel behind them) will still run > * Slight bit of interpreter feel: can execute draw/setup functions interactively > * uses MetaKernel magics [2] > > I like the setup for education because: > > * I can control/configure the environment exactly (what buttons are available, etc). I have a few nbextensions in the > newly supported system folder, and load them in the notebook.html template. > * I can update the system at any time > * I can track students' time-on-task > * Students will be able to submit their homework (as notebooks) via a button > * I can add more stuff as I think of it (like the "Publish" and "Submit Homework" buttons) > * All of the benefits of using a notebook: by students, for students ("runs like code; reads like a journal paper", > "literate computing", "reproducible research", etc) > > Things I don't like: > > * Processing -> Javascript error messages are terrible > > We'll have all of the code easily installable (via pip) as soon as Jupyter is out. Of course, you can also get the code > directly [3]. > > It could be that a Processing IPython magic could turn Processing code into a really fancy, low-level, dynamic widget > system for Python. Because rendered Processing sketches run in rendered notebooks, it also solves some of those issues. > There needs to be a way to get data to Processing. Also, for live kernels, it would be handy to have Processing be able > to set Python variables. > > -Doug > > [1] - http://www.apress.com/9781430244646 > [2] - https://github.com/Calysto/metakernel > [3] - https://github.com/Calysto/calysto/tree/master/calysto/language/processing > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg at gmail.com Fri Jan 23 12:56:41 2015 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 23 Jan 2015 09:56:41 -0800 Subject: [IPython-dev] Jupyter + Processing for education In-Reply-To: <54C27A03.6080408@elbonia.de> References: <CAAusYCjQYpYX5C62fjGrRHuUDaP5Wigt6CWj8=aLnumMyCnMEA@mail.gmail.com> <54C27A03.6080408@elbonia.de> Message-ID: <CAH4pYpRd9XRwg+uixWVcSuPOje+VtVAtw7uYKtn-6trzm6E3eA@mail.gmail.com> Love it! On Fri, Jan 23, 2015 at 8:42 AM, Juergen Hasch <python at elbonia.de> wrote: > Hey Doug, > this is nice! > > I always wanted to something like that, but never got around. You can be > sure I will play with it. > > Juergen > > Am 23.01.2015 um 15:01 schrieb Doug Blank: > > IPython community, > > > > Thought you might be interested in seeing how students can use the new > Jupyterhub with a new, external kernel. This > > semester I am using the language "Processing"... originally designed for > artists by artists. My colleagues have written > > a textbook on how to use it for teaching computer science [1]. > > > > Here is a 5 minute video on how it looks to the student: > > > > https://www.youtube.com/watch?v=V4TzARh-ClY > > > > and here is a live notebook: > > > > http://jupyter.cs.brynmawr.edu/hub/kohara/public/Untitled.ipynb > > > > (mouse over some of the sketches) > > > > If you know Processing, this is different from the regular Processing > IDE: > > > > * students log into the Jupyterhub system over the web > > * uses the Processing-Javascript system > > * each Jupyter notebook "cell" is a Processing "sketch" > > * Shift+Tab shows Processing on-line help > > * Tab will do completion (completes with Processing variables and > functions) > > * Rendered notebooks (eg, without a live computational engine/kernel > behind them) will still run > > * Slight bit of interpreter feel: can execute draw/setup functions > interactively > > * uses MetaKernel magics [2] > > > > I like the setup for education because: > > > > * I can control/configure the environment exactly (what buttons are > available, etc). I have a few nbextensions in the > > newly supported system folder, and load them in the notebook.html > template. > > * I can update the system at any time > > * I can track students' time-on-task > > * Students will be able to submit their homework (as notebooks) via a > button > > * I can add more stuff as I think of it (like the "Publish" and "Submit > Homework" buttons) > > * All of the benefits of using a notebook: by students, for students > ("runs like code; reads like a journal paper", > > "literate computing", "reproducible research", etc) > > > > Things I don't like: > > > > * Processing -> Javascript error messages are terrible > > > > We'll have all of the code easily installable (via pip) as soon as > Jupyter is out. Of course, you can also get the code > > directly [3]. > > > > It could be that a Processing IPython magic could turn Processing code > into a really fancy, low-level, dynamic widget > > system for Python. Because rendered Processing sketches run in rendered > notebooks, it also solves some of those issues. > > There needs to be a way to get data to Processing. Also, for live > kernels, it would be handy to have Processing be able > > to set Python variables. > > > > -Doug > > > > [1] - http://www.apress.com/9781430244646 > > [2] - https://github.com/Calysto/metakernel > > [3] - > https://github.com/Calysto/calysto/tree/master/calysto/language/processing > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/0a99ceea/attachment.html> From doug.blank at gmail.com Fri Jan 23 13:40:14 2015 From: doug.blank at gmail.com (Doug Blank) Date: Fri, 23 Jan 2015 13:40:14 -0500 Subject: [IPython-dev] Jupyter + Processing for education In-Reply-To: <CAH4pYpRd9XRwg+uixWVcSuPOje+VtVAtw7uYKtn-6trzm6E3eA@mail.gmail.com> References: <CAAusYCjQYpYX5C62fjGrRHuUDaP5Wigt6CWj8=aLnumMyCnMEA@mail.gmail.com> <54C27A03.6080408@elbonia.de> <CAH4pYpRd9XRwg+uixWVcSuPOje+VtVAtw7uYKtn-6trzm6E3eA@mail.gmail.com> Message-ID: <CAAusYCjLvWjV8sONX8=KwZL1Z60pL5NRKYjmdW_F9677XapDGQ@mail.gmail.com> On Fri, Jan 23, 2015 at 12:56 PM, Brian Granger <ellisonbg at gmail.com> wrote: > Love it! > Thanks, guys! BTW, the background music is "Jupiter's Descendants" ... which is so punny and appropriate for the Calysto project :) Anticipating "Jupiter Ascending" due in theatres on Feb 9th... maybe you can have the official beta launch then... 2 weeks three days! -Doug [1] - https://soundcloud.com/victory/cinematic-layout-jupiters > > On Fri, Jan 23, 2015 at 8:42 AM, Juergen Hasch <python at elbonia.de> wrote: > >> Hey Doug, >> this is nice! >> >> I always wanted to something like that, but never got around. You can be >> sure I will play with it. >> >> Juergen >> >> Am 23.01.2015 um 15:01 schrieb Doug Blank: >> > IPython community, >> > >> > Thought you might be interested in seeing how students can use the new >> Jupyterhub with a new, external kernel. This >> > semester I am using the language "Processing"... originally designed >> for artists by artists. My colleagues have written >> > a textbook on how to use it for teaching computer science [1]. >> > >> > Here is a 5 minute video on how it looks to the student: >> > >> > https://www.youtube.com/watch?v=V4TzARh-ClY >> > >> > and here is a live notebook: >> > >> > http://jupyter.cs.brynmawr.edu/hub/kohara/public/Untitled.ipynb >> > >> > (mouse over some of the sketches) >> > >> > If you know Processing, this is different from the regular Processing >> IDE: >> > >> > * students log into the Jupyterhub system over the web >> > * uses the Processing-Javascript system >> > * each Jupyter notebook "cell" is a Processing "sketch" >> > * Shift+Tab shows Processing on-line help >> > * Tab will do completion (completes with Processing variables and >> functions) >> > * Rendered notebooks (eg, without a live computational engine/kernel >> behind them) will still run >> > * Slight bit of interpreter feel: can execute draw/setup functions >> interactively >> > * uses MetaKernel magics [2] >> > >> > I like the setup for education because: >> > >> > * I can control/configure the environment exactly (what buttons are >> available, etc). I have a few nbextensions in the >> > newly supported system folder, and load them in the notebook.html >> template. >> > * I can update the system at any time >> > * I can track students' time-on-task >> > * Students will be able to submit their homework (as notebooks) via a >> button >> > * I can add more stuff as I think of it (like the "Publish" and "Submit >> Homework" buttons) >> > * All of the benefits of using a notebook: by students, for students >> ("runs like code; reads like a journal paper", >> > "literate computing", "reproducible research", etc) >> > >> > Things I don't like: >> > >> > * Processing -> Javascript error messages are terrible >> > >> > We'll have all of the code easily installable (via pip) as soon as >> Jupyter is out. Of course, you can also get the code >> > directly [3]. >> > >> > It could be that a Processing IPython magic could turn Processing code >> into a really fancy, low-level, dynamic widget >> > system for Python. Because rendered Processing sketches run in rendered >> notebooks, it also solves some of those issues. >> > There needs to be a way to get data to Processing. Also, for live >> kernels, it would be handy to have Processing be able >> > to set Python variables. >> > >> > -Doug >> > >> > [1] - http://www.apress.com/9781430244646 >> > [2] - https://github.com/Calysto/metakernel >> > [3] - >> https://github.com/Calysto/calysto/tree/master/calysto/language/processing >> > >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/c41a2c17/attachment.html> From chris at plot.ly Fri Jan 23 14:03:52 2015 From: chris at plot.ly (Chris P) Date: Fri, 23 Jan 2015 14:03:52 -0500 Subject: [IPython-dev] Plotly Interactive Graph Widget Message-ID: <CADQk6vh9k=4PVYH1MBT47h=nULzntAnhSv6TSe_iHu5sQ0gSLQ@mail.gmail.com> Hey IPython Community -- I wanted to share with you Plotly's new IPython graph widget. A couple GIFs: Contour Plot Explorer <http://nbviewer.ipython.org/gist/chriddyp/9827cb5086cfab8da092/Widgets%20-%20Contour%20Plotter.ipynb> - The bottom graph updates with the elevation and distance of the points that are clicked on in the top-graph. (Link to GIF: http://i.imgur.com/2Wbbo6l.gif) Dataframe Explorer of NYC 311 calls <http://nbviewer.ipython.org/gist/chriddyp/9827cb5086cfab8da092/Pandas%20Widget%20-%20Time%20Series,%20Search,%20and%20Filtering.ipynb>: Zooming and panning in the top time-series graph updates the bottom graph. Updating the search, dropdown, and slider values perform Pandas calls and Plotly restyle commands. (Link to GIF: http://i.imgur.com/h5GLIHc.gif) This involved a couple of neat interactions with the Widget framework: - Events from our JS graphing library (hover, click, zoom) get passed to the Python kernel. Python callbacks can be assigned with e.g. GraphWidget.on_click(callback) - All the functions from our JS graphing lib are exposed in the Python GraphWidget: restyle, add_traces, delete_traces, reorder_traces, relayout - We're embedded these graphs as IFrames and all communication from the IPython JS client happens through postMessage. Since the GraphWidget is initialized with the unique URL of the graph, it's possible to swap graphs in and out of IPython widgets. For example, that Contour Plot Explorer <http://nbviewer.ipython.org/9827cb5086cfab8da092/Widgets%20-%20Contour%20Plotter.ipynb> will work with any Contour Plot, the user just has to replace contour_plot = GraphWidget('https://plot.ly/~bronsolo/63') with a URL of their own contour plot. The widget code is here: https://github.com/plotly/python-api/tree/master/plotly/widgets. Any feedback or suggestions is greatly appreciated! Some comments from the development POV: - JS tracebacks were really difficult to debug. - For easy use by the user, I decided to inject the JS dynamically on import <https://github.com/plotly/python-api/blob/master/plotly/widgets/graph_widget.py#L17-L22>. Adding //@ source <https://github.com/plotly/python-api/blob/master/plotly/widgets/graphWidget.js#L129> to the end of the injected JS made the JS code appear in Chrome's dev tools. Without //@ source, the dynamic JS was lost in a Chrome VM. - I'm using Unicode(sync=True) to pass messages back and forth, but really I felt like I should just be using the underlying comm interface instead of the MVC framework. But, I had trouble finding good documentation on that interface. - Looking forward to a "knob" widget fora ipython etch-a-sketch <http://nbviewer.ipython.org/gist/chriddyp/9827cb5086cfab8da092/Widgets%20-%20Sliders%20and%20Etch-A-Sketch.ipynb> ;) Here are a few notebooks <http://nbviewer.ipython.org/gist/chriddyp/9827cb5086cfab8da092> demonstrating the graph widget, including an overview <http://nbviewer.ipython.org/gist/chriddyp/9827cb5086cfab8da092>. Finally, thanks a bunch to the IPython team for all the great work with widgets! Chris -- Chris Parmer 514 571 5897 https://plot.ly/team -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/432d3ade/attachment.html> From edsrahn at gmail.com Fri Jan 23 14:12:51 2015 From: edsrahn at gmail.com (Ed Rahn) Date: Fri, 23 Jan 2015 14:12:51 -0500 Subject: [IPython-dev] setting up public ipython notebook server Message-ID: <CADFkHPpFpMggaPowcGbDTONGkJfJWYSXH4+CSLY0Mi_3FohS7w@mail.gmail.com> When I follow the instructions at http://ipython.org/ipython-doc/dev/notebook/public_server.html I get: Exception in callback (<socket._socketobject object at 0x7f422f0baf30>, <function null_wrapper at 0x7f422f07b578>) Traceback (most recent call last): File "/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/ioloop.py", line 837, in start handler_func(fd_obj, events) File "/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/stack_context.py", line 275, in null_wrapper return fn(*args, **kwargs) File "/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/netutil.py", line 220, in accept_handler callback(connection, address) File "/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/tcpserver.py", line 225, in _handle_connection do_handshake_on_connect=False) File "/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/netutil.py", line 453, in ssl_wrap_socket context = ssl_options_to_context(ssl_options) File "/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/netutil.py", line 430, in ssl_options_to_context context.load_cert_chain(ssl_options['certfile'], ssl_options.get('keyfile', None)) TypeError: coercing to Unicode: need string or buffer, NoneType found I try setting keyfile but then I get ssl errors during requests. Ed -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/39c8f831/attachment.html> From jmhanly at me.com Fri Jan 23 14:14:36 2015 From: jmhanly at me.com (John Hanly) Date: Fri, 23 Jan 2015 14:14:36 -0500 Subject: [IPython-dev] Plotly Interactive Graph Widget In-Reply-To: <CADQk6vh9k=4PVYH1MBT47h=nULzntAnhSv6TSe_iHu5sQ0gSLQ@mail.gmail.com> References: <CADQk6vh9k=4PVYH1MBT47h=nULzntAnhSv6TSe_iHu5sQ0gSLQ@mail.gmail.com> Message-ID: <3CC5A1CA-459D-4998-8EF6-50834129B82F@me.com> wonderful!! thanks. Go Nats! jmhanly at me.com On Jan 23, 2015, at 2:03 PM, Chris P <chris at plot.ly> wrote: > Hey IPython Community -- > > I wanted to share with you Plotly's new IPython graph widget. A couple GIFs: > > Contour Plot Explorer - The bottom graph updates with the elevation and distance of the points that are clicked on in the top-graph. > > (Link to GIF: http://i.imgur.com/2Wbbo6l.gif) > > > Dataframe Explorer of NYC 311 calls: Zooming and panning in the top time-series graph updates the bottom graph. Updating the search, dropdown, and slider values perform Pandas calls and Plotly restyle commands. > > > (Link to GIF: http://i.imgur.com/h5GLIHc.gif) > > > > This involved a couple of neat interactions with the Widget framework: > - Events from our JS graphing library (hover, click, zoom) get passed to the Python kernel. Python callbacks can be assigned with e.g. GraphWidget.on_click(callback) > > - All the functions from our JS graphing lib are exposed in the Python GraphWidget: restyle, add_traces, delete_traces, reorder_traces, relayout > > - We're embedded these graphs as IFrames and all communication from the IPython JS client happens through postMessage. Since the GraphWidget is initialized with the unique URL of the graph, it's possible to swap graphs in and out of IPython widgets. For example, that Contour Plot Explorer will work with any Contour Plot, the user just has to replace contour_plot = GraphWidget('https://plot.ly/~bronsolo/63') with a URL of their own contour plot. > > The widget code is here: https://github.com/plotly/python-api/tree/master/plotly/widgets. Any feedback or suggestions is greatly appreciated! > > Some comments from the development POV: > - JS tracebacks were really difficult to debug. > - For easy use by the user, I decided to inject the JS dynamically on import. Adding //@ source to the end of the injected JS made the JS code appear in Chrome's dev tools. Without //@ source, the dynamic JS was lost in a Chrome VM. > - I'm using Unicode(sync=True) to pass messages back and forth, but really I felt like I should just be using the underlying comm interface instead of the MVC framework. But, I had trouble finding good documentation on that interface. > - Looking forward to a "knob" widget fora ipython etch-a-sketch ;) > > Here are a few notebooks demonstrating the graph widget, including an overview. > > Finally, thanks a bunch to the IPython team for all the great work with widgets! > > Chris > > -- > Chris Parmer > 514 571 5897 > https://plot.ly/team > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/753088bb/attachment.html> From ssanderson at quantopian.com Fri Jan 23 14:38:53 2015 From: ssanderson at quantopian.com (ssanderson) Date: Fri, 23 Jan 2015 11:38:53 -0800 (PST) Subject: [IPython-dev] setting up public ipython notebook server In-Reply-To: <CADFkHPpFpMggaPowcGbDTONGkJfJWYSXH4+CSLY0Mi_3FohS7w@mail.gmail.com> References: <CADFkHPpFpMggaPowcGbDTONGkJfJWYSXH4+CSLY0Mi_3FohS7w@mail.gmail.com> Message-ID: <1422041933630-5084131.post@n6.nabble.com> Hi Ed, I believe the default instructions you linked create a .pem file that contains both an SSL cert and a private key together in a single file. You can pass the path to that file via --certfile, and I believe the underlying Tornado server does the right thing. For the SSL errors you're seeing when you make request, are you connecting the server over HTTPS? It's expected that you'd get errors when connecting over HTTP. - Scott -- View this message in context: http://python.6.x6.nabble.com/setting-up-public-ipython-notebook-server-tp5084129p5084131.html Sent from the IPython - Development mailing list archive at Nabble.com. From bussonniermatthias at gmail.com Sat Jan 24 16:02:12 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Sat, 24 Jan 2015 22:02:12 +0100 Subject: [IPython-dev] Plotly Interactive Graph Widget In-Reply-To: <CADQk6vh9k=4PVYH1MBT47h=nULzntAnhSv6TSe_iHu5sQ0gSLQ@mail.gmail.com> References: <CADQk6vh9k=4PVYH1MBT47h=nULzntAnhSv6TSe_iHu5sQ0gSLQ@mail.gmail.com> Message-ID: <AF48F7A9-835F-4013-8D06-5E6CF910D42E@gmail.com> Thanks a lot for the announce, We are really happy to see what you have build with widgets ! Please also let us know any other feedback you have. We are converging on IPython 3.0, and once the beta is released we will try to improve documentation. If you have any blog post that describe some of the workaround you did, or even want to add things to our docs, please feel free to do so. Thanks a lot for this new release of Plot.ly ! Cheers, -- Matthias Le 23 janv. 2015 ? 20:03, Chris P <chris at plot.ly> a ?crit : > Hey IPython Community -- > > I wanted to share with you Plotly's new IPython graph widget. A couple GIFs: > > Contour Plot Explorer - The bottom graph updates with the elevation and distance of the points that are clicked on in the top-graph. > > (Link to GIF: http://i.imgur.com/2Wbbo6l.gif) > > > Dataframe Explorer of NYC 311 calls: Zooming and panning in the top time-series graph updates the bottom graph. Updating the search, dropdown, and slider values perform Pandas calls and Plotly restyle commands. > > > (Link to GIF: http://i.imgur.com/h5GLIHc.gif) > > > > This involved a couple of neat interactions with the Widget framework: > - Events from our JS graphing library (hover, click, zoom) get passed to the Python kernel. Python callbacks can be assigned with e.g. GraphWidget.on_click(callback) > > - All the functions from our JS graphing lib are exposed in the Python GraphWidget: restyle, add_traces, delete_traces, reorder_traces, relayout > > - We're embedded these graphs as IFrames and all communication from the IPython JS client happens through postMessage. Since the GraphWidget is initialized with the unique URL of the graph, it's possible to swap graphs in and out of IPython widgets. For example, that Contour Plot Explorer will work with any Contour Plot, the user just has to replace contour_plot = GraphWidget('https://plot.ly/~bronsolo/63') with a URL of their own contour plot. > > The widget code is here: https://github.com/plotly/python-api/tree/master/plotly/widgets. Any feedback or suggestions is greatly appreciated! > > Some comments from the development POV: > - JS tracebacks were really difficult to debug. > - For easy use by the user, I decided to inject the JS dynamically on import. Adding //@ source to the end of the injected JS made the JS code appear in Chrome's dev tools. Without //@ source, the dynamic JS was lost in a Chrome VM. > - I'm using Unicode(sync=True) to pass messages back and forth, but really I felt like I should just be using the underlying comm interface instead of the MVC framework. But, I had trouble finding good documentation on that interface. > - Looking forward to a "knob" widget fora ipython etch-a-sketch ;) > > Here are a few notebooks demonstrating the graph widget, including an overview. > > Finally, thanks a bunch to the IPython team for all the great work with widgets! > > Chris > > -- > Chris Parmer > 514 571 5897 > https://plot.ly/team > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150124/c2d4cdb4/attachment.html> From zaki.mughal at gmail.com Sun Jan 25 13:26:42 2015 From: zaki.mughal at gmail.com (Zakariyya Mughal) Date: Sun, 25 Jan 2015 12:26:42 -0600 Subject: [IPython-dev] Execution semantics and how to set them Message-ID: <20150125182642.GA25051@quadra> Hello, I'm a language kernel author and I'm looking at the execution semantics document[^1] where I see a reference to the "single" execution mode. It says that the code for i in range(10): i**2 running in "single" mode would display 10 lines. I looked through the IPython source and I see that the execution semantics are set as an argument for `run_ast_nodes`[^2], but I see no way that the given for-loop would give multiple outputs unless the `interactivity` parameter is set to `last` or `all`. Is there a way to activate this execution mode through a front-end and pass that information to the back-end using the messaging protocol? Or should I just implement the logic at the end of the document ("The code field is split into individual blocks...") in my language back-end? Cheers, - Zaki Mughal [^1]: Execution semantics spec <http://ipython.org/ipython-doc/dev/development/execution.html#execution-semantics> [^2]: run_ast_nodes() <https://github.com/ipython/ipython/blob/d0fbe137/IPython/core/interactiveshell.py#L2919> From takowl at gmail.com Sun Jan 25 13:57:01 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 25 Jan 2015 10:57:01 -0800 Subject: [IPython-dev] Execution semantics and how to set them In-Reply-To: <20150125182642.GA25051@quadra> References: <20150125182642.GA25051@quadra> Message-ID: <CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com> Hi Zaki, On 25 January 2015 at 10:26, Zakariyya Mughal <zaki.mughal at gmail.com> wrote: > > I'm a language kernel author and I'm looking at the execution semantics > document[^1] where I see a reference to the "single" execution mode. > > It says that the code > > for i in range(10): > i**2 > > running in "single" mode would display 10 lines. > > I looked through the IPython source and I see that the execution > semantics are set as an argument for `run_ast_nodes`[^2], but I see no > way that the given for-loop would give multiple outputs unless the > `interactivity` parameter is set to `last` or `all`. First, let me emphasise that all of this is only relevant for executing Python code. If you're building a kernel for another language, you should ignore all of this. That documentation page is only about IPython's own kernel. The three modes described on that page (exec, eval, single) are compilation modes of Python itself, passed as arguments to the built-in compile() function. The parameter passed to our run_ast_nodes() method (which can be all, none, last or last_expr) controls which of a list of AST nodes are compiled in 'single' mode. Our default of last_expr is chosen specifically so that the for loop example above does not produce output, because we think that's more likely to be annoying than useful. There is no way to control this from the frontend, and I don't think we have any intention to add such a feature. For your own kernel, you should implement whatever display of output makes sense given the language and the likely users. For Python, we decided that it makes sense to automatically display the result of the last statement if that statement is an expression, and implemented that using the details described above. For R, I implemented similar behaviour based on the evaluate package by creating an output_handler with a value= parameter. Which language are you working on a kernel for? Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150125/2c39eb42/attachment.html> From bussonniermatthias at gmail.com Sun Jan 25 14:06:39 2015 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Sun, 25 Jan 2015 20:06:39 +0100 Subject: [IPython-dev] Execution semantics and how to set them In-Reply-To: <CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com> References: <20150125182642.GA25051@quadra> <CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com> Message-ID: <56F40087-43D3-4290-A681-568E072E1D66@gmail.com> Le 25 janv. 2015 ? 19:57, Thomas Kluyver a ?crit : > First, let me emphasise that all of this is only relevant for executing Python code. If you're building a kernel for another language, you should ignore all of this. That documentation page is only about IPython's own kernel. > > The three modes described on that page (exec, eval, single) are compilation modes of Python itself, passed as arguments to the built-in compile() function. > > The parameter passed to our run_ast_nodes() method (which can be all, none, last or last_expr) controls which of a list of AST nodes are compiled in 'single' mode. Our default of last_expr is chosen specifically so that the for loop example above does not produce output, because we think that's more likely to be annoying than useful. There is no way to control this from the frontend, and I don't think we have any intention to add such a feature. > > For your own kernel, you should implement whatever display of output makes sense given the language and the likely users. For Python, we decided that it makes sense to automatically display the result of the last statement if that statement is an expression, and implemented that using the details described above. For R, I implemented similar behaviour based on the evaluate package by creating an output_handler with a value= parameter. > > Which language are you working on a kernel for? Shot in the dark, helped by google IPerl: https://github.com/zmughal/p5-Devel-IPerl -- M -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150125/c844670a/attachment.html> From zaki.mughal at gmail.com Sun Jan 25 14:16:59 2015 From: zaki.mughal at gmail.com (Zakariyya Mughal) Date: Sun, 25 Jan 2015 13:16:59 -0600 Subject: [IPython-dev] Execution semantics and how to set them In-Reply-To: <CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com> References: <20150125182642.GA25051@quadra> <CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com> Message-ID: <20150125191658.GB25051@quadra> On 2015-01-25 at 10:57:01 -0800, Thomas Kluyver wrote: > Hi Zaki, > > On 25 January 2015 at 10:26, Zakariyya Mughal <zaki.mughal at gmail.com> wrote: > > > > > I'm a language kernel author and I'm looking at the execution semantics > > document[^1] where I see a reference to the "single" execution mode. > > > > It says that the code > > > > for i in range(10): > > i**2 > > > > running in "single" mode would display 10 lines. > > > > I looked through the IPython source and I see that the execution > > semantics are set as an argument for `run_ast_nodes`[^2], but I see no > > way that the given for-loop would give multiple outputs unless the > > `interactivity` parameter is set to `last` or `all`. > > > First, let me emphasise that all of this is only relevant for executing > Python code. If you're building a kernel for another language, you should > ignore all of this. That documentation page is only about IPython's own > kernel. > > The three modes described on that page (exec, eval, single) are compilation > modes of Python itself, passed as arguments to the built-in compile() > function. > > The parameter passed to our run_ast_nodes() method (which can be all, none, > last or last_expr) controls which of a list of AST nodes are compiled in > 'single' mode. Our default of last_expr is chosen specifically so that the > for loop example above does not produce output, because we think that's > more likely to be annoying than useful. There is no way to control this > from the frontend, and I don't think we have any intention to add such a > feature. > > For your own kernel, you should implement whatever display of output makes > sense given the language and the likely users. For Python, we decided that > it makes sense to automatically display the result of the last statement if > that statement is an expression, and implemented that using the details > described above. For R, I implemented similar behaviour based on the > evaluate package by creating an output_handler with a value= parameter. I see. It's easy enough to just call print if a user wants that. > Which language are you working on a kernel for? I wrote one for Perl <https://github.com/zmughal/p5-Devel-IPerl>. I released it back in August and I've been meaning to write a blog post and announce it more widely, but I first wanted to make sure it installed everywhere. Cheers, - Zaki Mughal > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From takowl at gmail.com Sun Jan 25 14:31:28 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 25 Jan 2015 11:31:28 -0800 Subject: [IPython-dev] Execution semantics and how to set them In-Reply-To: <20150125191658.GB25051@quadra> References: <20150125182642.GA25051@quadra> <CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com> <20150125191658.GB25051@quadra> Message-ID: <CAOvn4qhyS6Yh8zTn8VE6_DH-N2F24yi1OykY7gEBi_xH0MaEWw@mail.gmail.com> On 25 January 2015 at 11:16, Zakariyya Mughal <zaki.mughal at gmail.com> wrote: > I wrote one for Perl <https://github.com/zmughal/p5-Devel-IPerl>. > Great! I see it's already been added to the list of available kernels, as well. https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages Feel free to ask if you have any other questions we can help with. Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150125/50e611a0/attachment.html> From markbak at gmail.com Mon Jan 26 11:17:27 2015 From: markbak at gmail.com (Mark Bakker) Date: Mon, 26 Jan 2015 17:17:27 +0100 Subject: [IPython-dev] ginput plans for IPython Notebook with inline figures Message-ID: <CAEX=yabBw-Uj78hki=-6EFmUFh8zhrw7zMox5vOhTuUMuPvchQ@mail.gmail.com> Dear IPython List, Are there any plans for getting ginput to work when figures are inline in a Notebook? It would be so cool to have that working together with the other widget machinery. Or is this something the matplotlib developers should pick up? Thanks, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150126/edc144ed/attachment.html> From takowl at gmail.com Mon Jan 26 12:47:03 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 26 Jan 2015 09:47:03 -0800 Subject: [IPython-dev] ginput plans for IPython Notebook with inline figures In-Reply-To: <CAEX=yabBw-Uj78hki=-6EFmUFh8zhrw7zMox5vOhTuUMuPvchQ@mail.gmail.com> References: <CAEX=yabBw-Uj78hki=-6EFmUFh8zhrw7zMox5vOhTuUMuPvchQ@mail.gmail.com> Message-ID: <CAOvn4qizD2-tydoWhk-BTM5LOcOu7Sp_-c7TxsD7wCZ7K-gsiA@mail.gmail.com> On 26 January 2015 at 08:17, Mark Bakker <markbak at gmail.com> wrote: > Are there any plans for getting ginput to work when figures are inline in > a Notebook? Do you mean something equivalent to the Matlab ginput() function? http://www.mathworks.com/help/matlab/ref/ginput.html I'd say that looks like an issue for Matplotlib. I wouldn't be surprised if something similar is already possible using the nbagg backend. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150126/8a2c4430/attachment.html> From markbak at gmail.com Mon Jan 26 14:41:56 2015 From: markbak at gmail.com (Mark Bakker) Date: Mon, 26 Jan 2015 20:41:56 +0100 Subject: [IPython-dev] ginput plans for IPython Notebook with inline figures Message-ID: <CAEX=yaZQX5E_xZOOA=ME9t=NvYBzgQtpojsBRXqaHj9iaK+wTQ@mail.gmail.com> Unfortunately, ginput doesn't work yet in the nbagg backend. I'll mail the matplotlib list. Thanks, Mark On 26 January 2015 at 08:17, Mark Bakker <markbak at gmail.com> wrote: > Are there any plans for getting ginput to work when figures are inline in > a Notebook? Do you mean something equivalent to the Matlab ginput() function? http://www.mathworks.com/help/matlab/ref/ginput.html I'd say that looks like an issue for Matplotlib. I wouldn't be surprised if something similar is already possible using the nbagg backend. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150126/0c901d13/attachment.html> From maidos93 at laposte.net Wed Jan 28 07:40:25 2015 From: maidos93 at laposte.net (thwiouz) Date: Wed, 28 Jan 2015 04:40:25 -0800 (PST) Subject: [IPython-dev] IPython parallel and objects Message-ID: <1422448825555-5084497.post@n6.nabble.com> Hi guys, I'd like to parallelize the following loop: m1 = np.arange(0, 10, 100) m2 = np.arange(0, 10, 100) for m_ in m1: for n_2 in m2: tmp = optimizer.compute_expectation([m_, n_]) res.append(tmp) with type(optimizer) some class that I've defined. How could I handle it? I tried to do the same as in http://stackoverflow.com/questions/9363118/parallelise-nested-for-loop-in-ipython so I created a lambda function f f = lambda x, y: optimizer.compute_expectation([x, y]) but it's not working with a load balanced view, saying that optimizer is not defined. Could someone help? Thanks, -- View this message in context: http://python.6.x6.nabble.com/IPython-parallel-and-objects-tp5084497.html Sent from the IPython - Development mailing list archive at Nabble.com. From john at omernik.com Wed Jan 28 10:08:14 2015 From: john at omernik.com (John Omernik) Date: Wed, 28 Jan 2015 09:08:14 -0600 Subject: [IPython-dev] Display Function from with in a Function Message-ID: <CAKOFcwrkU4TG-KUHV=4JY7GJDc0d01An=0hwZjsgLY6vo32tMw@mail.gmail.com> Greetings all, I am using iPython notebook, I can do this: from IPython.display import display, HTML h = "<H1>Oh My </H1>" HTML(h) and get the expected result But if I do this: from IPython.display import display, HTML def runme(html): HTML(html) h = "<H1>Oh My </H1> runme(h) I get no results. I've noticed this is true for other displays as well, how can I add displays from withi functions? Thanks! John From cyrille.rossant at gmail.com Wed Jan 28 10:17:11 2015 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Wed, 28 Jan 2015 16:17:11 +0100 Subject: [IPython-dev] Display Function from with in a Function In-Reply-To: <CAKOFcwrkU4TG-KUHV=4JY7GJDc0d01An=0hwZjsgLY6vo32tMw@mail.gmail.com> References: <CAKOFcwrkU4TG-KUHV=4JY7GJDc0d01An=0hwZjsgLY6vo32tMw@mail.gmail.com> Message-ID: <CA+-1RQQ3PjM=0RJ8yOGj8RDH2-B5cPUALGFN91NjzpkE+uuGPg@mail.gmail.com> HTML(...) will only display something if it's the last statement of a code cell. This is because IPython always displays the result of the last result of a cell (unless you terminate the line with a semicolon ';'). A more reliable way of displaying stuff is to use display() or display_html(). This will work from within functions. See http://ipython.org/ipython-doc/dev/api/generated/IPython.display.html for more details. 2015-01-28 16:08 GMT+01:00 John Omernik <john at omernik.com>: > Greetings all, I am using iPython notebook, I can do this: > > from IPython.display import display, HTML > > h = "<H1>Oh My </H1>" > HTML(h) > > and get the expected result > > But if I do this: > > from IPython.display import display, HTML > > def runme(html): > HTML(html) > > > h = "<H1>Oh My </H1> > runme(h) > > I get no results. > > I've noticed this is true for other displays as well, how can I add > displays from withi functions? > > Thanks! > > John > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From moritz.beber at gmail.com Wed Jan 28 10:18:29 2015 From: moritz.beber at gmail.com (Moritz Beber) Date: Wed, 28 Jan 2015 16:18:29 +0100 Subject: [IPython-dev] IPython parallel and objects In-Reply-To: <1422448825555-5084497.post@n6.nabble.com> References: <1422448825555-5084497.post@n6.nabble.com> Message-ID: <CAFOFTpSke6Oiwd3p4KOMPdbPjrGPD4qNHxEAZcoCG-79Q4Fr=g@mail.gmail.com> Hi, On Wed, Jan 28, 2015 at 1:40 PM, thwiouz <maidos93 at laposte.net> wrote: > Hi guys, > > I'd like to parallelize the following loop: > > m1 = np.arange(0, 10, 100) > m2 = np.arange(0, 10, 100) > > for m_ in m1: > for n_2 in m2: > tmp = optimizer.compute_expectation([m_, n_]) > res.append(tmp) > > with type(optimizer) some class that I've defined. How could I handle it? > > I tried to do the same as in > > http://stackoverflow.com/questions/9363118/parallelise-nested-for-loop-in-ipython > so I created a lambda function f > > f = lambda x, y: optimizer.compute_expectation([x, y]) > > but it's not working with a load balanced view, saying that optimizer is > not > defined. > IPython only automatically transmits the function definition used in a `map` call or similar. Your optimizer is inside your lambda function and thus unknown on the remote kernels. In order to overcome this, take a look here: http://ipython.org/ipython-doc/2/parallel/parallel_multiengine.html#remote-function-decorators and here: http://ipython.org/ipython-doc/2/parallel/parallel_task.html#dependencies this could be helpful, too: http://ipython.org/ipython-doc/2/parallel/parallel_multiengine.html#moving-python-objects-around HTH, Moritz -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150128/91860336/attachment.html> From john at omernik.com Wed Jan 28 10:34:26 2015 From: john at omernik.com (John Omernik) Date: Wed, 28 Jan 2015 09:34:26 -0600 Subject: [IPython-dev] Display Function from with in a Function In-Reply-To: <CA+-1RQQ3PjM=0RJ8yOGj8RDH2-B5cPUALGFN91NjzpkE+uuGPg@mail.gmail.com> References: <CAKOFcwrkU4TG-KUHV=4JY7GJDc0d01An=0hwZjsgLY6vo32tMw@mail.gmail.com> <CA+-1RQQ3PjM=0RJ8yOGj8RDH2-B5cPUALGFN91NjzpkE+uuGPg@mail.gmail.com> Message-ID: <CAKOFcwr4KPj99J51Y0Q8+GqwynuCiLDcJS9i=FzKUYQbd7TTdg@mail.gmail.com> So I tried this: from IPython.display import display, display_html html = "<H1>OH MY </H1>" def runme(html): display_html(html) runme(html) # This produced no results/output Then I tried def runme(html): display(html) runme(html) # This printed: '<H1> OH MY </H1>' Which isn't what HTML(html) displayed (it displayed it rendered). So I had a hunch... h = display_html(html) type(h) NoneType so then in my function I did this: def runme(html): h = HTML(html) display(h) print "did it work?" and then runme(html) produced the expected Heading 1 OH MY and the printed statement after.... thanks for the pointer in the right direction! John On Wed, Jan 28, 2015 at 9:17 AM, Cyrille Rossant <cyrille.rossant at gmail.com> wrote: > HTML(...) will only display something if it's the last statement of a > code cell. This is because IPython always displays the result of the > last result of a cell (unless you terminate the line with a semicolon > ';'). > > A more reliable way of displaying stuff is to use display() or > display_html(). This will work from within functions. See > http://ipython.org/ipython-doc/dev/api/generated/IPython.display.html > for more details. > > 2015-01-28 16:08 GMT+01:00 John Omernik <john at omernik.com>: >> Greetings all, I am using iPython notebook, I can do this: >> >> from IPython.display import display, HTML >> >> h = "<H1>Oh My </H1>" >> HTML(h) >> >> and get the expected result >> >> But if I do this: >> >> from IPython.display import display, HTML >> >> def runme(html): >> HTML(html) >> >> >> h = "<H1>Oh My </H1> >> runme(h) >> >> I get no results. >> >> I've noticed this is true for other displays as well, how can I add >> displays from withi functions? >> >> Thanks! >> >> John >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From cyrille.rossant at gmail.com Wed Jan 28 10:38:44 2015 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Wed, 28 Jan 2015 16:38:44 +0100 Subject: [IPython-dev] Display Function from with in a Function In-Reply-To: <CAKOFcwr4KPj99J51Y0Q8+GqwynuCiLDcJS9i=FzKUYQbd7TTdg@mail.gmail.com> References: <CAKOFcwrkU4TG-KUHV=4JY7GJDc0d01An=0hwZjsgLY6vo32tMw@mail.gmail.com> <CA+-1RQQ3PjM=0RJ8yOGj8RDH2-B5cPUALGFN91NjzpkE+uuGPg@mail.gmail.com> <CAKOFcwr4KPj99J51Y0Q8+GqwynuCiLDcJS9i=FzKUYQbd7TTdg@mail.gmail.com> Message-ID: <CA+-1RQThvE29NQLSNtREUiQWLdoFanTwX1PystNf6D1nGp+TTQ@mail.gmail.com> You're welcome! Also have a look at the "raw=True" option to 'display()'. 2015-01-28 16:34 GMT+01:00 John Omernik <john at omernik.com>: > So I tried this: > > from IPython.display import display, display_html > > > html = "<H1>OH MY </H1>" > def runme(html): > display_html(html) > > runme(html) > > # This produced no results/output Then I tried > def runme(html): > display(html) > > runme(html) > > # This printed: '<H1> OH MY </H1>' > > Which isn't what HTML(html) displayed (it displayed it rendered). So > I had a hunch... > > h = display_html(html) > > type(h) > NoneType > > so then in my function I did this: > > def runme(html): > h = HTML(html) > display(h) > print "did it work?" > > > and then runme(html) produced the expected Heading 1 OH MY and the > printed statement after.... thanks for the pointer in the right > direction! > > John > > > > On Wed, Jan 28, 2015 at 9:17 AM, Cyrille Rossant > <cyrille.rossant at gmail.com> wrote: >> HTML(...) will only display something if it's the last statement of a >> code cell. This is because IPython always displays the result of the >> last result of a cell (unless you terminate the line with a semicolon >> ';'). >> >> A more reliable way of displaying stuff is to use display() or >> display_html(). This will work from within functions. See >> http://ipython.org/ipython-doc/dev/api/generated/IPython.display.html >> for more details. >> >> 2015-01-28 16:08 GMT+01:00 John Omernik <john at omernik.com>: >>> Greetings all, I am using iPython notebook, I can do this: >>> >>> from IPython.display import display, HTML >>> >>> h = "<H1>Oh My </H1>" >>> HTML(h) >>> >>> and get the expected result >>> >>> But if I do this: >>> >>> from IPython.display import display, HTML >>> >>> def runme(html): >>> HTML(html) >>> >>> >>> h = "<H1>Oh My </H1> >>> runme(h) >>> >>> I get no results. >>> >>> I've noticed this is true for other displays as well, how can I add >>> displays from withi functions? >>> >>> Thanks! >>> >>> John >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From ctaf42 at gmail.com Thu Jan 29 11:57:45 2015 From: ctaf42 at gmail.com (cedric gestes) Date: Thu, 29 Jan 2015 08:57:45 -0800 (PST) Subject: [IPython-dev] Dockable widgets for IPython Message-ID: <1422550665175-5084622.post@n6.nabble.com> I hacked a prototype of dockable widgets using DockSpawn. see http://www.hostingpics.net/viewer.php?id=990121201501291741191600x900scrot.png for a screenshot. Before going further I would like to take your advice on the subject. Is that something wanted in IPython? We can imagine having cells and widgets in docks. The prototype is hackish and just a proof of concept. You can find it github on the branch dockspawn of: github.com/ctaf42/ipython and github.com/ctaf42/ipython-components just checkout, run IPython and open a notebook like this one: http://nbviewer.ipython.org/gist/ctaf42/e029c475b08bdfe43f6a Cedric -- View this message in context: http://python.6.x6.nabble.com/Dockable-widgets-for-IPython-tp5084622.html Sent from the IPython - Development mailing list archive at Nabble.com. From jeanbigboute at gmail.com Thu Jan 29 12:28:27 2015 From: jeanbigboute at gmail.com (Jean Bigboute) Date: Thu, 29 Jan 2015 09:28:27 -0800 Subject: [IPython-dev] Two basic Notebook questions (Python focus) Message-ID: <CAEen6ux7Fi4JNCg-uab2ePgs4g1J72XhhXAdcM4sGABTJscy5Q@mail.gmail.com> 1) I often go back and forth in cells, tweaking functions. I'd like to be able to show the current code for a function in a cell (not the pager) when I'm not sure which function definition I last executed %psource function works nicely but the result goes in the pager, not the cell. These two options work but require remembering things which gets tricky when I use different computers: import inspect print inspect.getsource(function) and the pager redefinition shown in http://nbviewer.ipython.org/gist/minrk/7715212 has a fix which requires imports I am sure there must be a way to do this and I just haven't searched on the correct terms... 2) Is it possible to paste images from the clipboard buffer into a cell without first creating a separate image file and then linking to that file? Thanks, JBB -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/92da9cb7/attachment.html> From nick.bollweg at gmail.com Thu Jan 29 13:32:03 2015 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Thu, 29 Jan 2015 13:32:03 -0500 Subject: [IPython-dev] Dockable widgets for IPython In-Reply-To: <1422550665175-5084622.post@n6.nabble.com> References: <1422550665175-5084622.post@n6.nabble.com> Message-ID: <CACejjWw+vbe=OU9X-c2VaWHne8HJ36Kt-hJjHJ8Fdy+0OEw71w@mail.gmail.com> cedric: this looks very cool: as the "ipython is not a platform" party line softens, ideas like this are important for pushing what "interactive computing" can be like. My immediate concern would be in not knowing which cell a particular widget came from. Additionally, from a purely stylistic sense, try using bootstrap panels and fontawesome to make it flow more seamlessly with the notebook UI. I think several folks have been looking for this in the past, especially with creating such controls at notebook load time: http://python.6.x6.nabble.com/How-To-create-persistent-interactive-UI-elements-outside-cells-td5079080.html#a5079562 I'd say near-term, see if you can repackage this into an nbextension that someone can optionally add at profile config/run-time. If it can be made into pure javascript that stores its data in the notebook/cell metadata, that lowers the barrier to entry... and makes it more feasible to use with other kernels. Otherwise, if you need the python part, there is the approach taken by cite2c <https://github.com/takluyver/cite2c>. Specifically, allowing the configuration of the web app (not the kernel) to include loading custom extensions <https://github.com/takluyver/cite2c/blob/master/install.py#L11>. i have been interested in alternate editable layouts for some time: of them i think live_reveal <https://github.com/damianavila/live_reveal>, while still a little fiddly to install, has the most potential for becoming something that would make sense to ship with the notebook experience, especially as it is cell-based, (really) front-end only and its metadata is already integrated into nvconvert and (soon, hopefully!) nbviewer. If your widgets could survive nbviewer (either with or without help from a custom converter) that would be really cool... once static widgets are supported. i have a bit of python widget code that will populate widgets on top of SVG regions (as defined by layers in inkscape): this makes it easy to build up full-screen capable dashboards without requiring a lot of CSS/JS knowledge. SVGLayout(svg="dashboard.svg", children={"chart": SomeChartWidget()}) i want to refactor it into pure javascript as well. keep the list posted! On Thu, Jan 29, 2015 at 11:57 AM, cedric gestes <ctaf42 at gmail.com> wrote: > I hacked a prototype of dockable widgets using DockSpawn. > > see > > http://www.hostingpics.net/viewer.php?id=990121201501291741191600x900scrot.png > for a screenshot. > > Before going further I would like to take your advice on the subject. > > Is that something wanted in IPython? > > We can imagine having cells and widgets in docks. > > The prototype is hackish and just a proof of concept. > > You can find it github on the branch dockspawn of: > github.com/ctaf42/ipython and github.com/ctaf42/ipython-components > > just checkout, run IPython and open a notebook like this one: > http://nbviewer.ipython.org/gist/ctaf42/e029c475b08bdfe43f6a > > Cedric > > > > > -- > View this message in context: > http://python.6.x6.nabble.com/Dockable-widgets-for-IPython-tp5084622.html > Sent from the IPython - Development mailing list archive at Nabble.com. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/517fdb2d/attachment.html> From mjandrews.net at gmail.com Thu Jan 29 18:59:49 2015 From: mjandrews.net at gmail.com (Mark Andrews) Date: Thu, 29 Jan 2015 23:59:49 +0000 Subject: [IPython-dev] problems with ipython qtconsole --pylab on arch linux Message-ID: <CABj652bJQuykLd5kdoTb7NXqHzs8n=7bAL01moTi_LiZqF8XMA@mail.gmail.com> After a recent upgrade on arch linux (where I have been using ipython for years), I just encountered the following problem when I start ipython (2.3.1) with "ipython2 qtconsole --pylab" Using matplotlib backend: Qt5Agg ERROR:tornado.application:Exception in callback <functools.partial object at 0x7fcac006c838> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/tornado/ioloop.py", line 565, in _run_callback ret = callback() File "/usr/lib/python2.7/site-packages/tornado/stack_context.py", line 275, in null_wrapper return fn(*args, **kwargs) File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/ipkernel.py", line 262, in enter_eventloop self.eventloop(self) File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py", line 71, in loop_qt5 return loop_qt4(kernel) File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py", line 60, in loop_qt4 kernel.app = get_app_qt4([" "]) File "/usr/lib/python2.7/site-packages/IPython/lib/guisupport.py", line 111, in get_app_qt4 from IPython.external.qt_for_kernel import QtGui File "/usr/lib/python2.7/site-packages/IPython/external/qt_for_kernel.py", line 84, in <module> from IPython.external.qt import QtCore, QtGui, QtSvg, QT_API File "/usr/lib/python2.7/site-packages/IPython/external/qt.py", line 17, in <module> (QT_API, QT_API_PYSIDE, QT_API_PYQT)) RuntimeError: Invalid Qt API 'pyqt5', valid values are: 'pyside', 'pyqt' If I start with "ipython2 qtconsole" and then do %matplotlib inline import matplotlib.pyplot then plotting works fine. I am guessing that this is something that arises only with arch, and has something to do with its packages for python Qt bindings. Here are some of other relevant packages I am using python2-matplotlib 1.4.2-3 python2-pyqt4 4.11.3-1 python2-pyqt5 5.4.0-2 It is not a major hassle but it would be good know what is causing this. -m ps. ipython2 is arch linux's name for ipython 2.x, while the name ipython is for ipython 3.x. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/e759b1d0/attachment.html> From takowl at gmail.com Thu Jan 29 19:09:37 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 29 Jan 2015 16:09:37 -0800 Subject: [IPython-dev] problems with ipython qtconsole --pylab on arch linux In-Reply-To: <CABj652bJQuykLd5kdoTb7NXqHzs8n=7bAL01moTi_LiZqF8XMA@mail.gmail.com> References: <CABj652bJQuykLd5kdoTb7NXqHzs8n=7bAL01moTi_LiZqF8XMA@mail.gmail.com> Message-ID: <CAOvn4qg4KJO76NmhSxG4ExfK6=4XnNNsM-Sri4p=_EO7fXcyuA@mail.gmail.com> I'd hazard a guess that Arch has set matplotlib's default backend to qt5agg, but that's not supported by the released version of IPython. 3.0 and 2.4 will support Qt 5. In the meantime, you should be able to work around it by specifying another backend, e.g. --pylab qt4 or --pylab inline. Thomas On 29 January 2015 at 15:59, Mark Andrews <mjandrews.net at gmail.com> wrote: > After a recent upgrade on arch linux (where I have been using ipython for > years), I just encountered the following problem when I start ipython > (2.3.1) with "ipython2 qtconsole --pylab" > > Using matplotlib backend: Qt5Agg > > ERROR:tornado.application:Exception in callback <functools.partial object > at 0x7fcac006c838> > > Traceback (most recent call last): > > File "/usr/lib/python2.7/site-packages/tornado/ioloop.py", line 565, in > _run_callback > > ret = callback() > > File "/usr/lib/python2.7/site-packages/tornado/stack_context.py", line > 275, in null_wrapper > > return fn(*args, **kwargs) > > File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/ipkernel.py", > line 262, in enter_eventloop > > self.eventloop(self) > > File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py", > line 71, in loop_qt5 > > return loop_qt4(kernel) > > File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py", > line 60, in loop_qt4 > > kernel.app = get_app_qt4([" "]) > > File "/usr/lib/python2.7/site-packages/IPython/lib/guisupport.py", line > 111, in get_app_qt4 > > from IPython.external.qt_for_kernel import QtGui > > File "/usr/lib/python2.7/site-packages/IPython/external/qt_for_kernel.py", > line 84, in <module> > > from IPython.external.qt import QtCore, QtGui, QtSvg, QT_API > > File "/usr/lib/python2.7/site-packages/IPython/external/qt.py", line 17, > in <module> > > (QT_API, QT_API_PYSIDE, QT_API_PYQT)) > > RuntimeError: Invalid Qt API 'pyqt5', valid values are: 'pyside', 'pyqt' > > > If I start with "ipython2 qtconsole" and then do > > %matplotlib inline > > import matplotlib.pyplot > > > then plotting works fine. > > > I am guessing that this is something that arises only with arch, and has > something to do with its packages for python Qt bindings. > > > Here are some of other relevant packages I am using > > python2-matplotlib 1.4.2-3 > > python2-pyqt4 4.11.3-1 > python2-pyqt5 5.4.0-2 > > > It is not a major hassle but it would be good know what is causing this. > > > -m > > > ps. ipython2 is arch linux's name for ipython 2.x, while the name ipython > is for ipython 3.x. > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/324cfc9e/attachment.html> From mjandrews.net at gmail.com Thu Jan 29 19:22:43 2015 From: mjandrews.net at gmail.com (Mark Andrews) Date: Fri, 30 Jan 2015 00:22:43 +0000 Subject: [IPython-dev] problems with ipython qtconsole --pylab on arch linux In-Reply-To: <CAOvn4qg4KJO76NmhSxG4ExfK6=4XnNNsM-Sri4p=_EO7fXcyuA@mail.gmail.com> References: <CABj652bJQuykLd5kdoTb7NXqHzs8n=7bAL01moTi_LiZqF8XMA@mail.gmail.com> <CAOvn4qg4KJO76NmhSxG4ExfK6=4XnNNsM-Sri4p=_EO7fXcyuA@mail.gmail.com> Message-ID: <CABj652ZrE8UVBEwf9UsrSKkboT06iuFR9Fg9iy318drbGOM+Pg@mail.gmail.com> That seems like a reasonable hypothesis. I am not too sure how to check if matplotlib's default backend to has been recently set to qt5agg, but I will try. In the meantime, I can confirm that ipython2 qtconsole --pylab qt4 ipython2 qtconsole --pylab inline work fine. Thanks! -m On Fri, Jan 30, 2015 at 12:09 AM, Thomas Kluyver <takowl at gmail.com> wrote: > I'd hazard a guess that Arch has set matplotlib's default backend to > qt5agg, but that's not supported by the released version of IPython. 3.0 > and 2.4 will support Qt 5. > > In the meantime, you should be able to work around it by specifying > another backend, e.g. --pylab qt4 or --pylab inline. > > Thomas > > On 29 January 2015 at 15:59, Mark Andrews <mjandrews.net at gmail.com> wrote: > >> After a recent upgrade on arch linux (where I have been using ipython for >> years), I just encountered the following problem when I start ipython >> (2.3.1) with "ipython2 qtconsole --pylab" >> >> Using matplotlib backend: Qt5Agg >> >> ERROR:tornado.application:Exception in callback <functools.partial object >> at 0x7fcac006c838> >> >> Traceback (most recent call last): >> >> File "/usr/lib/python2.7/site-packages/tornado/ioloop.py", line 565, in >> _run_callback >> >> ret = callback() >> >> File "/usr/lib/python2.7/site-packages/tornado/stack_context.py", line >> 275, in null_wrapper >> >> return fn(*args, **kwargs) >> >> File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/ipkernel.py", >> line 262, in enter_eventloop >> >> self.eventloop(self) >> >> File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py", >> line 71, in loop_qt5 >> >> return loop_qt4(kernel) >> >> File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py", >> line 60, in loop_qt4 >> >> kernel.app = get_app_qt4([" "]) >> >> File "/usr/lib/python2.7/site-packages/IPython/lib/guisupport.py", line >> 111, in get_app_qt4 >> >> from IPython.external.qt_for_kernel import QtGui >> >> File >> "/usr/lib/python2.7/site-packages/IPython/external/qt_for_kernel.py", line >> 84, in <module> >> >> from IPython.external.qt import QtCore, QtGui, QtSvg, QT_API >> >> File "/usr/lib/python2.7/site-packages/IPython/external/qt.py", line 17, >> in <module> >> >> (QT_API, QT_API_PYSIDE, QT_API_PYQT)) >> >> RuntimeError: Invalid Qt API 'pyqt5', valid values are: 'pyside', 'pyqt' >> >> >> If I start with "ipython2 qtconsole" and then do >> >> %matplotlib inline >> >> import matplotlib.pyplot >> >> >> then plotting works fine. >> >> >> I am guessing that this is something that arises only with arch, and has >> something to do with its packages for python Qt bindings. >> >> >> Here are some of other relevant packages I am using >> >> python2-matplotlib 1.4.2-3 >> >> python2-pyqt4 4.11.3-1 >> python2-pyqt5 5.4.0-2 >> >> >> It is not a major hassle but it would be good know what is causing this. >> >> >> -m >> >> >> ps. ipython2 is arch linux's name for ipython 2.x, while the name ipython >> is for ipython 3.x. >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/1b7e0021/attachment.html> From alexgarciac at gmail.com Thu Jan 29 22:07:13 2015 From: alexgarciac at gmail.com (Alexander Garcia Castro) Date: Thu, 29 Jan 2015 22:07:13 -0500 Subject: [IPython-dev] ipython in scholarly communication Message-ID: <CALAe=OLFhnyzXH0svqkX9kov_vbT-7QDnhJt7+RziNf6AqRWyg@mail.gmail.com> Dear all, Sepublica is particularly interested in getting submissions from ipython usrs/developers. we are very interested in seeing new ways of scholarly communication in action and we believe that ipython illustrates a path to be followed. SePublica 2015: Do show, do not tell! ===================================== Workshop website: http://sepublica.info/ Event hashtags: #SePublica #ESWC2015 in conjunction with ESWC 2015 The 12th European Semantic Web Conference http://2015.eswc-conferences.org/ At SePublica we are interested in addressing the question, how is Semantic Web technology being used as part of publication workflows. Challenge us with a creative submission. Let us see and experience your take on publications for the Web of data; we want to see NEW ideas in action, INNOVATION that makes our lives easier. Is it about scholarly communication? Then what problem are you solving? How are you making the content interoperable? How is this different/better from traditional approaches? Important Dates --------------- * Submission deadline: 2015-03-06 * Notifications: 2015-04-03 * Camera ready version: 2015-04-17 Submission ---------- Easychair: https://www.easychair.org/conferences/?conf=sepublica2015 Submissions will be peer reviewed. Accepted works have to be presented at the workshop (requires registering for the ESWC conference and the workshop). Every accepted submission will be published in the proceedings of the workshop. Submission Types ---------------- ### Call for Submissions Deadline: 2015-03-06. Authors may submit their content in whatever way they consider is best for their purposes (e.g., HTML, RDFS/OWL, IPython, PDF, or any publicly executable script, etc). Please feel free to contact us to discuss your particular case if necessary. We will be happy to assist authors and support them as much as we can. Submissions are format free, GIT URLs are accepted just like OWL, XML etc. However, reviewers should not have to install software in order to review the submissions. Datasets are also accepted. In all cases everything must be publicly available. For those of you submitting a PDF please use the LNCS format, 12 pages is the limit ?including references. Please, note that (X)HTML(+RDFa) submissions are also welcome as long as the layout complies with the LNCS style. For example, authors may use the template provided at https://github.com/csarven/linked-research ### Call for Polemics Deadline: 2015-04-24. We would like to invite authors to send us a one page manuscript, 20 lines max, describing their position with respect to new technologies supporting the publication workflow: What are the most pressing issues to be addressed? What is their position with respect to the overall problem? What innovation is needed? etc. Polemics authors will have only 5 minutes to present; the format of this session is sequential, after each presentation the next follows with no questions in between. There will be a discussion and summary of all the issues at the end of all the polemics session. Issues to be addressed ---------------------- * Should papers be semantically annotated? * How would you search or query for research objects? * How can research objects be reused? * What is an executable semantic paper? * How can we create interactive research documents to better communicate and educate? * Is there a format for semantic publishing or does it require a combination of formats? * How can research documents be persistent and archived? Which formats are preferable and why? * If we subscribe to science, free and open access to knowledge, what's the purpose of the arrangement between conferences and publishers? Or can big conferences manage the publication themselves? * Is there visible change towards semantic authoring and publishing? If not, what are some of the underlying challenges to get there? * Assume a 3-year plan to transition the SW research community to completely publish using SW technologies. Identify the milestones and explain how it would happen. * Backend use of semantics by publishers in book, news, and scholarly publishing * Publishing data-driven arguments for general news consumption or in scholarly papers * Historical changes in publishing since the advent of online publishing: what is going right? Organizing Committee -------------------- * Phillip Lord * Alexander Garcia Castro * Sarven Capadisli * Jodi Schneider -- Alexander Garcia http://www.alexandergarcia.name/ http://www.usefilm.com/photographer/75943.html http://www.linkedin.com/in/alexgarciac -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/c9a9aa39/attachment.html> From rgbkrk at gmail.com Thu Jan 29 22:10:46 2015 From: rgbkrk at gmail.com (Kyle Kelley) Date: Thu, 29 Jan 2015 21:10:46 -0600 Subject: [IPython-dev] ipython/ipython Docker image switching to debian:jessie Message-ID: <CA+tbMaWiHpytqyneOuZyVNPVM3ZxfBo__de7498-HE4xixq4Fw@mail.gmail.com> In https://github.com/ipython/ipython/pull/7391 we're switching our base Docker image over to debian:jessie. Let us know if you need more time before we merge this, which will result in triggering builds across all the ipython/* images on Docker Hub. -- Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/749b9d54/attachment.html> From bussonniermatthias at gmail.com Fri Jan 30 05:28:58 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Fri, 30 Jan 2015 11:28:58 +0100 Subject: [IPython-dev] Two basic Notebook questions (Python focus) In-Reply-To: <CAEen6ux7Fi4JNCg-uab2ePgs4g1J72XhhXAdcM4sGABTJscy5Q@mail.gmail.com> References: <CAEen6ux7Fi4JNCg-uab2ePgs4g1J72XhhXAdcM4sGABTJscy5Q@mail.gmail.com> Message-ID: <81C8AF0D-F5AB-4404-A83D-7F82A58B0D73@gmail.com> Le 29 janv. 2015 ? 18:28, Jean Bigboute <jeanbigboute at gmail.com> a ?crit : > 1) I often go back and forth in cells, tweaking functions. I'd like to be able to show the current code for a function in a cell (not the pager) when I'm not sure which function definition I last executed > > > %psource function > works nicely but the result goes in the pager, not the cell. > > These two options work but require remembering things which gets tricky when I use different computers: > > import inspect > print inspect.getsource(function) > > and the pager redefinition shown in > http://nbviewer.ipython.org/gist/minrk/7715212 has a fix which requires imports > > I am sure there must be a way to do this and I just haven't searched on the correct terms... In IPython master there is an option to redirect all pager things to output. The plan it to make output detachable. try --InteractiveShell.display_page=True see PR 7009. > 2) Is it possible to paste images from the clipboard buffer into a cell without first creating a separate image file and then linking to that file? No, I would suggest looking at github.com/IPython-contrib for an extension to do that. There are some complication to do that though, in particular when "uploading" do you want to upload to server or kernel ? Now that we have content API it should be easier to implement if it is not yet done. -- M > > Thanks, > > JBB > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/b4f16ff5/attachment.html> From bussonniermatthias at gmail.com Fri Jan 30 05:30:32 2015 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Fri, 30 Jan 2015 11:30:32 +0100 Subject: [IPython-dev] Dockable widgets for IPython In-Reply-To: <CACejjWw+vbe=OU9X-c2VaWHne8HJ36Kt-hJjHJ8Fdy+0OEw71w@mail.gmail.com> References: <1422550665175-5084622.post@n6.nabble.com> <CACejjWw+vbe=OU9X-c2VaWHne8HJ36Kt-hJjHJ8Fdy+0OEw71w@mail.gmail.com> Message-ID: <DF2B4B6D-485D-496A-80DB-017C1D923A39@gmail.com> Hi, To add to what Nicholas said, After IPython 3.0 we will split the widget in a separate repository. It will be easier to contribute/discuss than just before 3.0 release or while widget are still tightly entangled with IPython code. -- M Le 29 janv. 2015 ? 19:32, Nicholas Bollweg <nick.bollweg at gmail.com> a ?crit : > cedric: > > this looks very cool: as the "ipython is not a platform" party line softens, ideas like this are important for pushing what "interactive computing" can be like. My immediate concern would be in not knowing which cell a particular widget came from. Additionally, from a purely stylistic sense, try using bootstrap panels and fontawesome to make it flow more seamlessly with the notebook UI. > > I think several folks have been looking for this in the past, especially with creating such controls at notebook load time: > http://python.6.x6.nabble.com/How-To-create-persistent-interactive-UI-elements-outside-cells-td5079080.html#a5079562 > > I'd say near-term, see if you can repackage this into an nbextension that someone can optionally add at profile config/run-time. If it can be made into pure javascript that stores its data in the notebook/cell metadata, that lowers the barrier to entry... and makes it more feasible to use with other kernels. > > Otherwise, if you need the python part, there is the approach taken by cite2c. Specifically, allowing the configuration of the web app (not the kernel) to include loading custom extensions. > > i have been interested in alternate editable layouts for some time: of them i think live_reveal, while still a little fiddly to install, has the most potential for becoming something that would make sense to ship with the notebook experience, especially as it is cell-based, (really) front-end only and its metadata is already integrated into nvconvert and (soon, hopefully!) nbviewer. If your widgets could survive nbviewer (either with or without help from a custom converter) that would be really cool... once static widgets are supported. > > i have a bit of python widget code that will populate widgets on top of SVG regions (as defined by layers in inkscape): this makes it easy to build up full-screen capable dashboards without requiring a lot of CSS/JS knowledge. > > SVGLayout(svg="dashboard.svg", > children={"chart": SomeChartWidget()}) > > i want to refactor it into pure javascript as well. > > keep the list posted! > > On Thu, Jan 29, 2015 at 11:57 AM, cedric gestes <ctaf42 at gmail.com> wrote: > I hacked a prototype of dockable widgets using DockSpawn. > > see > http://www.hostingpics.net/viewer.php?id=990121201501291741191600x900scrot.png > for a screenshot. > > Before going further I would like to take your advice on the subject. > > Is that something wanted in IPython? > > We can imagine having cells and widgets in docks. > > The prototype is hackish and just a proof of concept. > > You can find it github on the branch dockspawn of: > github.com/ctaf42/ipython and github.com/ctaf42/ipython-components > > just checkout, run IPython and open a notebook like this one: > http://nbviewer.ipython.org/gist/ctaf42/e029c475b08bdfe43f6a > > Cedric > > > > > -- > View this message in context: http://python.6.x6.nabble.com/Dockable-widgets-for-IPython-tp5084622.html > Sent from the IPython - Development mailing list archive at Nabble.com. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/152431be/attachment.html> From doug.blank at gmail.com Fri Jan 30 05:38:52 2015 From: doug.blank at gmail.com (Doug Blank) Date: Fri, 30 Jan 2015 05:38:52 -0500 Subject: [IPython-dev] Two basic Notebook questions (Python focus) In-Reply-To: <81C8AF0D-F5AB-4404-A83D-7F82A58B0D73@gmail.com> References: <CAEen6ux7Fi4JNCg-uab2ePgs4g1J72XhhXAdcM4sGABTJscy5Q@mail.gmail.com> <81C8AF0D-F5AB-4404-A83D-7F82A58B0D73@gmail.com> Message-ID: <CAAusYCjxv+C0kbWK-mwgOE0gv1a6byABPjb44KqbbBzAaMyWyg@mail.gmail.com> Drag and drop images via new API is done by juhasch: https://github.com/ipython-contrib/IPython-notebook-extensions/wiki/drag-and-drop -Doug On Fri, Jan 30, 2015 at 5:28 AM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > > Le 29 janv. 2015 ? 18:28, Jean Bigboute <jeanbigboute at gmail.com> a ?crit : > > 1) I often go back and forth in cells, tweaking functions. I'd like to be > able to show the current code for a function in a cell (not the pager) when > I'm not sure which function definition I last executed > > > %psource function > works nicely but the result goes in the pager, not the cell. > > These two options work but require remembering things which gets tricky > when I use different computers: > > import inspect > print inspect.getsource(function) > > and the pager redefinition shown in > http://nbviewer.ipython.org/gist/minrk/7715212 has a fix which requires > imports > > I am sure there must be a way to do this and I just haven't searched on > the correct terms... > > > In IPython master there is an option to redirect all pager things to > output. > The plan it to make output detachable. > > try --InteractiveShell.display_page=True > > see PR 7009. > > > 2) Is it possible to paste images from the clipboard buffer into a cell > without first creating a separate image file and then linking to that file? > > > No, > > I would suggest looking at github.com/IPython-contrib for an extension to > do that. > There are some complication to do that though, in particular when > "uploading" do you want to upload > to server or kernel ? > > Now that we have content API it should be easier to implement if it is not > yet done. > -- > M > > > Thanks, > > JBB > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/e8996ec7/attachment.html> From nick.bollweg at gmail.com Fri Jan 30 14:32:28 2015 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Fri, 30 Jan 2015 14:32:28 -0500 Subject: [IPython-dev] ipython in scholarly communication In-Reply-To: <CALAe=OLFhnyzXH0svqkX9kov_vbT-7QDnhJt7+RziNf6AqRWyg@mail.gmail.com> References: <CALAe=OLFhnyzXH0svqkX9kov_vbT-7QDnhJt7+RziNf6AqRWyg@mail.gmail.com> Message-ID: <CACejjWye5sHBFhusOtKc7dSL=kwJET0TGevXWrG1SYaEY3-_7Q@mail.gmail.com> I'm pretty interested in this. Here's the last discussion I recall about SW on this list: http://python.6.x6.nabble.com/Notebook-format-quot-incompatible-quot-changes-td5076324.html#a5076589 We talked at one point about creating an IPEP for incrementally adding linked data to various things across the ecosystem, perhaps this is a good time to dust that off. Here is the IPEP that I never actually posted: http://nbviewer.ipython.org/gist/bollwyvl/6edda34649ba4b5cbaf9 Forks, comments welcome! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/be2e4a62/attachment.html> From jeanbigboute at gmail.com Fri Jan 30 18:26:40 2015 From: jeanbigboute at gmail.com (Jean Bigboute) Date: Fri, 30 Jan 2015 15:26:40 -0800 Subject: [IPython-dev] Two basic Notebook questions (Python focus) Message-ID: <CAEen6uz_OfHFL4JfKHBTdeCYqHUVdnAM4G8rjF-wMhoB4T=mdA@mail.gmail.com> Thanks for the suggestions on how to disable the pager and on the options for pasting images. I am not savvy enough with IPython to run from the master so I'll keep an eye out for the upcoming switch. A couple of followup questions: a) Listing the code of a function seems like a useful feature to have, especially for relatively new/occasional Python programmers such as myself. Should I request it through the appropriate channels? b) Is it possible to pass arguments to a magic function? e.g. import inspect print inspect.getsource(function) I am sure I can remember to import inspect. I tried to create an alias for the print inspect.getsource(function) but couldn't figure out how to pass the desired function name as an argument Thanks, JBB From benjaminrk at gmail.com Fri Jan 30 18:30:52 2015 From: benjaminrk at gmail.com (MinRK) Date: Fri, 30 Jan 2015 15:30:52 -0800 Subject: [IPython-dev] IPython 2.4, 3.0b1 Message-ID: <CAHNn8BXm6xtXNv2o0jf5uxgYrp1WUAtsYWqWCh_XnVadx=qu_w@mail.gmail.com> Hello, It?s been a long time coming, but we have begun the release cycle for IPython 3.0, with the first beta. You can take it for a spin with: pip install --pre 'ipython[all]' IPython 3.0 brings a lot of changes - optional terminals in the browser, editing non-notebook text files, major notebook format and message spec revisions, UI overhaul, etc. Take it for a spin, and let us know what you find. Check out the release notes <http://ipython.org/ipython-doc/dev/whatsnew/version3.html#release-3-0> for more info. We still have a few known bugs and UI issues to address before we get to 3.0 final, but we?ll get there soon. We also released IPython 2.4 today, which adds to 2.x the ability to *read* the new notebook format coming in IPython 3.0. 2.4 also fixes some compatibility issues with PyQt5 and pygments 2. With your help, 3.0 final will see the light of day before too long. -MinRK, on behalf of the IPython team ? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/25f06dd5/attachment.html> From takowl at gmail.com Fri Jan 30 18:48:59 2015 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 30 Jan 2015 15:48:59 -0800 Subject: [IPython-dev] Two basic Notebook questions (Python focus) In-Reply-To: <CAEen6uz_OfHFL4JfKHBTdeCYqHUVdnAM4G8rjF-wMhoB4T=mdA@mail.gmail.com> References: <CAEen6uz_OfHFL4JfKHBTdeCYqHUVdnAM4G8rjF-wMhoB4T=mdA@mail.gmail.com> Message-ID: <CAOvn4qjuysh_1z1tCJK1YWOC53QCjZdnCEwudS8Co1G-s1uK=w@mail.gmail.com> On 30 January 2015 at 15:26, Jean Bigboute <jeanbigboute at gmail.com> wrote: > b) Is it possible to pass arguments to a magic function? > e.g. > import inspect > print inspect.getsource(function) > This is probably easiest to do in a plain Python function. Create a file ~/.ipython/profile_default/startup/printsource.py (which IPython will automatically run when it starts, and put in it: import inspect def source(func) print inspect.getsource(function) Now you can just type 'source(f)' to see a function in your notebook. Be aware, though, that if you share that notebook, other people won't have the source() function defined. You could also define it at the top of the notebook if you prefer. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/abd3b6cb/attachment.html> From cpdiehl at gmail.com Sat Jan 31 11:07:50 2015 From: cpdiehl at gmail.com (diehl) Date: Sat, 31 Jan 2015 08:07:50 -0800 (PST) Subject: [IPython-dev] Complex UI Examples with Widgets Message-ID: <1422720470245-5084847.post@n6.nabble.com> Hi All, I'm looking for IPython notebooks containing complex UIs built with the widget framework. Just trying to get a sense of what is in the realm of the possible, especially when it comes to composition and positioning of the widgets in the notebook. Any examples you can refer me to would be greatly appreciated! Thanks, Chris -- View this message in context: http://python.6.x6.nabble.com/Complex-UI-Examples-with-Widgets-tp5084847.html Sent from the IPython - Development mailing list archive at Nabble.com. From hughesadam87 at gmail.com Sat Jan 31 13:05:24 2015 From: hughesadam87 at gmail.com (Adam Hughes) Date: Sat, 31 Jan 2015 13:05:24 -0500 Subject: [IPython-dev] Complex UI Examples with Widgets In-Reply-To: <1422720470245-5084847.post@n6.nabble.com> References: <1422720470245-5084847.post@n6.nabble.com> Message-ID: <CAMHV+dDkOAJahHX2Tfn4wGV-uh7u8bEig-QikgOnVC-WvdeQ8g@mail.gmail.com> We have a fairly involved one. The traitlets Model is in specgram.py and the GUI/Widget elements are in specgui.py https://github.com/hugadams/scikit-spectra/tree/master/skspec/interact/ipynbs Here's a video of it in action: http://hugadams.github.io/scikit-spectra/ On Sat, Jan 31, 2015 at 11:07 AM, diehl <cpdiehl at gmail.com> wrote: > Hi All, > > I'm looking for IPython notebooks containing complex UIs built with the > widget framework. Just trying to get a sense of what is in the realm of the > possible, especially when it comes to composition and positioning of the > widgets in the notebook. Any examples you can refer me to would be greatly > appreciated! > > Thanks, > Chris > > > > -- > View this message in context: > http://python.6.x6.nabble.com/Complex-UI-Examples-with-Widgets-tp5084847.html > Sent from the IPython - Development mailing list archive at Nabble.com. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150131/45d63d36/attachment.html> From cpdiehl at gmail.com Sat Jan 31 13:51:58 2015 From: cpdiehl at gmail.com (Chris Diehl) Date: Sat, 31 Jan 2015 10:51:58 -0800 Subject: [IPython-dev] Complex UI Examples with Widgets In-Reply-To: <CAMHV+dDkOAJahHX2Tfn4wGV-uh7u8bEig-QikgOnVC-WvdeQ8g@mail.gmail.com> References: <1422720470245-5084847.post@n6.nabble.com> <CAMHV+dDkOAJahHX2Tfn4wGV-uh7u8bEig-QikgOnVC-WvdeQ8g@mail.gmail.com> Message-ID: <CABYryNBVcsjieDsKE=S-u5cNGfbNeZwLT4qsf+Qe+yVBv17CtQ@mail.gmail.com> This is a great example Adam. Thanks! On Sat, Jan 31, 2015 at 10:05 AM, Adam Hughes <hughesadam87 at gmail.com> wrote: > We have a fairly involved one. The traitlets Model is in specgram.py and > the GUI/Widget elements are in specgui.py > > > https://github.com/hugadams/scikit-spectra/tree/master/skspec/interact/ipynbs > > Here's a video of it in action: > > http://hugadams.github.io/scikit-spectra/ > > On Sat, Jan 31, 2015 at 11:07 AM, diehl <cpdiehl at gmail.com> wrote: > >> Hi All, >> >> I'm looking for IPython notebooks containing complex UIs built with the >> widget framework. Just trying to get a sense of what is in the realm of >> the >> possible, especially when it comes to composition and positioning of the >> widgets in the notebook. Any examples you can refer me to would be greatly >> appreciated! >> >> Thanks, >> Chris >> >> >> >> -- >> View this message in context: >> http://python.6.x6.nabble.com/Complex-UI-Examples-with-Widgets-tp5084847.html >> Sent from the IPython - Development mailing list archive at Nabble.com. >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150131/5190a1ee/attachment.html> From hughesadam87 at gmail.com Sat Jan 31 18:19:43 2015 From: hughesadam87 at gmail.com (Adam Hughes) Date: Sat, 31 Jan 2015 18:19:43 -0500 Subject: [IPython-dev] Complex UI Examples with Widgets In-Reply-To: <CABYryNBVcsjieDsKE=S-u5cNGfbNeZwLT4qsf+Qe+yVBv17CtQ@mail.gmail.com> References: <1422720470245-5084847.post@n6.nabble.com> <CAMHV+dDkOAJahHX2Tfn4wGV-uh7u8bEig-QikgOnVC-WvdeQ8g@mail.gmail.com> <CABYryNBVcsjieDsKE=S-u5cNGfbNeZwLT4qsf+Qe+yVBv17CtQ@mail.gmail.com> Message-ID: <CAMHV+dB-cz4DCzRDzNTMkMky4HpLLsZZu2K2iqBj-f1TYebw6w@mail.gmail.com> No prob, feel free to pick from it. On Sat, Jan 31, 2015 at 1:51 PM, Chris Diehl <cpdiehl at gmail.com> wrote: > This is a great example Adam. Thanks! > > On Sat, Jan 31, 2015 at 10:05 AM, Adam Hughes <hughesadam87 at gmail.com> > wrote: > >> We have a fairly involved one. The traitlets Model is in specgram.py and >> the GUI/Widget elements are in specgui.py >> >> >> https://github.com/hugadams/scikit-spectra/tree/master/skspec/interact/ipynbs >> >> Here's a video of it in action: >> >> http://hugadams.github.io/scikit-spectra/ >> >> On Sat, Jan 31, 2015 at 11:07 AM, diehl <cpdiehl at gmail.com> wrote: >> >>> Hi All, >>> >>> I'm looking for IPython notebooks containing complex UIs built with the >>> widget framework. Just trying to get a sense of what is in the realm of >>> the >>> possible, especially when it comes to composition and positioning of the >>> widgets in the notebook. Any examples you can refer me to would be >>> greatly >>> appreciated! >>> >>> Thanks, >>> Chris >>> >>> >>> >>> -- >>> View this message in context: >>> http://python.6.x6.nabble.com/Complex-UI-Examples-with-Widgets-tp5084847.html >>> Sent from the IPython - Development mailing list archive at Nabble.com. >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150131/d4d97d1e/attachment.html> From jeanbigboute at gmail.com Sat Jan 31 22:41:41 2015 From: jeanbigboute at gmail.com (Jean Bigboute) Date: Sat, 31 Jan 2015 19:41:41 -0800 Subject: [IPython-dev] Two basic Notebook questions (Python focus) Message-ID: <CAEen6uyY6u=z95stNyJ5qxk=h=T62NWJoqdrA5S2_5RoJmCb1w@mail.gmail.com> On Sat, Jan 31, 2015 at 10:00 AM, <ipython-dev-request at scipy.org> wrote: > > Message: 4 > Date: Fri, 30 Jan 2015 15:48:59 -0800 > From: Thomas Kluyver <takowl at gmail.com> > Subject: Re: [IPython-dev] Two basic Notebook questions (Python focus) > On 30 January 2015 at 15:26, Jean Bigboute <jeanbigboute at gmail.com> wrote: > >> b) Is it possible to pass arguments to a magic function? >> e.g. >> import inspect >> print inspect.getsource(function) >> > > This is probably easiest to do in a plain Python function. Create a file > ~/.ipython/profile_default/startup/printsource.py (which IPython will > automatically run when it starts, and put in it: > > import inspect > def source(func) > print inspect.getsource(function) > > Now you can just type 'source(f)' to see a function in your notebook. Be > aware, though, that if you share that notebook, other people won't have the > source() function defined. You could also define it at the top of the > notebook if you prefer. > > Thomas > I tried this on Anaconda and Canopy without success. (I also included a colon at the end of the def source statement. The code never loaded into the Notebook interface. On Anaconda: I then tried plain ipython (no notebook) and got the error shown below. I systematically deleted lines in the startup file and eventually had just a blank file. Error still results. The Canopy distribution is on my work machine so I don't know what pure ipython would have done on it. ===================== [TerminalIPythonApp] WARNING | Error in loading extension: kernmagic Check your config files in /{PATH}/.ipython/profile_default --------------------------------------------------------------------------- ImportError Traceback (most recent call last) /{PATH}/anaconda/lib/python2.7/site-packages/IPython/core/extensions.pyc in load_extension(self, module_str) 96 if module_str not in sys.modules: 97 with prepended_to_syspath(self.ipython_extension_dir): ---> 98 __import__(module_str) 99 mod = sys.modules[module_str] 100 if self._call_load_ipython_extension(mod): ImportError: No module named kernmagic