From erik.tollerud at gmail.com Wed Feb 3 15:58:32 2010 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Wed, 3 Feb 2010 12:58:32 -0800 Subject: [IPython-dev] isympy doesn't work with the new ipython-dev In-Reply-To: References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com> Message-ID: I encountered some problems with isympy as well, but managed to get it to work (at least for my purposes) by changing /usr/local/bin/isympy (or wherever your script installs the isympy script) to just: #!/bin/bash ipython -p sympy and then I put a ipython_config_sympy.py file in the .ipython directory with the following: c = get_config() # This can be used at any point in a config file to load a sub config # and merge it into the current one. load_subconfig('ipython_config.py') lines = """ from __future__ import division from sympy import * x, y, z = symbols('xyz') k, m, n = symbols('kmn', integer=True) f, g, h = map(Function, 'fgh') """ # You have to make sure that attributes that are containers already # exist before using them. Simple assigning a new list will override # all previous values. if hasattr(c.Global, 'exec_lines'): c.Global.exec_lines.append(lines) else: c.Global.exec_lines = [lines] On Fri, Jan 22, 2010 at 4:58 PM, Fernando Perez wrote: > On Fri, Jan 22, 2010 at 1:20 PM, G?khan Sever wrote: >> >> I can't get the isympy working with the Fernando's trunk. I solved one of >> the missing attribute error but stuck at the other one: > > No, things have changed a bit. ?I'm insanely pressed for time right > now, the best I suggest is looking at this test code file, around line > 130: > > http://bazaar.launchpad.net/%7Efdo.perez/ipython/trunk-dev/annotate/head%3A/IPython/testing/globalipapp.py#L130 > > > That makes the kind of shell you're looking for, sort of. > > Sorry not to be able to give you a detailed example right now... > > cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From fperez.net at gmail.com Tue Feb 9 11:45:50 2010 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 9 Feb 2010 11:45:50 -0500 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: <46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com> References: <46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com> Message-ID: I've realized I let this linger without action, and we probably should do something about it while it's (relatively) fresh in our minds. On Wed, Jan 20, 2010 at 4:46 PM, Ville M. Vainio wrote: >> Now I could issue lp commands and upgrade the repos in lp itself, but >> that's going to cascade to everyone else, so I don't want to do that >> without further feedback. > > We did it for Leo and it didn't cause problems. People just needed to > get a fresh copy from launchpad and merge their changes there. > > Remote repository upgrade did take quite a bit of time, though > (possibly hours?). So, as a concrete proposal, I can update the remote repo say by this weekend, giving anyone with local changes in any branch time to merge them. By Sunday I will start a remote repo upgrade, and we'll all simply have to run local upgrades before doing any further operations. Does that sound OK, have I missed anything? >> It's nuts that bzr would make a change like this with such a nasty >> cascading effect... > > It's actually quite simple to get over this; I agree that launchpad > admins could have made a little bit more noise about this change. Yes, it's annoying because they more or less forced projects to deal with the issue to some extent right away, without any clear head warning. But it doesn't matter, we can do it and move on. >> Ideas from the experts? (I know, I want to switch to git too so we can >> get off the mess that is bzr and its never ending stream of weird >> problems, but that's probably too disruptive for now, so let's stick >> to solutions that use bzr...) > > Judging by my experience with git, I don't think it would be much of > an improvement. Mercurial, on the other hand... That's a topic for another day, but I'd be curious about your experiences. The more I use git the more I like it and I've only used hg very lightly, so I'd be interested in this. Cheers, f From jorgen.stenarson at bostream.nu Tue Feb 9 13:07:15 2010 From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=) Date: Tue, 09 Feb 2010 19:07:15 +0100 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: References: <46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com> Message-ID: <4B71A453.1060406@bostream.nu> Fernando Perez skrev 2010-02-09 17:45: > So, as a concrete proposal, I can update the remote repo say by this > weekend, giving anyone with local changes in any branch time to merge > them. By Sunday I will start a remote repo upgrade, and we'll all > simply have to run local upgrades before doing any further operations. > > Does that sound OK, have I missed anything? > Sounds reasonable to me. /J?rgen From nwagner at iam.uni-stuttgart.de Tue Feb 9 14:22:10 2010 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Tue, 09 Feb 2010 20:22:10 +0100 Subject: [IPython-dev] ImportError: No module named ipapp Message-ID: Hi all, I have installed ipython from scratch bzr branch lp:ipython cd ipython python setup.py install --prefix=$HOME/local If I start ipython ipython Traceback (most recent call last): File "/home/nwagner/local/bin/ipython", line 4, in from IPython.core.ipapp import launch_new_instance ImportError: No module named ipapp Any idea ? Nils From ellisonbg.net at gmail.com Tue Feb 9 23:34:48 2010 From: ellisonbg.net at gmail.com (Brian Granger) Date: Tue, 9 Feb 2010 20:34:48 -0800 Subject: [IPython-dev] ImportError: No module named ipapp In-Reply-To: References: Message-ID: <6ce0ac131002092034s7e6c27e6kdf34be3082af137b@mail.gmail.com> Can you try removing all traces of ipython (both from bin dirs and site-packages) and reinstall. It looks like you have multiple versions on your PATH/PYTHONPATH that are conflicting. Cheers, Brian On Tue, Feb 9, 2010 at 11:22 AM, Nils Wagner wrote: > Hi all, > > I have installed ipython from scratch > > bzr branch lp:ipython > cd ipython > python setup.py install --prefix=$HOME/local > > If I start ipython > > ipython > Traceback (most recent call last): > File "/home/nwagner/local/bin/ipython", line 4, in > > from IPython.core.ipapp import launch_new_instance > ImportError: No module named ipapp > > Any idea ? > > Nils > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg.net at gmail.com Tue Feb 9 23:37:29 2010 From: ellisonbg.net at gmail.com (Brian Granger) Date: Tue, 9 Feb 2010 20:37:29 -0800 Subject: [IPython-dev] isympy doesn't work with the new ipython-dev In-Reply-To: References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com> Message-ID: <6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com> Erik, This approach will also work. With the current trunk, you shouldn't even have to move the ipython_config_sympy.py to .ipython. But if you do move it over, you can change it to your taste. Can you mention this on the sympy list to see what approach they want to take? Cheers, Brian I encountered some problems with isympy as well, but managed to get it > to work (at least for my purposes) by changing /usr/local/bin/isympy > (or wherever your script installs the isympy script) to just: > > #!/bin/bash > ipython -p sympy > > > and then I put a ipython_config_sympy.py file in the .ipython > directory with the following: > > > c = get_config() > > # This can be used at any point in a config file to load a sub config > # and merge it into the current one. > load_subconfig('ipython_config.py') > > lines = """ > from __future__ import division > from sympy import * > x, y, z = symbols('xyz') > k, m, n = symbols('kmn', integer=True) > f, g, h = map(Function, 'fgh') > """ > > # You have to make sure that attributes that are containers already > # exist before using them. Simple assigning a new list will override > # all previous values. > if hasattr(c.Global, 'exec_lines'): > c.Global.exec_lines.append(lines) > else: > c.Global.exec_lines = [lines] > > > > On Fri, Jan 22, 2010 at 4:58 PM, Fernando Perez > wrote: > > On Fri, Jan 22, 2010 at 1:20 PM, G?khan Sever > wrote: > >> > >> I can't get the isympy working with the Fernando's trunk. I solved one > of > >> the missing attribute error but stuck at the other one: > > > > No, things have changed a bit. I'm insanely pressed for time right > > now, the best I suggest is looking at this test code file, around line > > 130: > > > > > http://bazaar.launchpad.net/%7Efdo.perez/ipython/trunk-dev/annotate/head%3A/IPython/testing/globalipapp.py#L130 > > > > > > That makes the kind of shell you're looking for, sort of. > > > > Sorry not to be able to give you a detailed example right now... > > > > cheers, > > > > f > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue Feb 9 23:45:34 2010 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 9 Feb 2010 23:45:34 -0500 Subject: [IPython-dev] isympy doesn't work with the new ipython-dev In-Reply-To: <6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com> References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com> <6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com> Message-ID: On Tue, Feb 9, 2010 at 11:37 PM, Brian Granger wrote: > Erik, > > This approach will also work.? With the current trunk, you shouldn't even > have > to move the ipython_config_sympy.py to .ipython.? But if you do move it > over, > you can change it to your taste.? Can you mention this on the sympy list to > see what approach they want to take? Erik, it's worth mentioning we've been mulling the idea of allowing profiles to reside in third-party projects. Something like iptyhon -p sympy:isympy would make IPython look for sympy/ipython/profile/ipython_config_isympy.py So that Sympy itself (using sympy just as an example) could ship its profiles/config files by including a directory named ipython/ in its own tree, with config/profile directories. We'd like to get feedback on this idea. It's not implemented yet, we've just been thinking about it to minimize the coupling between project schedules and make it really easy for projects to expose ipython-based functionality without having to ask their users to copy profile/config files. Cheers, f From ellisonbg.net at gmail.com Tue Feb 9 23:59:55 2010 From: ellisonbg.net at gmail.com (Brian Granger) Date: Tue, 9 Feb 2010 20:59:55 -0800 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS Message-ID: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> Hello all, As many of you know, we have been working hard on IPython 0.11. One of the big new features of 0.11 is a refactored integration with GUI event loops. We are very excited about this as it is much more stable and add new capabilities, like being able to enable pylab *after* starting IPython (it is a magic: %pylab). BUT, as Fernando and I have started to use the IPython 0.11 alpha for our daily work, we have found some subtle issue with IPython 0.11 and the current versions of matplotlib and Enthought tool suite (traits, chaco, mayavi). All of these packages rely on IPython's previous way of handling GUI integration. Because of this, current versions of matplotlib and ets don't work fully with IPython 0.11. In some cases, they work OK, but there are a whole class of bugs that keep popping up. We don't want to release IPython 0.11 until these issues are resolved. All of these bugs are related to a few aspects of creating the main GUI app objects and starting event loops. We think we understand the issues, but we will need to work with the devs of matplotlib and ets to decide on how we all want to handle these things. Here is where we are at... Current situation ============= Both matplotlib and ets have code that tries to: * See what GUI toolkit is being used * Get the global App object if it already exists, if not create it. * See if the main loop is running, if not possibly start it. All of this logic makes many assumptions about how IPython affects the answers to these questions. Because IPython's GUI support has changed in significant ways, current matplotlib and ets make incorrect decisions about these issues (such as trying to start the event loop a second time, creating a second main App ojbect, etc.) under IPython 0.11. This leads to crashes... Description of GUI support in 0.11 ========================== IPython allows GUI event loops to be run in an interactive IPython session. This is done using Python's PyOS_InputHook hook which Python calls when the :func:`raw_input` function is called and is waiting for user input. IPython has versions of this hook for wx, pyqt4 and pygtk. When the inputhook is called, it iterates the GUI event loop until a user starts to type again. When the user stops typing, the event loop iterates again. This is how tk works. When a GUI program is used interactively within IPython, the event loop of the GUI should *not* be started. This is because, the PyOS_Inputhook itself is responsible for iterating the GUI event loop. IPython has facilities for installing the needed input hook for each GUI toolkit and for creating the needed main GUI application object. Usually, these main application objects should be created only once and for some GUI toolkits, special options have to be passed to the application object to enable it to function properly in IPython. What we need to decide =================== We need to answer the following questions: * Who is responsible for creating the main GUI application object, IPython or third parties (matplotlib, enthought.traits, etc.)? * What is the proper way for third party code to detect if a GUI application object has already been created? If one has been created, how should the existing instance be retrieved? * In a GUI application object has been created, how should third party code detect if the GUI event loop is running. It is not sufficient to call the relevant function methods in the GUI toolkits (like ``IsMainLoopRunning``) because those don't know if the GUI event loop is running through the input hook. * We might need a way for third party code to determine if it is running in IPython or not. Currently, the only way of running GUI code in IPython is by using the input hook, but eventually, GUI based versions of IPython will allow the GUI event loop in the more traditional manner. We will need a way for third party code to distinguish between these two cases. Thanks for participating in this discussion! Cheers, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From hans_meine at gmx.net Wed Feb 10 03:06:56 2010 From: hans_meine at gmx.net (Hans Meine) Date: Wed, 10 Feb 2010 09:06:56 +0100 Subject: [IPython-dev] isympy doesn't work with the new ipython-dev In-Reply-To: References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com> <6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com> Message-ID: <201002100907.02504.hans_meine@gmx.net> On Mittwoch 10 Februar 2010, Fernando Perez wrote: > Erik, it's worth mentioning we've been mulling the idea of allowing > profiles to reside in third-party projects. Something like > > iptyhon -p sympy:isympy > > would make IPython look for > > sympy/ipython/profile/ipython_config_isympy.py > > [...] > We'd like to get feedback on this idea. What kind of feedback are you waiting for? That idea is terrific! :-) I am sure that *many* projects will be interested in this - I for sure will use it with some of my own software! (OK, I guess it would already be possible to install some minimal executable which does the equivalent of starting ipython and loading some profile, but this seems to be easier to set up.) Ciao, / / .o. /--/ ..o / / ANS ooo PS: I dislike the change of the executable name though. ;-P -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: From hans_meine at gmx.net Wed Feb 10 03:26:14 2010 From: hans_meine at gmx.net (Hans Meine) Date: Wed, 10 Feb 2010 09:26:14 +0100 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> Message-ID: <201002100926.15886.hans_meine@gmx.net> On Mittwoch 10 Februar 2010, Brian Granger wrote: > When a GUI program is used interactively within IPython, the event loop of > the GUI should *not* be started. This is because, the PyOS_Inputhook itself > is responsible for iterating the GUI event loop. From a user's perspective (i.e. not from an MPL/ETS point of view), the above also rings bells: I am used to running GUI apps from within ipython, where (with older IPython versions) I'll get at most an exception from running sys.exit(), but my main window pops up and the app is running while I still have the interpreter available. That's an incredibly important use case to me. Now if the mainloop must not be started with 0.11, how about monkey-patching the application classes (i.e. QCoreApplication.exec_) in order to allow running unpatched programs when the GUI toolkit integration is active? (In fact, in our own Qt3-based python terminal, we had monkeypatched QApplication in order to also ignore the creation of a second app object.) -- Ciao, / / .o. /--/ ..o / / ANS ooo -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: From Chris.Barker at noaa.gov Wed Feb 10 13:15:34 2010 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Wed, 10 Feb 2010 10:15:34 -0800 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <201002100926.15886.hans_meine@gmx.net> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002100926.15886.hans_meine@gmx.net> Message-ID: <4B72F7C6.9060109@noaa.gov> Hi folks, I'm new to the Ipython dev list, and don't really understand the internals, but this thread was posted elsewhere to solicit ideas, so here I am: > One of the > big new features of 0.11 is a refactored integration with GUI event loops. > We are very excited about this as it is much more stable and add new > capabilities, like being able to enable pylab *after* starting IPython (it > is a magic: %pylab). very nice. > Current situation > ============= > > Both matplotlib and ets have code that tries to: > > * See what GUI toolkit is being used > * Get the global App object if it already exists, if not create it. > * See if the main loop is running, if not possibly start it. Are these done in order to support IPython? or for other general usage? > Description of GUI support in 0.11 > ========================== > > IPython allows GUI event loops to be run in an interactive IPython session. > This is done using Python's PyOS_InputHook hook which Python calls > when the :func:`raw_input` function is called and is waiting for user input. So it doesn't use threads at all? that is good news. > some > GUI toolkits, special options have to be passed to the application object > to enable it to function properly in IPython. so the app itself needs to do something special? that is too bad -- so far, I've found IPython works pretty well with my own wxPython apps, with nothing changed in my app. > * Who is responsible for creating the main GUI application object, IPython > or third parties (matplotlib, enthought.traits, etc.)? I think the third parties -- i.e. you really want to be able to use IPython with ANY GUI app - not just one written to be used with IPython. > * What is the proper way for third party code to detect if a GUI application > object has already been created? If one has been created, how should > the existing instance be retrieved? This is why it seems better for IPython NOT to create the GUI app object. If it doesn't, would the third party code have to detect it? hmm, yes it would, or something -- if I want to call my program with "run" more than once, then a wxApp has been initialized already, and I think you can only do that once per process. Could Ipython monkey-patch the GUI toolkits to override the app creation and mainloop starting? I'm not sure this is possible, but a typical wx app might look like this: app = wx.App() app.MainLoop() if wx had been monkey-patched by IPython, the app creation and mainloop calls could be overridden with ones that check to see if they have been called before, and do something different if so. As for wx, I don't know if you can stop and re-start the mainloop anyway. I'm still confused how this all seems to work for me with IPython 0.9.1! I hope this is helpful, -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From erik.tollerud at gmail.com Wed Feb 10 17:11:07 2010 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Wed, 10 Feb 2010 14:11:07 -0800 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <201002100926.15886.hans_meine@gmx.net> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002100926.15886.hans_meine@gmx.net> Message-ID: I wholly concur with Hans that running GUI apps simultaneously (e.g. non-blocking) within IPython is crucial, but it's not necessarily the case that because IPython doesn't start the GUI it doesn't get started - as Christopher says, it could just be foisting the responsibility onto the third party... On the other hand, there are clearly some problems there (at least in the current designs) - I've been running on the bzr ipython for a while now along with traits GUIs, and I've noticed a big change in behavior from IPython < .11: it used to be when you called traitedobject.configure_traits() or traitedobject.edit_traits(), the GUIs would pop up and everything would run fine. But now, configure_traits is blocking (and it kills the app at the end, making edit_traits stop working), while edit_traits has the same behavior as before, so I use edit_traits... On the other hand, if I have a non-interactive script (e.g. not ipython) if I run the exact same code, edit_traits freaks out because no app object has been created. For now I've just ignored the problem, as I mostly work in interactive mode, but it would be way better if I could use a single function call in either situation. But that could be resolved in either traits or IPython, presumably. As for which layer that belongs on... personally, I think that anything that requires "two-way" communication between IPython and the third-party app is a bad idea do to the dangers of version desync. e.g. if Traits has to know it's in IPython to know that it should create the app, there's a huge potentially for the code getting out-of-sync between the two projects whenever some new feature gets implemented with the inevitable unintended consequences, causing headaches for all involved. So I would advocate either monkey-patching as some of the others here have suggested, or use some method of passing the information through the GUI toolkit instead of directly from IPython to the third-party if possible. I'm really only familiar with wx, but do the other toolkits have an equivalent wx.GetApp() function? And are there similar mechanisms that can leave messages embedded somewhere in the toolkit as to whether or not the PyOS_InputHook is calling as opposed to living in the event loop? If that's the case, most everything should be left to the 3rd party app... On Wed, Feb 10, 2010 at 12:26 AM, Hans Meine wrote: > On Mittwoch 10 Februar 2010, Brian Granger wrote: >> When a GUI program is used interactively within IPython, the event loop of >> the GUI should *not* be started. This is because, the PyOS_Inputhook itself >> is responsible for iterating the GUI event loop. > > From a user's perspective (i.e. not from an MPL/ETS point of view), the above > also rings bells: I am used to running GUI apps from within ipython, where > (with older IPython versions) I'll get at most an exception from running > sys.exit(), but my main window pops up and the app is running while I still > have the interpreter available. ?That's an incredibly important use case to > me. > > Now if the mainloop must not be started with 0.11, how about monkey-patching > the application classes (i.e. QCoreApplication.exec_) in order to allow > running unpatched programs when the GUI toolkit integration is active? > > (In fact, in our own Qt3-based python terminal, we had monkeypatched > QApplication in order to also ignore the creation of a second app object.) > > -- > Ciao, / ?/ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?.o. > ? ? /--/ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ..o > ? ?/ ?/ ANS ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ooo > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > From Chris.Barker at noaa.gov Wed Feb 10 18:23:26 2010 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Wed, 10 Feb 2010 15:23:26 -0800 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002100926.15886.hans_meine@gmx.net> Message-ID: <4B733FEE.1040003@noaa.gov> Erik Tollerud wrote: > As for which layer that belongs on... personally, I think that > anything that requires "two-way" communication between IPython and the > third-party app is a bad idea do to the dangers of version desync. And the fact that there are an infinite number of potential apps to communicate with. MPL and ETS-based apps is great, but it would be so much greater if IPython could be used with ANY app. > or use some > method of passing the information through the GUI toolkit instead of > directly from IPython to the third-party if possible. If monkey patching isn't possible, then this probably is, and there are only a handful of GUI toolkits. If IPython were to work well with PyGTK, PyQT, wxPython and TKinter (and maybe PyObjC), that would go a long way. > I'm really only familiar with wx, me too, but I'd be happy to try to help with it. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From termim at gmail.com Thu Feb 11 01:16:50 2010 From: termim at gmail.com (Mikhail Terekhov) Date: Thu, 11 Feb 2010 01:16:50 -0500 Subject: [IPython-dev] isympy doesn't work with the new ipython-dev In-Reply-To: References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com> <6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com> Message-ID: <12aaa0811002102216i2b1be3e0sb6350c0ad5ce41a4@mail.gmail.com> On Tue, Feb 9, 2010 at 11:45 PM, Fernando Perez wrote: > > Erik, it's worth mentioning we've been mulling the idea of allowing > profiles to reside in third-party projects. ?Something like > > iptyhon -p sympy:isympy > > would make IPython look for > > sympy/ipython/profile/ipython_config_isympy.py > > So that Sympy itself (using sympy just as an example) could ship its > profiles/config files by including a directory named ipython/ in its > own tree, with config/profile directories. > > We'd like to get feedback on this idea. ?It's not implemented yet, > we've just been thinking about it to minimize the coupling between > project schedules and make it really easy for projects to expose > ipython-based functionality without having to ask their users to copy > profile/config files. > I'd suggest to allow loading profiles from _arbitrary_ place. Why it is forbidden to have some specialized profile just somewhere? For example, I would benefit if it was possible to have profile somewhere on NFS and my group could use it from different computers. This would allow centralized profile management in big organizations. Regards, -- Mikhail Terekhov From ellisonbg.net at gmail.com Thu Feb 11 15:07:06 2010 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 11 Feb 2010 12:07:06 -0800 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <201002100926.15886.hans_meine@gmx.net> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002100926.15886.hans_meine@gmx.net> Message-ID: <6ce0ac131002111207w4c806528w58c926d6e064e8c1@mail.gmail.com> Hans, > From a user's perspective (i.e. not from an MPL/ETS point of view), the > above > also rings bells: I am used to running GUI apps from within ipython, where > (with older IPython versions) I'll get at most an exception from running > sys.exit(), but my main window pops up and the app is running while I still > have the interpreter available. That's an incredibly important use case to > me. Yes and we absolutely need to support this usage case. > Now if the mainloop must not be started with 0.11, how about monkey-patching > the application classes (i.e. QCoreApplication.exec_) in order to allow > running unpatched programs when the GUI toolkit integration is active? Fernando and I spend a full day at Scipy 2009 trying this monkey patching approach. Summary: it doesn't work. Full details: each GUI toolkits has additional ways of running the event loop (other than just calling "run") for a short period of time. If everyone avoided these other calls the monkey patch approach would work. But *many* people (including matplotlib and traits) do need to spin the event loop by hand and call these functions. All of our monkeypatch approaches broke these things. > (In fact, in our own Qt3-based python terminal, we had monkeypatched > QApplication in order to also ignore the creation of a second app object.) Yes, this type of monkey patch is not too difficult to handle. But *any* GUI app should first call getApp or its equivalent before creating an App object to make sure a 2nd one is created. The more difficult one is to figure out if the event loop is running. This is difficult because there are so many different ways of running the event loop and the "isMainLoopRunning" type functions don't account for all of these. Cheers, Brian > -- > Ciao, / / .o. > /--/ ..o > / / ANS ooo > From ellisonbg.net at gmail.com Thu Feb 11 15:19:04 2010 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 11 Feb 2010 12:19:04 -0800 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <4B72F7C6.9060109@noaa.gov> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002100926.15886.hans_meine@gmx.net> <4B72F7C6.9060109@noaa.gov> Message-ID: <6ce0ac131002111219u568441f9n7d1bd431d685afe4@mail.gmail.com> Chris, > I'm new to the Ipython dev list, and don't really understand the > internals, but this thread was posted elsewhere to solicit ideas, so > here I am: Thanks for joining the discussion, it helps alot. >> Both matplotlib and ets have code that tries to: >> >> * See what GUI toolkit is being used >> * Get the global App object if it already exists, if not create it. >> * See if the main loop is running, if not possibly start it. > > Are these done in order to support IPython? or for other general usage? Yes, as I understand it that is why this logic exists (for IPython's -wthread or -pylab). > So it doesn't use threads at all? that is good news. Yep, no threads at all. Other than the bugs we are talking about here it is *much* more robust and stable. >> some >> GUI toolkits, special options have to be passed to the application object >> to enable it to function properly in IPython. > > so the app itself needs to do something special? that is too bad -- so > far, I've found IPython works pretty well with my own wxPython apps, > with nothing changed in my app. Yes, you are understanding the issue well. We don't see anyway around this. If you want to use GUI code both in IPython and outside of IPython, you will need to have a small amount of custom logic. Fernando and I spent an entire day at Scipy 2009 trying to figure out a way to prevent this...no luck.. I am afraid this is a fundamental limitation unless all the GUI toolkits radically change their APIs. >> * Who is responsible for creating the main GUI application object, IPython >> or third parties (matplotlib, enthought.traits, etc.)? > > I think the third parties -- i.e. you really want to be able to use > IPython with ANY GUI app - not just one written to be used with IPython. Again, unless qt/wx/gtk change their code, I don't think this is possible. Plus even if they did, Ipython still has to work with current/older versions. By third party here I am thinking of projects like matplotlib/enthought that, in general, are commited to maintaining good integration with Ipython. >> * What is the proper way for third party code to detect if a GUI >> application >> object has already been created? If one has been created, how should >> the existing instance be retrieved? > > This is why it seems better for IPython NOT to create the GUI app > object. If it doesn't, would the third party code have to detect it? In general there are reliable ways for third party code to detect if an app object has been created already. But, creating an app is not always trivial. In some cases (like wx) in order to work properly with IPython, specific arguments have to be passed (non-default ones) to the app. In these cases, it makes sense for IPython to handle it. > hmm, yes it would, or something -- if I want to call my program with > "run" more than once, then a wxApp has been initialized already, and I > think you can only do that once per process. Yep. > Could Ipython monkey-patch the GUI toolkits to override the app creation > and mainloop starting? I'm not sure this is possible, but a typical wx > app might look like this: > > app = wx.App() > app.MainLoop() The app creation step is pretty easy to handle. Most GUI code should call getApp first to make sure the app has not already been created before creating an app from scratch. The more difficult question is knowing if you should start the main loop. For example, if IPython is already running the event loop using pyos_inputhook, calling app.MainLoop will lead to bad things. Unfortunately, calling isMainLoopRunning() returns False, so you don't have a good way of discerning what to do....currently. > if wx had been monkey-patched by IPython, the app creation and mainloop > calls could be overridden with ones that check to see if they have been > called before, and do something different if so. As for wx, I don't know > if you can stop and re-start the mainloop anyway. See my other post, but the mainloop monkeypatch approach does not work. It breaks qt/wx/etc. > I'm still confused how this all seems to work for me with IPython 0.9.1! It doesn't *really* work :) The pre 0.11 code is not thread safe and does very subtle tricks to get cross thread signals working....which is why it crashes often and breaks if you touch the code. > I hope this is helpful, Very much, thank you. Cheers, Brian > -Chris > > > > -- > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Thu Feb 11 15:20:04 2010 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 11 Feb 2010 12:20:04 -0800 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <4B733FEE.1040003@noaa.gov> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002100926.15886.hans_meine@gmx.net> <4B733FEE.1040003@noaa.gov> Message-ID: <6ce0ac131002111220v4610b49ey4294e65cf5954fc5@mail.gmail.com> > If monkey patching isn't possible, then this probably is, and there are > only a handful of GUI toolkits. If IPython were to work well with PyGTK, > PyQT, wxPython and TKinter (and maybe PyObjC), that would go a long way. Yes, those are our targets and all of these GUI toolkits support the pyos_inputhook approach. Cheers, Brian > > I'm really only familiar with wx, > > me too, but I'd be happy to try to help with it. > > -Chris > > > > -- > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Thu Feb 11 15:27:06 2010 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 11 Feb 2010 12:27:06 -0800 Subject: [IPython-dev] isympy doesn't work with the new ipython-dev In-Reply-To: <201002100907.02504.hans_meine@gmx.net> References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com> <6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com> <201002100907.02504.hans_meine@gmx.net> Message-ID: <6ce0ac131002111227n111df2a6s4a323c56db4621e9@mail.gmail.com> On 2/10/10, Hans Meine wrote: > On Mittwoch 10 Februar 2010, Fernando Perez wrote: >> Erik, it's worth mentioning we've been mulling the idea of allowing >> profiles to reside in third-party projects. Something like >> >> iptyhon -p sympy:isympy >> >> would make IPython look for >> >> sympy/ipython/profile/ipython_config_isympy.py >> >> [...] >> We'd like to get feedback on this idea. > > What kind of feedback are you waiting for? > > That idea is terrific! :-) This type of feedback :) > I am sure that *many* projects will be interested in this - I for sure will > use it with some of my own software! (OK, I guess it would already be > possible to install some minimal executable which does the equivalent of > starting ipython and loading some profile, but this seems to be easier to > set > up.) Thanks! Brian > Ciao, / / .o. > /--/ ..o > / / ANS ooo > > PS: I dislike the change of the executable name though. ;-P > From vivainio at gmail.com Thu Feb 11 15:27:28 2010 From: vivainio at gmail.com (Ville M. Vainio) Date: Thu, 11 Feb 2010 22:27:28 +0200 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: References: <46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com> Message-ID: <46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com> On Tue, Feb 9, 2010 at 6:45 PM, Fernando Perez wrote: > I've realized I let this linger without action, and we probably should > do something about it while it's (relatively) fresh in our minds. You may be into the same entertaining chore soon again, I'm running Lucid beta's and am seeing: """ Doing on-the-fly conversion from to . This may take some time. Upgrade the repositories to the same format for better performance. """ >> Judging by my experience with git, I don't think it would be much of >> an improvement. Mercurial, on the other hand... > > That's a topic for another day, but I'd be curious about your > experiences. ?The more I use git the more I like it and I've only used > hg very lightly, so I'd be interested in this. I'm using git with gitorious myself, but basically it's just that I'm unable to feel confident with the program at all (as in, when problems appear, I'm not sure I can intuitively understand them, like I do with mercurial and bzr). I guess it has everything to do with the bad "ui" of git, cryptic error messages etc. So far, I haven't really hit much problems, but I've seen a lot of other people do, and on those situations they tend to need expert support. Not so with hg, people can basically use it after they understand dvcs in general. I also like the fact that it's trivial to put up a "hg serve" session for quick collaborative hacking, and that you get time based "simple" revision numbers instead of relying only on hashes. -- Ville M. Vainio http://tinyurl.com/vainio From ellisonbg.net at gmail.com Thu Feb 11 15:28:54 2010 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 11 Feb 2010 12:28:54 -0800 Subject: [IPython-dev] isympy doesn't work with the new ipython-dev In-Reply-To: <12aaa0811002102216i2b1be3e0sb6350c0ad5ce41a4@mail.gmail.com> References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com> <6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com> <12aaa0811002102216i2b1be3e0sb6350c0ad5ce41a4@mail.gmail.com> Message-ID: <6ce0ac131002111228w4213c16bs61ec57e8468cf421@mail.gmail.com> Profiles can be loaded from arbitrary places currently, but Fernando's approach would allow profiles to be found using the regular import mechanisms in IPython. Basically any module or package in sys.path could declare the location of an IPython profile using this API. Cheers, Brian > I'd suggest to allow loading profiles from _arbitrary_ place. > Why it is forbidden to have some specialized profile just somewhere? > For example, I would benefit if it was possible to have profile somewhere > on NFS and my group could use it from different computers. This would > allow centralized profile management in big organizations. > > Regards, > -- > Mikhail Terekhov > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From hans_meine at gmx.net Fri Feb 12 04:31:57 2010 From: hans_meine at gmx.net (Hans Meine) Date: Fri, 12 Feb 2010 10:31:57 +0100 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: <46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com> References: <46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com> Message-ID: <201002121031.57250.hans_meine@gmx.net> On Thursday 11 February 2010 21:27:28 Ville M. Vainio wrote: > [...] Not so with hg, people can basically use it > after they understand dvcs in general. > > I also like the fact that it's trivial to put up a "hg serve" session > for quick collaborative hacking, and that you get time based "simple" > revision numbers instead of relying only on hashes. OK, my 2 cents: About 2 years ago, I made the choice myself. I went with hg, because I saw the majority of (python) projects I watched making this choice. I did never regret it, to the contrary. From the discussions on this list and on many other OSS development lists, my impression is that - bzr's strenth is launchpad, but it sometimes tries to be too clever and it's behavior is not completely trivial to predict, while - git is for professionals. Its UI is said to have improved, but still it seems to be the swiss army knife which you need to know very well in order to use the correct part of it. hg's big strength is its simplicity, really. Out of the box (i.e. without any third-party extensions), it is incredibly simple to use and understand. After a while you start to become more brave and try out some extensions which offer dangerous commands, but I have always stayed in harmony with mercurial. The mercurial mailing list offers help for everyone, often discussing either optimized ways to perform certain tasks, or general DVCS workflows, often with posters trying to convince their bosses to make the switch from SVN or (shudder) CVS. But I have never had the impression that there's anything hard to understand with mercurial. Oh, and BTW: the repository format is *incredibly* stable, with only one minor (filename mangling) change in all the years. Basically, Matt chose the correct design from the beginning. I do not plan to convince anybody that hg is better than bzr, but I want to report that I am very happy with it (and I am willing to answer hg-related questions if you want more input). HTH, Hans From hans_meine at gmx.net Fri Feb 12 04:42:32 2010 From: hans_meine at gmx.net (Hans Meine) Date: Fri, 12 Feb 2010 10:42:32 +0100 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <6ce0ac131002111207w4c806528w58c926d6e064e8c1@mail.gmail.com> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002100926.15886.hans_meine@gmx.net> <6ce0ac131002111207w4c806528w58c926d6e064e8c1@mail.gmail.com> Message-ID: <201002121042.32877.hans_meine@gmx.net> On Thursday 11 February 2010 21:07:06 you wrote: > > Now if the mainloop must not be started with 0.11, how about > > monkey-patching the application classes (i.e. QCoreApplication.exec_) in > > order to allow running unpatched programs when the GUI toolkit > > integration is active? > > Fernando and I spend a full day at Scipy 2009 trying this monkey > patching approach. > Summary: it doesn't work. That's very interesting, I'd like to know more details. > Full details: each GUI toolkits has additional > ways of running the event loop (other than just calling "run") for a short > period of time. Right. For Qt, would that be qApp.processEvents() (i.e. "handle [some] pending events"), or QDialog.exec_ (which actually waits for user input in a recursively entered eventloop)? Both are commonly used and must work indeed. > If everyone avoided these other calls the monkey > patch approach would work. But *many* people (including matplotlib > and traits) do need to spin the event loop by hand and call these > functions. Right, but which of the above methods cause problems, and why exactly? > > (In fact, in our own Qt3-based python terminal, we had monkeypatched > > QApplication in order to also ignore the creation of a second app > > object.) > > Yes, this type of monkey patch is not too difficult to handle. But > *any* GUI app should first call getApp or its equivalent before > creating an App object to make sure a 2nd one is created. Huh? Honestly, why should applications modify their __main__ to check for *existing* global objects - I think we're talking about a special case here which should be handled transparently, don't you agree? (OK, I *do* occasionally check for existing objects in my __main__, in order be able to quickly reload a program with "%run -i", without losing the results of some expensive computations, but that's really a special case IMO.) Have a nice day, Hans From gael.varoquaux at normalesup.org Fri Feb 12 05:02:52 2010 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 12 Feb 2010 11:02:52 +0100 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <201002121042.32877.hans_meine@gmx.net> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002100926.15886.hans_meine@gmx.net> <6ce0ac131002111207w4c806528w58c926d6e064e8c1@mail.gmail.com> <201002121042.32877.hans_meine@gmx.net> Message-ID: <20100212100252.GA20392@phare.normalesup.org> On Fri, Feb 12, 2010 at 10:42:32AM +0100, Hans Meine wrote: > Huh? Honestly, why should applications modify their __main__ to check for > *existing* global objects - Because nobody should code thinking they own the main. It makes debugging and reusing much harder. That's my point of view, and it comes from a long experience of trying to help people run code, debug code, or simply get the job done using third party code. Ga?l From hans_meine at gmx.net Fri Feb 12 05:08:18 2010 From: hans_meine at gmx.net (Hans Meine) Date: Fri, 12 Feb 2010 11:08:18 +0100 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <20100212100252.GA20392@phare.normalesup.org> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002121042.32877.hans_meine@gmx.net> <20100212100252.GA20392@phare.normalesup.org> Message-ID: <201002121108.19174.hans_meine@gmx.net> Hi Ga?l, On Friday 12 February 2010 11:02:52 you wrote: > On Fri, Feb 12, 2010 at 10:42:32AM +0100, Hans Meine wrote: > > Huh? Honestly, why should applications modify their __main__ to check > > for *existing* global objects - > > Because nobody should code thinking they own the main. It makes debugging > and reusing much harder. But I am thinking of mains like the following: if __name__ == "__main__": import sys app = QtGui.QApplication(sys.argv) mw = BeverageMainWindow() mw.resize(800, 600) mw.show() sys.exit(app.exec_()) This is a prototypical __main__ which I'd like to be able to %run from IPython as well as from the commandline. If I want to reuse code, I'll import the file and reuse BeverageMainWindow, but not the __main__ code. > That's my point of view, and it comes from a long experience of trying to > help people run code, debug code, or simply get the job done using third > party code. Maybe then those people put too much code into __main__? Have a nice day, Hans From gael.varoquaux at normalesup.org Fri Feb 12 05:17:22 2010 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 12 Feb 2010 11:17:22 +0100 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <201002121108.19174.hans_meine@gmx.net> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002121042.32877.hans_meine@gmx.net> <20100212100252.GA20392@phare.normalesup.org> <201002121108.19174.hans_meine@gmx.net> Message-ID: <20100212101722.GB20392@phare.normalesup.org> On Fri, Feb 12, 2010 at 11:08:18AM +0100, Hans Meine wrote: > > > Huh? Honestly, why should applications modify their __main__ to check > > > for *existing* global objects - > > Because nobody should code thinking they own the main. It makes debugging > > and reusing much harder. > But I am thinking of mains like the following: > if __name__ == "__main__": > import sys > app = QtGui.QApplication(sys.argv) > mw = BeverageMainWindow() > mw.resize(800, 600) > mw.show() > sys.exit(app.exec_()) > This is a prototypical __main__ which I'd like to be able to %run from > IPython as well as from the commandline. If I want to reuse code, I'll > import the file and reuse BeverageMainWindow, but not the __main__ > code. OK, but in the above code, you are saying that you want to own the application object. What if you want to interact with other code that does the same thing? There can only be one like this. > > That's my point of view, and it comes from a long experience of trying to > > help people run code, debug code, or simply get the job done using third > > party code. > Maybe then those people put too much code into __main__? Most probably, but if you follow this logic, then you end up deciding that nobody should ever create a new application object without checking that one already exists, unless they want exactly that (eg for explicit threading reasons). What I am trying to say is that you have contradictory requirements: you are asking for full ownership of the application, but you would also like to play nice with other environments. This is going to fail, and not only in IPython. I'd be interested: how does such code behave when run through an execfile in a PyQt IDE, for instance eric4 or spyder? Basically, whoever requires control and ownership of a global state will be limited in its interaction with other programs, and I don't believe there is a way around it. Ga?l From gael.varoquaux at normalesup.org Fri Feb 12 05:37:31 2010 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 12 Feb 2010 11:37:31 +0100 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: <201002121031.57250.hans_meine@gmx.net> References: <46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com> <201002121031.57250.hans_meine@gmx.net> Message-ID: <20100212103731.GC20392@phare.normalesup.org> On Fri, Feb 12, 2010 at 10:31:57AM +0100, Hans Meine wrote: > On Thursday 11 February 2010 21:27:28 Ville M. Vainio wrote: > > [...] Not so with hg, people can basically use it > > after they understand dvcs in general. > > I also like the fact that it's trivial to put up a "hg serve" session > > for quick collaborative hacking, and that you get time based "simple" > > revision numbers instead of relying only on hashes. > OK, my 2 cents: And mine... I use heavily only bzr, and I see its limitations. I agree with Hans that its main strength is nowadays Launchpad, although that fact that it is easy to get going under Windows is a pro too. Out of the 3 main DVCSs, I am also most tempted by hg, because I feels it strikes the right balance between power and simplicity. However, for me hg has a killer feature: subrepositories (or the forest extension, previously): http://mercurial.selenic.com/wiki/subrepos I work in many projects where the natural setting is actually a group of projects evolving together. Sub-repositories are the natural answer to this situation and a big pro for me because the reduce technical and social difficulties with loosely coupled projects: better than a fully-integrated repo as projects have some independence, but better than separate repos, as it is possible to have comon history. Maybe git has a similar feature. I haven't seen it yet. Ga?l From hans_meine at gmx.net Fri Feb 12 06:06:50 2010 From: hans_meine at gmx.net (Hans Meine) Date: Fri, 12 Feb 2010 12:06:50 +0100 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <20100212101722.GB20392@phare.normalesup.org> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002121108.19174.hans_meine@gmx.net> <20100212101722.GB20392@phare.normalesup.org> Message-ID: <201002121206.51045.hans_meine@gmx.net> On Friday 12 February 2010 11:17:22 you wrote: > On Fri, Feb 12, 2010 at 11:08:18AM +0100, Hans Meine wrote: > > if __name__ == "__main__": > > import sys > > > > app = QtGui.QApplication(sys.argv) > > > > mw = BeverageMainWindow() > > mw.resize(800, 600) > > mw.show() > > sys.exit(app.exec_()) > > > > This is a prototypical __main__ which I'd like to be able to %run from > > IPython as well as from the commandline. If I want to reuse code, I'll > > import the file and reuse BeverageMainWindow, but not the __main__ > > code. > > OK, but in the above code, you are saying that you want to own the > application object. Do I? I see it more like a minimal setup function for opening the main window, which is all I care about. The QApplication line is only there to make the stuff below work (with Qt, as with most toolkits AFAIK, I would otherwise get an error because the widgets will rely on global state being initialized by the application constructor). > What if you want to interact with other code that > does the same thing? There can only be one like this. Actually, the most important use case I see is that I repeatedly %run the same program. That would even work if I called app.setApplicationName(), .setVersion, ... which I did not in my above minimal example. > [...] if you follow this logic, then you end up deciding > that nobody should ever create a new application object without checking > that one already exists, unless they want exactly that (eg for explicit > threading reasons). OK, but then my __main__ becomes twice as long, because I must also not enter the main loop and not call sys.exit() in theory. I would simply prefer if IPython automatically handled that for me. I fully understand that this will not work in general, with any application. (For instance, Qt's database handling includes global state about the "default database" for your convenience etc.) OTOH, ipython mostly is about hacks for convenient interactive experiments, and most of my programs - do not rely on global state at all (i.e. the main window is all I actually care about), or - just set up some "more or less constant" global state (like app name & version) which would still work if I %run the same app several times (and probably degrade quite gracefully if I %run another app). > What I am trying to say is that you have contradictory requirements: you > are asking for full ownership of the application, I do not mean to. > but you would also like > to play nice with other environments. Not really either. ;-) Greetings, Hans From gael.varoquaux at normalesup.org Fri Feb 12 06:11:04 2010 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 12 Feb 2010 12:11:04 +0100 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <201002121206.51045.hans_meine@gmx.net> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002121108.19174.hans_meine@gmx.net> <20100212101722.GB20392@phare.normalesup.org> <201002121206.51045.hans_meine@gmx.net> Message-ID: <20100212111104.GA7228@phare.normalesup.org> On Fri, Feb 12, 2010 at 12:06:50PM +0100, Hans Meine wrote: > Do I? I see it more like a minimal setup function for opening the main > window, which is all I care about. The QApplication line is only there to > make the stuff below work (with Qt, as with most toolkits AFAIK, I would > otherwise get an error because the widgets will rely on global state being > initialized by the application constructor). Use: app = QtGui.QApplication.instance() That should suit your needs and avoid creating an QApplication if it already exists. Cheers, Ga?l From hans_meine at gmx.net Fri Feb 12 06:12:22 2010 From: hans_meine at gmx.net (Hans Meine) Date: Fri, 12 Feb 2010 12:12:22 +0100 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: <20100212103731.GC20392@phare.normalesup.org> References: <201002121031.57250.hans_meine@gmx.net> <20100212103731.GC20392@phare.normalesup.org> Message-ID: <201002121212.22530.hans_meine@gmx.net> On Friday 12 February 2010 11:37:31 you wrote: > On Fri, Feb 12, 2010 at 10:31:57AM +0100, Hans Meine wrote: > > On Thursday 11 February 2010 21:27:28 Ville M. Vainio wrote: > > > [...] Not so with hg, people can basically use it > > > after they understand dvcs in general. > > > > > > I also like the fact that it's trivial to put up a "hg serve" session > > > for quick collaborative hacking, and that you get time based "simple" > > > revision numbers instead of relying only on hashes. > > > > OK, my 2 cents: > > And mine... > > I use heavily only bzr, and I see its limitations. I agree with Hans that > its main strength is nowadays Launchpad, although that fact that it is > easy to get going under Windows is a pro too. I haven't had problems with hg's windows installer either. Or are you talking about TortoiseHg? AFAIK, that's also well-maintained and rapidly improving, but I don't have much experience with it (at least not on Windows *). HTH, Hans * Yes, contrary to my former belief, TortoiseHg can be used on Linux, too (for example as a convenient commit GUI). From hans_meine at gmx.net Fri Feb 12 06:25:04 2010 From: hans_meine at gmx.net (Hans Meine) Date: Fri, 12 Feb 2010 12:25:04 +0100 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <20100212111104.GA7228@phare.normalesup.org> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002121206.51045.hans_meine@gmx.net> <20100212111104.GA7228@phare.normalesup.org> Message-ID: <201002121225.05301.hans_meine@gmx.net> On Friday 12 February 2010 12:11:04 Gael Varoquaux wrote: > On Fri, Feb 12, 2010 at 12:06:50PM +0100, Hans Meine wrote: > > Do I? I see it more like a minimal setup function for opening the main > > window, which is all I care about. The QApplication line is only there > > to make the stuff below work (with Qt, as with most toolkits AFAIK, I > > would otherwise get an error because the widgets will rely on global > > state being initialized by the application constructor). > > Use: > > app = QtGui.QApplication.instance() > > That should suit your needs and avoid creating an QApplication if it > already exists. Thanks for the hint, but the docs say: "If no instance has been allocated, null is returned." Furthermore, what about app.exec_? I thought I needed sth. like: if __name__ == "__main__": hasApp = QtGui.QApplication.instance() if not hasApp: import sys app = QtGui.QApplication(sys.argv) mw = BeverageMainWindow() mw.resize(800, 600) mw.show() if not hasApp: sys.exit(app.exec_()) That's OK, I can do that, but I would still prefer not to have to change every application just for the occasional debugging session in ipython. And how do you want to communicate this to all ipython users? I mean - there needs to be some documentation about this boilerplate code for every toolkit, and every user needs to be aware of this and get it right. I fully agree that the above code has a much better design, is more well- behaved, but you know very well that most programs are not so well-behaved, and consequently you'll lock out a large fraction of existing programs from being %run from IPython. And from a user's POV, this will even appear to be a regression, even if previous support for that was buggy - there *was* support for it. I'd say not looking at other program's deficiencies, but closing the eyes and adding some proper hack would be much more in the spirit of IPython. Best, Hans From matthieu.brucher at gmail.com Fri Feb 12 06:32:18 2010 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Fri, 12 Feb 2010 12:32:18 +0100 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: <20100212103731.GC20392@phare.normalesup.org> References: <46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com> <201002121031.57250.hans_meine@gmx.net> <20100212103731.GC20392@phare.normalesup.org> Message-ID: 2010/2/12 Gael Varoquaux : > On Fri, Feb 12, 2010 at 10:31:57AM +0100, Hans Meine wrote: >> On Thursday 11 February 2010 21:27:28 Ville M. Vainio wrote: >> > [...] Not so with hg, people can basically ?use it >> > after they understand dvcs in general. > >> > I also like the fact that it's trivial to put up a "hg serve" session >> > for quick collaborative hacking, and that you get time based "simple" >> > revision numbers instead of relying only on hashes. > >> OK, my 2 cents: > > And mine... And mine ;) I've started with Bazaar and I'm still using it for my personal projects, but professionnally I switched to Hg, because I didn't need Launchpad there. Multiple branches are a extraordinary thing, but it can be messy if you have several different branches yourself. > I use heavily only bzr, and I see its limitations. I agree with Hans that > its main strength is nowadays Launchpad, although that fact that it is > easy to get going under Windows is a pro too. +1 > Out of the 3 main DVCSs, I am also most tempted by hg, because I feels it > strikes the right balance between power and simplicity. However, for me > hg has a killer feature: subrepositories (or the forest extension, > previously): http://mercurial.selenic.com/wiki/subrepos It's not that great. Bzr also has the same feature. The main issue is when you copy something from one subrepos to another: you loose the history. Basically, it is useless IMHO. > I work in many projects where the natural setting is actually a group of > projects evolving together. Sub-repositories are the natural answer to > this situation and a big pro for me because the reduce technical and > social difficulties with loosely coupled projects: better than a > fully-integrated repo as projects have some independence, but better than > separate repos, as it is possible to have comon history. I think the ultimate solution to this issue (that I also have) is not there yet :( Matthieu -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher From gael.varoquaux at normalesup.org Fri Feb 12 06:39:03 2010 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 12 Feb 2010 12:39:03 +0100 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <201002121225.05301.hans_meine@gmx.net> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002121206.51045.hans_meine@gmx.net> <20100212111104.GA7228@phare.normalesup.org> <201002121225.05301.hans_meine@gmx.net> Message-ID: <20100212113903.GB7228@phare.normalesup.org> On Fri, Feb 12, 2010 at 12:25:04PM +0100, Hans Meine wrote: > > Use: > > app = QtGui.QApplication.instance() > > That should suit your needs and avoid creating an QApplication if it > > already exists. > Thanks for the hint, but the docs say: > "If no instance has been allocated, null is returned." Hehe, you caught me bullshiting, I am afraid. :) How about: QtGui.qApp I believe that works, but I am not sure (being cautious this time). > And how do you want to communicate this to all ipython users? I mean - there > needs to be some documentation about this boilerplate code for every toolkit, > and every user needs to be aware of this and get it right. Honestly, I don't really care about how things are done. I am no longer really involved in IPython development. All I know is that trying to solve impossible problems leads to hair loss and disappointment. I believe you are trying to solve an impossible problem: making sure a user can use a main event loop without being aware it is around. IPython used to try and solve that problem using threads. It lead to segfaults because every single piece of code executed had to be injected in the main loop. So what was supposed to be transparent worked 99.9% of the time, and the rest of the time forced people to modify their code in subtle ways and debug race conditions, which is hard. The developers have now stepped back and decided that they would make a choice that forces users to be aware of the problems upfront, but has a simple solution. I am not saying this is a good or bad choice. I haven't had time to try out the new IPython, and I am worried of the problems with Mayavi and Traits. However, I am saying that there is some reasonning behind the choice, and that whatever we do, we are sitting between a rock and a hard place. > I fully agree that the above code has a much better design, is more > well- behaved, but you know very well that most programs are not so > well-behaved, and consequently you'll lock out a large fraction of > existing programs from being %run from IPython. And from a user's POV, > this will even appear to be a regression, even if previous support for > that was buggy - there *was* support for it. Well, yes, but there were threading problems. You had to be aware of how threading works under IPython, which is much more technical. > I'd say not looking at other program's deficiencies, but closing the eyes and > adding some proper hack would be much more in the spirit of IPython. Hacks lead to what we've all been experimenting: segfaults. For somebody like me who spends most of my day in IPython, it is nice no longer to have any segfaults. Ga?l From drewm1980 at gmail.com Fri Feb 12 12:23:48 2010 From: drewm1980 at gmail.com (Andrew Wagner) Date: Fri, 12 Feb 2010 11:23:48 -0600 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: <20100212103731.GC20392@phare.normalesup.org> References: <46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com> <201002121031.57250.hans_meine@gmx.net> <20100212103731.GC20392@phare.normalesup.org> Message-ID: <795840fe1002120923y372163f4t25ab0aa7553aee03@mail.gmail.com> Hello- I've only been lurking for a few days, but since I use git, I can share a bit of useful info to help your evaluation. (I have not used hg; went straight from SVN to git) On Fri, Feb 12, 2010 at 4:37 AM, Gael Varoquaux wrote: > Out of the 3 main DVCSs, I am also most tempted by hg, because I feels it > strikes the right balance between power and simplicity. However, for me > hg has a killer feature: subrepositories (or the forest extension, > previously): http://mercurial.selenic.com/wiki/subrepos > > I work in many projects where the natural setting is actually a group of > projects evolving together. Sub-repositories are the natural answer to > this situation and a big pro for me because the reduce technical and > social difficulties with loosely coupled projects: better than a > fully-integrated repo as projects have some independence, but better than > separate repos, as it is possible to have comon history. > > Maybe git has a similar feature. I haven't seen it yet. The command you seek is called git-submodule, but I have not used it. For those who are new to git, I recommend first reading the git community "book": http://book.git-scm.com/ especially the page about git's object model: http://book.git-scm.com/1_the_git_object_model.html I also recommend gitk for inspecting your history (usually distributed with git) and the eg wrapper script for improved command-line help: http://www.gnome.org/~newren/eg/ Cheers, Drew From JDM at MarchRay.net Fri Feb 12 14:12:58 2010 From: JDM at MarchRay.net (Jonathan March) Date: Fri, 12 Feb 2010 13:12:58 -0600 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: <201002121031.57250.hans_meine@gmx.net> References: <46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com> <201002121031.57250.hans_meine@gmx.net> Message-ID: On Fri, Feb 12, 2010 at 3:31 AM, Hans Meine wrote: > On Thursday 11 February 2010 21:27:28 Ville M. Vainio wrote: > > [...] Not so with hg, people can basically use it > > after they understand dvcs in general. > > OK, my 2 cents: > I switched from bzr to hg over a year ago and have no regrets. FWIW I'd be very happy if scipy went hg. -------------- next part -------------- An HTML attachment was scrubbed... URL: From JDM at MarchRay.net Fri Feb 12 14:25:07 2010 From: JDM at MarchRay.net (Jonathan March) Date: Fri, 12 Feb 2010 13:25:07 -0600 Subject: [IPython-dev] scipy at pycon? Message-ID: I plan to come to scipy in Austin in June, but hoped/expected that there would also be some scipy-related activity in Atlanta this month... but see no sprints, almost no talks or posters... anybody going? Jonathan March -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Feb 13 23:55:40 2010 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 13 Feb 2010 23:55:40 -0500 Subject: [IPython-dev] scipy at pycon? In-Reply-To: References: Message-ID: Hi Jonathan, On Fri, Feb 12, 2010 at 2:25 PM, Jonathan March wrote: > I plan to come to scipy in Austin in June, but hoped/expected that there > would also be some scipy-related activity in Atlanta this month... but see > no sprints, almost no talks or posters... anybody going? I won't be attending, but I think the Enthought guys are holding a tutorial there (I saw it mentioned at the numpy/stats webinar on Friday). I don't know about anyone else who might be going from the local Berkeley crowd either. Cheers, f From fperez.net at gmail.com Sun Feb 14 02:26:27 2010 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 14 Feb 2010 02:26:27 -0500 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <20100212113903.GB7228@phare.normalesup.org> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002121206.51045.hans_meine@gmx.net> <20100212111104.GA7228@phare.normalesup.org> <201002121225.05301.hans_meine@gmx.net> <20100212113903.GB7228@phare.normalesup.org> Message-ID: Hi all, Thanks to everyone for the feedback, it's indeed very useful and we are aware this is a hard problem. The last thing we want to do is cause what people will effectively see as massive regressions, no matter how good our intentions or the reasons behind our choices. Gael summarizes the situation very well here: On Fri, Feb 12, 2010 at 6:39 AM, Gael Varoquaux wrote: > Honestly, I don't really care about how things are done. I am no longer > really involved in IPython development. All I know is that trying to > solve impossible problems leads to hair loss and disappointment. I > believe you are trying to solve an impossible problem: making sure a user > can use a main event loop without being aware it is around. IPython used > to try and solve that problem using threads. It lead to segfaults because > every single piece of code executed had to be injected in the main loop. > So what was supposed to be transparent worked 99.9% of the time, and the > rest of the time forced people to modify their code in subtle ways and > debug race conditions, which is hard. The developers have now stepped > back and decided that they would make a choice that forces users to be > aware of the problems upfront, but has a simple solution. > > I am not saying this is a good or bad choice. I haven't had time to try > out the new IPython, and I am worried of the problems with Mayavi and > Traits. However, I am saying that there is some reasonning behind the > choice, and that whatever we do, we are sitting between a rock and a hard > place. As we go through this, I am pondering the possibility of eventually adding the threading-style hacks back, but NOT as default. This isn't final by any means, I'm just thinking out loud here, about a plan along these lines: - non-threaded gui support with pyos_inputhook, that would require some code changes. If this can give a really robust solution and the needed changes aren't too intrusive, I could see matplotlib, ETS and possibly others adding the necessary adaptation (especially if we provide it in ipython already so for the users it ends up being almost a one-liner). - adding back the threading hacks, which could be enabled with special command-line flags. These would carry the known fragility we are trying to get away from, but would let users continue to use them in codes that have not been modified, just like they do today. No better, but no worse either. IPython is an eminently practical tool, so we'll definitely remember the zen's stanza on purity and practicality, or at least we'll try. Regards, f From fperez.net at gmail.com Sun Feb 14 23:33:34 2010 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 14 Feb 2010 23:33:34 -0500 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: <4B71A453.1060406@bostream.nu> References: <46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com> <4B71A453.1060406@bostream.nu> Message-ID: Hi all, [ thanks a lot for the dvcs comments, there was actually a lot of good info there. But for now I'm sticking to the migration topic here, I'm a little pressed for time] On Tue, Feb 9, 2010 at 1:07 PM, J?rgen Stenarson wrote: >> So, as a concrete proposal, I can update the remote repo say by this >> weekend, giving anyone with local changes in any branch time to merge >> them. ?By Sunday I will start a remote repo upgrade, and we'll all >> simply have to run local upgrades before doing any further operations. >> >> Does that sound OK, have I missed anything? >> > Sounds reasonable to me. OK, I tried to get this ball rolling, but the process looks more complex than I thought, and I'm having second thoughts. This is what I have found so far in terms of instructions: http://doc.bazaar.canonical.com/latest/en/upgrade-guide/index.html#data-migration It's a very involved process, that seems to require lots of manual checks and attention to detail, with multiple chances for screwing up. Ville, you mentioned doing this for Leo, could you provide us with some info on how you went about it? Is that guide overly complicated and do you have a simpler alternative? I am seriously, seriously annoyed that using a version control tool becomes a chore like this. We have precious little time as it is for ipython, the last thing I want to do is to waste it babysitting the VCS (especially when it's slow and crash-prone to begin with). This thread on the upgrade topic is another one that I found when searching instructions for this: http://omgili.com/mailinglist/bazaar/lists/canonical/com/20090930081159735b276dlimelightwoozorg.html and it also has lots of things in it that give me pause. So before I pull a trigger I might regret, should we reconsider this? Ville, if you have tips on how to do this cleanly I'd love to hear them, perhaps I'm being unnecessarily paranoid and fearful and in practice it really isn't that bad. But one thing that is true is that in the next few weeks I do not have a full 4-6 open hours to devote to this. I thought I could simply activate 'repo upgrade' somewhere on LP and be done with it, but that doesn't seem to be the case. If it requires a full day of my time to happen cleanly, I simply can't do it now, I'm afraid. Advice will be much welcome. Thanks! f From vivainio at gmail.com Mon Feb 15 02:52:23 2010 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 15 Feb 2010 09:52:23 +0200 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: References: <46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com> <4B71A453.1060406@bostream.nu> Message-ID: <46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com> On Mon, Feb 15, 2010 at 6:33 AM, Fernando Perez wrote: > Ville, you mentioned doing this for Leo, could you provide us with > some info on how you went about it? ?Is that guide overly complicated > and do you have a simpler alternative? I did "bzr upgrade lp:leo-editor" and waited it out. Safer alternative seems to be: bzr branch lp:ipython cd ipython bzr upgrade (this takes a long time) + push to a new branch, and change the "development focus" to be that branch. -- Ville M. Vainio http://tinyurl.com/vainio From fperez.net at gmail.com Mon Feb 15 11:13:42 2010 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 15 Feb 2010 11:13:42 -0500 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: <46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com> References: <46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com> <4B71A453.1060406@bostream.nu> <46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com> Message-ID: On Mon, Feb 15, 2010 at 2:52 AM, Ville M. Vainio wrote: > I did "bzr upgrade lp:leo-editor" and waited it out. > > Safer alternative seems to be: > > bzr branch lp:ipython > cd ipython > bzr upgrade (this takes a long time) > > + push to a new branch, and change the "development focus" to be that branch. OK, thanks. I'll try to fire that process tonight then. I'll ping with an email to the list when I start. Cheers, f From gael.varoquaux at normalesup.org Mon Feb 15 13:20:40 2010 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Mon, 15 Feb 2010 19:20:40 +0100 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: <46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com> References: <46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com> <4B71A453.1060406@bostream.nu> <46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com> Message-ID: <20100215182040.GC20321@phare.normalesup.org> On Mon, Feb 15, 2010 at 09:52:23AM +0200, Ville M. Vainio wrote: > Safer alternative seems to be: > bzr branch lp:ipython > cd ipython > bzr upgrade (this takes a long time) > + push to a new branch, and change the "development focus" to be that branch. Yes, I would advise to do this. I did this for at least one project I work on (can't remember which). And I agree that this is a mess... Ga?l From fperez.net at gmail.com Tue Feb 16 00:56:31 2010 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 16 Feb 2010 00:56:31 -0500 Subject: [IPython-dev] What do we do about repository formats??? In-Reply-To: References: <46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com> <4B71A453.1060406@bostream.nu> <46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com> Message-ID: On Mon, Feb 15, 2010 at 11:13 AM, Fernando Perez wrote: > >> bzr branch lp:ipython >> cd ipython >> bzr upgrade (this takes a long time) >> >> + push to a new branch, and change the "development focus" to be that branch. > > OK, thanks. > > I'll try to fire that process tonight then. ?I'll ping with an email > to the list when I start. For some reason this is taking a long time on my system, and I need to crash now. I'll be traveling for a few days, so I won't touch anything on the repo until next week at best, when I return. Cheers, f From ondrej at certik.cz Tue Feb 16 21:21:47 2010 From: ondrej at certik.cz (Ondrej Certik) Date: Tue, 16 Feb 2010 18:21:47 -0800 Subject: [IPython-dev] using reST for representing the notebook cells+text Message-ID: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> Hi, has anyone given a deeper thought how to best implement this functionality? Currently the Sage notebook seems to remember the worksheet using: -------------- sdsd system:sage

Well, some text

{{{id=5| awe /// }}} {{{id=1| 4 /// 4 }}} {{{id=2| 5 /// 5 }}} {{{id=3| /// }}} --------------------- E.g. each cell is either a text cell, or a cell that can be evaluated. Is that right? I know that Fernando suggested here: http://groups.google.com/group/sage-devel/browse_thread/thread/3f1f5a0c26975519/ to use reST, e.g. something like: ---------- Well, some text :: >>> print 3 3 ------------ I am implementing a notebook prototype using pure Python (automatically translated to javascript using pyjamas) and I would like to run it on the google app engine (the javascript in your browser would then communicate either with the app engine, or your own hardware that would host Sage/FEMhub/your own code): http://gamma.sympy.org/nb/ and here is my development version with logins and settings implemented: http://2.latest.sympy-gamma.appspot.com/nb/ now I need to implement the worksheets, so I am thinking how to best represent it. Did the syntax for input/output blocks in Sphinx settled already? I guess internally I will implement the worksheet as a set of cells, either text cells, or "evaluatable" cells (that contain both input/output blocks) and then support conversion to and from reST/Sphinx. I would be interested in any feedback. Ondrej From Chris.Barker at noaa.gov Thu Feb 18 17:24:40 2010 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 18 Feb 2010 14:24:40 -0800 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002121206.51045.hans_meine@gmx.net> <20100212111104.GA7228@phare.normalesup.org> <201002121225.05301.hans_meine@gmx.net> <20100212113903.GB7228@phare.normalesup.org> Message-ID: <4B7DBE28.90702@noaa.gov> Thanks for everyone's work on this. I had a thought. It seems that it's really impossible (or at least impractical) to monkey-patch the GUI toolkits such that you can run a regular old unmodified app exactly the same way inside and outside of ipython. Gael's right -- that way lies loss of hair (and I"ve lost enough in the last ten years as it is). However, as it happens, I've been using ipython for a wx GUI app of mine (with no ETS or MPL), and it sort-of-works, and also locks up and crashes fairly regularly. So the threading solution isn't really robust either. Also, when wx starts using Cocoa on OS-X, putting the GUI in another thread won't be an option anymore anyway. So: > - non-threaded gui support with pyos_inputhook, that would require > some code changes. If this can give a really robust solution and the > needed changes aren't too intrusive, I could see matplotlib, ETS and > possibly others adding the necessary adaptation (especially if we > provide it in ipython already so for the users it ends up being almost > a one-liner). Exactly -- while you can't expect unmodified regular old wx code to work, we could provide alternative app startup code that a user can simply cut and past in, and then have an app that will work with and without ipython -- I think that's a fine solution. So for now, is there a sample, minimal wx app that works with the new pyos_inputhook code? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From noamraph at gmail.com Sun Feb 21 04:56:35 2010 From: noamraph at gmail.com (Noam Yorav-Raphael) Date: Sun, 21 Feb 2010 11:56:35 +0200 Subject: [IPython-dev] DreamPie - a new graphical Python shell Message-ID: Hello, My name is Noam Yorav-Raphael. I wanted to let you know that I wrote a new graphical Python shell called DreamPie, and I would like to have your feedback as the developers of a mature Python shell. DreamPie is quite similar to IDLE, in that it uses the graphical user interface to give you automatic completions and tooltips showing function arguments. It's similar to IPython in that it has a result history, and in that it supports using matplotlib. It also offers these unique features: * Long output can be folded, so as not to distract you from the important results. Also, textual results can be saved to external files or viewed in an external viewer. * You can type any amount of code and execute it at once. The Copy Code Only command lets you easily copy successful commands into a Python module you're editing. * History can be saved to HTML files and loaded back, so you can record how you got your results and also quickly re-execute commands from a previous session. Would you care to give it a try, at http://dreampie.sourceforge.net/ , and tell me what you think about it? Thanks, Noam From vivainio at gmail.com Sun Feb 21 07:51:41 2010 From: vivainio at gmail.com (Ville M. Vainio) Date: Sun, 21 Feb 2010 14:51:41 +0200 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: References: Message-ID: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> On Sun, Feb 21, 2010 at 11:56 AM, Noam Yorav-Raphael wrote: > DreamPie is quite similar to IDLE, in that it uses the graphical user > interface to give you automatic completions and tooltips showing > function arguments. It's similar to IPython in that it has a result > history, and in that it supports using matplotlib. Out of curiosity, are you familiar with Spyder: http://packages.python.org/spyder/ That project shares some of the same aims. -- Ville M. Vainio http://tinyurl.com/vainio From noamraph at gmail.com Sun Feb 21 08:51:15 2010 From: noamraph at gmail.com (Noam Yorav-Raphael) Date: Sun, 21 Feb 2010 15:51:15 +0200 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> References: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> Message-ID: Thanks for the link! I wasn't familiar with Spyder. It looks very interesting. It seems to me that it is less oriented toward using the shell - for example, I couldn't find a way to recall a block (more than one line) from history, and there was no way to interrupt either the internal or the external console. I think DreamPie is better in that respect. Noam On Sun, Feb 21, 2010 at 2:51 PM, Ville M. Vainio wrote: > > Out of curiosity, are you familiar with Spyder: > > http://packages.python.org/spyder/ > > That project shares some of the same aims. > > -- > Ville M. Vainio > http://tinyurl.com/vainio > From gael.varoquaux at normalesup.org Sun Feb 21 09:04:49 2010 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sun, 21 Feb 2010 15:04:49 +0100 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: References: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> Message-ID: <20100221140449.GA32024@phare.normalesup.org> On Sun, Feb 21, 2010 at 03:51:15PM +0200, Noam Yorav-Raphael wrote: > Thanks for the link! I wasn't familiar with Spyder. It looks very interesting. > It seems to me that it is less oriented toward using the shell - for > example, I couldn't find a way to recall a block (more than one line) > from history, and there was no way to interrupt either the internal or > the external console. I think DreamPie is better in that respect. I am sure that DreamPie is better than Spyder in some respects, and vice versa. However, the question is: is there an interest in collecting many different efforts that have their own strengths and weaknesses? Note that I am a user of neither, because they don't offer all the functionalities that I need from IPython... Best, Ga?l From noamraph at gmail.com Sun Feb 21 09:34:32 2010 From: noamraph at gmail.com (Noam Yorav-Raphael) Date: Sun, 21 Feb 2010 16:34:32 +0200 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: <20100221140449.GA32024@phare.normalesup.org> References: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> <20100221140449.GA32024@phare.normalesup.org> Message-ID: On Sun, Feb 21, 2010 at 4:04 PM, Gael Varoquaux wrote: > > I am sure that DreamPie is better than Spyder in some respects, and vice > versa. > > However, the question is: is there an interest in collecting many > different efforts that have their own strengths and weaknesses? > That's always a problem with projects which may have common goals. However, in this case, I don't feel that there was a lot of redundant work done, since Spyder seems to be more focused on visualization while DreamPie is more focused on the shell itself. > Note that I am a user of neither, because they don't offer all the > functionalities that I need from IPython... Can you say what these functionalities are? I'm interested to know which features people are missing from DreamPie. Thanks, Noam From gael.varoquaux at normalesup.org Sun Feb 21 09:39:37 2010 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sun, 21 Feb 2010 15:39:37 +0100 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: References: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> <20100221140449.GA32024@phare.normalesup.org> Message-ID: <20100221143937.GB32024@phare.normalesup.org> On Sun, Feb 21, 2010 at 04:34:32PM +0200, Noam Yorav-Raphael wrote: > > Note that I am a user of neither, because they don't offer all the > > functionalities that I need from IPython... > Can you say what these functionalities are? I'm interested to know > which features people are missing from DreamPie. Most of the IPython magics. Go history management. The ability to have several GUI event loop. Fancy tracebacks and interactive debuger support. Output history. Shell support. Tab completion on file names. The '?' and '??' support. Does 'Ctrl-C' work in Dreampie? There might be other things, but that's what I can think of. Ga?l From noamraph at gmail.com Sun Feb 21 10:02:20 2010 From: noamraph at gmail.com (Noam Yorav-Raphael) Date: Sun, 21 Feb 2010 17:02:20 +0200 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: <20100221143937.GB32024@phare.normalesup.org> References: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> <20100221140449.GA32024@phare.normalesup.org> <20100221143937.GB32024@phare.normalesup.org> Message-ID: On Sun, Feb 21, 2010 at 4:39 PM, Gael Varoquaux wrote: > On Sun, Feb 21, 2010 at 04:34:32PM +0200, Noam Yorav-Raphael wrote: >> > Note that I am a user of neither, because they don't offer all the >> > functionalities that I need from IPython... > >> Can you say what these functionalities are? I'm interested to know >> which features people are missing from DreamPie. > > Most of the IPython magics. Go history management. The ability to have > several GUI event loop. Fancy tracebacks and interactive debuger support. > Output history. Shell support. Tab completion on file names. The '?' and > '??' support. Thanks for the list! * History management is there - the recent values computed are saved, and all your command and output history is saved too. You can even save it to a HTML file. * GUI event loops are there - GTK, Tkinter, and QT event loops are automatically supported, in a way very similar to Tkinter in the traditional interpreter - handling events while idle. * Tracebacks are not really fancy, but are colored in red so that it's easy to distinguish them from normal output. * Completion of file names is there too, using a popup list. This leaves these features: * Fancy tracebacks and debugger support - right now pdb works like in the classic interpreter, but I'm sure IPython supports better debugging. * Shell support. (you can use os.system, but it's not the same thing) * '?' and '??'. I guess that in a graphical environment it's best if the help appears in a new window. That's a great list. I hope I will be able to implement those. Thanks! > > Does 'Ctrl-C' work in Dreampie? > Works like charm... It was hard getting it right on Windows, but I managed it. Thanks, Noam From gael.varoquaux at normalesup.org Sun Feb 21 10:07:16 2010 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sun, 21 Feb 2010 16:07:16 +0100 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: References: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> <20100221140449.GA32024@phare.normalesup.org> <20100221143937.GB32024@phare.normalesup.org> Message-ID: <20100221150716.GA8338@phare.normalesup.org> On Sun, Feb 21, 2010 at 05:02:20PM +0200, Noam Yorav-Raphael wrote: > * History management is there - the recent values computed are saved, > and all your command and output history is saved too. You can even > save it to a HTML file. What I specificaly want is recalling based on the begining of a line. > * GUI event loops are there - GTK, Tkinter, and QT event loops are > automatically supported, in a way very similar to Tkinter in the > traditional interpreter - handling events while idle. Cool. I missed that. That's nice. > * Tracebacks are not really fancy, but are colored in red so that it's > easy to distinguish them from normal output. :( . IPython's traceback really improve my productivity, because they often allow me to pin-point the problem immediatly. > * Completion of file names is there too, using a popup list. Cool! > * Fancy tracebacks and debugger support - right now pdb works like in > the classic interpreter, but I'm sure IPython supports better > debugging. > * Shell support. (you can use os.system, but it's not the same thing) > * '?' and '??'. I guess that in a graphical environment it's best if > the help appears in a new window. And the magics too :). > That's a great list. I hope I will be able to implement those. Thanks! Good luck. > > Does 'Ctrl-C' work in Dreampie? > Works like charm... It was hard getting it right on Windows, but I managed it. Good work. That's indeed hard. By the way, DreamPie is licensed under GPL. This means that commercial users (even those doing open source) are quite likely to reimplement it, rather than to reuse it. That's a pitty, if you are putting so much effort in it. Best, Ga?l From Chris.Barker at noaa.gov Sun Feb 21 15:48:55 2010 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Sun, 21 Feb 2010 12:48:55 -0800 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: References: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> <20100221140449.GA32024@phare.normalesup.org> <20100221143937.GB32024@phare.normalesup.org> Message-ID: <4B819C37.9000108@noaa.gov> Noam Yorav-Raphael wrote: > * GUI event loops are there - GTK, Tkinter, and QT event loops are > automatically supported, in a way very similar to Tkinter in the > traditional interpreter - handling events while idle. what about wxPython? What GUI toolkit is Dreampie written in? It doesn't matter much, I suppose as long as it works with the others, but I'm wondering. Also, if it's GTK, then native MAc support isn't an option ( :-( ). Can it run your app in a different process? Given the jython and iton python support, it looks like yes. > * Shell support. (you can use os.system, but it's not the same thing) yes -- that's really nice. > * '?' and '??'. I guess that in a graphical environment it's best if > the help appears in a new window. not really -- there is a real efficiency to being able whip stuff out at the command line. It might be nice to have the help show up in a new window, but don't make me pick up the mouse to get it. typing ? is really easy. Gael Varoquaux wrote: > What I specificaly want is recalling based on the begining of a line. yes - this is VERY handy. I'm looking forward to giving it a real try. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From noamraph at gmail.com Sun Feb 21 16:17:57 2010 From: noamraph at gmail.com (Noam Yorav-Raphael) Date: Sun, 21 Feb 2010 23:17:57 +0200 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: <20100221150716.GA8338@phare.normalesup.org> References: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> <20100221140449.GA32024@phare.normalesup.org> <20100221143937.GB32024@phare.normalesup.org> <20100221150716.GA8338@phare.normalesup.org> Message-ID: On Sun, Feb 21, 2010 at 5:07 PM, Gael Varoquaux wrote: > On Sun, Feb 21, 2010 at 05:02:20PM +0200, Noam Yorav-Raphael wrote: >> * History management is there - the recent values computed are saved, >> and all your command and output history is saved too. You can even >> save it to a HTML file. > > What I specificaly want is recalling based on the begining of a line. > Already there! >> * Tracebacks are not really fancy, but are colored in red so that it's >> easy to distinguish them from normal output. > > :( . IPython's traceback really improve my productivity, because they often > allow me to pin-point the problem immediatly. > I think that adding an option for extended exceptions won't be too hard. It's good to know what helps people. >> * '?' and '??'. I guess that in a graphical environment it's best if >> the help appears in a new window. > > And the magics too :). Ok. Perhaps the best way would be to add an IPython mode... > By the way, DreamPie is licensed under GPL. This means that commercial > users (even those doing open source) are quite likely to reimplement it, > rather than to reuse it. That's a pitty, if you are putting so much > effort in it. > I don't think that's a real problem. Since DreamPie is a program, not a library, LGPL isn't really applicable. It can be used for commercial purposes, it's just that others can't add a feature and sell their version. Thanks for the feedback, Noam From noamraph at gmail.com Sun Feb 21 16:24:02 2010 From: noamraph at gmail.com (Noam Yorav-Raphael) Date: Sun, 21 Feb 2010 23:24:02 +0200 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: <4B819C37.9000108@noaa.gov> References: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> <20100221140449.GA32024@phare.normalesup.org> <20100221143937.GB32024@phare.normalesup.org> <4B819C37.9000108@noaa.gov> Message-ID: On Sun, Feb 21, 2010 at 10:48 PM, Christopher Barker wrote: > Noam Yorav-Raphael wrote: >> * GUI event loops are there - GTK, Tkinter, and QT event loops are >> automatically supported, in a way very similar to Tkinter in the >> traditional interpreter - handling events while idle. > > what about wxPython? > I haven't implemented it, but it shouldn't be a problem. > What GUI toolkit is Dreampie written in? It doesn't matter much, I > suppose as long as it works with the others, but I'm wondering. Also, if > it's GTK, then native MAc support isn't an option ( :-( ). It's PyGTK. I've heard that there is a way to use PyGTK on Mac, but I'm sure that the support is not as good as linux and Windows. I've chosen PyGTK because its Python bindings are MUCH better than PyQT. > > Can it run your app in a different process? Given the jython and iton > python support, it looks like yes. Yes. It actually always runs a Python subprocess which executes your code, and the main one only handles the GUI. > >> * Shell support. (you can use os.system, but it's not the same thing) > > yes -- that's really nice. > >> * '?' and '??'. I guess that in a graphical environment it's best if >> the help appears in a new window. > > not really -- there is a real efficiency to being able whip stuff out at > the command line. It might be nice to have the help show up in a new > window, but don't make me pick up the mouse to get it. typing ? is > really easy. Indeed. I thought of something like an extended tooltip. There's currently a tooltip showing function arguments, and I guess it can be extended to show more documentation. > > I'm looking forward to giving it a real try. > I will be glad to hear what you think once you do! Noam From warren.weckesser at enthought.com Sun Feb 21 16:38:03 2010 From: warren.weckesser at enthought.com (Warren Weckesser) Date: Sun, 21 Feb 2010 15:38:03 -0600 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: References: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> <20100221140449.GA32024@phare.normalesup.org> <20100221143937.GB32024@phare.normalesup.org> <4B819C37.9000108@noaa.gov> Message-ID: <4B81A7BB.5010906@enthought.com> Noam Yorav-Raphael wrote: >>> * '?' and '??'. I guess that in a graphical environment it's best if >>> the help appears in a new window. >>> >> not really -- there is a real efficiency to being able whip stuff out at >> the command line. It might be nice to have the help show up in a new >> window, but don't make me pick up the mouse to get it. typing ? is >> really easy. >> > > Indeed. I thought of something like an extended tooltip. There's > currently a tooltip showing function arguments, and I guess it can be > extended to show more documentation. > Actually, while I find the ? *very* convenient and helpful, there is one behavior about it that often irks me. It uses a "more"-like viewer for the text, but the text *goes away* when I hit 'q'. Many times I want to keep the docstring visible while I proceed to enter a command. I then have to resort to something like "print foo.__doc__". (Or have I missed a key that exits the docstring view but leaves the text visible?) Having the docstring in a window would be much nicer in this case. Warren From listsin at integrateddevcorp.com Sun Feb 21 16:53:44 2010 From: listsin at integrateddevcorp.com (Steve Steiner (listsin)) Date: Sun, 21 Feb 2010 16:53:44 -0500 Subject: [IPython-dev] DreamPie - a new graphical Python shell In-Reply-To: References: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> <20100221140449.GA32024@phare.normalesup.org> <20100221143937.GB32024@phare.normalesup.org> <4B819C37.9000108@noaa.gov> Message-ID: <28B74AAD-3467-4768-AE1D-7B5B645BACDA@integrateddevcorp.com> On Feb 21, 2010, at 4:24 PM, Noam Yorav-Raphael wrote: >> What GUI toolkit is Dreampie written in? It doesn't matter much, I >> suppose as long as it works with the others, but I'm wondering. Also, if >> it's GTK, then native MAc support isn't an option ( :-( ). > > It's PyGTK. I've heard that there is a way to use PyGTK on Mac, I'm running it under PyGTK under the 64 bit MacPorts Python. Assuming you have Python installed under MacPorts: # sudo port install py26-gtk2 # sudo port install py26-pygtksourceview # dreampie And off to the races. Found the font dialog so I could make it so I could see it and there you have it! S From drewm1980 at gmail.com Mon Feb 22 01:24:21 2010 From: drewm1980 at gmail.com (Andrew Wagner) Date: Mon, 22 Feb 2010 00:24:21 -0600 Subject: [IPython-dev] Having trouble using TaskClient; exception inside twisted Message-ID: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com> Hello- I'm trying to use the TaskClient infrastructure, and I'm hitting some exceptions I don't understand. My python and twisted are from debian testing. I believe the versions in stable are too old to support task clients, etc, which was my main motive for switching to testing. On importing I get the warnings: ____________ from IPython.kernel import client ...: /usr/lib/pymodules/python2.5/IPython/kernel/clientconnector.py:43: DeprecationWarning: Importing class Tub directly from 'foolscap' is deprecated since Foolscap 0.4.3. Please import foolscap.api.Tub instead self.tub = Tub() __________ On instantiation of a task client I get: ____________ In [7]: tc = client.TaskClient() /usr/lib/pymodules/python2.5/IPython/kernel/taskfc.py:79: DeprecationWarning: Importing class Referenceable1 directly from 'foolscap' is deprecated since Foolscap 0.4.3. Please import foolscap.api.Referenceable instead class FCTaskControllerFromTaskController(Referenceable): ____________ Something similar for a multiengine client: ______________ In [8]: mec = client.MultiEngineClient() /usr/lib/pymodules/python2.5/IPython/kernel/multienginefc.py:86: DeprecationWarning: Importing class Referenceable1 directly from 'foolscap' is deprecated since Foolscap 0.4.3. Please import foolscap.api.Referenceable instead class FCSynchronousMultiEngineFromMultiEngine(Referenceable): _____________ The following example from the website and docs runs cleanly. __________ serial_result = map(lambda x:x**10, range(200)) parallel_result = tc.map(lambda x:x**10, range(200)) assert serial_result==parallel_result __________ However, when I run my code, at the point where I run tc.map() I'm getting the following: ______________ Traceback (most recent call last): File "./objectives.py", line 165, in e = objective_multithread(s) File "./objectives.py", line 150, in objective_multithread norms = tc.map(f, range(testingImageCount)) File "/usr/lib/pymodules/python2.5/IPython/kernel/taskclient.py", line 154, in map return self.mapper().map(func, *sequences) File "/usr/lib/pymodules/python2.5/IPython/kernel/mapper.py", line 230, in map task_results = [self.task_controller.get_task_result(tid) for tid in task_ids] File "/usr/lib/pymodules/python2.5/IPython/kernel/taskclient.py", line 93, in get_task_result taskid, block) File "/usr/lib/pymodules/python2.5/IPython/kernel/twistedutil.py", line 72, in blockingCallFromThread return twisted.internet.threads.blockingCallFromThread(reactor, f, *a, **kw) File "/usr/lib/python2.5/site-packages/twisted/internet/threads.py", line 112, in blockingCallFromThread result.raiseException() File "/usr/lib/python2.5/site-packages/twisted/python/failure.py", line 326, in raiseException raise self.type, self.value, self.tb AttributeError: 'int' object has no attribute 'rfind' ________________ Any ideas/advice would be greatly appreciated. Thanks, Drew From ellisonbg at gmail.com Mon Feb 22 01:52:59 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Sun, 21 Feb 2010 22:52:59 -0800 Subject: [IPython-dev] Having trouble using TaskClient; exception inside twisted In-Reply-To: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com> References: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com> Message-ID: Andrew, > I'm trying to use the TaskClient infrastructure, and I'm hitting some > exceptions I don't understand. ?My python and twisted are from debian > testing. ?I believe the versions in stable are too old to support task > clients, etc, which was my main motive for switching to testing. What version of IPython are you using? Trunk or latest stable (0.10)? > On importing I get the warnings: > ____________ > from IPython.kernel import client > ? ...: > > /usr/lib/pymodules/python2.5/IPython/kernel/clientconnector.py:43: > DeprecationWarning: Importing class Tub directly from 'foolscap' is > deprecated since Foolscap 0.4.3. Please import foolscap.api.Tub > instead > ?self.tub = Tub() > __________ > > On instantiation of a task client I get: > > ____________ > In [7]: tc = client.TaskClient() > /usr/lib/pymodules/python2.5/IPython/kernel/taskfc.py:79: > DeprecationWarning: Importing class Referenceable1 directly from > 'foolscap' is deprecated since Foolscap 0.4.3. Please import > foolscap.api.Referenceable instead > ?class FCTaskControllerFromTaskController(Referenceable): > ____________ > > Something similar for a multiengine client: > > ______________ > In [8]: mec = client.MultiEngineClient() > /usr/lib/pymodules/python2.5/IPython/kernel/multienginefc.py:86: > DeprecationWarning: Importing class Referenceable1 directly from > 'foolscap' is deprecated since Foolscap 0.4.3. Please import > foolscap.api.Referenceable instead > ?class FCSynchronousMultiEngineFromMultiEngine(Referenceable): > _____________ Don't worry to much about these errors. I am aware of them, but need to make a small change to get rid of the warning. Could you file a bug report as a reminder for me: https://bugs.launchpad.net/ipython > > The following example from the website and docs runs cleanly. > __________ > serial_result = map(lambda x:x**10, range(200)) > parallel_result = tc.map(lambda x:x**10, range(200)) > assert serial_result==parallel_result > __________ Great, so things are basically working... > > However, when I run my code, at the point where I run tc.map() I'm > getting the following: > > ______________ > Traceback (most recent call last): > ?File "./objectives.py", line 165, in > ? ?e = objective_multithread(s) > ?File "./objectives.py", line 150, in objective_multithread > ? ?norms = tc.map(f, range(testingImageCount)) > ?File "/usr/lib/pymodules/python2.5/IPython/kernel/taskclient.py", > line 154, in map > ? ?return self.mapper().map(func, *sequences) > ?File "/usr/lib/pymodules/python2.5/IPython/kernel/mapper.py", line 230, in map > ? ?task_results = [self.task_controller.get_task_result(tid) for tid > in task_ids] > ?File "/usr/lib/pymodules/python2.5/IPython/kernel/taskclient.py", > line 93, in get_task_result > ? ?taskid, block) > ?File "/usr/lib/pymodules/python2.5/IPython/kernel/twistedutil.py", > line 72, in blockingCallFromThread > ? ?return twisted.internet.threads.blockingCallFromThread(reactor, f, *a, **kw) > ?File "/usr/lib/python2.5/site-packages/twisted/internet/threads.py", > line 112, in blockingCallFromThread > ? ?result.raiseException() > ?File "/usr/lib/python2.5/site-packages/twisted/python/failure.py", > line 326, in raiseException > ? ?raise self.type, self.value, self.tb > AttributeError: 'int' object has no attribute 'rfind' > ________________ Hmm, haven't seen this last one. Could you put together a small, self contained example that demonstrates the error. Then I can track it down. Cheers, Brian > Any ideas/advice would be greatly appreciated. > > Thanks, > Drew > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From drewm1980 at gmail.com Mon Feb 22 10:28:28 2010 From: drewm1980 at gmail.com (Andrew Wagner) Date: Mon, 22 Feb 2010 09:28:28 -0600 Subject: [IPython-dev] Having trouble using TaskClient; exception inside twisted In-Reply-To: References: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com> Message-ID: <795840fe1002220728m62fb2e3bvd4ef38258e1ced34@mail.gmail.com> On Mon, Feb 22, 2010 at 12:52 AM, Brian Granger wrote: > What version of IPython are you using? ?Trunk or latest stable (0.10)? IPython 0.10 Python 2.5.2 (r252:60911, Jan 24 2010, 17:44:40) twisted.version Version('twisted', 9, 0, 0) > Don't worry to much about these errors. ?I am aware of them, but need > to make a small change > to get rid of the warning. ?Could you file a bug report as a reminder for me: Done: https://bugs.launchpad.net/ipython/+bug/525805 > Hmm, haven't seen this last one. ?Could you put together a small, self > contained example that > demonstrates the error. ?Then I can track it down. Sure thing; I'll try to hammer one out asap. Thanks Brian! From ellisonbg at gmail.com Mon Feb 22 13:37:46 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 22 Feb 2010 10:37:46 -0800 Subject: [IPython-dev] Having trouble using TaskClient; exception inside twisted In-Reply-To: <795840fe1002220728m62fb2e3bvd4ef38258e1ced34@mail.gmail.com> References: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com> <795840fe1002220728m62fb2e3bvd4ef38258e1ced34@mail.gmail.com> Message-ID: Andrew, > IPython 0.10 I will be fixing these things in trunk. We are about to release 0.11, and it will have these fixes. Cheers, Brian > Python 2.5.2 (r252:60911, Jan 24 2010, 17:44:40) > twisted.version Version('twisted', 9, 0, 0) > >> Don't worry to much about these errors. ?I am aware of them, but need >> to make a small change >> to get rid of the warning. ?Could you file a bug report as a reminder for me: > Done: > https://bugs.launchpad.net/ipython/+bug/525805 > >> Hmm, haven't seen this last one. ?Could you put together a small, self >> contained example that >> demonstrates the error. ?Then I can track it down. > Sure thing; I'll try to hammer one out asap. > > Thanks Brian! > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From drewm1980 at gmail.com Mon Feb 22 20:01:48 2010 From: drewm1980 at gmail.com (Andrew Wagner) Date: Mon, 22 Feb 2010 19:01:48 -0600 Subject: [IPython-dev] Having trouble using TaskClient; exception inside twisted In-Reply-To: References: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com> <795840fe1002220728m62fb2e3bvd4ef38258e1ced34@mail.gmail.com> Message-ID: <795840fe1002221701s79dcb989i8a19a895ff40a843@mail.gmail.com> >>> Hmm, haven't seen this last one. ?Could you put together a small, self >>> contained example that >>> demonstrates the error. ?Then I can track it down. >> Sure thing; I'll try to hammer one out asap. I have replicated the error on the trunk: IPython 0.11.alpha1.bzr.r1223 TaskClient.map is giving the same error as before. I tried MultiEngineClient.map: CompositeError: one or more exceptions from call to method: execute [0:execute]: AttributeError: 'int' object has no attribute 'rfind' [1:execute]: AttributeError: 'int' object has no attribute 'rfind' [2:execute]: AttributeError: 'int' object has no attribute 'rfind' [3:execute]: TypeError: map() requires at least two args [4:execute]: TypeError: map() requires at least two args [5:execute]: TypeError: map() requires at least two args [6:execute]: TypeError: map() requires at least two args [7:execute]: TypeError: map() requires at least two args engines 0-2 did have a job to work on, and gave the same error as the TaskClient engines 3-7 did not have a job to work on. My network setup: * client on machine 0 * ipcontroller on machine 1 * 8 ipengine instances on machine 2 All of the machines 0 and 2 mount my home directory from an NFS share on machine 1, which is how the furl files get distributed. I'm using /etc/inittab to keep the controller and engine instances alive on machines 1 and 2. By changing in the type of the second map parameter I figured out that the 'int' in "[0:execute]: AttributeError: 'int' object has no attribute 'rfind'" is the type inside the list that I send. i.e. for norms = mec.map(f, [(i,) for i in range(3)]) I get: [0:execute]: AttributeError: 'tuple' object has no attribute 'rfind' ... It's odd that it complains about the type of my list of int's, but not the list of int's in the example. I'm still working on a small standalone version; just sending out info as I go. From ellisonbg.net at gmail.com Wed Feb 24 02:52:03 2010 From: ellisonbg.net at gmail.com (Brian Granger) Date: Tue, 23 Feb 2010 23:52:03 -0800 Subject: [IPython-dev] using reST for representing the notebook cells+text In-Reply-To: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> Message-ID: <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> Ondrej, I have have thought about possible notebook formats many times. IPython has partially implemented at least two different notebook formats. Both of them have not been very successful. The first was XML based and the second was based on an sqlalchemy db. Both were insanely complex. Fernando and I have talked many times about a notebook format that is simply restructured text. There are many advantages of this approach: * All of us use reST for lots of things any ways. * There are existing parsers for reST. * It naturally knows about code + text. * Latex support, graphics. * And more. In many respects it seems almost perfect. But, one day recently I was thinking about how successful Mathematica's notebook continues to be. So I began to look more at the Mathematica notebook format. Amazingly, the Mathematica notebook format is a plain text file that itself is *valid Mathematica code*. This id documented here: http://reference.wolfram.com/mathematica/guide/LowLevelNotebookProgramming.html For examples a simple notebook with one text cell is just: Notebook[{Cell['Here is my text', 'Text']}] Everything - input cells, output cells, static images and all are represented in this way and embedded in the plain text notebook file. The Python generalization of this would be the following: * A Python notebook is plain text, importable Python code. * That code is simply a tree of objects that declare the relevant parts of the notebook. This has a number of advantages: * A notebook can be imported, manipulated and run by anyone who has the support code (the notebook module that defines the relevant classes). * A notebook doesn't need to be parsed. It is valid Python and can be imported or exec'd. Once that is done, you have the full notebook in memory. You can immediately do anything you want with it. * The various Notebook, Cell, Image, etc. classes can know about how to output to various formats, latex, html, reST, XML, etc: import mynotebook mynotebook.notebook.export('rest') * Each individual format (HTML, reST, latex) has weaknesses. If you pick any one to be *the* notebook format, you are building those weaknesses into your design. A pure python based notebook format won't suffer from that syndrome. * It is a clean separation of the model (Notebook, Cell, Image, etc.) and the view (HTML, reST, etc.). Picking HTML or reST for the notebook format confuses (at some level) the model and view... * Third party code can define new Notebook elements that specify how they can be rendered in different contexts. For example, matplotlib could ship a Figure element that knows how to render itself as a native PyQt GUI, a static image, a web page, etc. * A notebook remains a single plain text file that anyone can edit - even if it has embedded images. Neither HTML nor reST have the ability to inline graphics in plain text files. While I love reST, it is a pain that I need an entire directory of files to render a single Sphinx doc. Anyways, that is where my brain is wandering these days. Cheers, Brian On Tue, Feb 16, 2010 at 6:21 PM, Ondrej Certik wrote: > Hi, > > has anyone given a deeper thought how to best implement this functionality? > > Currently the Sage notebook seems to remember the worksheet using: > > -------------- > sdsd > system:sage > > >

Well, some text

> > {{{id=5| > awe > /// > }}} > > {{{id=1| > 4 > /// > 4 > }}} > > {{{id=2| > 5 > /// > 5 > }}} > > {{{id=3| > > /// > }}} > --------------------- > > E.g. each cell is either a text cell, or a cell that can be evaluated. > Is that right? > > I know that Fernando suggested here: > > > http://groups.google.com/group/sage-devel/browse_thread/thread/3f1f5a0c26975519/ > > to use reST, e.g. something like: > > ---------- > Well, some text > > :: > >>> print 3 > 3 > > ------------ > > > I am implementing a notebook prototype using pure Python > (automatically translated to javascript using pyjamas) and I would > like to run it on the google app engine (the javascript in your > browser would then communicate either with the app engine, or your own > hardware that would host Sage/FEMhub/your own code): > > http://gamma.sympy.org/nb/ > > and here is my development version with logins and settings implemented: > > http://2.latest.sympy-gamma.appspot.com/nb/ > > now I need to implement the worksheets, so I am thinking how to best > represent it. > > > Did the syntax for input/output blocks in Sphinx settled already? I > guess internally I will implement the worksheet as a set of cells, > either text cells, or "evaluatable" cells (that contain both > input/output blocks) and then support conversion to and from > reST/Sphinx. > > I would be interested in any feedback. > > Ondrej > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vivainio at gmail.com Wed Feb 24 03:39:23 2010 From: vivainio at gmail.com (Ville M. Vainio) Date: Wed, 24 Feb 2010 10:39:23 +0200 Subject: [IPython-dev] using reST for representing the notebook cells+text In-Reply-To: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> Message-ID: <46cb515a1002240039r57a62acdh3d94d7df50c0e379@mail.gmail.com> On Wed, Feb 17, 2010 at 4:21 AM, Ondrej Certik wrote: > I am implementing a notebook prototype using pure Python > (automatically translated to javascript using pyjamas) and I would You may want to draw some inspiration from caliopy: http://www.caliopywork.org/ -- Ville M. Vainio http://tinyurl.com/vainio From robert.kern at gmail.com Wed Feb 24 10:49:56 2010 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 24 Feb 2010 09:49:56 -0600 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> Message-ID: <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> On Wed, Feb 24, 2010 at 01:52, Brian Granger wrote: > In many respects it seems almost perfect.? But, one day recently I was > thinking about how successful Mathematica's notebook continues to be.? So I > began to look more at the Mathematica notebook format.? Amazingly, the > Mathematica notebook format is a plain text file that itself is *valid > Mathematica code*. The problem with transferring this to Python is that Mathematica's language is a very Lispy one. ExpressionCell[] contains the actual expression, not a string representing a Mathematica expression. You can't do that with Python. You would have to put the Python code into strings. You just have a "dumb" tree structure with text leaf nodes. At that point, you might as well use XML to describe the exact same structure. With XML, you can push it across to other languages, JavaScript being the hugely important player here. Certainly, you are going to have a Python API that will represent that tree of text nodes as Python objects, but I just don't see the point of making the repr() of that be the lingua franca format of the notebook file. It's just a wasted opportunity. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From fperez.net at gmail.com Wed Feb 24 11:39:13 2010 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 24 Feb 2010 11:39:13 -0500 Subject: [IPython-dev] using reST for representing the notebook cells+text In-Reply-To: <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> Message-ID: On Wed, Feb 24, 2010 at 2:52 AM, Brian Granger wrote: > > In many respects it seems almost perfect.? But, one day recently I was > thinking about how successful Mathematica's notebook continues to be.? So I > began to look more at the Mathematica notebook format.? Amazingly, the > Mathematica notebook format is a plain text file that itself is *valid > Mathematica code*.? This id documented here: > > http://reference.wolfram.com/mathematica/guide/LowLevelNotebookProgramming.html > > For examples a simple notebook with one text cell is just: > > Notebook[{Cell['Here is my text', 'Text']}] > > Everything - input cells, output cells, static images and all are > represented in this way and embedded in the plain text notebook file.? The > Python generalization of this would be the following: > > * A Python notebook is plain text, importable Python code. > * That code is simply a tree of objects that declare the relevant parts of > the notebook. > > This has a number of advantages: > > * A notebook can be imported, manipulated and run by anyone who has the > support code (the notebook module that defines the relevant classes). > * A notebook doesn't need to be parsed.? It is valid Python and can be > imported or exec'd.? Once that is done, you have the full notebook in > memory.? You can immediately do anything you want with it. > * The various Notebook, Cell, Image, etc. classes can know about how to > output to various formats, latex, html, reST, XML, etc: > > import mynotebook > mynotebook.notebook.export('rest') > > * Each individual format (HTML, reST, latex) has weaknesses.? If you pick > any one to be *the* notebook format, you are building those weaknesses into > your design.? A pure python based notebook format won't suffer from that > syndrome. > * It is a clean separation of the model (Notebook, Cell, Image, etc.) and > the view (HTML, reST, etc.).? Picking HTML or reST for the notebook format > confuses (at some level) the model and view... > * Third party code can define new Notebook elements that specify how they > can be rendered in different contexts.? For example, matplotlib could ship a > Figure element that knows how to render itself as a native PyQt GUI, a > static image, a web page, etc. > * A notebook remains a single plain text file that anyone can edit - even if > it has embedded images.? Neither HTML nor reST have the ability to inline > graphics in plain text files.? While I love reST, it is a pain that I need > an entire directory of files to render a single Sphinx doc. I had just composed a reply offline (I had no network for a bit) but I just now saw that in the interim, Robert said pretty much what I had written. Since python isn't lisp(ish), I'm also not convinced that a code-as-data approach will be all that useful for us. I do think rest can be the next best thing for us, since at least we have parsing machinery (docutils), extensible document tools (sphinx), and any tools on top of rest/sphinx would likely get a lot of reuse. I added your notes to the ipython docs for reference though, it will be good to have these notes all in one place. Cheers, f From ellisonbg at gmail.com Wed Feb 24 11:48:06 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 24 Feb 2010 08:48:06 -0800 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> Message-ID: [Happy to get a lively response] > The problem with transferring this to Python is that Mathematica's > language is a very Lispy one. ExpressionCell[] contains the actual > expression, not a string representing a Mathematica expression. You > can't do that with Python. You would have to put the Python code into > strings. Absolutely....almost. You are right that the Notebook, Cell, etc. in the Mathematica notebook format are full Mathematica expressions. But, to represent Mathematica code, they use strings: Here is a simple "a=10" input cell: Cell[BoxData[ RowBox[{"a", " ", "=", " ", "10"}]], "Input", CellChangeTimes->{{3.4759886492031307`*^9, 3.475988649761156*^9}}] The reason (I am guessing) that Wolfram did this is that the Notebook lives in the frontend, the "code" (a=10) lives in the kernel. Thus it doesn't make sense to "instantiate" the code into a full expression until it is in the backend. Which is *exactly* how this would work in Python/IPython. By this, I mean that the internal representation of "code" in IPython is a string. > You just have a "dumb" tree structure with text leaf nodes. At some level I agree with you. You would have a tree in Text, Input, Output cells that simply have plain text leaf nodes like Text("Here is the text"). But that does not mean that the Cell and Notebook objects could not have powerful APIs for working with these cells. Things like: * Representation (reST, HTML, latex, etc.) * Insert, delete, move, etc. * Get all the strings of python code to execute and inject their outputs. * Whatever else you need. Because the Notebook would live in the frontend, it would *not* contain methods for code execution. But it could have an API for working with objects that interface with the kernel. > At that point, you might as well use XML to describe the exact same > structure. With XML, you can push it across to other languages, > JavaScript being the hugely important player here. Certainly, you are > going to have a Python API that will represent that tree of text nodes > as Python objects, but I just don't see the point of making the repr() > of that be the lingua franca format of the notebook file. It's just a > wasted opportunity. Again, I think the biggest issues with XML are: * No one wants to edit XML by hand. * The result is not importable in a regular Python shell. Obviously, reST doesn't suffer from the first of these issues, but it does share the second (as well as not being able to embed binary data like images. Cheers, Brian > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma that is made terrible by our own mad attempt to interpret it as > though it had an underlying truth." > ?-- Umberto Eco > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From robert.kern at gmail.com Wed Feb 24 11:55:43 2010 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 24 Feb 2010 10:55:43 -0600 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> Message-ID: <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> On Wed, Feb 24, 2010 at 10:48, Brian Granger wrote: > [Happy to get a lively response] > >> The problem with transferring this to Python is that Mathematica's >> language is a very Lispy one. ExpressionCell[] contains the actual >> expression, not a string representing a Mathematica expression. You >> can't do that with Python. You would have to put the Python code into >> strings. > > Absolutely....almost. ?You are right that the Notebook, Cell, etc. in > the Mathematica > notebook format are full Mathematica expressions. ?But, to represent Mathematica > code, they use strings: > > Here is a simple "a=10" input cell: > > Cell[BoxData[ > ?RowBox[{"a", " ", "=", " ", "10"}]], "Input", > ?CellChangeTimes->{{3.4759886492031307`*^9, 3.475988649761156*^9}}] > > The reason (I am guessing) that Wolfram did this is that the Notebook > lives in the > frontend, the "code" (a=10) lives in the kernel. ?Thus it doesn't make sense > to "instantiate" the code into a full expression until it is in the > backend. ?Which is > *exactly* how this would work in Python/IPython. ?By this, I mean that > the internal > representation of "code" in IPython is a string. > >> You just have a "dumb" tree structure with text leaf nodes. > > At some level I agree with you. ?You would have a tree in Text, Input, > Output cells > that simply have plain text leaf nodes like Text("Here is the text"). ?But that > does not mean that the Cell and Notebook objects could not have powerful APIs > for working with these cells. I agree, but that doesn't imply that the main text serialization of the objects in the API should be Python code. >> At that point, you might as well use XML to describe the exact same >> structure. With XML, you can push it across to other languages, >> JavaScript being the hugely important player here. Certainly, you are >> going to have a Python API that will represent that tree of text nodes >> as Python objects, but I just don't see the point of making the repr() >> of that be the lingua franca format of the notebook file. It's just a >> wasted opportunity. > > Again, I think the biggest issues with XML are: > > * No one wants to edit XML by hand. I don't want to write Python code in that tree style by hand, either. Too many things to get wrong. I don't see anyone editing notebooks outside of the notebook apps, honestly. Does anyone edit Mathematica notebooks in a text editor? I don't think the success of Mathematica's notebook is the file format; it's the notebook GUI itself. > * The result is not importable in a regular Python shell. So you use a function call instead. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg at gmail.com Wed Feb 24 13:39:09 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 24 Feb 2010 10:39:09 -0800 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> Message-ID: >> * No one wants to edit XML by hand. > > I don't want to write Python code in that tree style by hand, either. Yes, for the most part, but the argument for reST is that people can edit it by hand. Python is closer to that, but I agree that most people won't probably do that. > Too many things to get wrong. I don't see anyone editing notebooks > outside of the notebook apps, honestly. Does anyone edit Mathematica > notebooks in a text editor? I don't think the success of Mathematica's > notebook is the file format; it's the notebook GUI itself. > >> * The result is not importable in a regular Python shell. > > So you use a function call instead. True, that works for getting an in memory rep of the notebook (parsing). But the importability feature of a pure python notebook format has a broader impact on the design. This is related to validation and extensibility. validation: not all XML files are valid notebook files. Thus, you have to validate the XML at some level. This means 1) either developing a schema (which is very rigid) or 2) doing the validation in Python once you parse the XML. In the case of a pure python notebook, validation is done by Python itself. If the notebook can be exec'd or imported, it is a valid notebook extensibility: using XML requires specifying the XML schema in a central location. That schema is fixed and thus hard to extend. Using Python allows the notebook format to be extended by simple subclassing: class MatplotlibFigureCell(Cell): Sure, if your notebook uses the MatplotlibCell > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma that is made terrible by our own mad attempt to interpret it as > though it had an underlying truth." > ?-- Umberto Eco > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From ellisonbg at gmail.com Wed Feb 24 13:46:33 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 24 Feb 2010 10:46:33 -0800 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> Message-ID: Ooopps, hit keyboard shortcut for send... > Using Python allows the notebook format to be extended by simple > subclassing: > > class MatplotlibFigureCell(Cell): This subclass could contain all the logic for representing the matplotlib figure in different formats: html, jpeg, svg, native pyqt gui, etc and could be distributed with matplotlib. Sure, if your notebook uses the MatplotlibCell, you will need matplotlib, but the validation of that aspect is handled by python itself, not an XML schema. Doing these types of things using XML would require all of this to be put into the centralized XML schema, making it difficult for third parties to extend. Plus all of the additional logic associated with that Cell type has to be put somewhere as well. [I am thinking aloud here...not sure if all of this would pan out in practice...only actual code would tell...] [For what it is worth, while I *love* the idea of having a reST based notebook format, I think it is simply too limiting in the long run. Between reST and XML, I think XML wins...] Cheers, Brian > > > >> -- >> Robert Kern >> >> "I have come to believe that the whole world is an enigma, a harmless >> enigma that is made terrible by our own mad attempt to interpret it as >> though it had an underlying truth." >> ?-- Umberto Eco >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > Brian E. Granger, Ph.D. > Assistant Professor of Physics > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu > ellisonbg at gmail.com > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From ondrej at certik.cz Wed Feb 24 13:55:13 2010 From: ondrej at certik.cz (Ondrej Certik) Date: Wed, 24 Feb 2010 10:55:13 -0800 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> Message-ID: <85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com> On Wed, Feb 24, 2010 at 10:46 AM, Brian Granger wrote: > Ooopps, hit keyboard shortcut for send... > >> Using Python allows the notebook format to be extended by simple >> subclassing: >> >> class MatplotlibFigureCell(Cell): > > This subclass could contain all the logic for representing the > matplotlib figure in different formats: html, jpeg, svg, native pyqt > gui, etc and could be distributed with matplotlib. > > Sure, if your notebook uses the MatplotlibCell, you will need > matplotlib, but the validation of that aspect is handled by python > itself, not an XML schema. > > Doing these types of things using XML would require all of this to be > put into the centralized XML schema, making it difficult for third > parties to extend. ?Plus all of the additional logic associated with > that Cell type has to be put somewhere as well. > > [I am thinking aloud here...not sure if all of this would pan out in > practice...only actual code would > tell...] > > [For what it is worth, while I *love* the idea of having a reST based > notebook format, I think it is simply > too limiting in the long run. ?Between reST and XML, I think XML wins...] I will do reST import/export, but having it the default format, that I think sucks. With xml, anyone can easily write a parsers for it in any language. Besides, I need to push the worksheet to the browser. In my case, I use pyjamas, so it's pure Python, however, exec/eval are not evailable there (as far as I know), but I am sure xml would work, here is some code to handle xml in pyjamas: http://pyjamas.sourceforge.net/examples/xmldoc/XMLload.py I think, that it might work to push the whole Python objects somehow (that's the nice thing about pyjamas, that since you have python on both sides, you can do lots of cool stuff), but maybe not. I like Brian's idea to be able to just import it in Python. But I think I'll just stick to xml for the first iteration of my notebook, to get the job done. Ondrej From robert.kern at gmail.com Wed Feb 24 13:57:12 2010 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 24 Feb 2010 12:57:12 -0600 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> Message-ID: <3d375d731002241057n3f7bc5b4h5e4ae3153586b083@mail.gmail.com> On Wed, Feb 24, 2010 at 12:39, Brian Granger wrote: >>> * No one wants to edit XML by hand. >> >> I don't want to write Python code in that tree style by hand, either. > > Yes, for the most part, but the argument for reST is that people can > edit it by hand. ?Python is closer to that, but I agree that most people > won't probably do that. No, Python is *not* closer to reST than XML. reST is very different from either XML or the Python object tree. The latter are much more similar to each other than they are to reST. >> Too many things to get wrong. I don't see anyone editing notebooks >> outside of the notebook apps, honestly. Does anyone edit Mathematica >> notebooks in a text editor? I don't think the success of Mathematica's >> notebook is the file format; it's the notebook GUI itself. >> >>> * The result is not importable in a regular Python shell. >> >> So you use a function call instead. > > True, that works for getting an in memory rep of the notebook > (parsing). ?But the > importability feature of a pure python notebook format has a broader > impact on the design. ?This is related > to validation and extensibility. > > validation: not all XML files are valid notebook files. ?Thus, you > have to validate the XML at some level. ?This means 1) either > developing a schema (which is very rigid) or 2) doing the validation > in Python once you parse the XML. ?In the case of a pure python > notebook, validation is done by Python itself. ?If the notebook can be > exec'd or imported, it is a valid notebook And this is what I think is most wrong with using Python to do this. It's too flexible and too easy to create things that are incompatible with other representations. It's fine to build things that are incompatible (and I wouldn't want to prevent you from creating a Notebook object that was incompatible with some of its representations), but the main file format is a bad place to make that limitation. > extensibility: using XML requires specifying the XML schema in a > central location. ?That schema is fixed and thus hard to extend. The X is there for a reason. :-) It stands for eXtensible. It really isn't hard to add tags or to write systems that do sensible things when they see tags that they don't handle. This is exactly what XML was designed for. > Using Python allows the notebook format to be extended by simple > subclassing: > > class MatplotlibFigureCell(Cell): ... > This subclass could contain all the logic for representing the > matplotlib figure in different formats: html, jpeg, svg, native pyqt > gui, etc and could be distributed with matplotlib. > > Sure, if your notebook uses the MatplotlibCell, you will need > matplotlib, but the validation of that aspect is handled by python > itself, not an XML schema. *You don't need XML schemas to validate XML.* They are often entirely superfluous. Now, you will need a Python API that builds Notebook instances with Cells and all that jazz. And you will need a way to build that object tree from XML. *That* is your validation. Algorithm succeeds == valid file. The same is actually true for any other format. > Doing these types of things using XML would require all of this to be > put into the centralized XML schema, making it difficult for third > parties to extend. ?Plus all of the additional logic associated with > that Cell type has to be put somewhere as well. Even if you do develop an XML schema to use, these are easily made extensible. Providing a mapping from tag names to classes is a straightforward way to extend the parser. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg at gmail.com Wed Feb 24 14:15:15 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 24 Feb 2010 11:15:15 -0800 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> <85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com> Message-ID: > I will do reST import/export, but having it the default format, that I > think sucks. With xml, anyone can easily write a parsers for it in any > language. > > Besides, I need to push the worksheet to the browser. In my case, I > use pyjamas, so it's pure Python, however, exec/eval are not evailable > there (as far as I know), but I am sure xml would work, here is some > code to handle xml in pyjamas: A point of clarification. The Notebook and Cell objects would not be responsible for the execution of python code. The execution logic would simply pull InputCells out of the Notebook, extract the python code (as a str) and return the OutputCell. [Thinking out loud about what this might look like in a web browser....more thought needed] The Python Notebook and Cell objects would not be pyjamas entities - that part of the python code would not be compiled to Javascript. The Notebook and Cell objects would live in the web server and be capable of rendering themselves as json or html. The pyjamas classes would contain the logic for the user interface, and calling out to the Notebook/Cell web service and pass XML/json back and forth.... > I think, that it might work to push the whole Python objects somehow > (that's the nice thing about pyjamas, that since you have python on > both sides, you can do lots of cool stuff), but maybe not. Yes, that might work. I have been thinking about the Cell objects as having all the logic needed to render itself in various formats. But maybe a Cell object itself could be lighter weight - the rendering logic could be a separate class. Then the Cell objects could be compiled to JS using pyjamas. But I am not familiar with how pyjamas works enough to see if this would work. > I like Brian's idea to be able to just import it in Python. But I > think I'll just stick to xml for the first iteration of my notebook, > to get the job done. Definitely don't wait for me to implement my idea...I do agree with your statement about the ease of parsing XML though. Cheers, Brian > Ondrej > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From ondrej at certik.cz Wed Feb 24 15:10:11 2010 From: ondrej at certik.cz (Ondrej Certik) Date: Wed, 24 Feb 2010 12:10:11 -0800 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> <85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com> Message-ID: <85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com> On Wed, Feb 24, 2010 at 11:15 AM, Brian Granger wrote: >> I will do reST import/export, but having it the default format, that I >> think sucks. With xml, anyone can easily write a parsers for it in any >> language. >> >> Besides, I need to push the worksheet to the browser. In my case, I >> use pyjamas, so it's pure Python, however, exec/eval are not evailable >> there (as far as I know), but I am sure xml would work, here is some >> code to handle xml in pyjamas: > > A point of clarification. ?The Notebook and Cell objects would not be > responsible > for the execution of python code. ?The execution logic would simply > pull InputCells > out of the Notebook, extract the python code (as a str) and return the > OutputCell. > > [Thinking out loud about what this might look like in a web > browser....more thought needed] > > The Python Notebook and Cell objects would not be pyjamas entities - that > part of the python code would not be compiled to Javascript. ?The > Notebook and Cell > objects would live in the web server and be capable of rendering > themselves as json > or html. > > The pyjamas classes would contain the logic for the user interface, > and calling out to the > Notebook/Cell web service and pass XML/json back and forth.... That's right. I just did more studying of this and I don't want to parse xml by hand in javascript/pyjamas. I would like to use the builtin functionality for RPC, essentially, you just call python methods on both sides and things get automatically transfered (using json/xml RPC) behind the scene, it's very convenient. > >> I think, that it might work to push the whole Python objects somehow >> (that's the nice thing about pyjamas, that since you have python on >> both sides, you can do lots of cool stuff), but maybe not. > > Yes, that might work. ?I have been thinking about the Cell objects as having > all the logic needed to render itself in various formats. ?But maybe a > Cell object > itself could be lighter weight - the rendering logic could be a separate class. > Then the Cell objects could be compiled to JS using pyjamas. ?But I am > not familiar > with how pyjamas works enough to see if this would work. > >> I like Brian's idea to be able to just import it in Python. But I >> think I'll just stick to xml for the first iteration of my notebook, >> to get the job done. > > Definitely don't wait for me to implement my idea...I do agree with > your statement about > the ease of parsing XML though. Here is my current code, that runs in the browser: http://github.com/certik/sympy_gamma/blob/login/templates/nb.py and here is the code, that runs on the server, listens at some url (/eval_cell/...) and evaluates the cell: http://github.com/certik/sympy_gamma/blob/login/app/views.py#L126 it's just 15 lines, 7 of those are debug prints. So there is no way to remember sessions, worksheets anything yet. So I will have Notebook/Cells on the server, the eval_cell() method will (besides the actual evaluation) keep the tree structure of Notebook/Cells updated. This structure will know how to export/import to/from many formats (reST, xml, ...), that part is clear. The rendering itself is done using pyjamas, so the Notebook/Cells classes don't need to know anything about it, they just contain the logic. On the browser part, I will also need some Notebook_pyjamas/Cell_pyjamas classes, that are purely responsible for rendering and I need somehow to initialize them from the server. I think I will do that by implementing Notebook_pyjamas.add_cell() method, that will be exported using RPC to the server, so the server will call this method as many times as needed, and add there parameters (input_text, output_text, type_of_the_cell="text"/"eval"). This would also solve my problem with passing data to the pyjamas app. So far I passed it by saving it to the "meta" tags in the header of the html page, and then getting it from inside pyjamas. But it sucks, it's not very versatile. See e.g. here the code to read it: http://github.com/certik/sympy_gamma/blob/login/templates/nb.py#L453 it reads the html elements and returns a Python dict with the options. It's very fragile. So this pyjamas development is a bit different to what one was used to before (e.g. generating the html on the server), essentially the philosophy is to initialize "Python" in the browser as soon as possible and then do everything on the Python level. And generate all html in the browser itself, from Python by constructing appropriate pyjamas GUI widgets (e.g. I subclassed the TextArea widget that implements the html textarea, with backspace/tab, shift+enter, correctly resizing it, calculating (x, y) coordinates of the cursor /nontrivial.../, etc, etc). It's kind of a fake, since there is no Python in the browser, just javascript, so it took my a while to realize that I can think of this really as having a Python interpreter in the browser (resp. a static subset of Python). Ondrej From ellisonbg at gmail.com Wed Feb 24 15:24:18 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 24 Feb 2010 12:24:18 -0800 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> <85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com> <85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com> Message-ID: Ondrej, >> The pyjamas classes would contain the logic for the user interface, >> and calling out to the >> Notebook/Cell web service and pass XML/json back and forth.... > > That's right. I just did more studying of this and I don't want to > parse xml by hand in javascript/pyjamas. I would like to use the > builtin functionality > ?for RPC, essentially, you just call python methods on both sides and > things get automatically transfered (using json/xml RPC) behind the > scene, it's very convenient. Nice, that was my impression with pyjamas. Sounds cool. > Here is my current code, that runs in the browser: > > http://github.com/certik/sympy_gamma/blob/login/templates/nb.py > > and here is the code, that runs on the server, listens at some url > (/eval_cell/...) and evaluates the cell: > > http://github.com/certik/sympy_gamma/blob/login/app/views.py#L126 > > it's just 15 lines, 7 of those are debug prints. This is super nice code. Very easy to understand. I really like how pyjamas works. > > So there is no way to remember sessions, worksheets anything yet. So I > will have Notebook/Cells on the server, the eval_cell() method will > (besides the actual evaluation) keep the tree structure of > Notebook/Cells updated. This structure will know how to export/import > to/from many formats (reST, xml, ...), that part is clear. Yes, you will have to add urls and handlers that know how to add cells, delete cells, etc. I would follow a RESTful design of those URLS: http://en.wikipedia.org/wiki/Representational_State_Transfer > The rendering itself is done using pyjamas, so the Notebook/Cells > classes don't need to know anything about it, they just contain the > logic. OK, this is the aspect of this model I still don't quite see - where the representation stuff is handled. > On the browser part, I will also need some > Notebook_pyjamas/Cell_pyjamas classes, that are purely responsible for > rendering and I need somehow to initialize them from the server. I > think I will do that by implementing Notebook_pyjamas.add_cell() > method, that will be exported using RPC to the server, so the server > will call this method as many times as needed, and add there > parameters (input_text, output_text, type_of_the_cell="text"/"eval"). Yep. > This would also solve my problem with passing data to the pyjamas app. > So far I passed it by saving it to the "meta" tags in the header of > the html page, and then getting it from inside pyjamas. But it sucks, > it's not very versatile. See e.g. here the code to read it: > > http://github.com/certik/sympy_gamma/blob/login/templates/nb.py#L453 > > it reads the html elements and returns a Python dict with the options. > It's very fragile. Right using json rpc will be much better. > So this pyjamas development is a bit different to what one was used to > before (e.g. generating the html on the server), essentially the > philosophy is to initialize "Python" in the browser as soon as > possible and then do everything on the Python level. And generate all > html in the browser itself, from Python by constructing appropriate > pyjamas GUI widgets (e.g. I subclassed the TextArea widget that > implements the html textarea, with backspace/tab, shift+enter, > correctly resizing it, calculating (x, y) coordinates of the cursor > /nontrivial.../, etc, etc). It's kind of a fake, since there is no > Python in the browser, just javascript, so it took my a while to > realize that I can think of this really as having a Python interpreter > in the browser (resp. a static subset of Python). This is great ,I am curious to see how this evolves. Brian > Ondrej > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From ondrej at certik.cz Wed Feb 24 15:36:57 2010 From: ondrej at certik.cz (Ondrej Certik) Date: Wed, 24 Feb 2010 12:36:57 -0800 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> <85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com> <85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com> Message-ID: <85b5c3131002241236o257efd87v9f3d1c6f37bd105c@mail.gmail.com> On Wed, Feb 24, 2010 at 12:10 PM, Ondrej Certik wrote: > On Wed, Feb 24, 2010 at 11:15 AM, Brian Granger wrote: >>> I will do reST import/export, but having it the default format, that I >>> think sucks. With xml, anyone can easily write a parsers for it in any >>> language. >>> >>> Besides, I need to push the worksheet to the browser. In my case, I >>> use pyjamas, so it's pure Python, however, exec/eval are not evailable >>> there (as far as I know), but I am sure xml would work, here is some >>> code to handle xml in pyjamas: >> >> A point of clarification. ?The Notebook and Cell objects would not be >> responsible >> for the execution of python code. ?The execution logic would simply >> pull InputCells >> out of the Notebook, extract the python code (as a str) and return the >> OutputCell. >> >> [Thinking out loud about what this might look like in a web >> browser....more thought needed] >> >> The Python Notebook and Cell objects would not be pyjamas entities - that >> part of the python code would not be compiled to Javascript. ?The >> Notebook and Cell >> objects would live in the web server and be capable of rendering >> themselves as json >> or html. >> >> The pyjamas classes would contain the logic for the user interface, >> and calling out to the >> Notebook/Cell web service and pass XML/json back and forth.... > > That's right. I just did more studying of this and I don't want to > parse xml by hand in javascript/pyjamas. I would like to use the > builtin functionality > ?for RPC, essentially, you just call python methods on both sides and > things get automatically transfered (using json/xml RPC) behind the > scene, it's very convenient. > >> >>> I think, that it might work to push the whole Python objects somehow >>> (that's the nice thing about pyjamas, that since you have python on >>> both sides, you can do lots of cool stuff), but maybe not. >> >> Yes, that might work. ?I have been thinking about the Cell objects as having >> all the logic needed to render itself in various formats. ?But maybe a >> Cell object >> itself could be lighter weight - the rendering logic could be a separate class. >> Then the Cell objects could be compiled to JS using pyjamas. ?But I am >> not familiar >> with how pyjamas works enough to see if this would work. >> >>> I like Brian's idea to be able to just import it in Python. But I >>> think I'll just stick to xml for the first iteration of my notebook, >>> to get the job done. >> >> Definitely don't wait for me to implement my idea...I do agree with >> your statement about >> the ease of parsing XML though. > > Here is my current code, that runs in the browser: > > http://github.com/certik/sympy_gamma/blob/login/templates/nb.py > > and here is the code, that runs on the server, listens at some url > (/eval_cell/...) and evaluates the cell: > > http://github.com/certik/sympy_gamma/blob/login/app/views.py#L126 > > it's just 15 lines, 7 of those are debug prints. > > > So there is no way to remember sessions, worksheets anything yet. So I > will have Notebook/Cells on the server, the eval_cell() method will > (besides the actual evaluation) keep the tree structure of > Notebook/Cells updated. This structure will know how to export/import > to/from many formats (reST, xml, ...), that part is clear. > > The rendering itself is done using pyjamas, so the Notebook/Cells > classes don't need to know anything about it, they just contain the > logic. > > On the browser part, I will also need some > Notebook_pyjamas/Cell_pyjamas classes, that are purely responsible for > rendering and I need somehow to initialize them from the server. I > think I will do that by implementing Notebook_pyjamas.add_cell() > method, that will be exported using RPC to the server, so the server > will call this method as many times as needed, and add there > parameters (input_text, output_text, type_of_the_cell="text"/"eval"). The browser part is actually already implemented (see the nb.py file above), here is how I call it: CellWidget Worksheet I think I will rename Worksheet to WorksheetWidget, then it will be clear that it is just a widget. Hm, so it is more complex. The WorksheetWidget currently contains methods to join cells, etc. Currently all this "technical" logic is determined in pyjamas (e.g. the cells can only by joined sometimes, etc.). So I think the WorksheetWidget would just call Notebook.join_cell() RPC call on the server, when the cells were joined. Note sure now, what to do with synchronization. E.g. let's say the RPC call fails (no net connection), should the user wait until the RPC call finishes to make sure the cells can be joined, or should they be joined in the browser immediately and then somehow remember all the pending changes to push them to the server? I will ignore this problem for now. I will start by implementing this RPC logic --- currently I don't use RPC to evaluate cells, here is what I do now: http://github.com/certik/sympy_gamma/blob/login/templates/nb.py#L335 I simply do an asynchronous html request, pass in my Loader() instance, and its onCompletion() method gets called, I decode the JSON data and that's it. So I will start by rewriting this part of the code to use RPC, and then slowly increase the RPC interface on both sides and implement Notebook and other things on the server and simply communicate between the server and the browser more often. Here is a demo how to use RPC in pyjamas: http://pyjs.org/examples/jsonrpc/output/JSONRPCExample.html and here is the code: http://pyjs.org/examples/jsonrpc/JSONRPCExample.py And more documentation is here: http://pyjs.org/book/output/Bookreader.html#Rest%20of%20the%20World So far this is how to call methods (implemented on the server) in pyjamas, e.g. stuff like for evaluating cells. So after pyjamas starts, I will call the server method "give_me_the_cells", and I guess it can just pass me a python dictionary with the cells data, and I initialize it according to it. So it seems the RPC is always initiated from the browser. > Yes, you will have to add urls and handlers that know how to add > cells, delete cells, > etc. ?I would follow a RESTful design of those URLS: > > http://en.wikipedia.org/wiki/Representational_State_Transfer Actually, from the pyjamas side, I will just call methods, no need to bother with URLs, and on the server side (django), there is also a functionality in django, so that I don't have to bother with URLs either. It's explained in the pyjamas docs link, above. Ok, I think I know what to do now, there are still couple details that are not clear to me, but it will sort itself out, once I find some time to implement it. Ondrej From ellisonbg at gmail.com Wed Feb 24 15:48:35 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 24 Feb 2010 12:48:35 -0800 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <85b5c3131002241236o257efd87v9f3d1c6f37bd105c@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> <85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com> <85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com> <85b5c3131002241236o257efd87v9f3d1c6f37bd105c@mail.gmail.com> Message-ID: Ondrej, > The browser part is actually already implemented (see the nb.py file > above), here is how I call it: > > CellWidget > Worksheet > > I think I will rename Worksheet to WorksheetWidget, then it will be > clear that it is just a widget. > > Hm, so it is more complex. The WorksheetWidget currently contains > methods to join cells, etc. Currently all this "technical" logic is > determined in pyjamas (e.g. the cells can only by joined sometimes, > etc.). So I think the WorksheetWidget would just call > Notebook.join_cell() RPC call on the server, when the cells were > joined. Yes. > Note sure now, what to do with synchronization. E.g. let's say the RPC > call fails (no net connection), should the user wait until the RPC > call finishes to make sure the cells can be joined, or should they be > joined in the browser immediately and then somehow remember all the > pending changes to push them to the server? I will ignore this problem > for now. I think the browser should wait until it gets the server response - obviously doing the browser side pending changes might speed it up, but it more work... > I will start by implementing this RPC logic --- currently I don't use > RPC to evaluate cells, here is what I do now: > > http://github.com/certik/sympy_gamma/blob/login/templates/nb.py#L335 > > I simply do an asynchronous html request, pass in my Loader() > instance, and its onCompletion() method gets called, I decode the JSON > data and that's it. So I will start by rewriting this part of the code > to use RPC, and then slowly increase the RPC interface on both sides > and implement Notebook and other things on the server and simply > communicate between the server and the browser more often. The RPC will definitely be an improvement. > Here is a demo how to use RPC in pyjamas: > > http://pyjs.org/examples/jsonrpc/output/JSONRPCExample.html > > and here is the code: > > http://pyjs.org/examples/jsonrpc/JSONRPCExample.py > > And more documentation is here: > > http://pyjs.org/book/output/Bookreader.html#Rest%20of%20the%20World Nice, thanks for the links. > > So far this is how to call methods (implemented on the server) in > pyjamas, e.g. stuff like for evaluating cells. So after pyjamas > starts, I will call the server method "give_me_the_cells", and I guess > it can just pass me a python dictionary with the cells data, and I > initialize it according to it. > > So it seems the RPC is always initiated from the browser. Yes, that is the traditional "AJAX" limitation. Allowing the server to initiate the RPC goes under the name "Comet" and is much more difficult. >> Yes, you will have to add urls and handlers that know how to add >> cells, delete cells, >> etc. ?I would follow a RESTful design of those URLS: >> >> http://en.wikipedia.org/wiki/Representational_State_Transfer > > Actually, from the pyjamas side, I will just call methods, no need to > bother with URLs, and on the server side (django), there is also a > functionality in django, so that I don't have to bother with URLs > either. It's explained in the pyjamas docs link, above. This is very nice. I wish I had some time to play with pyjamas... > Ok, I think I know what to do now, there are still couple details that > are not clear to me, but it will sort itself out, once I find some > time to implement it. Cheers, Brian -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From termim at gmail.com Wed Feb 24 18:29:11 2010 From: termim at gmail.com (Mikhail Terekhov) Date: Wed, 24 Feb 2010 18:29:11 -0500 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> Message-ID: <12aaa0811002241529k76b509f3g7b9f1e4230e39452@mail.gmail.com> On Wed, Feb 24, 2010 at 4:04 PM, Robert Kern wrote: > [Did you mean to post this to one of the lists?] > Robert, yes I meant to send this to the list, sorry - hit the wrong button. Apology for the list members if this discussion is a closed matter already, I just thought it is interesting. I'm attaching our conversation with Robert in hope it will be somehow useful. On Wed, Feb 24, 2010 at 14:31, Mikhail Terekhov wrote: > On Wed, Feb 24, 2010 at 10:49 AM, Robert Kern wrote: >> On Wed, Feb 24, 2010 at 01:52, Brian Granger wrote: >> >>> In many respects it seems almost perfect. But, one day recently I was >>> thinking about how successful Mathematica's notebook continues to be. So I >>> began to look more at the Mathematica notebook format. Amazingly, the >>> Mathematica notebook format is a plain text file that itself is *valid >>> Mathematica code*. >> >> The problem with transferring this to Python is that Mathematica's >> language is a very Lispy one. ExpressionCell[] contains the actual >> expression, not a string representing a Mathematica expression. You > > Not quite so, see for example: > http://reference.wolfram.com/mathematica/ref/format/NB.html > > It is true that nb format contains a valid Mathematica's code but it is not > the code user usually deals with but a lower level one where expressions > are constructed from strings and API calls. This is exactly as Brian suggests > AFAICS. > >> can't do that with Python. You would have to put the Python code into >> strings. You just have a "dumb" tree structure with text leaf nodes. > > That is exactly as Mathematica does that, at least as it is described in > the link above. Quite so. >> At that point, you might as well use XML to describe the exact same >> structure. > > Sure you can but you have to pay a price for that and the price is XML > parsing. While it looks negligible when your nb is just a couple of dozen > lines it will be huge when you have to load many big nbs at once (think > library) and when performance and/or memory are important. Subversion > actually is a good example here. They started to use XML to store their > internal workspace data but then were forced to switch to some simple > text format due to memory/performance issues. I am almost certain that their use cases and workloads are much different than the notebook's would be. Python's parser isn't exactly a speed demon, either. A general statement like "XML is slow" followed by an unrelated anecdote is not terribly convincing. Show me experiments. I've attached mine. Python ends up being about 3 times slower than the equivalent XML for a variety of file sizes. >> With XML, you can push it across to other languages, > > This is very ambitious and most of the time is just an illusion :( > The chances that other projects throw away their's own formats (XML > or not) and embrace this one are quite slim IMHO. I'm not talking about other projects adopting anything. I'm talking about basic capabilities of other languages, like JavaScript's builtin support for parsing XML. That enables *us* to build things in JavaScript. > BTW the fact that > everyone can parse XML doesn't mean that every one can _use_ the > data right away. Nor am I saying that. I am saying that it is enormously easier to build the JavaScript parser for the XML representation rather than the Python one. > One have to have an internal logic/library/API specific > to the data represented by some particular XML document. If you take > this into account then the value of the exchange document format > somewhat reduces. It is still not zero though and IMHO it is easy to > teach classes proposed by Brian to produce XML representation just > for the mythical interchange with something :) The need for interchange is not at all mythical. Web frontends are exactly what we are talking about in this thread. >> JavaScript being the hugely important player here. Certainly, you are > > Again, it is important to define to what degree the interoperability with > something like JavaScript is needed. If you plan to work on/modify/execute > the same nbs in Python and in JavaScript then you have to implement > compatible engine/API in Python _and_ in JavaScript. Are you sure you > want to do that? If only the representation or "computed" notebook is > needed for display purposes by JavaScript, then it is something different > and could be implemented through specialized repr methods. Or you could use the same mechanism for both instead of duplicating efforts. >> going to have a Python API that will represent that tree of text nodes >> as Python objects, but I just don't see the point of making the repr() >> of that be the lingua franca format of the notebook file. It's just a >> wasted opportunity. > > The point is that nb became a first class python object - just a module, > no need for specialized parser and you can work with it as with regular > Python module - just import and use it. The only difference is that nb is > mutable - if you modified it then you have to save it. I really don't see why having the file format be Python code makes it any more of a first class object. The objects are the first class objects. As long as loading to those objects is easy, the format just doesn't matter. Loading an object by importing is actually a very inflexible and difficult to work with method compared to a function call. - Show quoted text - -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco notebook.py notebook.py 2K Download -------------- next part -------------- A non-text attachment was scrubbed... Name: notebook.py Type: text/x-python Size: 1558 bytes Desc: not available URL: From termim at gmail.com Wed Feb 24 18:35:38 2010 From: termim at gmail.com (Mikhail Terekhov) Date: Wed, 24 Feb 2010 18:35:38 -0500 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> Message-ID: <12aaa0811002241535w4df29502vefde93842dd92e10@mail.gmail.com> On Wed, Feb 24, 2010 at 4:04 PM, Robert Kern wrote: > [Did you mean to post this to one of the lists?] > Robert, yes I meant to send this to the list, sorry - hit the wrong button. Apology for the list members if this discussion is a closed matter already, I just thought it is interesting. I'm attaching our conversation with Robert in hope it will be somehow useful. On Wed, Feb 24, 2010 at 14:31, Mikhail Terekhov wrote: > On Wed, Feb 24, 2010 at 10:49 AM, Robert Kern wrote: >> On Wed, Feb 24, 2010 at 01:52, Brian Granger wrote: >> >>> In many respects it seems almost perfect. But, one day recently I was >>> thinking about how successful Mathematica's notebook continues to be. So I >>> began to look more at the Mathematica notebook format. Amazingly, the >>> Mathematica notebook format is a plain text file that itself is *valid >>> Mathematica code*. >> >> The problem with transferring this to Python is that Mathematica's >> language is a very Lispy one. ExpressionCell[] contains the actual >> expression, not a string representing a Mathematica expression. You > > Not quite so, see for example: > http://reference.wolfram.com/mathematica/ref/format/NB.html > > It is true that nb format contains a valid Mathematica's code but it is not > the code user usually deals with but a lower level one where expressions > are constructed from strings and API calls. This is exactly as Brian suggests > AFAICS. > >> can't do that with Python. You would have to put the Python code into >> strings. You just have a "dumb" tree structure with text leaf nodes. > > That is exactly as Mathematica does that, at least as it is described in > the link above. Quite so. >> At that point, you might as well use XML to describe the exact same >> structure. > > Sure you can but you have to pay a price for that and the price is XML > parsing. While it looks negligible when your nb is just a couple of dozen > lines it will be huge when you have to load many big nbs at once (think > library) and when performance and/or memory are important. Subversion > actually is a good example here. They started to use XML to store their > internal workspace data but then were forced to switch to some simple > text format due to memory/performance issues. I am almost certain that their use cases and workloads are much different than the notebook's would be. Python's parser isn't exactly a speed demon, either. A general statement like "XML is slow" followed by an unrelated anecdote is not terribly convincing. Show me experiments. I've attached mine. Python ends up being about 3 times slower than the equivalent XML for a variety of file sizes. >> With XML, you can push it across to other languages, > > This is very ambitious and most of the time is just an illusion :( > The chances that other projects throw away their's own formats (XML > or not) and embrace this one are quite slim IMHO. I'm not talking about other projects adopting anything. I'm talking about basic capabilities of other languages, like JavaScript's builtin support for parsing XML. That enables *us* to build things in JavaScript. > BTW the fact that > everyone can parse XML doesn't mean that every one can _use_ the > data right away. Nor am I saying that. I am saying that it is enormously easier to build the JavaScript parser for the XML representation rather than the Python one. > One have to have an internal logic/library/API specific > to the data represented by some particular XML document. If you take > this into account then the value of the exchange document format > somewhat reduces. It is still not zero though and IMHO it is easy to > teach classes proposed by Brian to produce XML representation just > for the mythical interchange with something :) The need for interchange is not at all mythical. Web frontends are exactly what we are talking about in this thread. >> JavaScript being the hugely important player here. Certainly, you are > > Again, it is important to define to what degree the interoperability with > something like JavaScript is needed. If you plan to work on/modify/execute > the same nbs in Python and in JavaScript then you have to implement > compatible engine/API in Python _and_ in JavaScript. Are you sure you > want to do that? If only the representation or "computed" notebook is > needed for display purposes by JavaScript, then it is something different > and could be implemented through specialized repr methods. Or you could use the same mechanism for both instead of duplicating efforts. >> going to have a Python API that will represent that tree of text nodes >> as Python objects, but I just don't see the point of making the repr() >> of that be the lingua franca format of the notebook file. It's just a >> wasted opportunity. > > The point is that nb became a first class python object - just a module, > no need for specialized parser and you can work with it as with regular > Python module - just import and use it. The only difference is that nb is > mutable - if you modified it then you have to save it. I really don't see why having the file format be Python code makes it any more of a first class object. The objects are the first class objects. As long as loading to those objects is easy, the format just doesn't matter. Loading an object by importing is actually a very inflexible and difficult to work with method compared to a function call. - Show quoted text - -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco notebook.py notebook.py 2K Download -------------- next part -------------- A non-text attachment was scrubbed... Name: notebook.py Type: text/x-python Size: 1558 bytes Desc: not available URL: From termim at gmail.com Wed Feb 24 18:35:50 2010 From: termim at gmail.com (Mikhail Terekhov) Date: Wed, 24 Feb 2010 18:35:50 -0500 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> Message-ID: <12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com> On Wed, Feb 24, 2010 at 4:04 PM, Robert Kern wrote: > > I am almost certain that their use cases and workloads are much > different than the notebook's would be. Python's parser isn't exactly > a speed demon, either. A general statement like "XML is slow" followed > by an unrelated anecdote is not terribly convincing. Show me > experiments. I've attached mine. Python ends up being about 3 times > slower than the equivalent XML for a variety of file sizes. > Believe it or not, I can't find any example of a python project that started implementing scientific notebook as an XML document and then switched to something else :) I've used a "scientific analogy" principle. Seriously, the Subversion is a real project and they really suffered from the decision to use XML as a storage for the workspace meta data and they really switched away from XML. No anecdotes. IMHO the relation is quite simple - the things like Mathematica's notebooks tend to multiply and form libraries or collections. In this case XML parsing could became a problem. Your example is not quite correct but it is a good start :} It actually illustrates two important points. First is that writing serializer that produces XML representation is easy. The second and more important is that after parsing XML you've got nothing but internal XML representation and the only thing you can do with it is to write back to a file, you still need to implement all the functionality as a python objects and convert the XML tree into python objects tree. Only after that there will be any point in benchmarking. At the contrary, the python's version is complete and if you had the real Notebook implementation then you were just ready to use it. Also note that there is no need to write, debug and support _any_ parser/reader, python provides that for free. > > I'm not talking about other projects adopting anything. I'm talking > about basic capabilities of other languages, like JavaScript's builtin > support for parsing XML. That enables *us* to build things in > JavaScript. > >> BTW the fact that >> everyone can parse XML doesn't mean that every one can _use_ the >> data right away. > > Nor am I saying that. I am saying that it is enormously easier to > build the JavaScript parser for the XML representation rather than the > Python one. > That is the real question - why JavaScript needs to read _interenal_ representation of the nb if it is not going to implement all the needed functionality to use it? >> One have to have an internal logic/library/API specific >> to the data represented by some particular XML document. If you take >> this into account then the value of the exchange document format >> somewhat reduces. It is still not zero though and IMHO it is easy to >> teach classes proposed by Brian to produce XML representation just >> for the mythical interchange with something :) > > The need for interchange is not at all mythical. Web front ends are > exactly what we are talking about in this thread. > Sure, and It looks like in his very interesting approach the JavaScript part is a client that queries python server for information about nb it needs and there is no need for JS to read nb or even know how it is stored on disk. More general: internal representation does not have to be tightly coupled to interfaces to external systems. Simplicity and reliability of the internal representation (in this case - just a regular python compiler versus custom XML parser) outweighs the need to write relatively simple export/interface functions that give a view on the nb. As Ondrej's work shows they are needed anyway and of coarse they can use XML if it is easy for the client. >>> JavaScript being the hugely important player here. Certainly, you are >> >> Again, it is important to define to what degree the interoperability with >> something like JavaScript is needed. If you plan to work on/modify/execute >> the same nbs in Python and in JavaScript then you have to implement >> compatible engine/API in Python _and_ in JavaScript. Are you sure you >> want to do that? If only the representation or "computed" notebook is >> needed for display purposes by JavaScript, then it is something different >> and could be implemented through specialized repr methods. > > Or you could use the same mechanism for both instead of duplicating efforts. > Unfortunately one have to duplicate something in either case. nb->XML would duplicate nb->repr, but as your example shows the nb->XML is quite straightforward. In case XML->nb one have to duplicate python compiler which is unnecessary in case repr->nb. >>> going to have a Python API that will represent that tree of text nodes >>> as Python objects, but I just don't see the point of making the repr() >>> of that be the lingua franca format of the notebook file. It's just a >>> wasted opportunity. >> >> The point is that nb became a first class python object - just a module, >> no need for specialized parser and you can work with it as with regular >> Python module - just import and use it. The only difference is that nb is >> mutable - if you modified it then you have to save it. > > I really don't see why having the file format be Python code makes it > any more of a first class object. The objects are the first class You are right - not the first class, just a native python object. > objects. As long as loading to those objects is easy, the format just In a sense I agree, the only difference is that from the programming POV loading cost for repr->nb is zero (all is done by the regular python compiler) and XML->nb requires a special loader that should be maintained and updated when the application changes. > doesn't matter. Loading an object by importing is actually a very > inflexible and difficult to work with method compared to a function > call. > If one prefers functions one can always to use __import__() or imp.load_module() functions instead of import statement. Regards, -- Mikhail Terekhov From robert.kern at gmail.com Wed Feb 24 18:47:24 2010 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 24 Feb 2010 17:47:24 -0600 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com> Message-ID: <3d375d731002241547m24d710b7y88db30e694946da3@mail.gmail.com> On Wed, Feb 24, 2010 at 17:04, Mikhail Terekhov wrote: > On Wed, Feb 24, 2010 at 4:04 PM, Robert Kern wrote: >> >> I am almost certain that their use cases and workloads are much >> different than the notebook's would be. Python's parser isn't exactly >> a speed demon, either. A general statement like "XML is slow" followed >> by an unrelated anecdote is not terribly convincing. Show me >> experiments. I've attached mine. Python ends up being about 3 times >> slower than the equivalent XML for a variety of file sizes. > > Believe it or not, I can't find any example of a python project that started > implementing scientific notebook as an XML document and then switched to > something else :) I've used a "scientific analogy" principle. Seriously, > the Subversion is a real project and they really suffered from the decision to > use XML as a storage for the workspace meta data and they really switched away > from XML. No anecdotes. Yes, that is an anecdote. Anecdotes are true stories, but they are not convincing data. This comparison is by no means scientific. There are large differences between the use cases here. The simple text format that Subversion moved to is not comparable to the Python format being discussed here. Subversion faced different read/write loads than a notebook will. I'm happy to consider other examples of projects finding that their XML parsers were too slow, but you need to make a more considered argument that the circumstances are similar enough to the one we are talking about. XML is not slow. XML is not fast. It cannot be either thing because it is a file format, not an implementation. Parsers can be slow or fast. cElementTree is particularly fast and rather faster than the Python parser on equivalent data. > IMHO the relation is quite simple - the things like Mathematica's > notebooks tend to > multiply and form libraries or collections. In this case XML parsing > could became a > problem. I'm sorry, but this does not follow. > Your example is not quite correct but it is a good start :} It > actually illustrates two > important points. First is that writing serializer that produces XML > representation > is easy. The second and more important is that after parsing XML you've got > nothing but internal XML representation and the only thing you can do with it is > to write back to a file, you still need to implement all the > functionality as a python > objects and convert the XML tree into python objects tree. Only after > that there > will be any point in benchmarking. At the contrary, the python's > version is complete > and if you had the real Notebook implementation then you were just > ready to use it. > Also note that there is no need to write, debug and support _any_ > parser/reader, > python provides that for free. I intended to measure the performance of the parsing, not the difficulty of implementation of anything else. The performance of constructing the objects is rather smaller, at least for XML. It is a negligible cost on top of the XML parsing, but executing the Python code actually imposes a much larger burden on the Python implementation. See the attached updated benchmark. The Python method now takes about 4x as much time to construct the Notebook instance than the XML. >> I'm not talking about other projects adopting anything. I'm talking >> about basic capabilities of other languages, like JavaScript's builtin >> support for parsing XML. That enables *us* to build things in >> JavaScript. >> >>> BTW the fact that >>> everyone can parse XML doesn't mean that every one can _use_ the >>> data right away. >> >> Nor am I saying that. I am saying that it is enormously easier to >> build the JavaScript parser for the XML representation rather than the >> Python one. > > That is the real question - why JavaScript needs to read _interenal_ > representation of the nb if it is not going to implement all the > needed functionality > to use it? File formats are not internal. They are external. They are the primary method of interchange. >>> One have to have an internal logic/library/API specific >>> to the data represented by some particular XML document. If you take >>> this into account then the value of the exchange document format >>> somewhat reduces. It is still not zero though and IMHO it is easy to >>> teach classes proposed by Brian to produce XML representation just >>> for the mythical interchange with something :) >> >> The need for interchange is not at all mythical. Web front ends are >> exactly what we are talking about in this thread. > > Sure, and It looks like in his very interesting approach the > JavaScript part is a > client that queries python server for information about nb it needs and there is > no need for JS to read nb or even know how it is stored on disk. > > More general: internal representation does not have to be tightly coupled to > interfaces to external systems. Exactly. The Python code file format is a greater coupling to the internal implementation, not a looser one. > Simplicity and reliability of the > internal representation > (in this case - just a regular python compiler versus custom XML > parser) outweighs > the need to write relatively simple export/interface functions that > give a view on the nb. I disagree with that judgement and with the characterization of the Python format as more simple and reliable. > As Ondrej's work shows they are needed anyway and of coarse they can use XML if > it is easy for the client. > >>>> JavaScript being the hugely important player here. Certainly, you are >>> >>> Again, it is important to define to what degree the interoperability with >>> something like JavaScript is needed. If you plan to work on/modify/execute >>> the same nbs in Python and in JavaScript then you have to implement >>> compatible engine/API in Python _and_ in JavaScript. Are you sure you >>> want to do that? If only the representation or "computed" notebook is >>> needed for display purposes by JavaScript, then it is something different >>> and could be implemented through specialized repr methods. >> >> Or you could use the same mechanism for both instead of duplicating efforts. > > Unfortunately one have to duplicate something in either case. nb->XML would > duplicate nb->repr, but as your example shows the nb->XML is quite > straightforward. > In case XML->nb one have to duplicate python compiler which is unnecessary > in case repr->nb. Honestly, it's pretty trivial stuff. >>>> going to have a Python API that will represent that tree of text nodes >>>> as Python objects, but I just don't see the point of making the repr() >>>> of that be the lingua franca format of the notebook file. It's just a >>>> wasted opportunity. >>> >>> The point is that nb became a first class python object - just a module, >>> no need for specialized parser and you can work with it as with regular >>> Python module - just import and use it. The only difference is that nb is >>> mutable - if you modified it then you have to save it. >> >> I really don't see why having the file format be Python code makes it >> any more of a first class object. The objects are the first class > > You are right - not the first class, just a native python object. A file with Python code is not a native Python object. From the API user's perspective, they call a function or execute a statement and then they have an object. It's exactly the same for any format. There is no benefit for either case. >> objects. As long as loading to those objects is easy, the format just > > In a sense I agree, the only difference is that from the programming POV > loading cost for repr->nb is zero (all is done by the regular python compiler) > and XML->nb requires a special loader that should be maintained and updated > when the application changes. Actually, that raises another objection I have to using Python code as the file format. The file format is intimately tied to the internal implementation. What if we want to change the internal implementation? It *will* happen. With a more neutral representation, you can read older files as long as you update the reader appropriately. If you are just executing code, you are stuck with maintaining the classes with backwards compatible argument specs forever and won't be able to make some of the changes that you want down the road. Format versioning is a *huge* issue whenever you design file formats. XML and other generic representations permit this. The reason that Mathematica can get away with this is that it is a Lispish language. Although you correctly point out that being able to do things like ExpressionCell aren't particularly important, being able to load the code into a neutral tree structure and manipulate it before actually instantiating your API classes it is. >> doesn't matter. Loading an object by importing is actually a very >> inflexible and difficult to work with method compared to a function >> call. > > If one prefers functions one can always to use __import__() or imp.load_module() > functions instead of import statement. Abusing a Python-internal API as your main file loading API is just not a good practice. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -------------- next part -------------- A non-text attachment was scrubbed... Name: notebook.py Type: application/octet-stream Size: 1620 bytes Desc: not available URL: From dsdale24 at gmail.com Thu Feb 25 12:51:46 2010 From: dsdale24 at gmail.com (Darren Dale) Date: Thu, 25 Feb 2010 12:51:46 -0500 Subject: [IPython-dev] question about generics.complete_object Message-ID: How is generics.complete_object supposed to work in the trunk? I have a custom completer that uses it, and it used to work, but now complete_object keeps returning TryNext: In [12]: generics.complete_object('foo', dir('foo')) --------------------------------------------------------------------------- TryNext Traceback (most recent call last) /Users/darren/Projects/phynx.git/ in () /Users/darren/Projects/ipython/IPython/external/simplegeneric.pyc in dispatch(*args, **kw) 73 f = _gbt(t, _sentinel) 74 if f is not _sentinel: ---> 75 return f(*args, **kw) 76 else: 77 return func(*args, **kw) /Users/darren/Projects/ipython/IPython/utils/generics.pyc in complete_object(obj, prev_completions) 60 own_attrs + prev_completions. 61 """ ---> 62 raise TryNext 63 64 TryNext: Is this a bug, or am I expecting functionality that no longer exists? Thanks, Darren From ellisonbg at gmail.com Thu Feb 25 12:56:26 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 25 Feb 2010 09:56:26 -0800 Subject: [IPython-dev] question about generics.complete_object In-Reply-To: References: Message-ID: Darren, > How is generics.complete_object supposed to work in the trunk? I have > a custom completer that uses it, and it used to work, but now > complete_object keeps returning TryNext: It should still work. This looks like a bug. If you don't have time to look into this yourself, could you submit a bug report. Thanks! Brian > In [12]: generics.complete_object('foo', dir('foo')) > --------------------------------------------------------------------------- > TryNext ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Traceback (most recent call last) > > /Users/darren/Projects/phynx.git/ in () > > /Users/darren/Projects/ipython/IPython/external/simplegeneric.pyc in > dispatch(*args, **kw) > ? ? 73 ? ? ? ? ? ? ? ? f = _gbt(t, _sentinel) > ? ? 74 ? ? ? ? ? ? ? ? if f is not _sentinel: > ---> 75 ? ? ? ? ? ? ? ? ? ? return f(*args, **kw) > ? ? 76 ? ? ? ? ? ? else: > ? ? 77 ? ? ? ? ? ? ? ? return func(*args, **kw) > > /Users/darren/Projects/ipython/IPython/utils/generics.pyc in > complete_object(obj, prev_completions) > ? ? 60 ? ? own_attrs + prev_completions. > ? ? 61 ? ? """ > ---> 62 ? ? raise TryNext > ? ? 63 > ? ? 64 > > TryNext: > > > Is this a bug, or am I expecting functionality that no longer exists? > > Thanks, > Darren > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From ellisonbg at gmail.com Thu Feb 25 13:32:54 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 25 Feb 2010 10:32:54 -0800 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <3d375d731002241547m24d710b7y88db30e694946da3@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com> <3d375d731002241547m24d710b7y88db30e694946da3@mail.gmail.com> Message-ID: I would expect that parsing an XML notebook would be faster than importing/exec'ing a pure python based notebook (as Robert's tests shows). It is not clear to me that performance is the central concern, but it could be in some situations. We all know that long import times are an issue for some projects. Robert, do you have any thoughts about the validation/extensibility issues related to XML that I brought up? I think at this point, those are my main concerns with XML versus pure python. Cheers, Brian On Wed, Feb 24, 2010 at 3:47 PM, Robert Kern wrote: > On Wed, Feb 24, 2010 at 17:04, Mikhail Terekhov wrote: >> On Wed, Feb 24, 2010 at 4:04 PM, Robert Kern wrote: >>> >>> I am almost certain that their use cases and workloads are much >>> different than the notebook's would be. Python's parser isn't exactly >>> a speed demon, either. A general statement like "XML is slow" followed >>> by an unrelated anecdote is not terribly convincing. Show me >>> experiments. I've attached mine. Python ends up being about 3 times >>> slower than the equivalent XML for a variety of file sizes. >> >> Believe it or not, I can't find any example of a python project that started >> implementing scientific notebook as an XML document and then switched to >> something else :) I've used a "scientific analogy" principle. Seriously, >> the Subversion is a real project and they really suffered from the decision to >> use XML as a storage for the workspace meta data and they really switched away >> from XML. No anecdotes. > > Yes, that is an anecdote. Anecdotes are true stories, but they are not > convincing data. This comparison is by no means scientific. There are > large differences between the use cases here. The simple text format > that Subversion moved to is not comparable to the Python format being > discussed here. Subversion faced different read/write loads than a > notebook will. I'm happy to consider other examples of projects > finding that their XML parsers were too slow, but you need to make a > more considered argument that the circumstances are similar enough to > the one we are talking about. > > XML is not slow. XML is not fast. It cannot be either thing because it > is a file format, not an implementation. Parsers can be slow or fast. > cElementTree is particularly fast and rather faster than the Python > parser on equivalent data. > >> IMHO the relation is quite simple - the things like Mathematica's >> notebooks tend to >> multiply and form libraries or collections. In this case XML parsing >> could became a >> problem. > > I'm sorry, but this does not follow. > >> Your example is not quite correct but it is a good start :} It >> actually illustrates two >> important points. First is that writing serializer that produces XML >> representation >> is easy. The second and more important is that after parsing XML you've got >> nothing but internal XML representation and the only thing you can do with it is >> to write back to a file, you still need to implement all the >> functionality as a python >> objects and convert the XML tree into python objects tree. Only after >> that there >> will be any point in benchmarking. At the contrary, the python's >> version is complete >> and if you had the real Notebook implementation then you were just >> ready to use it. >> Also note that there is no need to write, debug and support _any_ >> parser/reader, >> python provides that for free. > > I intended to measure the performance of the parsing, not the > difficulty of implementation of anything else. The performance of > constructing the objects is rather smaller, at least for XML. It is a > negligible cost on top of the XML parsing, but executing the Python > code actually imposes a much larger burden on the Python > implementation. See the attached updated benchmark. The Python method > now takes about 4x as much time to construct the Notebook instance > than the XML. > >>> I'm not talking about other projects adopting anything. I'm talking >>> about basic capabilities of other languages, like JavaScript's builtin >>> support for parsing XML. That enables *us* to build things in >>> JavaScript. >>> >>>> BTW the fact that >>>> everyone can parse XML doesn't mean that every one can _use_ the >>>> data right away. >>> >>> Nor am I saying that. I am saying that it is enormously easier to >>> build the JavaScript parser for the XML representation rather than the >>> Python one. >> >> That is the real question - why JavaScript needs to read _interenal_ >> representation of the nb if it is not going to implement all the >> needed functionality >> to use it? > > File formats are not internal. They are external. They are the primary > method of interchange. > >>>> One have to have an internal logic/library/API specific >>>> to the data represented by some particular XML document. If you take >>>> this into account then the value of the exchange document format >>>> somewhat reduces. It is still not zero though and IMHO it is easy to >>>> teach classes proposed by Brian to produce XML representation just >>>> for the mythical interchange with something :) >>> >>> The need for interchange is not at all mythical. Web front ends are >>> exactly what we are talking about in this thread. >> >> Sure, and It looks like in his very interesting approach the >> JavaScript part is a >> client that queries python server for information about nb it needs and there is >> no need for JS to read nb or even know how it is stored on disk. >> >> More general: internal representation does not have to be tightly coupled to >> interfaces to external systems. > > Exactly. The Python code file format is a greater coupling to the > internal implementation, not a looser one. > >> Simplicity and reliability of the >> internal representation >> (in this case - just a regular python compiler versus custom XML >> parser) outweighs >> the need to write relatively simple export/interface functions that >> give a view on the nb. > > I disagree with that judgement and with the characterization of the > Python format as more simple and reliable. > >> As Ondrej's work shows they are needed anyway and of coarse they can use XML if >> it is easy for the client. >> >>>>> JavaScript being the hugely important player here. Certainly, you are >>>> >>>> Again, it is important to define to what degree the interoperability with >>>> something like JavaScript is needed. If you plan to work on/modify/execute >>>> the same nbs in Python and in JavaScript then you have to implement >>>> compatible engine/API in Python _and_ in JavaScript. Are you sure you >>>> want to do that? If only the representation or "computed" notebook is >>>> needed for display purposes by JavaScript, then it is something different >>>> and could be implemented through specialized repr methods. >>> >>> Or you could use the same mechanism for both instead of duplicating efforts. >> >> Unfortunately one have to duplicate something in either case. nb->XML would >> duplicate nb->repr, but as your example shows the nb->XML is quite >> straightforward. >> In case XML->nb one have to duplicate python compiler which is unnecessary >> in case repr->nb. > > Honestly, it's pretty trivial stuff. > >>>>> going to have a Python API that will represent that tree of text nodes >>>>> as Python objects, but I just don't see the point of making the repr() >>>>> of that be the lingua franca format of the notebook file. It's just a >>>>> wasted opportunity. >>>> >>>> The point is that nb became a first class python object - just a module, >>>> no need for specialized parser and you can work with it as with regular >>>> Python module - just import and use it. The only difference is that nb is >>>> mutable - if you modified it then you have to save it. >>> >>> I really don't see why having the file format be Python code makes it >>> any more of a first class object. The objects are the first class >> >> You are right - not the first class, just a native python object. > > A file with Python code is not a native Python object. From the API > user's perspective, they call a function or execute a statement and > then they have an object. It's exactly the same for any format. There > is no benefit for either case. > >>> objects. As long as loading to those objects is easy, the format just >> >> In a sense I agree, the only difference is that from the programming POV >> loading cost for repr->nb is zero (all is done by the regular python compiler) >> and XML->nb requires a special loader that should be maintained and updated >> when the application changes. > > Actually, that raises another objection I have to using Python code as > the file format. The file format is intimately tied to the internal > implementation. What if we want to change the internal implementation? > It *will* happen. With a more neutral representation, you can read > older files as long as you update the reader appropriately. If you are > just executing code, you are stuck with maintaining the classes with > backwards compatible argument specs forever and won't be able to make > some of the changes that you want down the road. Format versioning is > a *huge* issue whenever you design file formats. XML and other generic > representations permit this. > > The reason that Mathematica can get away with this is that it is a > Lispish language. Although you correctly point out that being able to > do things like ExpressionCell aren't particularly important, being > able to load the code into a neutral tree structure and manipulate it > before actually instantiating your API classes it is. > >>> doesn't matter. Loading an object by importing is actually a very >>> inflexible and difficult to work with method compared to a function >>> call. >> >> If one prefers functions one can always to use __import__() or imp.load_module() >> functions instead of import statement. > > Abusing a Python-internal API as your main file loading API is just > not a good practice. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma that is made terrible by our own mad attempt to interpret it as > though it had an underlying truth." > ?-- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From robert.kern at gmail.com Thu Feb 25 13:37:24 2010 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 25 Feb 2010 12:37:24 -0600 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com> <3d375d731002241547m24d710b7y88db30e694946da3@mail.gmail.com> Message-ID: <3d375d731002251037x68dd56c3u707bb00534d312c6@mail.gmail.com> On Thu, Feb 25, 2010 at 12:32, Brian Granger wrote: > Robert, do you have any thoughts about the validation/extensibility > issues related to XML that I brought up? Yes: http://mail.scipy.org/pipermail/ipython-dev/2010-February/005830.html -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg at gmail.com Thu Feb 25 13:54:41 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 25 Feb 2010 10:54:41 -0800 Subject: [IPython-dev] [sympy] Re: using reST for representing the notebook cells+text In-Reply-To: <3d375d731002251037x68dd56c3u707bb00534d312c6@mail.gmail.com> References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> <12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com> <3d375d731002241547m24d710b7y88db30e694946da3@mail.gmail.com> <3d375d731002251037x68dd56c3u707bb00534d312c6@mail.gmail.com> Message-ID: Opps, I missed that. Thanks. Brian On Thu, Feb 25, 2010 at 10:37 AM, Robert Kern wrote: > On Thu, Feb 25, 2010 at 12:32, Brian Granger wrote: > >> Robert, do you have any thoughts about the validation/extensibility >> issues related to XML that I brought up? > > Yes: > > http://mail.scipy.org/pipermail/ipython-dev/2010-February/005830.html > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma that is made terrible by our own mad attempt to interpret it as > though it had an underlying truth." > ?-- Umberto Eco > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From dsdale24 at gmail.com Thu Feb 25 14:43:21 2010 From: dsdale24 at gmail.com (Darren Dale) Date: Thu, 25 Feb 2010 14:43:21 -0500 Subject: [IPython-dev] question about generics.complete_object In-Reply-To: References: Message-ID: On Thu, Feb 25, 2010 at 12:56 PM, Brian Granger wrote: > Darren, > >> How is generics.complete_object supposed to work in the trunk? I have >> a custom completer that uses it, and it used to work, but now >> complete_object keeps returning TryNext: > > It should still work. ?This looks like a bug. ?If you don't have time > to look into this > yourself, could you submit a bug report. > > Thanks! I tried to locate the problem, but was unsuccessful. The bug report is at https://bugs.edge.launchpad.net/ipython/+bug/527968 Darren From ellisonbg at gmail.com Thu Feb 25 15:19:44 2010 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 25 Feb 2010 12:19:44 -0800 Subject: [IPython-dev] question about generics.complete_object In-Reply-To: References: Message-ID: Darren, Thanks, we will make sure that is fixed before 0.11 is released. Cheers, Brian On Thu, Feb 25, 2010 at 11:43 AM, Darren Dale wrote: > On Thu, Feb 25, 2010 at 12:56 PM, Brian Granger wrote: >> Darren, >> >>> How is generics.complete_object supposed to work in the trunk? I have >>> a custom completer that uses it, and it used to work, but now >>> complete_object keeps returning TryNext: >> >> It should still work. ?This looks like a bug. ?If you don't have time >> to look into this >> yourself, could you submit a bug report. >> >> Thanks! > > I tried to locate the problem, but was unsuccessful. The bug report is > at https://bugs.edge.launchpad.net/ipython/+bug/527968 > > Darren > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From ellisonbg.net at gmail.com Fri Feb 26 12:47:00 2010 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 26 Feb 2010 09:47:00 -0800 Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and Matplotlib/ETS In-Reply-To: <4B7DBE28.90702@noaa.gov> References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> <201002121206.51045.hans_meine@gmx.net> <20100212111104.GA7228@phare.normalesup.org> <201002121225.05301.hans_meine@gmx.net> <20100212113903.GB7228@phare.normalesup.org> <4B7DBE28.90702@noaa.gov> Message-ID: <6ce0ac131002260947y52aa366dgf9c33dd15554dee1@mail.gmail.com> All, I had a thought. It seems that it's really impossible (or at least > impractical) to monkey-patch the GUI toolkits such that you can run a > regular old unmodified app exactly the same way inside and outside of > ipython. Gael's right -- that way lies loss of hair (and I"ve lost > enough in the last ten years as it is). > > However, as it happens, I've been using ipython for a wx GUI app of mine > (with no ETS or MPL), and it sort-of-works, and also locks up and > crashes fairly regularly. So the threading solution isn't really robust > either. Also, when wx starts using Cocoa on OS-X, putting the GUI in > another thread won't be an option anymore anyway. > > The old threaded code is a train wreck - it is not thread safe in the least bit and we have to do crazy hack to get Ctrl-C to work across threads. The result is, well, an eventual train wreck. So: > > > - non-threaded gui support with pyos_inputhook, that would require > > some code changes. If this can give a really robust solution and the > > needed changes aren't too intrusive, I could see matplotlib, ETS and > > possibly others adding the necessary adaptation (especially if we > > provide it in ipython already so for the users it ends up being almost > > a one-liner). > > Exactly -- while you can't expect unmodified regular old wx code to > work, we could provide alternative app startup code that a user can > simply cut and past in, and then have an app that will work with and > without ipython -- I think that's a fine solution. > > Yes, this should be fairly easy to do. Cheers, Brian > So for now, is there a sample, minimal wx app that works with the new > pyos_inputhook code? > > -Chris > > > > > > -- > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: