From christopher.drexler at siemens.com  Mon May 12 03:28:55 2003
From: christopher.drexler at siemens.com (Drexler Christopher)
Date: Mon, 12 May 2003 09:28:55 +0200
Subject: [IPython-dev] RE: [Fwd: [IPython-user] re: Fwd: readline for Win32]
Message-ID: <CB0365D517B7D611B5E100508B9498B6022A9B50@erlh904a.med.siemens.de>

Dear List,

I'm working with IPython since a few weeks on a win32 system
and as I was missing the readline support I tried to get
a working readline.pyd. Fernando Perez made it already 
available and as I'm now subscribed I can (hopefully) answer
the problemes related to the package. Also I'm going to write
a more detailed README this week. 


> -------- Original Message --------
> Subject: [IPython-user] re: Fwd: readline for Win32
> Date: Wed, 07 May 2003 16:21:49 EDT
> From: Gary Bishop <gb at cs.unc.edu>
> To: ipython-user at scipy.net
>
> I tried first with the native Cygwin stuff I use all the time. Then I
> downloaded MinGW 2.0.0-3 which appears to have the correct compiler as
> listed in the readme for PyReadline*.

I can't say anything about other MingW-versions as I've only tested the
mentioned one. 

> With the MinGW world I get the following output from the second make
> (the first for libreadline.a and libhistory.a goes fine).

At least :-)

> [...]
> readline.o(.text+0x8b):readline.c: undefined reference to
> `_imp___Py_NoneStruct'
> [...]

It seems that gcc can't find the right python lib. I'm using the
ActiveState python distribution (2.2) and you have to link against
the 'libpython22.a' in the 'libs' directory of the installation 
directory.

I installed python in 'c:/Python22'. Therefore the path in the 
'makefile' is set via '-Lc:/Python22/libs'. Having the flag
'-lpython22' then causes the library 'c:/Python22/libs/libpython22.a'
to be linked. I don't know (and don't hope) that this library is special
to the ActiveState distribution of python. 

So the way to go is: find this library and change the 'LIBS' line in the
makefile e.g. to:

  LIBS=c:/Python22/libs/libpython22.a -lreadline

replacing the path with the one to your library.
This should solve the problem.

Best Regards,
Chris

--
Christopher Drexler, Siemens AG MED AX E 1
Siemensstr. 1 D-91301 Forchheim Germany
Phone: +49 9191 18 9776 Fax: +49 9191 18 9723



From gb at cs.unc.edu  Mon May 12 08:34:48 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Mon, 12 May 2003 08:34:48 EDT
Subject: [IPython-dev] RE: [Fwd: [IPython-user] re: Fwd: readline for Win32]
Message-ID: <200305121234.h4CCYmXo027167@wren.cs.unc.edu>

Thanks Chris,

With that hint and some googling for building extensions with MinGW I 
found success. The problem as you correctly identified was I did not 
have a libpython22.a. The standard python distribution does not ship with one.

So, I downloaded lib2def.py and replaced python15 with python22 
everywhere (I can make a version of this script that does this 
automagically if one doesn't already exist). Then I:

python lib2def.py python22.lib python22.def
dlltool --dllname python22.dll --def python22.def --output-lib libpython22.a

Then over in the PyReadline/src folder I can now:

PATH=/c/MinGW/bin:$PATH make

I had to adjust the path because I have the cygwin stuff installed.

Thanks for your efforts on readline for windows.

gb



From fperez at colorado.edu  Tue May 13 15:13:27 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Tue, 13 May 2003 13:13:27 -0600
Subject: [IPython-dev] Re: IPython Crash Report
In-Reply-To: <200305131849.h4DInjXo018909@wren.cs.unc.edu>
References: <200305131849.h4DInjXo018909@wren.cs.unc.edu>
Message-ID: <3EC143D7.8050907@colorado.edu>

Hi Gary,

> The idea is simple. I assume that during the exec in 
> InteractiveShell.runcode, calls to sys.excepthook are just another normal
> exception rather than a crash in IPython. I do this by swapping in a local
> excepthook just before the exec and swapping it back out just after. I
> added 3 methods to InteractiveShell: excepthook (the local handler),
> setexcepthook (to swap it in) and clearexcepthook (to swap it out).

Great, thanks for your effort.  Right now I'm busy with 'real' work, so I
won't have time to look at this in detail.  Maybe tonight or tomorrow.  But
please keep using it with your own stuff and see if it breaks anything.  I'll
study it in detail, and if all looks ok, I'll commit the patches.  I have to
think hard about possible corner cases.

> 
> I also modified AutoFormattedTB to 1) respect etype,value, and tb args if
> they are given, and 2) to accept an offset argument to temporarily override
> the tb_offset. There is probably a better way to accomplish this...

This actually sounds ok.  The beauty of python's optional arguments is that
one can tweak functions for corner cases without breaking their 'main' 
functionality.  And it's necessary to have fine control over stack frame 
offsets, b/c of the need to properly 'hit' the right user frame through 
ipython's own code.

Let's keep in touch on this.

Cheers,

f.

ps.  I'm cc-ing the list on my replies to keep this stuff archived/searchable. 
  I'm going to switch to cc-ing the dev list, as this became far more 
technical than most user issues.



From gb at cs.unc.edu  Sat May 17 07:49:39 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Sat, 17 May 2003 07:49:39 EDT
Subject: [IPython-dev] re: 0.4.0 ready for Monday
Message-ID: <200305171149.h4HBneXo024735@wren.cs.unc.edu>

It still says it is 0.2.15.pre5, I guess that is just a version string 
somewhere.

Also, installing from CVS appears to have blown away my html manual. 
Now when I try to open it Windows says it is searching for it.

I worry a tiny bit about the trick of saving the old sys.eventhook and 
restoring it (same as I did). If for some reason we don't get to 
restore it, then it will never get restored (because the wrong one will 
then be the old one from then on). There appears to be only 1 that 
IPython uses. We could always restore it. Thus if things went wrong, 
they'd be fixed after the next command.

May not be worth worrying about. I don't think we come back by any road 
except that restore code...

gb


From fperez at colorado.edu  Tue May 20 18:12:53 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Tue, 20 May 2003 16:12:53 -0600
Subject: [IPython-dev] [ANN] IPython 0.4.0 has been released!
Message-ID: <3ECAA865.9090109@colorado.edu>

Hi all,

I've just released IPython 0.4.0.  The NEWS file and full changelog are linked 
to on the webpage, so I won't bother you with all of that here.

As always, the download link is at: http://ipython.scipy.org/dist

I changed the version number to start a 0.4 series, as I feel that the 
stability and maturity of the code warrants a number higher than 0.2.x. 
Numbers that low tend to scare users into thinking that it's pre-pre-alpha 
code, which it certainly isn't :)

I'd like to thank all the users who contributed patches and ideas for this 
release, and especially Jack Moffitt, who has done a TON of work to help with 
multiple areas of the project.  Gary Bishop has also been doing very good work 
with Windows-related things, which as you all know are my weakest spot.  Stay 
tuned for more on that front, as Gary irons out his new developments.

If you feel that your name should be on the credits section and it isn't, 
please drop me a line so I can correct the omission.  It was certainly not 
intentional.

I think IPython is really stabilizing quite nicely, since with a growing user 
base the bug/crash reports don't seem to be multiplying :)  So perhaps soon 
(after having 0.4.x out for a while), I can begin making more invasive 
architectural changes.  An internal cleanup of ipython will be necessary for 
it to continue growing, since the current codebase is simply too messy for 
most people to deal with.

A brief highlight of the most important changes since the last official 
release, which was 0.2.14 (as usual, the changelog is very detailed):


Bug Fixes
---------

- Fixed many crashes caused by the auto-eval code.  The new code is much 
safer, and free of potential side-effects.  Autocalling became controllable at 
runtime via @autocall.  This should be considered a critical fix.

- Fix namespace handling in @run which could cause problems with pickle.

- Fixed color handling across many terminal types.  Except for MS-Windows 
native cmd.exe windows, all terminals seem to handle color fine now.

- Fixed the installer removing many ugly distutils-related hacks/bugs.  It 
doesn't generate spurious errors and warnings anymore.

- Various fixes to the Gnuplot support (an important one in hardcopy).

- Fixed crash when pdb.pm() was called.

- Fixed potentially destructive change of files in ~ when ipython ran first.

- Fixed crash for user-defined prompt strings with unmatched '%' in them.

- Better handling of sys.excepthook to protect against spurious errors when 
exceptions are generated by WxWindows.

- Internal workaround in case 'kinds' is not found (recently removed from 
Numeric's standard distribution).

- Added workaround for SWIG crashes when inspecting objects.


New features/enhancements
-------------------------

- Optional (off by default) auto-indenting.  It can be toggled at runtime.

- Better support for mouse/non mouse versions of Gnuplot.

- Better timing routines (under Unix) in genutils, which use getresource(). 
These avoid the wraparound problems inherent in time.clock().

- IPython now supports -c option, just like regular python.

- IPython now has manpages (for ipython and pycolor).

- RPMS are provided by me, and Debian/Fink packages are being contributed by 
Jack Moffitt and Andrea Riciputi (respectively).

- Improved (X)Emacs integration.  It now seems to work almost perfectly (using 
Alex Schmolck's support files, available at ipython's site).




So please grab 0.4.0 and enjoy!

Thanks to all,

Fernando.



From gb at cs.unc.edu  Thu May 22 15:24:14 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Thu, 22 May 2003 15:24:14 EDT
Subject: [IPython-dev] Small change to MagicCompleter fixes filename completion on Windows
Message-ID: <200305221924.h4MJOEXo018537@wren.cs.unc.edu>

Now that I've got my Python readline starting to work, I've found a 
problem that prevents filename completion on windows. Unfixed, it does this

In [1]: cd ~/pyt<TAB>
@ddir   @dhist  @dir    @dirs

In[1]: cd @d

The problem is in MagicCompleter. It only tests to see if the string 
begins with os.sep but a Windows file name may well begin with d: 
(after ~ expansion). I put in a small hack to fix this. I think a 
better patch would test the string to see if it looks like a legal 
python variable before subjecting it to these other completions. But my 
simple change fixes things well enough...

$ diff -Naur iplib.py~ iplib.py
--- iplib.py~   2003-05-13 16:24:11.000000000 -0400
+++ iplib.py    2003-05-22 15:15:21.000000000 -0400
@@ -123,7 +123,9 @@
             This is called successively with state == 0, 1, 2, ... until it
             returns None.  The completion should begin with 'text'.  """

+            doing_magic = False
             if text.startswith('@'):
+                doing_magic = True
                 text = text.replace('@','__IP.magic_')
             if text.startswith('~'):
                 text = os.path.expanduser(text)
@@ -138,7 +140,7 @@
                 else:
                     self.matches = self.global_matches(text)
                     # this is so completion finds magics when automagic is on:
-                    if self.matches == [] and not text.startswith(os.sep):
+                    if self.matches == [] and doing_magic:
                         self.matches = self.attr_matches('__IP.magic_'+text)
             try:
                 return self.matches[state].replace('__IP.magic_','@')


From gb at cs.unc.edu  Thu May 22 15:27:00 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Thu, 22 May 2003 15:27:00 EDT
Subject: [IPython-dev] Small change to MagicCompleter fixes filename completion on Windows
Message-ID: <200305221927.h4MJR0Xo018707@wren.cs.unc.edu>

Ignore that previous patch. That code should get activated even when 
the string didn't begin with @. I'll try something else.

Sorry for the noise
gb


From gb at cs.unc.edu  Thu May 22 15:55:40 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Thu, 22 May 2003 15:55:40 EDT
Subject: [IPython-dev] A patch to fix filename completion on Windows
Message-ID: <200305221955.h4MJteXo020281@wren.cs.unc.edu>

OK, sorry for the first one. Here is another try. I think this one

* fixes the problem
* doesn't break other cases
* and protects from other such problems
* fixes another bug.

$ diff -Naur FlexCompleter.py~ FlexCompleter.py
--- FlexCompleter.py~   2002-07-04 14:33:26.000000000 -0400
+++ FlexCompleter.py    2003-05-22 15:46:19.000000000 -0400
@@ -167,10 +167,10 @@
        #m = re.match(r"([\w\[\]]+(\.[\w\[\]]+)*)\.(\w*)", text)

         # Another option, seems to work great. Catches things like ''.<tab>
-        m = re.match(r"(\S+(\.\w+)*)\.(\w*)", text)
+        m = re.match(r"(\S+(\.\w+)*)\.(\w*)$", text)

         if not m:
-            return
+            return []
         expr, attr = m.group(1, 3)
         object = eval(expr, self.namespace)
         words = dir(object)

The change is adding a $ to the end of the regular expression for 
matching completions. It seems wrong to allow other stuff hanging off 
the end of it to be ignored. The $ requires the re to match the entire 
string. That won't happen with filenames...

The second bit is fixing a bug that is provoked by this. If the re 
doesn't match it formerly returned None but should have returned an 
empty list because that is what other code is testing for.

The net is I added 3 characters! I hope they are the right 3. See what 
you think.

gb


From fperez at colorado.edu  Thu May 22 17:16:47 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Thu, 22 May 2003 15:16:47 -0600
Subject: [IPython-dev] A patch to fix filename completion on Windows
In-Reply-To: <200305221955.h4MJteXo020281@wren.cs.unc.edu>
References: <200305221955.h4MJteXo020281@wren.cs.unc.edu>
Message-ID: <3ECD3E3F.8010205@colorado.edu>

Gary Bishop wrote:
> OK, sorry for the first one. Here is another try. I think this one

Just put in CVS, it looks ok  (and the return [] bugfix was definitely 
needed).  Thanks.

ps.  I just committed another fix which is essentially the sys.excepthook work 
you'd done, but at startup time.  It turns out that if files given on the 
command line tap into sys.excepthook directly, we get similar behavior as what 
you were seeing with WxWindows.

Kasper reported exactly this today with PyQt.  If it weren't because of having 
worked on this issue with you a few days ago, I would have NEVER figured out 
what was going on (in the command-line case, the crash report looks even weirder).

So thanks to your previous work, this buglet (it doesn't really crash ipython, 
but it's annoying) was solved in minutes.  Thanks!

I've opened 0.4.1-cvs and will keep it going like that until either:

- we accumulate enough stuff to warrant putting 0.1 out

OR

- a CRITICAL fix comes in

So keep working on the Windows stuff and sending patches my way :)

Cheers,

f.



From prabhu at aero.iitm.ernet.in  Fri May 23 14:40:11 2003
From: prabhu at aero.iitm.ernet.in (Prabhu Ramachandran)
Date: Sat, 24 May 2003 00:10:11 +0530
Subject: [IPython-dev] Making docs and installing from CVS.
Message-ID: <16078.27403.669231.313029@monster.linux.in>

Hi,

I just got IPython off CVS and when installing it the docs are not
installed properly.  So the question is how does one build the docs
and get a complete install?  If its a little too complex its OK, I'll
just download the release.

cheers,
prabhu




From gb at cs.unc.edu  Fri May 23 16:45:08 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Fri, 23 May 2003 16:45:08 EDT
Subject: [IPython-dev] IPython bug?
Message-ID: <200305232045.h4NKj9Xo019149@wren.cs.unc.edu>

With readline mark-directories set to "on" (the default value), on 
windows when I try:

In [3]: cd d:\msl\gb\
   ...:
   ...:
   ...:
   ...:
   ...:

The trailing \ on a magic command seems to make it request 
continuations forever. I agree that \ as the separator on windows is 
horrible. I don't know why the Python guys use it. Virtually everything 
works with /. I might try changing it somewhere. In the meantime, it 
breaks IPython.

gb


From gb at cs.unc.edu  Fri May 23 16:58:17 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Fri, 23 May 2003 16:58:17 EDT
Subject: [IPython-dev] IPython with Color on windows
Message-ID: <200305232058.h4NKwHXo019746@wren.cs.unc.edu>

I *can* make color work on Windows with my Python port of GNU Readline 
but it will require changes to  IPython. Specifically, I need to get 
access to everything that IPython  prints before it goes out. I tried 
replacing sys.stdout with a "file  like" object but IPython doesn't 
want to play with that for reasons I  don't understand. Probably a bad 
idea for a shell to fool with that anyway.

So, if internally IPython had a function console_print or whatever,  
normally it would just print but the windows version could eat the  
escape codes (I've got that part written) and tell the Windows console  
to do the right thing.

I'm not real excited about color myself but am willing to do it while  
I'm bashing on things if folks are interested.

gb


From gb at cs.unc.edu  Fri May 23 17:02:21 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Fri, 23 May 2003 17:02:21 EDT
Subject: [IPython-dev] What readline features do people use?
Message-ID: <200305232102.h4NL2LXo019939@wren.cs.unc.edu>

I've got a Python implementation of GNU readline that seems to make 
IPython happy. It does filename and set_completer style completion, has 
most of Emacs mode, and generally feels to me like I think IPython should.

But GNU readline is an incredibly feature-full beast. I believe all of 
it *could* be implemented in my framework. But it will likely be more 
work than I'm willing/able to do.

For example "vi" mode. It can be done, but I don't need it. 
Multi-character bindings. Conditionals in init files, lots of 
options... There is a ton of stuff I just don't use.

Perhaps I should release something and see what people holler about?

gb


From fperez at colorado.edu  Fri May 23 17:58:38 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Fri, 23 May 2003 15:58:38 -0600
Subject: [IPython-dev] Making docs and installing from CVS.
In-Reply-To: <16078.27403.669231.313029@monster.linux.in>
References: <16078.27403.669231.313029@monster.linux.in>
Message-ID: <3ECE998E.4030107@colorado.edu>

Prabhu Ramachandran wrote:
> Hi,
> 
> I just got IPython off CVS and when installing it the docs are not
> installed properly.  So the question is how does one build the docs
> and get a complete install?  If its a little too complex its OK, I'll
> just download the release.

It's not too complex, but you'd need a script called lyxport I wrote a while 
back.  That's b/c I keep the sources in lyx, and the html/pdf are built from 
there.

lyxport is at:  http://www-hep.colorado.edu/~fperez/lyxport

I also use a few other scripts to build everything, but those are included in 
the CVS sources.

Basically, you can do the build by grabbing the tools/release shell script and 
commenting out the lines (at the end) which upload the released files to 
scipy.org.  It will do everything for you, leaving a built rpm/tgz file in 
ipython/dist.

Or, grab the docs from my release tarballs :)

Cheers,

f.



From fperez at colorado.edu  Fri May 23 21:56:56 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Fri, 23 May 2003 19:56:56 -0600
Subject: [IPython-dev] IPython with Color on windows
In-Reply-To: <200305232058.h4NKwHXo019746@wren.cs.unc.edu>
References: <200305232058.h4NKwHXo019746@wren.cs.unc.edu>
Message-ID: <3ECED168.9000603@colorado.edu>

Gary Bishop wrote:
> I *can* make color work on Windows with my Python port of GNU Readline 
> but it will require changes to  IPython. Specifically, I need to get 
> access to everything that IPython  prints before it goes out. I tried 
> replacing sys.stdout with a "file  like" object but IPython doesn't 
> want to play with that for reasons I  don't understand. Probably a bad 
> idea for a shell to fool with that anyway.
> 
> So, if internally IPython had a function console_print or whatever,  
> normally it would just print but the windows version could eat the  
> escape codes (I've got that part written) and tell the Windows console  
> to do the right thing.

Ipython prints all output via the Prompts classes.  Take a look at that file, 
it's fairly simple.  It uses the fact that python itself honors whatever is 
hooked into sys.displayhook:

ipmaker.py:678:    sys.displayhook = IP.outputcache

where:

ipmaker.py:650:    IP.outputcache = CachedOutput(IP.rc.cache_size...

and CachedOutput is a class defined in Prompts.  You'll see that this class 
already has a display() method implemented by default as _display, but you 
could overwrite it as needed with your own methods under Windows.

The whole point of this design is to make it possible to overwrite these 
methods to satisfy specific needs without actually changing ipython's code 
itself too much.

Obviously, if you find that the current architecture _prevents_ you from 
conveniently putting in the hooks you need, I'm happy to make any necessary 
changes.  But before we go into trying to change ipython itself, let's see how 
far we can get by properly subclassing/overriding methods at runtime.

You could start playing with this as a profile, which at startup time modifies 
the display() method with whatever you need.  This would be a user-level, 100% 
unintrusive modification.  If it passes the test of some extended use by you 
and others, it becomes trivial to fold it into the ipython codebase to be 
loaded at startup under Windows.

Thanks for this work, and keep us posted.

Best,

f.



From fperez at colorado.edu  Fri May 23 21:58:37 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Fri, 23 May 2003 19:58:37 -0600
Subject: [IPython-dev] What readline features do people use?
In-Reply-To: <200305232102.h4NL2LXo019939@wren.cs.unc.edu>
References: <200305232102.h4NL2LXo019939@wren.cs.unc.edu>
Message-ID: <3ECED1CD.7020109@colorado.edu>

Gary Bishop wrote:
> I've got a Python implementation of GNU readline that seems to make 
> IPython happy. It does filename and set_completer style completion, has 
> most of Emacs mode, and generally feels to me like I think IPython should.
> 
[snip...]
> Perhaps I should release something and see what people holler about?

That's probably the best approach.  As long as the basic stuff works 
(attribute/keyword/filename completion, basic line editing...), most people 
will be super happy.  As requests come by, you can always add more stuff. 
Don't over-engineer it now, there's plenty more to do :)

Cheers,

f



From fperez at colorado.edu  Fri May 23 22:01:54 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Fri, 23 May 2003 20:01:54 -0600
Subject: [IPython-dev] IPython bug?
In-Reply-To: <200305232045.h4NKj9Xo019149@wren.cs.unc.edu>
References: <200305232045.h4NKj9Xo019149@wren.cs.unc.edu>
Message-ID: <3ECED292.1050307@colorado.edu>

Gary Bishop wrote:
> With readline mark-directories set to "on" (the default value), on 
> windows when I try:
> 
> In [3]: cd d:\msl\gb\
>    ...:
>    ...:
>    ...:
>    ...:
>    ...:
> 
> The trailing \ on a magic command seems to make it request 
> continuations forever. I agree that \ as the separator on windows is 
> horrible. I don't know why the Python guys use it. Virtually everything 
> works with /. I might try changing it somewhere. In the meantime, it 
> breaks IPython.

 From the ipythonrc file:

# ii. readline_remove_delims: a string of characters to be removed from the
# default word-delimiters list used by readline, so that completions may be
# performed on strings which contain them. Do not change the default value
# unless you know what you're doing.
readline_remove_delims '"[]{}-/~


Try removing '\' from the delimiters list to see what happens.  Some escaping 
might be necessary.  If this fixes it, we can always put a note in for users.

Cheers,

f.



From gb at cs.unc.edu  Fri May 23 22:04:49 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Fri, 23 May 2003 22:04:49 EDT
Subject: [IPython-dev] IPython bug?
Message-ID: <200305240204.h4O24nXo029923@wren.cs.unc.edu>

On Fri, 23 May 2003 20:01:54 -0600 "Fernando Perez" 
<fperez at colorado.edu> wrote:
>  From the ipythonrc file:

> # ii. readline_remove_delims: a string of characters to be removed from the
> # default word-delimiters list used by readline, so that completions may be
> # performed on strings which contain them. Do not change the default value
> # unless you know what you're doing.
> readline_remove_delims '"[]{}-/~


> Try removing '\' from the delimiters list to see what happens.  Some 
> escaping might be necessary.  If this fixes it, we can always put a 
> note in for users.

I tried that. Doesn't help. I don't think \ was in the set to begin with.

gb


From fperez at colorado.edu  Fri May 23 22:12:18 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Fri, 23 May 2003 20:12:18 -0600
Subject: [IPython-dev] IPython bug?
In-Reply-To: <200305240204.h4O24nXo029923@wren.cs.unc.edu>
References: <200305240204.h4O24nXo029923@wren.cs.unc.edu>
Message-ID: <3ECED502.4010703@colorado.edu>

Gary Bishop wrote:
> 
>> Try removing '\' from the delimiters list to see what happens.  Some 
>> escaping might be necessary.  If this fixes it, we can always put a note
>> in for users.
> 
> 
> I tried that. Doesn't help. I don't think \ was in the set to begin with.

Bummer.  Without access to Windows, I'm a bit stumped then.  You might try to 
see if there is any difference here between ipython and a plain python shell 
using your readline stuff.

FWIW, if I type something similar in Linux, there's no problem.  So it _might_ 
be a readline issue, perhaps.

Sorry not to be of more use on this one, but it's hard without having access 
to a testing system myself.  As you are quickly realizing, you'll be the one 
doing the heavy lifting on Win-specific stuff, not me :)

Best,

f.



From gb at cs.unc.edu  Fri May 23 22:29:52 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Fri, 23 May 2003 22:29:52 EDT
Subject: [IPython-dev] IPython bug?
Message-ID: <200305240229.h4O2TqXo000725@wren.cs.unc.edu>

On Fri, 23 May 2003 20:12:18 -0600 "Fernando Perez" 
<fperez at colorado.edu> wrote:
> Bummer.  Without access to Windows, I'm a bit stumped then.  You might 
> try to see if there is any difference here between ipython and a plain 
> python shell using your readline stuff.

> FWIW, if I type something similar in Linux, there's no problem.  So it 
> _might_ be a readline issue, perhaps.

> Sorry not to be of more use on this one, but it's hard without having 
> access to a testing system myself.  As you are quickly realizing, 
> you'll be the one doing the heavy lifting on Win-specific stuff, not me :)

It has the same problem on cygwin if I end the line with \ so I don't 
*think* it is a problem in my readline.

I'll try to track it down when I get a chance.
gb




From fperez at colorado.edu  Fri May 23 22:33:21 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Fri, 23 May 2003 20:33:21 -0600
Subject: [IPython-dev] IPython bug?
In-Reply-To: <200305240229.h4O2TqXo000725@wren.cs.unc.edu>
References: <200305240229.h4O2TqXo000725@wren.cs.unc.edu>
Message-ID: <3ECED9F1.8060505@colorado.edu>

Gary Bishop wrote:
> On Fri, 23 May 2003 20:12:18 -0600 "Fernando Perez" 
> <fperez at colorado.edu> wrote:
> 
>>Bummer.  Without access to Windows, I'm a bit stumped then.  You might 
>>try to see if there is any difference here between ipython and a plain 
>>python shell using your readline stuff.
> 
> 
>>FWIW, if I type something similar in Linux, there's no problem.  So it 
>>_might_ be a readline issue, perhaps.
> 
> 
>>Sorry not to be of more use on this one, but it's hard without having 
>>access to a testing system myself.  As you are quickly realizing, 
>>you'll be the one doing the heavy lifting on Win-specific stuff, not me :)
> 
> 
> It has the same problem on cygwin if I end the line with \ so I don't 
> *think* it is a problem in my readline.

Well, I wonder then if it's something generic to the readline completion in 
python, which is missing the Win-specific cases.  Since typically that code is 
_not_ used under windows, I wouldn't be surprised if it has bugs which under 
*nix don't get triggered.

Keep in mind that I use FlexCompleter, not the normal rlcompleter code. 
FlexCompleter is basically rlcompleter + my own patches.  The good side of 
this is that we have full control over the completion algorithms in ipython, 
and any of those can be fixed if need be.

Best, and good luck with this one,

f.



From prabhu at aero.iitm.ernet.in  Fri May 23 23:22:01 2003
From: prabhu at aero.iitm.ernet.in (Prabhu Ramachandran)
Date: Sat, 24 May 2003 08:52:01 +0530
Subject: [IPython-dev] Making docs and installing from CVS.
In-Reply-To: <3ECE998E.4030107@colorado.edu>
References: <16078.27403.669231.313029@monster.linux.in>
	<3ECE998E.4030107@colorado.edu>
Message-ID: <16078.58713.483180.843822@monster.linux.in>

>>>>> "FP" == Fernando Perez <fperez at colorado.edu> writes:

    FP> It's not too complex, but you'd need a script called lyxport I
    FP> wrote a while back.  That's b/c I keep the sources in lyx, and
    FP> the html/pdf are built from there.

    FP> lyxport is at: http://www-hep.colorado.edu/~fperez/lyxport

Nice.

    FP> I also use a few other scripts to build everything, but those
    FP> are included in the CVS sources.

Yes, I noticed.

    FP> Basically, you can do the build by grabbing the tools/release
    FP> shell script and commenting out the lines (at the end) which
    FP> upload the released files to scipy.org.  It will do everything
    FP> for you, leaving a built rpm/tgz file in ipython/dist.

Well, I dont need to build the dists I simply want to do python
setup.py instal --prefix=... to have everything installed.  I think
I'll stick with the releases for the docs now.

Thanks.
cheers,
prabhu



From gb at cs.unc.edu  Sat May 24 07:20:14 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Sat, 24 May 2003 07:20:14 EDT
Subject: [IPython-dev] IPython with Color on windows
Message-ID: <200305241120.h4OBKEXo019027@wren.cs.unc.edu>

On Fri, 23 May 2003 19:56:56 -0600 "Fernando Perez" 
<fperez at colorado.edu> wrote:
> Ipython prints all output via the Prompts classes.  Take a look at that 
> file, it's fairly simple.  It uses the fact that python itself honors 
> whatever is hooked into sys.displayhook:

The appears to only apply to prompts 'In[0]' and results from 
computations. If I look at ultraTB.py I see

    def __call__(self, etype, value, elist):
        print >> sys.stderr, self.text(etype,value,elist)

greping for 'print' is see many but few that involve color so most are OK.

PyColorize.py appears to write to stdout.

Does stdout get mapped into some file-like object that sends text 
through code in Prompts?

I can already handle color in prompts because they come to readline for 
printing. But if I'm going to make color work for tracebacks, or 
listings, or other places, I'll have to have access to the strings 
before they go out.

Likely I'm missing something.

Thanks for your help,
gb


From gb at cs.unc.edu  Sat May 24 08:46:18 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Sat, 24 May 2003 08:46:18 EDT
Subject: [IPython-dev] IPython bug?
Message-ID: <200305241246.h4OCkIXo021028@wren.cs.unc.edu>

On Fri, 23 May 2003 20:33:21 -0600 "Fernando Perez" 
<fperez at colorado.edu> wrote:
>>> FWIW, if I type something similar in Linux, there's no problem.  So it 
>>> _might_ be a readline issue, perhaps.

It *has* to happen on Linux. Try:

In [1]: @cd foo\<RET>

I bet you'll get the never ending prompt. The problem starts in 
iplib.py line 1085 where you convert the input into a parameter:

            return shell+scommand+'(parameter_s="""%s""")' % (parameter_s,)

If parameter_s ends with a \ then it escapes one of the triple quotes! 
So the string doesn't end... Pretty twisted. Reverse slash is a sick, 
sick character.

Seems to me that at the point you go from a typed line to a string 
parameter you have to take care to escape \ and possibly other characters.

This patch appears to fix the immediate problem:

$ diff -Naur iplib.py~ iplib.py
--- iplib.py~   2003-05-21 06:09:54.000000000 -0400
+++ iplib.py    2003-05-24 08:43:49.000000000 -0400
@@ -1082,6 +1082,7 @@
                 parameter_s = ' ' + parameter_s
             if parameter_s.endswith('"'):
                 parameter_s += ' '
+            parameter_s = parameter_s.replace('\\', '\\\\')
             return shell+scommand+'(parameter_s="""%s""")' % (parameter_s,)
         else:
             return shell+line+'()'

gb


From fperez at colorado.edu  Sat May 24 14:53:40 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Sat, 24 May 2003 12:53:40 -0600
Subject: [IPython-dev] IPython with Color on windows
In-Reply-To: <200305241120.h4OBKEXo019027@wren.cs.unc.edu>
References: <200305241120.h4OBKEXo019027@wren.cs.unc.edu>
Message-ID: <3ECFBFB4.5020003@colorado.edu>

Gary Bishop wrote:
> On Fri, 23 May 2003 19:56:56 -0600 "Fernando Perez" 
> <fperez at colorado.edu> wrote:
> 
>>Ipython prints all output via the Prompts classes.  Take a look at that 
>>file, it's fairly simple.  It uses the fact that python itself honors 
>>whatever is hooked into sys.displayhook:
> 
> 
> The appears to only apply to prompts 'In[0]' and results from 
> computations. If I look at ultraTB.py I see
> 
>     def __call__(self, etype, value, elist):
>         print >> sys.stderr, self.text(etype,value,elist)
> 
> greping for 'print' is see many but few that involve color so most are OK.
> 

Ah, ok.  Ipython has basically three areas where color is supported:

- in/out prompts
- object info (via ?)
- exception tracebacks.

I guess that in order for you to get coloring to work under Win, some 
refactoring will be indeed needed, at least to make the output channels not 
default to stdout, but be runtime configurable.

This is probably a good thing in the long run, as it will make it easier to 
plug ipython into whatever other system which has its own idea of handling output.

But it also means it's a bigger task I'd initially thought.  I'm perfectly 
happy to give it a go as part of the 0.4 series, since it's part of bringing 
feature-parity for Windows.  But I have so much other work to tend to, that I 
will unfortunately be rather slow.  If you can make progress faster on your 
own, don't hesitate to send me the patches.  We could even open up a separate 
CVS branch for this if you want.  Otherwise, we'll move along, albeit a bit slow.

Best,

f.



From fperez at colorado.edu  Sat May 24 15:08:15 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Sat, 24 May 2003 13:08:15 -0600
Subject: [IPython-dev] IPython bug?
In-Reply-To: <200305241246.h4OCkIXo021028@wren.cs.unc.edu>
References: <200305241246.h4OCkIXo021028@wren.cs.unc.edu>
Message-ID: <3ECFC31F.5040605@colorado.edu>

Gary Bishop wrote:
> It *has* to happen on Linux. Try:
> 
> In [1]: @cd foo\<RET>

AH!  Ok, I see it now.  I was using <tab> instead of <ret>, and it all looked 
fine there.

> 
> I bet you'll get the never ending prompt. The problem starts in 
> iplib.py line 1085 where you convert the input into a parameter:
> 
>             return shell+scommand+'(parameter_s="""%s""")' % (parameter_s,)
> 
> If parameter_s ends with a \ then it escapes one of the triple quotes! 
> So the string doesn't end... Pretty twisted. Reverse slash is a sick, 
> sick character.

Indeed, having \ as part of day-to-day objects like paths makes many things 
unnecessarily, stupidly hard and annoying.

> Seems to me that at the point you go from a typed line to a string 
> parameter you have to take care to escape \ and possibly other characters.
> 
> This patch appears to fix the immediate problem:

[snip]

Could you try instead the following, and see what happens?  I prefer it b/c it 
breaks less situations where the user may be honestly trying to use escapes to 
handle special characters in his filenames.  I simply pad the input string 
around the edges with whitespace to prevent the \'s from screwing the quotes 
up.  It seems to work ok for me in most cases, and it actually simplifies the 
code with 2 less ifs/function calls per user input.

[IPython]> diff -Naur iplib.py~ iplib.py
--- iplib.py~   2003-05-16 21:55:33.000000000 -0600
+++ iplib.py    2003-05-24 13:05:05.000000000 -0600
@@ -1078,11 +1078,8 @@
              parameter_s = ''
              scommand = line
          if hasattr(self, scommand):
-            if parameter_s.startswith('"'):
-                parameter_s = ' ' + parameter_s
-            if parameter_s.endswith('"'):
-                parameter_s += ' '
-            return shell+scommand+'(parameter_s="""%s""")' % (parameter_s,)
+            parameter_s = " %s " % parameter_s
+            return shell+scommand+'(parameter_s="""%s""")' % parameter_s
          else:
              return shell+line+'()'

Cheers,

f.



From gb at cs.unc.edu  Sat May 24 15:46:04 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Sat, 24 May 2003 15:46:04 EDT
Subject: [IPython-dev] IPython bug?
Message-ID: <200305241946.h4OJk4Xo003761@wren.cs.unc.edu>

On Sat, 24 May 2003 13:08:15 -0600 "Fernando Perez" 
<fperez at colorado.edu> wrote:
> Could you try instead the following, and see what happens?  I prefer it 
> b/c it breaks less situations where the user may be honestly trying to 
> use escapes to handle special characters in his filenames.  I simply 
> pad the input string around the edges with whitespace to prevent the 
> \'s from screwing the quotes up.  It seems to work ok for me in most 
> cases, and it actually simplifies the code with 2 less ifs/function 
> calls per user input.

> [IPython]> diff -Naur iplib.py~ iplib.py
> --- iplib.py~   2003-05-16 21:55:33.000000000 -0600
> +++ iplib.py    2003-05-24 13:05:05.000000000 -0600
> @@ -1078,11 +1078,8 @@
>               parameter_s = ''
>               scommand = line
>           if hasattr(self, scommand):
> -            if parameter_s.startswith('"'):
> -                parameter_s = ' ' + parameter_s
> -            if parameter_s.endswith('"'):
> -                parameter_s += ' '
> -            return shell+scommand+'(parameter_s="""%s""")' % (parameter_s,)
> +            parameter_s = " %s " % parameter_s
> +            return shell+scommand+'(parameter_s="""%s""")' % parameter_s
>           else:
>               return shell+line+'()'


That won't do the job. Consider a line like:

In[1] cd foo\next

A perfectly legal thing for a Windows user to type, or for completion 
to return. When you convert that into a string, you'll have a newline 
character in the middle! Ack!

I see what you're saying about my fix though. If a magic command took a 
string argument, and the user wanted to escape something in the string. 
It would be bad. Combined with automatic quoting this seems to really 
produce a dilemma. Can you think of a case where this happens?

Maybe we should avoid \ as much as possible. I can make file name 
completion return /. We can encourage users to type / and warn them 
they better type \\ if they want to use it. I'd like to, one day, make 
a version of the commands that deal with paths that use / instead of \. 
They pretty much all accept / but they produce \.

Oh, but consider a ! escape. Sometimes the windows shell insists on \. 
It is pretty rare though.

The complicated rule would be outside quotation marks (of any kind) 
characters stand for themselves. There are no escapes. Inside quotation 
marks the usual escape conventions apply.

This would require users to quote strings they wanted to use escapes 
in, which seems reasonable to me. And it would allow us to change \ to 
\\ everywhere in unquoted strings (like file names).

Getting the quotes right probably requires character by character 
processing of the line.

What do you think?
gb


From gb at cs.unc.edu  Sat May 24 16:14:54 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Sat, 24 May 2003 16:14:54 EDT
Subject: [IPython-dev] IPython with Color on windows
Message-ID: <200305242014.h4OKEsXo004568@wren.cs.unc.edu>

On Sat, 24 May 2003 12:53:40 -0600 "Fernando Perez" 
<fperez at colorado.edu> wrote:

> Ah, ok.  Ipython has basically three areas where color is supported:

> - in/out prompts

I already do in prompts down in my readline code.

> - object info (via ?)
> - exception tracebacks.

> I guess that in order for you to get coloring to work under Win, some 
> refactoring will be indeed needed, at least to make the output channels 
> not default to stdout, but be runtime configurable.

Agreed.

> This is probably a good thing in the long run, as it will make it 
> easier to plug ipython into whatever other system which has its own 
> idea of handling output.

Agreed.

> But it also means it's a bigger task I'd initially thought.  I'm 
> perfectly happy to give it a go as part of the 0.4 series, since it's 
> part of bringing feature-parity for Windows.  But I have so much other 
> work to tend to, that I will unfortunately be rather slow.  If you can 
> make progress faster on your own, don't hesitate to send me the 
> patches.  We could even open up a separate CVS branch for this if you 
> want.  Otherwise, we'll move along, albeit a bit slow.

I *should* be working on something else but having fun with this. I'm 
going to have to quit (for a while at least) pretty soon.

There is a stylistic issue in the change that I think you should 
answer, even if I hack the code. This change could happen at least 2 
ways that I can think of.

1) Add a module that contains a file-like object that everyone uses to 
write to the console. Pretty easy to implement and small changes to 
modules. Outside windows the file-like object is just sys.stdout. 
Inside windows it is some filter code that uses my console code.

2) Pass the object down into the various other classes (ultraTB, 
Inspect) using their init methods. Might be doable but I'd have to 
understand the code a lot better than I do now.

Other possibilities? Preferences?

> Best,

> f.



From fperez at colorado.edu  Sat May 24 16:20:37 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Sat, 24 May 2003 14:20:37 -0600
Subject: [IPython-dev] IPython bug?
In-Reply-To: <200305241946.h4OJk4Xo003761@wren.cs.unc.edu>
References: <200305241946.h4OJk4Xo003761@wren.cs.unc.edu>
Message-ID: <3ECFD415.1000307@colorado.edu>

Gary Bishop wrote:
> 
> That won't do the job. Consider a line like:
> 
> In[1] cd foo\next
> 
> A perfectly legal thing for a Windows user to type, or for completion 
> to return. When you convert that into a string, you'll have a newline 
> character in the middle! Ack!
> 
> I see what you're saying about my fix though. If a magic command took a 
> string argument, and the user wanted to escape something in the string. 
> It would be bad. Combined with automatic quoting this seems to really 
> produce a dilemma. Can you think of a case where this happens?

Many.  If users in Unix need to escape spaces or other special characters in 
filenames, for instance.  In particular, I want to guarantee that the ipython 
syntax and the shell one look the same:

*** SHELL

[test]> d
/usr/local/home/fperez/test/test
total 8
drwxr-xr-x    2 fperez       4096 May 24 14:11 a space/
drwxr-xr-x    2 fperez       4096 May 24 13:58 b\ad/
[test]> ls a\ space
foo
[test]> ls b\\ad
foo

***IPython
In [15]: d
total 8
drwxr-xr-x    2 fperez       4096 May 24 14:11 a space/
drwxr-xr-x    2 fperez       4096 May 24 13:58 b\ad/

In [16]: ls a\ space
foo

In [17]: ls b\\ad
foo

I can get this to work with:

[IPython]> diff -Naur iplib_ori.py iplib.py
--- iplib_ori.py        2003-05-24 13:10:58.000000000 -0600
+++ iplib.py    2003-05-24 14:09:11.000000000 -0600
@@ -1078,11 +1078,8 @@
              parameter_s = ''
              scommand = line
          if hasattr(self, scommand):
-            if parameter_s.startswith('"'):
-                parameter_s = ' ' + parameter_s
-            if parameter_s.endswith('"'):
-                parameter_s += ' '
-            return shell+scommand+'(parameter_s="""%s""")' % (parameter_s,)
+            parameter_s = " %s " % parameter_s # Protect quotes at the end
+            return shell+scommand+'(parameter_s=r"""%s""")' % parameter_s
          else:
              return shell+line+'()'


It's basically the same patch as before, with an added raw string literal for 
the string as passed.

> Maybe we should avoid \ as much as possible. I can make file name 
> completion return /. We can encourage users to type / and warn them 
> they better type \\ if they want to use it. I'd like to, one day, make 
> a version of the commands that deal with paths that use / instead of \. 
> They pretty much all accept / but they produce \.
> 
> Oh, but consider a ! escape. Sometimes the windows shell insists on \. 
> It is pretty rare though.
> 
> The complicated rule would be outside quotation marks (of any kind) 
> characters stand for themselves. There are no escapes. Inside quotation 
> marks the usual escape conventions apply.
> 
> This would require users to quote strings they wanted to use escapes 
> in, which seems reasonable to me. And it would allow us to change \ to 
> \\ everywhere in unquoted strings (like file names).
> 
> Getting the quotes right probably requires character by character 
> processing of the line.
> 
> What do you think?

That down this path lies insanity :)

Seriously, I'm not going to write a nightmarish string escaping code like 
this.  If you want to do it, feel free to write it.  We can always hook it at 
runtime as a user extension via the prefilter() method, in a similar way to 
what the current 'tutorial' profile does.


The patch above works for me, and this is one of those cases where I will go 
for simplicity at the sacrifice of features for Windows, if it comes down to 
that.  I think we can come up with a reasonable solution via a combination of 
raw strings and other simple things.  But if the only way out is a 
full-fledged string escape code, it will have to be a separate prefilter() 
function which gets distributed in a  separate file and loaded only under 
Windows.  I'd be happy to include something like that, but NOT in the inner 
core of ipython.

Let's see how far we can get with simple, maintaninable solutions first. 
Remember Jamie Zawinski's wisdom:

Some people, when confronted with a problem, think ?I know, I?ll use regular 
expressions.? Now they have two problems.
	
--Jamie Zawinski, in comp.lang.emacs 	


Cheers,

f.



From gb at cs.unc.edu  Sat May 24 16:33:59 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Sat, 24 May 2003 16:33:59 EDT
Subject: [IPython-dev] IPython bug?
Message-ID: <200305242033.h4OKXxXo005099@wren.cs.unc.edu>

On Sat, 24 May 2003 14:20:37 -0600 "Fernando Perez" 
<fperez at colorado.edu> wrote:

Yes, your text cases convince me.

> I can get this to work with:

> [IPython]> diff -Naur iplib_ori.py iplib.py
> --- iplib_ori.py        2003-05-24 13:10:58.000000000 -0600
> +++ iplib.py    2003-05-24 14:09:11.000000000 -0600
> @@ -1078,11 +1078,8 @@
>               parameter_s = ''
>               scommand = line
>           if hasattr(self, scommand):
> -            if parameter_s.startswith('"'):
> -                parameter_s = ' ' + parameter_s
> -            if parameter_s.endswith('"'):
> -                parameter_s += ' '
> -            return shell+scommand+'(parameter_s="""%s""")' % (parameter_s,)
> +            parameter_s = " %s " % parameter_s # Protect quotes at the end
> +            return shell+scommand+'(parameter_s=r"""%s""")' % parameter_s
>           else:
>               return shell+line+'()'


> It's basically the same patch as before, with an added raw string 
> literal for the string as passed.

Yes! This seems to work fine. It is hard for me to think about the 
possibilities with an extra level of interpretation going on. Like 
working on a macro processor.

>> Maybe we should avoid \ as much as possible.

>> Getting the quotes right probably requires character by character 
>> processing of the line.

>> What do you think?

> That down this path lies insanity :)

Agreed. Your raw string change appear to fix it and leaves open all the 
possibilities. Good deal.

gb


From fperez at colorado.edu  Sat May 24 18:05:43 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Sat, 24 May 2003 16:05:43 -0600
Subject: [IPython-dev] IPython with Color on windows
In-Reply-To: <200305242014.h4OKEsXo004568@wren.cs.unc.edu>
References: <200305242014.h4OKEsXo004568@wren.cs.unc.edu>
Message-ID: <3ECFECB7.6020407@colorado.edu>

Gary Bishop wrote:

> I *should* be working on something else but having fun with this. I'm 
> going to have to quit (for a while at least) pretty soon.

No prob.  Things under windows have never moved much, so all we get from you 
is definitely progress :)

> There is a stylistic issue in the change that I think you should 
> answer, even if I hack the code. This change could happen at least 2 
> ways that I can think of.
> 
> 1) Add a module that contains a file-like object that everyone uses to 
> write to the console. Pretty easy to implement and small changes to 
> modules. Outside windows the file-like object is just sys.stdout. 
> Inside windows it is some filter code that uses my console code.
> 
> 2) Pass the object down into the various other classes (ultraTB, 
> Inspect) using their init methods. Might be doable but I'd have to 
> understand the code a lot better than I do now.
> 
> Other possibilities? Preferences?

Well, I favor something like 2.  It seems to make sense to instantiate the 
main ipython shell object with a pair of optional stdout/stderr arguments.  If 
not given, they default to sys.stdout/err as usual.  If given, these are used 
by the controlling class to pass down to other objects it uses which offer the 
same protocol.  Currently most write directly to stdout/err, but this would be 
easy (though time-consuming) to change.

Something like 1 would only work if we write a file-like object and set 
sys.stdout to it.  The moment user code resets sys.stdout, we're toast.  I've 
already seen failure cases like this at initialization time, were I have 
file-like objects for trapping output.  It's a disaster, and I'm actually gong 
to remove all that code.  I'll leave the OutputTrap modules in the 
distribution in case someone uses them, but I'll stop using them inside ipython.

So we need to have _our_ functions write to their special channels, without 
messing with what the users expect to be normal sys.std* behavior.  ipython 
should mimic, as much as possible, a plain vanilla python interpreter as far 
as how user code behaves under it.

As I said, this can be quite time-consuming.  If you plan on diving head first 
into it, opening a separate cvs branch may be a good idea so that this effort 
can progress at its own pace, without halting regular bugfixing/small 
improvements.  This is good, but a somewhat 'big architecture' kind of job.

Best,

f.



From fperez at colorado.edu  Sat May 24 18:11:44 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Sat, 24 May 2003 16:11:44 -0600
Subject: [IPython-dev] IPython with Color on windows
In-Reply-To: <3ECFECB7.6020407@colorado.edu>
References: <200305242014.h4OKEsXo004568@wren.cs.unc.edu> <3ECFECB7.6020407@colorado.edu>
Message-ID: <3ECFEE20.4050407@colorado.edu>

Fernando Perez wrote:

> Well, I favor something like 2.  It seems to make sense to instantiate the 
> main ipython shell object with a pair of optional stdout/stderr arguments.  If 
> not given, they default to sys.stdout/err as usual.  If given, these are used 
> by the controlling class to pass down to other objects it uses which offer the 
> same protocol.  Currently most write directly to stdout/err, but this would be 
> easy (though time-consuming) to change.

Just a clarification.  In order to avoid over-engineering this solution, I 
would only use the special output channels for printing 'special' strings, 
such as colored stuff.  This would keep the changes to a minimum (rather than 
having to change every single 'print' statement).

Later, if this same approach is deemed necessary for other things (such as GUI 
integration), we can always extend it.  But let's do 'the simplest thing that 
can work' first :)

best,

f.



From gb at cs.unc.edu  Sat May 24 18:27:47 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Sat, 24 May 2003 18:27:47 EDT
Subject: [IPython-dev] IPython with Color on windows
Message-ID: <200305242227.h4OMRlXo008805@wren.cs.unc.edu>

Option 1 would not require changing sys.stdout. I wouldn't want to do 
that. It would work just like option 2 except instead of passing 
arguments, you import a module with the info. I'd do something like a 
module "ConsoleFilter" with a variable "ConsoleOut".

Now any module that wants to output to the console, just

from ConsoleFilter import ConsoleOut

print >>ConsoleOut, whatever

instead of

print whatever

The ConsoleFilter (or whatever) module takes care of initializing 
"ConsoleFilter" based on the OS and available modules, etc.

Very minimal changes to the current structure and the change is nicely 
encapsulated in the "terminal" module.

See what I mean? It would be much easier (for me at least) to implement 
than 2 which requires changing every place in the call chain to provide 
the object.

gb



From gb at cs.unc.edu  Sat May 24 21:16:38 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Sat, 24 May 2003 21:16:38 EDT
Subject: [IPython-dev] IPython bug?
Message-ID: <200305250116.h4P1GcXo013459@wren.cs.unc.edu>

Doh! Padding with spaces breaks at least one other place. With the patch in try

@pfile afilename

I get

Object 'afilename' not found.
File ' afilename '.py not found.

We'll need a strip somewhere in there.

gb



From fperez at colorado.edu  Sun May 25 00:09:01 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Sat, 24 May 2003 22:09:01 -0600
Subject: [IPython-dev] IPython bug?
In-Reply-To: <200305250116.h4P1GcXo013459@wren.cs.unc.edu>
References: <200305250116.h4P1GcXo013459@wren.cs.unc.edu>
Message-ID: <3ED041DD.3000508@colorado.edu>

Gary Bishop wrote:
> Doh! Padding with spaces breaks at least one other place. With the patch in try
> 
> @pfile afilename
> 
> I get
> 
> Object 'afilename' not found.
> File ' afilename '.py not found.

Argh.  Ok, a different approach will be necessary.  sorry but I'm leaving for 
a couple of days, so I'll be offline til Tuesday at the earliest.

best,

f.

ps.  this breakage is why we need unittests everywhere _badly_.



From gb at cs.unc.edu  Sun May 25 07:58:37 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Sun, 25 May 2003 07:58:37 EDT
Subject: [IPython-dev] IPython bug?
Message-ID: <200305251158.h4PBwcXo005372@wren.cs.unc.edu>

I believe I've got it now. With a trick from the FAQ.

        if hasattr(self, scommand):
            parameter_s = " %s " % parameter_s # Protect quotes at the end
            return shell+scommand+'(parameter_s=r"""%s"""[1:-1])' % parameter_s
        else:
            return shell+line+'()'

You put extra spaces on, then pull them off with the slice after 
quoting is complete.

gb


From gb at cs.unc.edu  Sun May 25 21:12:57 2003
From: gb at cs.unc.edu (Gary Bishop)
Date: Sun, 25 May 2003 21:12:57 EDT
Subject: [IPython-dev] IPython with Color on windows
Message-ID: <200305260112.h4Q1CvXo000014@wren.cs.unc.edu>

OK. I *think* I've got IPython working on windows with color, history, 
completion (file and variable), emacs-style line editing and history 
search, paging with less (in color). The changes to the CVS version are 
pretty small. When you get back, let me know and I'll send them. I 
going to install all this on my machine and use it for a few days. That 
will, no doubt, turn up problems.

I've got to see if I can make readline a package and still have 
automatic loading work. I think it should.

gb