Mailman 3 May 2000 - Python-Dev

Updated curses module in CVS
by Andrew M. Kuchling May 25, 2000

May 25, 2000

Today I checked in a new version of the curses module that will only work with ncurses and/or SYSV curses. I've tried compiling it on Linux with ncurses 5.0, and on Solaris; there are also #ifdef's to make it work with some version of SGI's curses. I'd appreciate it if people could try the module with the curses implementations on other platforms: Tru64, AIX, *BSDs (though they use ncurses, maybe they're some versions behind), etc. Please let me know of your results through e-mail. And if you have code that used the old curses module, and breaks with the new module, please let me know; the goal is to have 100% backward-compatibility. Also, here's a list of ncurses functions that aren't yet supported; should I make adding them a priority. (Most of them seem to be pretty marginal, except for the mouse-related functions which I want to add next.) addchnstr addchstr chgat color_set copywin define_key del_curterm delscreen dupwin getmouse inchnstr inchstr innstr keyok mcprint mouseinterval mousemask mvaddchnstr mvaddchstr mvchgat mvcur mvinchnstr mvinchstr mvinnstr mmvwaddchnstr mvwaddchstr mvwchgat mvwgetnstr mvwinchnstr mvwinchstr mvwinnstr napms newterm overlay overwrite resetty resizeterm restartterm ripoffline savetty scr_dump scr_init scr_restore scr_set scrl set_curterm set_term setterm setupterm slk_attr slk_attr_off slk_attr_on slk_attr_set slk_attroff slk_attron slk_attrset slk_clear slk_color slk_init slk_label slk_noutrefresh slk_refresh slk_restore slk_set slk_touch tgetent tgetflag tgetnum tgetstr tgoto tigetflag tigetnum tigetstr timeout tparm tputs tputs typeahead ungetmouse use_default_colors vidattr vidputs waddchnstr waddchstr wchgat wcolor_set wcursyncup wenclose winchnstr winchstr winnstr wmouse_trafo wredrawln wscrl wtimeout -- A.M. Kuchling http://starship.python.net/crew/amk/ ..signature has giant ASCII graphic: Forced to read "War And Peace" at 110 baud on a Braille terminal after having fingers rubbed with sandpaper. -- Kibo, in the Happynet Manifesto

6 6

1.6 release date
by Ka-Ping Yee May 25, 2000

May 25, 2000

Sorry if i missed an earlier announcement on this topic. The web page about 1.6 currently says that Python 1.6 will be released on June 1. Is that still the target date? -- ?!ng

2 1

String encoding
by M.-A. Lemburg May 25, 2000

May 25, 2000

The recent discussion about repr() et al. brought up the idea of a locale based string encoding again. A support module for querying the encoding used in the current locale together with the experimental hook to set the string encoding could yield a compromise which satisfies ASCII, Latin-1 and UTF-8 proponents. The idea is to use the site.py module to customize the interpreter from within Python (rather than making the encoding a compile time option). This is easily doable using the (yet to be written) support module and the sys.setstringencoding() hook. The default encoding would be 'ascii' and could then be changed to whatever the user or administrator wants it to be on a per site basis. Furthermore, the encoding should be settable on a per thread basis inside the interpreter (Python threads do not seem to inherit any per-thread globals, so the encoding would have to be set for all new threads). E.g. a site.py module could look like this: """ import locale,sys # Get encoding, defaulting to 'ascii' in case it cannot be # determined defenc = locale.get_encoding('ascii') # Set main thread's string encoding sys.setstringencoding(defenc) This would result in the Unicode implementation to assume defenc as encoding of strings. """ Minor nit: due to the implementation, the C parser markers "s" and "t" and the hash() value calculation will still need to work with a fixed encoding which still is UTF-8. C APIs which want to support Unicode should be fixed to use "es" or query the object directly and then apply proper, possibly OS dependent conversion. Before starting off into implementing the above, I'd like to hear some comments... Thanks, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

6 15

Larry's need for metacharacters...
by Greg Stein May 25, 2000

May 25, 2000

[ paraphrased from a LWN letter to the editor ] Regarding the note posted here last week about Perl development stopping cuz Larry can't figure out any more characters to place after the '$' character (to create "special" things) ... Note that Larry became interested in Unicode a few years ago... Note that Perl now supports Unicode throughout... *including* variable names... Coincidence? I think not! $\uAB56 = 1; :-) Cheers, -g -- Greg Stein, http://www.lyra.org/

1 0

Some more on the 'tempfile' naming security issue
by pf＠artcom-gmbh.de May 24, 2000

May 24, 2000

[Guido] > Every few months I receive patches that purport to make the tempfile > module more secure. I've never felt that it is a problem. What is > with these people? [Tim] > Doing a google search on > > tempfile security > > turns up hundreds of rants. Have fun <wink>. There does appear to be a > real vulnerability here somewhere (not necessarily Python), but the closest > I found to a clear explanation in 10 minutes was an annoyed paragraph, > saying that if I didn't already understand the problem I should turn in my > Unix Security Expert badge immediately. Unfortunately, Bill Gates never > issued one of those to me. On <http://www.insecure.org/sploits/gcc.tmpfiles.html> you can find a working example which exploits this vulnerability in older versions of GCC. The basic idea is indeed very simple: Since the /tmp directory is writable for any user, the bad guy can create a symbolic link in /tmp pointing to some arbitrary file (e.g. to /etc/passwd). The attacked program will than overwrite this arbitrary file (where the programmer really wanted to write something to his tempfile instead). Since this will happen with the access permissions of the process running this program, this opens a bunch of vulnerabilities in many programs writing something into temporary files with predictable file names. www.cert.org is another great place to look for security related info. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)

3 3

Supporting non-Microsoft compilers
by Greg Ward May 24, 2000

May 24, 2000

A couple of people are working on support in the Distutils for building extensions on Windows with non-Microsoft compilers. I think this is crucial; I hate the idea of requiring people to either download a binary or shell out megabucks (and support Chairman Bill's monopoly) just to use some handy Python extension. (OK, OK, more likely they'll go without the extension, or go without Python. But still...) However, it seems like it would be nice if people could build Python itself with (eg.) cygwin's gcc or Borland's compiler. (It might be essential to properly support building extensions with gcc.) Has anyone one anything towards that goal? It appears that there is at least one patch floating around that advises people to hack up their installed config.h, and drop a libpython.a somewhere in the installation, in order to compile extensions with cygwin gcc and/or mingw32. This strikes me as sub-optimal: can at least the required changes to config.h be made to allow building Python with one of the Windows gcc ports? I would be willing to hold my nose and struggle with cygwin for a little while in Windows in dull moments at work -- had to reboot my Linux box into Windows today in order to test try building CXX (since my VMware trial license expired), so I might as well leave it there until it crashes and play with cygwin. Greg -- Greg Ward - software developer gward(a)mems-exchange.org MEMS Exchange / CNRI voice: +1-703-262-5376 Reston, Virginia, USA fax: +1-703-262-5367

2 1

homer-dev, anyone?
by Fredrik Lundh May 24, 2000

May 24, 2000

http://www.segfault.org/story.phtml?mode=2&id=391ae457-08fa7b40 </F>

7 7

Unicode
by Martin v. Loewis May 23, 2000

May 23, 2000

> comments? (for obvious reasons, I'm especially interested in comments > from people using non-ASCII characters on a daily basis...) > nobody? Hi Frederik, I think the problem you try to see is not real. My guideline for using Unicode in Python 1.6 will be that people should be very careful to *not* mix byte strings and Unicode strings. If you are processing text data, obtained from a narrow-string source, you'll always have to make an explicit decision what the encoding is. If you follow this guideline, I think the Unicode type of Python 1.6 will work just fine. If you use Unicode text *a lot*, you may find the need to combine them with plain byte text in a more convenient way. This is the time you should look at the implicit conversion stuff, and see which of the functionality is useful. You then don't need to memorize *all* the rules where implicit conversion would work - just the cases you care about. That may all look difficult - it probably is. But then, it is not more difficult than tuples vs. lists: why does >>> [a,b,c] = (1,2,3) work, and >>> [1,2]+(3,4) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: illegal argument type for built-in operation does not? Regards, Martin

9 17

repr vs. str and locales again
by Guido van Rossum May 23, 2000

May 23, 2000

The email below suggests a simple solution to a problem that e.g. Fran\347ois Pinard brought up long ago; repr() of a string turns all non-ASCII chars into \oct escapes. Jyrki's solution: use isprint(), which makes it locale-dependent. I can live with this. It needs a Py_CHARMASK() call but otherwise seems to be fine. Anybody got an opinion on this? I'm +0. I would even be +0 on a similar patch for unicode strings (once the ASCII proposal is implemented). --Guido van Rossum (home page: http://www.python.org/~guido/) ------- Forwarded Message Date: Fri, 19 May 2000 10:48:29 +0300 From: Jyrki Kuoppala <jkp(a)kaapeli.fi> To: guido(a)python.org Subject: python bug?: python 1.5.2 fails to print printable 8-bit characters in strings I'm not sure if this exactly is a bug, ie. whether python 1.5.2 is supposed to support locales and 8-bit characters. However, on Linux Debian "unstable" distribution the diff below makes python 1.5.2 handle printable 8-bit characters as one would expect. Problem description: python doesn't properly print printable 8-bit characters for the current locale . Details: With no locale set, 8-bit characters in quoted strings print as backslash-escapes, which I guess is OK: $ unset LC_ALL $ python Python 1.5.2 (#0, Apr 3 2000, 14:46:48) [GCC 2.95.2 20000313 (Debian GNU/Linu x)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> a=('foo','kääk') >>> print a ('foo', 'k\344\344k') >>> But with a locale with a printable 'ä' character (octal 344) I get: $ export LC_ALL=fi_FI $ python Python 1.5.2 (#0, Apr 3 2000, 14:46:48) [GCC 2.95.2 20000313 (Debian GNU/Linu x)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> a=('foo','kääk') >>> print a ('foo', 'k\344\344k') >>> I should be getting (output from python patched with the enclosed patch): $ export LC_ALL=fi_FI $ python Python 1.5.2 (#0, May 18 2000, 14:43:46) [GCC 2.95.2 20000313 (Debian GNU/Linu x)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> a=('foo','kääk') >>> print a ('foo', 'kääk') >>> This hits for example when Zope with squishdot weblog (squishdot 0.3.2-3 with zope 2.1.6-1) creates a text index from posted articles - strings with valid Latin1 characters get indexed as backslash-escaped octal codes, and thus become unsearchable. I am using debian unstable, kernels 2.2.15pre10 and 2.0.36, libc 2.1.3. I suggest that the test for printability in python-1.5.2 /Objects/stringobject.c be fixed to use isprint() which takes the locale into account: - --- python-1.5.2/Objects/stringobject.c.orig Thu Oct 8 05:17:48 1998 +++ python-1.5.2/Objects/stringobject.c Thu May 18 14:36:28 2000 @@ -224,7 +224,7 @@ c = op->ob_sval[i]; if (c == quote || c == '\\') fprintf(fp, "\\%c", c); - - else if (c < ' ' || c >= 0177) + else if (! isprint (c)) fprintf(fp, "\\%03o", c & 0377); else fputc(c, fp); @@ -260,7 +260,7 @@ c = op->ob_sval[i]; if (c == quote || c == '\\') *p++ = '\\', *p++ = c; - - else if (c < ' ' || c >= 0177) { + else if (! isprint (c)) { sprintf(p, "\\%03o", c & 0377); while (*p != '\0') p++; //Jyrki ------- End of Forwarded Message

9 31

Re: Python multiplexing is too hard (was: Network statistics program)
by Guido van Rossum May 23, 2000

May 23, 2000

[Quoting the entire mail because I've added python-dev to the cc: list] > Subject: Re: Python multiplexing is too hard (was: Network statistics program) > From: Alexandre Ferrieux <alexandre.ferrieux(a)cnet.francetelecom.fr> > To: Guido van Rossum <guido(a)python.org> > Cc: claird(a)starbase.neosoft.com > Date: Fri, 19 May 2000 11:25:43 +0200 > Delivery-Date: Fri May 19 05:26:59 2000 > > Guido van Rossum wrote: > > > > Cameron Laird wrote: > > > . > > > Right. asyncore is nice--but restricted to socket > > > connections. For many applications, that's not a > > > restriction at all. However, it'd be nice to have > > > such a handy interface for communication with > > > same-host processes; that's why I mentioned popen*(). > > > Does no one else perceive a gap there, in convenient > > > asynchronous piped IPC? Do folks just fall back on > > > select() for this case? > > > > Hm, really? For same-host processes, threads would > > do the job nicely I'd say. > > Overkill. > > > Or you could probably > > use unix domain sockets (popen only really works on > > Unix, so that's not much of a restriction). > > Overkill. > > > Also note that often this is needed in the context > > of a GUI app; there something integrated in the GUI > > main loop is recommended. (E.g. the file events that > > Moshe mentioned.) > > Okay so your answer is, The Python Way of doing it is to use Tcl. > That's pretty disappointing, I'm sorry to say... > > Consider: > > - In Tcl, as you said, this is nicely integrated with the GUI's > event queue: > - on unix, by a an additional bit on X's fd (socket) in > the select() > - on 'doze, everything is brought back to messages > anyway. > > And, in both cases, it works with pipes, sockets, serial or other > devices. Uniform, clean. > > - In python "popen only really works on Unix": are you satisfied with > that state of affairs ? I understand (and value) Python's focus on > algorithms and data structures, and worming around OS misgivings is a > boring, ancillary task. But what about the potential gain ? > > I'm an oldtime Tcler, firmly decided to switch to Python, 'cause it is > just so beautiful inside. But while Tcl is weaker in the algorithms, it > is stronger in the os-wrapping library, and taught me to love high-level > abstractions. [fileevent] shines in this respect, and I'll miss it in > Python. > > -Alex Alex, it's disappointing to me too! There just isn't anything currently in the library to do this, and I haven't written apps that needs this often enough to have a good feel for what kind of abstraction is needed. However perhaps we can come up with a design for something better? Do you have a suggestion here? I agree with your comment that higher-level abstractions around OS stuff are needed -- I learned system programming long ago, in C, and I'm "happy enough" with the current state of affairs, but I agree that for many people this is a problem, and there's no reason why Python couldn't do better... --Guido van Rossum (home page: http://www.python.org/~guido/)

11 30