Mailman 3 June 2000 - Python-Dev

Towards 1.6 Final
by bwarsaw＠python.org 20 Jun '00

20 Jun '00

Okay, let me just preface this by saying none of the following has been blessed by Guido in any way. I'm sure he's still getting nice and tan and drinking exotic tropical alcohol-based beverages, and not thinking about Python one little bit. :) Fred and Jeremy and I had a meeting today and thought about a schedule /we'd/ like to see for Python 1.6 final. This may or may not be feasible but I throw it out here for discussion anyway. The O'Reilly Open Source conference[1] starts July 17th and it would be a nice target to release 1.6 final on the 14th, the Friday before the conference. We all plan on being there.n What do we need to do to hit this target? If we figure a 2 week beta cycle, 1.6beta1 would have to be released June 30, with probably the last alpha on June 23. This is a tight schedule; is it even feasible?. Looking at Andrew's TODO list[2], here's what we came up with: - import hooks revamped (push this to 1.7) - update docos to match 1.6 changes (Fred is going to integrate what's sitting in his inbox) - document more undocumented modules (codes - done; unicodedata - needs work; mmap & pyexpat - Fred to integrate; curses - needs updates; regrtest - push to 1.7, see below) - Unicode: Compress the size of unicodedatabase (waiting for Chris Tismer patches?) - Unicode: Write \N{SMILEY} codec for Unicode (Bill Tutt has a patch, but none of us know what the state of this is -- is it crucial anyway?). - Unicode: the various XXX items in Misc/unicode.txt (we didn't actually find such outstanding issues in this file). - Delete obsolete subdirs in Demo/ directory (done) - Refurbish Demo subdirs to be properly documented, match modern coding style, etc. (???) - Fix ./ld_so_aix installation problem on AIX (anybody know more about this???) - Make test/regrtest.py more usable outside of the Python test suite & document it. (push to 1.7) - Conservative GC, of cycles (this is basically done but needs a lot more testing) - Changes to PyExpat module (Fred says this is done) - Test the hell out of SRE. (waiting for /F's complete implementation) - enable threads by default if OS supports it - integrate intl/gettext module - clear the patches and bugs backlog As we saw it the most critical things that need to get done are testing the GC stuff and the SRE module. Of those, the most critical is SRE since AFAIK, we're still waiting for the final complete code from /F. I could be wrong about that. The last three weren't on Andrew's list. I'm on the hook for enabling threads support by default and integrating intl/gettext. I think those will just be a couple of days work. The final one requires Us to go through the outstanding bug and patches lists and address the critical bugs. Thoughts? What have I missed? -Barry [1] http://conferences.oreilly.com/oscon2000/ [2] http://starship.python.net/crew/amk/python/1.6-jobs.html

8 23

Re: [Python-Dev] Proposal: standard way of defining and executing "atexit" functions...
by pf＠artcom-gmbh.de 20 Jun '00

20 Jun '00

Hi, I wrote: > def register_exit_func(func, *args, **kw): > import sys > previous_func = sys.exit_func this should have been 'sys.exitfunc'. I've also misssed to note, that I placed the following line into the module init code: import sys; def _nop(): pass; sys.exitfunc = _nop Otherwise I had to use getattr(sys, 'exitfunc', None) as David suggested. Next time I should actually cut'n'paste working code instead typing from blurred memory. :-( Another point I forgot, was that you usually want to install signal handlers on unices, that handle the signal.SIGTERM, signal.SIGHUP and signal.SIGINT signals and invoke exit function to do some final cleanup. Regards, Peter

2 1

Re: [Python-checkins] CVS: python/dist/src/Modules mmapmodule.c,2.10,2.11
by Vladimir.Marangozov＠inrialpes.fr 20 Jun '00

20 Jun '00

A.M. Kuchling wrote: > > Update of /cvsroot/python/python/dist/src/Modules > In directory slayer.i.sourceforge.net:/tmp/cvs-serv2060 > > Modified Files: > mmapmodule.c > Log Message: > Fix the size() method to return the size of the file on Unix, not the > size of the mapped area. This seems to be what the Windows version does. > This change requires keeping around the fd of the mapped file. On AIX, the test_mmap fails with errno=EINVAL on the m.flush() method. It works okay without the MS_INVALIDATE flag though, so I'd suggest removing it from msync in the flush and dealloc methods. Also, it would probably be a good idea to allow optional flags for flush, as it's done in new_mmap_object, the default being only MS_SYNC. Given that this module is enabled by default, I think that it needs some more stress-testing in test_mmap.py. I'll try to add a couple of tests shortly. -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252

3 3

Turning on sre?
by A.M. Kuchling 19 Jun '00

19 Jun '00

Before 1.6a3, should the sre module be turned on, replacing re? That way it'll start getting tested; better we find problems sooner rather than later... --amk

6 5

Re: [Python-Dev] CVS: python/dist/src/Modules _tkinter.c,1.99,1.100
by Michael Hudson 18 Jun '00

18 Jun '00

"Fredrik Lundh" <effbot(a)telia.com> writes: > michael wrote: > > > this patch adds a fast _flatten function to the _tkinter > > > module, and imports it from Tkinter.py (if available). > > > > Unfortunately this introduces another Way To Make Python Core: > > well, there are lots of way to dump core through Tkinter/Tk, so I > convinced myself that it wasn't really worth the effort to prevent > it in _flatten... Are there? (I hardly know Tkinter). Oh well. I don't know of many "core" methods of doing it; the only one I can think of is marshalling a recursive list, and as there seems to be a loud party going on next door to me, I may write a patch for that this evening... Cheers, M.

1 0

Re: [Python-checkins] CVS: python/dist/src/Modules _tkinter.c,1.99,1.100
by Michael Hudson 18 Jun '00

18 Jun '00

"A.M. Kuchling" <akuchling(a)users.sourceforge.net> writes: > Update of /cvsroot/python/python/dist/src/Modules > In directory slayer.i.sourceforge.net:/tmp/cvs-serv10064/Modules > > Modified Files: > _tkinter.c > Log Message: > Patch from /F: > this patch adds a fast _flatten function to the _tkinter > module, and imports it from Tkinter.py (if available). > > this speeds up canvas operations like create_line and > create_polygon. for example, a create_line with 5000 > vertices runs about 50 times faster with this patch in > place. Unfortunately this introduces another Way To Make Python Core: [mwh21@atrus build]$ ./python Python 1.6a2 (#4, Jun 18 2000, 23:57:36) [GCC 2.95.1 19990816/Linux (release)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam Copyright 1995-2000 Corporation for National Research Initiatives (CNRI) >>> import Tkinter >>> l = [] >>> l.append(l) >>> Tkinter._flatten(l) Segmentation fault (core dumped) Here's a simple solution: Index: _tkinter.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_tkinter.c,v retrieving revision 1.100 diff -u -r1.100 _tkinter.c --- _tkinter.c 2000/06/18 18:45:50 1.100 +++ _tkinter.c 2000/06/18 23:13:22 @@ -2001,13 +2001,16 @@ } static int -_flatten1(FlattenContext* context, PyObject* item) +_flatten1(FlattenContext* context, PyObject* item, int depth) { /* add tuple or list to argument tuple (recursively) */ int i, size; - if (PyList_Check(item)) { + if (depth > 1000) { + PyErr_SetString(PyExc_ValueError,"nesting too deep in _flatten"); + return 0; + } else if (PyList_Check(item)) { size = PyList_GET_SIZE(item); /* preallocate (assume no nesting) */ if (context->size + size > context->maxsize && !_bump(context, size)) @@ -2016,7 +2019,7 @@ for (i = 0; i < size; i++) { PyObject *o = PyList_GET_ITEM(item, i); if (PyList_Check(o) || PyTuple_Check(o)) { - if (!_flatten1(context, o)) + if (!_flatten1(context, o, depth + 1)) return 0; } else if (o != Py_None) { if (context->size + 1 > context->maxsize && !_bump(context, 1)) @@ -2033,7 +2036,7 @@ for (i = 0; i < size; i++) { PyObject *o = PyTuple_GET_ITEM(item, i); if (PyList_Check(o) || PyTuple_Check(o)) { - if (!_flatten1(context, o)) + if (!_flatten1(context, o, depth + 1)) return 0; } else if (o != Py_None) { if (context->size + 1 > context->maxsize && !_bump(context, 1)) @@ -2068,7 +2071,7 @@ context.size = 0; - if (!_flatten1(&context, item)) + if (!_flatten1(&context, item,0)) return NULL; if (_PyTuple_Resize(&context.tuple, context.size, 0)) "seems to work"; I've not tested it for performance, but I can't believe this is too hard a hit. Cheers, M.

2 1

Re: [Python-checkins] CVS: python/dist/src/Include unicodeobject.h,2.9,2.10
by Fredrik Lundh 18 Jun '00

18 Jun '00

mal wrote: > > #define Py_UNICODE_MATCH(string, offset, substring)\ > ! ((*((string)->str + (offset)) == *((substring)->str)) &&\ > ! !memcmp((string)->str + (offset), (substring)->str,\ > (substring)->length*sizeof(Py_UNICODE))) > I completely forgot that this has different semantics if the substring happens to be empty. (substrings are null terminated, but "\0" isn't the same thing as ""...) don't have time to screen unicodeobject.c right now, but maybe someone else can do it? if not, maybe amk's patch was a better idea after all... </F>

2 1

Towards selective compilation to native code?
by Vladimir.Marangozov＠inrialpes.fr 18 Jun '00

18 Jun '00

There was a cool seminar the other day here, focusing on embedded & real-time Java, so I'll throw here some impressions while they're fresh. One of the presenters talked extensively about compilation strategies and techniques for Java bytecode to native code, and while I know that Java is not Python, it would be interesting to, at least, try to project some of the ideas to Python in the long run, where applicable. Needless to say, the guy & his environment (ex-OSF, now Silicomp) are experts in compiler technology and I'm not in a position to comment on everything, but you may find the second half of the slides enlightening: http://cgi-serv.inrialpes.fr/InTech/2000-06-15/ejava-silicomp.ppt So, the interesting part of the talk was about a "new" compiler architecture, built upon the concept of "Flash compiling" (quite unappropriate name), which produces a mixture of the following: 1. Bytecode 2. Native code 3. Native code is compiled selectively on demand and/or ahead of time (!= JIT) for "expensive" methods. 4. Expensive methods are, on one hand, detected by dynamic profiling of the bytecode, and OTOH, determined ahead of time according to the following 2 simple creteria: - methods containing loops - methods calling native code from bytecode (to avoid bytecode to native and vice versa arg conversion paths which are expensive -- btw Python has the same perf. bottleneck) 5. Everything relies on dyn loading & late binding of native code (front-end loads dynamically the back-end, native code is stored in the code stream, but linked incrementally & dynamically during execution) 6. The whole thing is designed to avoid code explosion and to preserve the advantage of bytecode == small code size, and the presented preliminary results are quite optimistic -- the figures differ depending on the code size / speed tradeoff. Of course, this is Java and there's static typing in there, but it would be interesting to see whether Python can steal something from these ideas. For instance, I'm trying to foresee what kind of static typing (i.e. how much static info) would be appropriate in Python from this point of view but I fail to obtain any realistic image :). I see most parts of the dyn loading / late binding aspect though -- they are typical for Python. Hm. Sounds like I'll be harrassing Tim to make him finally write something valuable for Python ;-) -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252

2 1

Re: Almost insignificant patch for mmapmodule
by Andrew Kuchling 18 Jun '00

18 Jun '00

On Sat, Jun 17, 2000 at 04:29:14PM +0200, Lorenzo M. Catucci wrote: >therefore, here is the change: if we are compiling for linux, define >_GNU_SOURCE before including mman.h, and all is done. It seems the Hmm... IMHO this points out a bug; if MREMAP_MAYMOVE is not present, that doesn't imply that mremap() doesn't work at all; instead, a resize may fail if the mapping would have to be moved to a different virtual address. Quoting from the Linux man page: MREMAP_MAYMOVE indicates if the operation should fail, or change the virtual address if the resize cannot be done at the current virtual address. I think the correct fix is to simply specify MREMAP_MAYMOVE if it's defined, but still support attempting to resize. Possibly the configure script should also check for mremap()'s existence. --amk

2 1

RE: [Python-checkins] CVS: python/dist/src/Doc/lib libmmap.tex,NONE,1.1
by Mark Hammond 18 Jun '00

18 Jun '00

Sorry about that - I meant to send it direct to Andrew... Mark.

1 0