Python-Dev January 2005

python-dev@python.org

101 participants
123 discussions

Python Interpreter Thread Safety?
by Evan Jones 29 Jan '05

29 Jan '05

Due to the issue of thread safety in the Python memory allocator, I have been wondering about thread safety in the rest of the Python interpreter. I understand that the interpreter is not thread safe, but I'm not sure that I have seen a discussion of the all areas where this is an issue. Here are the areas I know of: 1. The memory allocator. 2. Reference counts. 3. The cyclic garbage collector. 4. Current interpreter state is pointed to by a single shared pointer. 5. Many modules may not be thread safe (?). Ignoring the issue of #5 for the moment, are there any other areas where this is a problem? I'm curious about how much work it would be to allow concurrent execution of Python code. Evan Jones Note: One of the reasons I am asking is that my memory allocator patch is that it changes the current allocator from "sort of" thread safe to obviously unsafe. One way to eliminate this issue is to make the allocator completely thread safe, but that would require some fairly significant changes to avoid a major performance penalty. However, if it was one of the components that permitted the interpreter to go multi-threaded, then it would be worth it.

4 7

RE: [Python-Dev] Python Interpreter Thread Safety?
by Michael Chermside 29 Jan '05

29 Jan '05

Martin v. Löwis writes: > Due to some > unfortunate historical reasons, there is code which enters free() > without holding the GIL - and that is what the allocator specifically > deals with. Except for this single case, all callers of the allocator > are required to hold the GIL. Donovan Baarda writes: > Just curious; is that "one case" a bug that needs fixing, or is the some > reason this case can't be changed to use the GIL? Surely making it > mandatory for all free() calls to hold the GIL is easier than making the > allocator deal with the one case where this isn't done. What Martin is trying to say here is that it _IS_ mandatory to hold the GIL when calling free(). However, there is some very old code in existance (written by other people) which calls free() without holding the GIL. We work very hard to provide backward compatibility, so we are jumping through hoops to ensure that even this old code which is violating the rules doesn't get broken. -- Michael Chermside

1 0

Patch review: [ 1009811 ] Add missing types to __builtin__
by Alan Green 29 Jan '05

29 Jan '05

Last August, James Knight posted to python-dev, "There's a fair number of classes that claim they are defined in __builtin__, but do not actually appear there". There was a discussion and James submitted this patch: http://sourceforge.net/tracker/index.php?func=detail&aid=1009811&group_id=5… The final result of the discussion is unclear. Guido declared himself +0.5 on the concept, but nobody has reviewed the patch in detail yet. The original email thread starts here: http://mail.python.org/pipermail/python-dev/2004-August/047477.html The patch still applies, and test cases still run OK afterwards. Now that 2.4 has been released it is perhaps a good time to discuss in on python-dev again. If it isn't discussed, then the patch should be closed due to lack of interest. Alan. -- Alan Green alan.green(a)cardboard.nu - http://cardboard.nu

6 10

Let's get rid of unbound methods
by Guido van Rossum 28 Jan '05

28 Jan '05

In my blog I wrote: Let's get rid of unbound methods. When class C defines a method f, C.f should just return the function object, not an unbound method that behaves almost, but not quite, the same as that function object. The extra type checking on the first argument that unbound methods are supposed to provide is not useful in practice (I can't remember that it ever caught a bug in my code) and sometimes you have to work around it; it complicates function attribute access; and the overloading of unbound and bound methods on the same object type is confusing. Also, the type checking offered is wrong, because it checks for subclassing rather than for duck typing. This is a really simple change to begin with: *** funcobject.c 28 Oct 2004 16:32:00 -0000 2.67 --- funcobject.c 4 Jan 2005 18:23:42 -0000 *************** *** 564,571 **** static PyObject * func_descr_get(PyObject *func, PyObject *obj, PyObject *type) { ! if (obj == Py_None) ! obj = NULL; return PyMethod_New(func, obj, type); } --- 564,573 ---- static PyObject * func_descr_get(PyObject *func, PyObject *obj, PyObject *type) { ! if (obj == NULL || obj == Py_None) { ! Py_INCREF(func); ! return func; ! } return PyMethod_New(func, obj, type); } There are some test suite failures but I suspect they all have to do with checking this behavior. Of course, more changes would be needed: docs, the test suite, and some simplifications to the instance method object implementation in classobject.c. Does anyone think this is a bad idea? Anyone want to run with it? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

21 52

Getting rid of unbound methods: patch available
by Guido van Rossum 28 Jan '05

28 Jan '05

https://sourceforge.net/tracker/index.php?func=detail&aid=1103689&group_id=… Here's a patch that gets rid of unbound methods, as discussed here before. A function's __get__ method now returns the function unchanged when called without an instance, instead of returning an unbound method object. I couldn't remove support for unbound methods completely, since they were used by the built-in exceptions. (We can get rid of that use once we convert to new-style exceptions.) For backward compatibility, functions now have read-only im_self and im_func attributes; im_self is always None, im_func is always the function itself. (These should issue warnings, but I haven't added that yet.) The test suite passes. (I have only tried "make test" on a Linux box.) What do people think? (My main motivation for this, as stated before, is that it adds complexity without much benefit.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

11 24

Re: [PyCon] Reg: Registration
by Steve Holden 28 Jan '05

28 Jan '05

Aahz, writing as pycon(a)python.org, wrote: > It's still January 28 here -- register now! I don't know if we'll be > able to extend the registration price beyond that. Just in case anybody else might be wondering when the early bird registration deadline is, I've asked the registration team to allow the early bird price as long as it's January 28th somewhere in the world. There have been rumors that Guido will not be attending PyCon this year. I am happy to scotch them by pointing out that Guido van Rossum's keynote address will be on its traditional Thursday morning. I look forward to joining you all to hear Guido speak on "The State of Python". regards Steve -- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/ Holden Web LLC +1 703 861 4237 +1 800 494 3119

1 0

Re: [Python-checkins] python/dist/src/Lib/idlelib EditorWindow.py, 1.65, 1.66 NEWS.txt, 1.53, 1.54 config-keys.def, 1.21, 1.22 configHandler.py, 1.37, 1.38
by Guido van Rossum 28 Jan '05

28 Jan '05

Thanks!!! On Thu, 27 Jan 2005 16:16:19 -0800, kbk(a)users.sourceforge.net <kbk(a)users.sourceforge.net> wrote: > Update of /cvsroot/python/python/dist/src/Lib/idlelib > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5316 > > Modified Files: > EditorWindow.py NEWS.txt config-keys.def configHandler.py > Log Message: > Add keybindings for del-word-left and del-word-right. > > M EditorWindow.py > M NEWS.txt > M config-keys.def > M configHandler.py > > Index: EditorWindow.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/EditorWindow.py,v > retrieving revision 1.65 > retrieving revision 1.66 > diff -u -d -r1.65 -r1.66 > --- EditorWindow.py 19 Jan 2005 00:22:54 -0000 1.65 > +++ EditorWindow.py 28 Jan 2005 00:16:15 -0000 1.66 > @@ -141,6 +141,8 @@ > text.bind("<<change-indentwidth>>",self.change_indentwidth_event) > text.bind("<Left>", self.move_at_edge_if_selection(0)) > text.bind("<Right>", self.move_at_edge_if_selection(1)) > + text.bind("<<del-word-left>>", self.del_word_left) > + text.bind("<<del-word-right>>", self.del_word_right) > > if flist: > flist.inversedict[self] = key > @@ -386,6 +388,14 @@ > pass > return move_at_edge > > + def del_word_left(self, event): > + self.text.event_generate('<Meta-Delete>') > + return "break" > + > + def del_word_right(self, event): > + self.text.event_generate('<Meta-d>') > + return "break" > + > def find_event(self, event): > SearchDialog.find(self.text) > return "break" > > Index: NEWS.txt > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/NEWS.txt,v > retrieving revision 1.53 > retrieving revision 1.54 > diff -u -d -r1.53 -r1.54 > --- NEWS.txt 19 Jan 2005 00:22:57 -0000 1.53 > +++ NEWS.txt 28 Jan 2005 00:16:16 -0000 1.54 > @@ -3,17 +3,24 @@ > > *Release date: XX-XXX-2005* > > +- Add keybindings for del-word-left and del-word-right. > + > - Discourage using an indent width other than 8 when using tabs to indent > Python code. > > - Restore use of EditorWindow.set_indentation_params(), was dead code since > - Autoindent was merged into EditorWindow. > + Autoindent was merged into EditorWindow. This allows IDLE to conform to the > + indentation width of a loaded file. (But it still will not switch to tabs > + even if the file uses tabs.) Any change in indent width is local to that > + window. > > - Add Tabnanny check before Run/F5, not just when Checking module. > > - If an extension can't be loaded, print warning and skip it instead of > erroring out. > > +- Improve error handling when .idlerc can't be created (warn and exit). > + > - The GUI was hanging if the shell window was closed while a raw_input() > was pending. Restored the quit() of the readline() mainloop(). > http://mail.python.org/pipermail/idle-dev/2004-December/002307.html > > Index: config-keys.def > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/config-keys.def,v > retrieving revision 1.21 > retrieving revision 1.22 > diff -u -d -r1.21 -r1.22 > --- config-keys.def 17 Aug 2004 08:01:19 -0000 1.21 > +++ config-keys.def 28 Jan 2005 00:16:16 -0000 1.22 > @@ -55,6 +55,8 @@ > untabify-region=<Alt-Key-6> <Meta-Key-6> > toggle-tabs=<Alt-Key-t> <Meta-Key-t> <Alt-Key-T> > change-indentwidth=<Alt-Key-u> <Meta-Key-u> <Alt-Key-U> > +del-word-left=<Control-Key-BackSpace> > +del-word-right=<Control-Key-Delete> > > [IDLE Classic Unix] > copy=<Alt-Key-w> <Meta-Key-w> > @@ -104,6 +106,8 @@ > untabify-region=<Alt-Key-6> > toggle-tabs=<Alt-Key-t> > change-indentwidth=<Alt-Key-u> > +del-word-left=<Alt-Key-BackSpace> > +del-word-right=<Alt-Key-d> > > [IDLE Classic Mac] > copy=<Command-Key-c> > @@ -153,3 +157,5 @@ > untabify-region=<Control-Key-6> > toggle-tabs=<Control-Key-t> > change-indentwidth=<Control-Key-u> > +del-word-left=<Control-Key-BackSpace> > +del-word-right=<Control-Key-Delete> > > Index: configHandler.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/configHandler.py,v > retrieving revision 1.37 > retrieving revision 1.38 > diff -u -d -r1.37 -r1.38 > --- configHandler.py 13 Jan 2005 17:37:38 -0000 1.37 > +++ configHandler.py 28 Jan 2005 00:16:16 -0000 1.38 > @@ -579,7 +579,9 @@ > '<<tabify-region>>': ['<Alt-Key-5>'], > '<<untabify-region>>': ['<Alt-Key-6>'], > '<<toggle-tabs>>': ['<Alt-Key-t>'], > - '<<change-indentwidth>>': ['<Alt-Key-u>'] > + '<<change-indentwidth>>': ['<Alt-Key-u>'], > + '<<del-word-left>>': ['<Control-Key-BackSpace>'], > + '<<del-word-right>>': ['<Control-Key-Delete>'] > } > if keySetName: > for event in keyBindings.keys(): > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > -- --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

Patch review: [ 1100942 ] datetime.strptime constructor added
by Alan Green 27 Jan '05

27 Jan '05

I see a need for this patch - I've had to write "datetime(*(time.strptime(date_string, format)[0:6]))" far too many times. I don't understand the C API well enough to check if reference counts are handled properly, but otherwise the implementation looks straight forward. Documentation looks good and the test passes on my machine. Two suggestions: 1. In the time module, the strptime() function's format parameter is optional. For consistency's sake, I'd expect datetime.strptime()'s format parameter also to be optional. (On the other hand, the default value for the format is not very useful.) 2. Since strftime is supported by datetime.time, datetime.date and datetime.datetime, I'd also expect strptime to be supported by all three classes. Could you add that now, or would it be better to do it as a separate patch? Alan. -- Alan Green alan.green(a)cardboard.nu - http://cardboard.nu

2 1

Strange segfault in Python threads and linux kernel 2.6
by Donovan Baarda 27 Jan '05

27 Jan '05

G'day, I've Cc'ed this to zope-coders as it might affect other Zope developers and it had me stumped for ages. I couldn't find anything on it anywhere, so I figured it would be good to get something into google :-). We are developing a Zope2.7 application on Debian GNU/Linux that is using fop to generate pdf's from xml-fo data. fop is a java thing, and we are using popen2.Popen3(), non-blocking mode, and select loop to write/read stdin/stdout/stderr. This was all working fine. Then over the Christmas chaos, various things on my development system were apt-get updated, and I noticed that java/fop had started segfaulting. I tried running fop with the exact same input data from the command line; it worked. I wrote a python script that invoked fop in exactly the same way as we were invoking it inside zope; it worked. It only segfaulted when invoked inside Zope. I googled and tried everything... switched from j2re1.4 to kaffe, rolled back to a previous version of python, re-built Zope, upgraded Zope from 2.7.2 to 2.7.4, nothing helped. Then I went back from a linux 2.6.8 kernel to a 2.4.27 kernel; it worked! After googling around, I found references to recent attempts to resolve some signal handling problems in Python threads. There was one post that mentioned subtle differences between how Linux 2.4 and Linux 2.6 did signals to threads. So it seems this is a problem with Python threads and Linux kernel 2.6. The attached program demonstrates that it has nothing to do with Zope. Using it to run "fop-test /usr/bin/fop </dev/null" on a Debian box with fop installed will show the segfault. Running the same thing on a machine with 2.4 kernel will instead get the fop "usage" message. It is not a generic fop/java problem with 2.6 because the commented un-threaded line works fine. It doesn't seem to segfault for any command... "cat -" works OK, so it must be something about java contributing. After searching the Python bugs, the closest I could find was #971213 <http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=9…>. Is this the same bug? Should I submit a new bug report? Is there any other way I can help resolve this? BTW, built in file objects really could use better non-blocking support... I've got a half-drafted PEP for it... anyone interested in it? -- Donovan Baarda <abo(a)minkirri.apana.org.au> http://minkirri.apana.org.au/~abo/

3 9

I think setup.py needs major rework
by Skip Montanaro 26 Jan '05

26 Jan '05

I just submitted a bug report for setup.py: http://python.org/sf/1109602 It begins: Python's setup.py has grown way out of control. I'm trying to build and install Python 2.4.0 on a Solaris system with Tcl/Tk installed in a non-standard place and I can't figure out the incantation to tell setup.py to look where they are installed. ... and ends: This might be an excellent sprint topic for PyCon. Skip

4 3

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev January 2005