I just started looking over the GIL pep
http://www.python.org/peps/pep-0311.html. Looks pretty nice to me so
far! I have these remarks:
"This API will not perform automatic initialization of Python, or
initialize Python for multi-threaded operation. Extension authors
must continue to call Py_Initialize(), and for multi-threaded
This is liable to confuse people (like me) who have never had to do
Py_Initialize or PyEval_InitThreads in their extension modules.
"It is intended that this API be all that is necessary to acquire
the Python GIL. Apart from the existing, standard
Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros, it is
assumed that no additional thread state API functions will be used
by the extension. Extensions with such complicated requirements
are free to continue to use the existing thread state API."
This sounds like it's saying you can't use any of the old thread API
functions if you use the new ones. Is that right? If so, I think it
should be made more explicit. Seems a bit draconian, though I admit I
don't know what other thread API functions people might want.
Good job, Mark!
> Need to make sure that preprocessor directives start in first column.
> This means we can't indent code which has preprocessor directives,
> nor have a space between [ #include for example.
What does the C standard say about this? I'm curious.
I notice that test_re is failing with a recursion depth limit error on Mac
OS X, but only when run under regrtest.py. Running plain works:
% ./python.exe ../Lib/test/test_re.py
Running re_tests test suite
Running from regrtest fails:
% ./python.exe ../Lib/test/regrtest.py test_re
test test_re produced unexpected output:
*** line 2 of expected output missing:
- maximum recursion limit exceeded
1 test failed:
Cranking the stack limit up as high as 32MB didn't help, though I don't
suppose it should have, given the error message. Bumping the recursion
limit up as high as 20000 didn't help either.
Anybody else seen this?
In message: <15963.57541.244513.169979(a)montanaro.dyndns.org>
Skip Montanaro <skip(a)pobox.com> writes:
>It's probably somewhat invalid to equate number of system calls with
>application runtime. I redumped my last ktrace file just now with
>timestamps. Here are some computed intervals:
> interval time
> -------- ----
> start -> open hammiefilter.pyc 0.071
> open hammiefilter.pyc -> open hammie.db 0.516
> open hammie.db -> close hammie.db 0.084
> close hammie.db -> program end 0.011
This is good info. Can you add in the time intervals between
loading each of the modules? That might point out which modules
are actually expensive (or if it's none in particular).
Executive summary for python-dev folks seeing this for the first time:
This thread started at
Running in a single interpreter loop, I can score roughly 46
messages per second. Running from the shell using hammiefilter.py
(which takes a msg on stdin and spits a scored message to stdout)
performance drops to roughly 2 messages per second. Neil
Schmenenauer noted all the failed open() calls during import
lookup, which got me started trying to whittle them down.
Two more things to try before abandoning this quixotic adventure...
It appears $prefix/python23.zip is left in sys.path even if it doesn't exist
(Just van Rossum explained to me in a bug report I filed that nonexistent
directories might actually be URLs or other weird hacks which import hooks
could make use of), so I went with the flow and created it, populating it
with the contents of $prefix/python2.3. My averate wallclock time went from
0.5 seconds to 0.47 seconds and user+sys times went from 0.43 seconds to
0.41 seconds. A modest improvement.
One more little tweak. I moved the lib-dynload directory to the front of
sys.path (obviously only safe if nothing there appears earlier in sys.path).
Wall clock average stayed at 0.47 seconds and user+sys at 0.41 seconds,
though the total number of system calls as measured by ktrace went from 3454
Hammiefilter itself really does very little. Looking at the last
ktrace/kdump output, I see 3042 system calls. The hammie.db file isn't
opened until line 2717. All the rest before that is startup stuff, the
largest chunk of which are nami operations (731) and open (557) calls, most
of them involving nonexistent files (as evidenced by seeing only 164 calls
to close()). In contrast, only 278 system calls appear to be directly
related to manipulating the hammie database.
This is still somewhat off-topic for this list (except for the fact that my
intention was to get hammiefilter to run faster), so I'll cc python-dev to
keep Tim happy, and perhaps mildly irritate Guido by discussing specific
apps on python-dev.
Baymountain, our host for mail.python.org (and mail.zope.org), will be
performing some short duration maintenance on the machine on Wednesday
26-Feb-2003 at 10pm EST. Expected downtime is about 10 minutes.
I think Modules/bsddbmodule.c needs to be moved to something like
Modules/bsddb185.c or bsddb185module.c otherwise it won't get found by the
import process. What's the best way to make that change in CVS?
I am trying to create a mutable object for a persistent object
database, and I've ran into a problem. The semantics are:
Object gets created as a 'Swizzle'. Swizzle is a stub that
intercepts any attempt to access object properties. When code
tries to access any of swizzle's properties, the swizzle should
automatically mutate into the stored object, by changing its
class and properties.
With the __set/get/del/attr__ traps this seemed feasible in
Python. But it turns out to be trickier than I've thought, my
naive implementation is not working.
What happens is:
swizzle.__getattr__() gets called
swizzle.__class__ = stored_class
swizzle.__dict__ = stored_data
calls getattr(self, attr_name)
calls swizzle.__getattr__() gets called
and I enter an infinite loop and blow the stack
Since swizzle's __class__ and __dict__ have changed, shouldn't
getattr use the new class to get attributes?
Why this is puzzling is that if I call swizzle.load() directly
(bypassing __getattr__ trap), and then try referencing swizzle's
fields, everything works. The object mutates successfully, and
__getattr__ never gets called again.
Any hints on what might be going wrong? Hopefully others have
struggled with the same problem before me.
Details of Python's class implementation and method dispatch
would also be interesting. I've read Guido's Unifying Types
essay, but I think I lack Python history required for its full
I've tried many different approaches:
- Swizzle inheriting from object (so that I can call super
methods directly). This one would not let me assign into the
- Swizzle calling different combinations of
Here is my current code.
Thanks in advance,
""" A swizzle is a placeholder for an object that has not
been loaded. It mutates into the loaded object upon access
def __init__(self, oid):
def __getattr__(self, name):
print "swizzle getattr called"
# oid is the only attribute that does not cause a load
if (name == 'oid'):
def __setattr__(self, name, value):
""" setattr passes class & dict because these are called
when we are mutating the object """
print "swizzle setattr called"
if (name == '__class__' or name == '__dict__' or name ==
self.__dict__[name] = value
return setattr(self,name, value)
def __delattr__(self, name):
print "swizzle delattr called"
return delattr(self, name)
""" TODO LOCK"""
print "Trying to load ", self.oid
anItem.__dict__ = getProperties(anItem.oid)
anItem.__class__ = getClass(anItem.oid)
While trying to figure out why so many stat() calls fail before locating a
module, it occurred to me that since most modules are written in Python it
makes sense to consider ".py" files before ".so" and "module.so" files.
Would changing the search order be possible or would it break something?
Can anybody explain the point of the iconv_codec module to me? I see
no docs for it, there's a continued stream of bug reports related to
it. I've heard that it's a wrapper around some 3rd party library, but
that doesn't help me -- I don't know anything about that 3rd party
library module either.
--Guido van Rossum (home page: http://www.python.org/~guido/)