Mailman 3 January 2003 - Python-Dev

Re: [Python-Dev] Extension modules, Threading, and the GIL
by James Henstridge 26 Jan '03

26 Jan '03

I originally sent this as private mail to David, but he suggested I repost it here as well. The Python GIL has been causing trouble getting threading to work correctly in my Python bindings for GTK 2.x. Both Python and GTK have an idea about how threading should work, and meshing the two together isn't quite successful. It would be great if Python made this kind of thing easier. James. -- Email: james(a)daa.com.au WWW: http://www.daa.com.au/~james/

3 2

Weekly Python Bug/Patch Summary
by Skip Montanaro 26 Jan '03

26 Jan '03

Bug/Patch Summary ----------------- 336 open / 3246 total bugs (-1) 109 open / 1923 total patches (+2) New Bugs -------- StringIO doc doesn't say it's sometimes read-only (2003-01-20) http://python.org/sf/671447 time module: time tuple not returned by certain functions (2003-01-21) http://python.org/sf/671731 Py_Main() does not perform to spec (2003-01-21) http://python.org/sf/672035 Three __contains__ implementations (2003-01-21) http://python.org/sf/672098 Assignment to __bases__ of direct object subclasses (2003-01-21) http://python.org/sf/672115 registry functions don't handle null characters (2003-01-21) http://python.org/sf/672132 2.3a1 computes lastindex incorrectly (2003-01-22) http://python.org/sf/672491 python -S still displays 'Type "help", "cop...' (2003-01-22) http://python.org/sf/672614 setting socket timeout crashes SSL? (2003-01-23) http://python.org/sf/673797 re.compile fails for verbose re with more than one group (2003-01-24) http://python.org/sf/674264 Access to serial devices through Carbon.CF (2003-01-25) http://python.org/sf/674574 New Patches ----------- Compile _randommodule.c and datetimemodule.c under cygwin (2003-01-20) http://python.org/sf/671176 test_pty hanging on hpux11 (2003-01-20) http://python.org/sf/671384 Optimize dictionary resizing (2003-01-20) http://python.org/sf/671454 Py_Main() removal of exit() calls. Return value instead (2003-01-21) http://python.org/sf/672053 securing pydoc server (2003-01-22) http://python.org/sf/672656 improve pydoc handling of extension types (2003-01-22) http://python.org/sf/672855 Expand dbshelve to have full shelve/dict interface (2003-01-24) http://python.org/sf/674396 test_htmlparser.py -- "," in attributes (2003-01-24) http://python.org/sf/674448 Closed Bugs ----------- test_unicode fails in wide unicode build (2002-05-11) http://python.org/sf/554916 inspect.getsource shows incorrect output (2002-08-14) http://python.org/sf/595018 logging package undocumented (2002-11-13) http://python.org/sf/637847 py_compile does not return error code (2002-12-13) http://python.org/sf/653301 typeobject provides incorrect __mul__ (2002-12-30) http://python.org/sf/660144 datetimetz constructors behave counterintuitively (2.3a1) (2003-01-01) http://python.org/sf/660872 test_httplib fails on the mac (2003-01-02) http://python.org/sf/661340 test_signal hang on some Solaris boxes (2003-01-05) http://python.org/sf/662787 test_bsddb3 fails when run directly (2003-01-08) http://python.org/sf/664581 test_ossaudiodev fails to run (2003-01-08) http://python.org/sf/664584 Py_NewInterpreter() doesn't work (2003-01-15) http://python.org/sf/668708 socket.inet_aton() succeeds on invalid input (2003-01-17) http://python.org/sf/669859 Closed Patches -------------- Use builtin boolean if present (2002-05-22) http://python.org/sf/559288 Better inspect.BlockFinder fixes bug (2002-11-06) http://python.org/sf/634557 general corrections to 2.2.2 refman, p.1 (2002-11-07) http://python.org/sf/634866 Filter unicode into unicode (2002-11-09) http://python.org/sf/636005 LaTeX documentation for logging package (2002-11-14) http://python.org/sf/638299 logging SysLogHandler proto type wrong (2002-11-23) http://python.org/sf/642974 fix memory (ref) leaks (2003-01-16) http://python.org/sf/669553 Patched test harness for logging (2003-01-18) http://python.org/sf/670390

1 0

xmlparse.c no longer compiles on Windows
by Tim Peters 26 Jan '03

26 Jan '03

It starts with this: /* Handle the case where memmove() doesn't exist. */ #ifndef HAVE_MEMMOVE #ifdef HAVE_BCOPY #define memmove(d,s,l) bcopy((s),(d),(l)) #else #error memmove does not exist on this platform, nor is a substitute available #endif /* HAVE_BCOPY */ #endif /* HAVE_MEMMOVE */ memmove() is a std C function, and Python requires std C, so there's no point to this block. Neither of the symbols are defined on Windows, and I don't want to define them -- other parts of Python use memmove() freely (like listobject.c). If that's ripped out, there are lots of syntax errors in xmlparse.c (over 100, at which point the compiler gives up on it). They have the character of cascading msgs due to some macro confusion; the first one is here: typedef struct prefix { const XML_Char *name; BINDING *binding; } PREFIX; The error msg points to this line and gives the unhepful msg C:\Code\python\Modules\expat\xmlparse.c(142) : error C2059: syntax error : 'string' I assume this is because PREFIX is a macro used by Modules/getpath.c, and PC/pyconfig.h defines it like so: #define PREFIX ""

3 3

ann: Minimal PyPython sprint
by holger krekel 25 Jan '03

25 Jan '03

------------------------------------------------------ Not-So-Mini Sprint towards Minimal PyPython 17th-23rd Feb. 2003 "Trillke-Gut", Hildesheim, Germany ------------------------------------------------------ Everybody is invited to join our first sprint for PyPython. Or was it Minimal Python? Nevermind! We will have one or - if needed - two big rooms, beamers for presentations (and movies?), a kitchen, internet and a piano. There is a big park and some forest in case you need some fresh air. Short Ad-Hoc presentations about our areas of interest, projects or plain code will certainly be appreciated. ------------------------------------------------------ Goals of the first Minimal PyPython Marathon ------------------------------------------------------ - codify ideas that were recently discussed on the pypy-dev codespeak list. - port your favorite C-module to Python (and maintain it :-) - rebuild CPython C-implementations in pure python, e.g. the interpreter, import mechanism, builtins ... - build & enhance infrastructure (python build system, webapps, email, subversion/cvs, automated testing, ...) - further explore the 'ctypes' approach from Thomas Heller to perform C/Machine-level calls from Python without extra C-bindings. - settle on concepts - focus on having releasable results at the end - have a lot of fun meeting like minded people. ------------------------------------------------------ Current Underlying Agreements (TM) ------------------------------------------------------ Please note, that we have reached some agreement on a number of basic points: a) the Python language is not to be modified. b) simplicity wins. especially against optimization. c) if in doubt we follow the CPython/PEP lead. d) CPython core abstractions are to be cleanly formulated in simple Python. e) Macro-techniques may be used at some levels if there really is no good way to avoid them. But a) to c) still hold. f) our Pythonic core "specification" is intended to be used for all kinds of generational steps e.g to C, Bytecode, Objective-C, assembler and last-but-not-least PSYCO-Specialization. g) if in doubt we follow Armins and Christians lead regarding the right abstraction levels. And any other python(dev) people taking responsibility. h) there are a lot of wishes, code, thoughts, suggestions and ... names :-) ------------------------------------------------------ How to proceed if you are interested ------------------------------------------------------ If you want to come - even part-time - then please subscribe at http://codespeak.net/mailman/listinfo/pypy-sprint where organisational stuff will be communicated. The sprint-location's city is called Hildesheim which is close to Hannover (200 km south of Hamburg). Code-related discussions continue to take place on the pypy-dev list. For people who don't want or can't spend the money for external rooms i can probably arrange something but it will not be luxurious. FYI there are already 7-8 people who will come among them Christian Tismer, Armin Rigo and Michael Hudson. ------------------------------------------------------ Disclaimer ------------------------------------------------------ There is no explicit commercial intention behind the organisation of the sprint (yet). On the implicit hand socializing with like-minded people tends to bring up future opportunities. The physical sprint location is provided by the people living in this house: http://www.trillke.net/images/trillke_schnee.png although snow is gone for the moment. and-who-said-the-announcement-would-be-minimal'ly y'rs, holger

1 0

logging package -- spurious package contents
by Guido van Rossum 25 Jan '03

25 Jan '03

I have another issue with the logging package. The submodule logging/config.py contains code that I feel should not be there. I experimented with the config file format it implements, and it appears very painful. It appears mostly undocumented, typos in the file are not always reported, and you seem to have to specify the filename and mode twice. (Example: [handler_normal] class=FileHandler level=NOTSET formatter=common args=('z3.log', 'a') <----\ filename=z3.log <---- >- this is ugly mode=a <----/ .) Since configuring the logging package with a few programmatic calls is so easy, and applications that need serious logging configurability typically already have some configuration mechanism, I propose to drop this from the Python distribution. I'm similarly not convinced of the utility of the logging/handlers.py submodule, but I've never tried to use it, so lacking any particular negative experience all I can say against it is YAGNI. Does anyone on python-dev think the logging package would lose utility if I removed those submodules? --Guido van Rossum (home page: http://www.python.org/~guido/)

5 8

Mixed-type datetime comparisons
by Tim Peters 25 Jan '03

25 Jan '03

I checked in changes so that datetime and date comparison return NotImplemented (instead of raising TypeError) if "the other" argument has a timetuple attribute. This gives other kinds of datetime objects a chance to intercept the comparison and implement it themselves. Note that this doesn't help for mixed-type time or timedelta comparison: datetime's time and timedelta objects don't have timetuple methods themselves, and their comparison implementations still raise TypeError whenever they don't recognize the other comparand's type (this is needed to prevent comparison against objects of arbitrary types from falling back to the default comparison of object addresses).

2 4

Looking for a Windows bsddb fan
by Tim Peters 25 Jan '03

25 Jan '03

Barry said I should switch Python 2.3 on Windows to using Sleepycat 4.1.25. A patch for doing so is attached. The full test_bsddb3 looks like a disaster then, and I don't have time to dig into it: C:\Code\python\PCbuild>python ../lib/test/regrtest.py -v -u bsddb test_bsddb3 test_bsddb3 test01_associateWithDB (bsddb.test.test_associate.AssociateHashTestCase) ... ERROR test02_associateAfterDB (bsddb.test.test_associate.AssociateHashTestCase) ... ERROR test01_associateWithDB (bsddb.test.test_associate.AssociateBTreeTestCase) ... ERROR test02_associateAfterDB (bsddb.test.test_associate.AssociateBTreeTestCase) ... ERROR test01_associateWithDB (bsddb.test.test_associate.AssociateRecnoTestCase) ... ERROR test02_associateAfterDB (bsddb.test.test_associate.AssociateRecnoTestCase) ... ERROR test01_associateWithDB (bsddb.test.test_associate.ShelveAssociateHashTestCase) ... ERROR etc. It's been running a long time, and unittest doesn't show what's wrong before it's all done (and it's not done yet, so no clue here). Maybe it's shallow. Plain test_bsddb passes. Later: Ran 182 tests in 588.640s FAILED (errors=22) test test_bsddb3 failed -- errors occurred; run in verbose mode for details 1 test failed: test_bsddb3 All the errors are of this form (this is the last one): ====================================================================== ERROR: test01_basics (bsddb.test.test_dbshelve.EnvThreadHashShelveTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\CODE\PYTHON\lib\bsddb\test\test_dbshelve.py", line 70, in test01_basics self.do_open() File "C:\CODE\PYTHON\lib\bsddb\test\test_dbshelve.py", line 233, in do_open self.env.open(homeDir, self.envflags | db.DB_INIT_MPOOL | db.DB_CREATE) DBAgainError: (11, 'Resource temporarily unavailable -- unable to join the environment') Huh? Win98SE, in case that's a clue.

4 7

the new 2.3a1 settimeout() with httplib and SSL
by Geoffrey Talvola 25 Jan '03

25 Jan '03

I'm trying to get the new socket.settimeout() in Python 2.3a1 to work in conjunction with httplib and SSL. This code seems to work fine: import httplib conn = httplib.HTTPConnection('ncsdevtest.nameconnector.com', 80) conn.connect() conn.sock.settimeout(90) conn.request('GET', '/cgi-bin/Pause30.cgi') response = conn.getresponse() print response.status, response.reason data = response.read() print 'read', len(data), 'bytes' conn.close() Where Pause30.cgi is a cgi script that simply sleeps for 30 seconds then sends back a simple response. As-is, this program returns after 30 seconds. If I adjust the timeout of 90 to be, lets say, 5 seconds, I correctly get a timeout exception after 5 seconds. So far, so good. But if I change HTTPConnection to HTTPSConnection and change 80 to 443, I have trouble -- my CPU usage goes up to 100%, the python process sucks up more and more memory, and it doesn't time out at all. It does still returns the correct response after 30 seconds. Is there a way to do this? Should I enter a bug report? - Geoff

2 1

Re: Extended Function syntax
by Manuel Garcia 24 Jan '03

24 Jan '03

I always considered stat_m = staticmethod(stat_m) class_m = classmethod(class_m) to be less about an awkward notation, and more about the standard Python idiom for one technique to keep namespaces clean of unused identifiers. Like this: i = int(i) # i held a string, now holds an int Instead of: i = int(i_string) # will never use i_string again With the benefit of clean namespaces being less identifiers to trip over when using any tool for introspection, preventing accidental references to identifiers we really wanted to be temporary, etc. In my mind, staticmethod(), classmethod(), and property() are the same, they take functions and "package" them in a way suitable for implementing methods or properties. Since everyone loves functional programming :-/ , functions that act on functions are nothing to be ashamed of, and don't deserve to be hidden behind syntactical sugar. (Things like stat_m = staticmethod(stat_m) is one reason of why C++ and Java people consider Python "not really object oriented", as if throwing keywords and keyword phrases at every damn OO construct is the best way to be object oriented. Yes, syntactical sugar like def name() [staticmethod]: helps Python gain more respect with the C++/Java crowd.) The most explicit way of using staticmethod(), classmethod(), property() forces you to take extra steps to keep your namespace clean: class klass01(object): def _pure_set_x(x): klass01.x = x static_set_x = staticmethod(_pure_set_x) def _pure_get_x(): return klass01.x static_get_x = staticmethod(_pure_get_x) def _pure_set_y(cls, y): cls.y = y class_set_y = classmethod(_pure_set_y) def _pure_get_y(cls): return cls.y class_get_y = classmethod(_pure_get_y) del _pure_set_x, _pure_get_x, _pure_set_y, _pure_get_y def _get_j(self): return self._j def _set_j(self, j): self._j = j j = property(_get_j, _set_j, None, 'dynamite!') del _get_j, _set_j (<sheepishly> This argument is more about property() than staticmethod(), classmethod(). I have to admit "stat_m = staticmethod(stat_m)" doesn't really bother me very much. I have a little more to say about staticmethod below.) I always wanted some syntactical sugar for this idiom: def _junk(): a = long_expression01(x0, y0) b = long_expression02(x1, y1) c = long_expression03(x2, y2) return min(a,b,c), max(a,b,c) d, e = _junk() del _junk() with the purpose being keeping a,b,c out of our nice clean namespace. The notation would be: d, e = block: a = long_expression01(x0, y0) b = long_expression02(x1, y1) c = long_expression03(x2, y2) return min(a,b,c), max(a,b,c) We would set up a property like this (if there is no benefit to having _get_j and _set_j in the namespace): j = block: def _get_j(self): return self._j def _set_j(self, j): self._j = j return property(_get_j, _set_j, None, 'dynamite!') I would like this way to define properties. Everything is for "j" is kept together, with the temporary names that make everything clear disappearing when we leave the block. Considering the most general form of the proposed extended function syntax: def name(arg, ...) [expr1, expr2, expr3]: ...body... we would have instead: name = block: def _(arg, ...): ...body... return expr3(expr2(expr1(_))) It takes 2 more lines, but you have the benefit of not having to explain that [expr1, expr2, expr3] means expr3(expr2(expr1(name))) Also less likely to need a lambda, etc. I admit this is almost no help at all for staticmethod(), classmethod() except perhaps in this case: stat1, stat2, stat3 = block: def pure1(): ...body... def pure2(): ...body... def pure3(): ...body... return staticmethod( (pure1, pure2, pure3) ) where staticmethod() now also accepts a tuple of functions. Contrived, but if you had more than one staticmethod, they would probably be grouped together anyway. We only saved 1 lousy line for 3 staticmethods. However, we have made the idea of turning pure functions into staticmethods explicit. I admit we have gained very little. We gain nothing here (except dubious explicitness): stat1 = block: def pure1(): ...body... return staticmethod(pure1) instead of: def stat1(): ...body... stat1 = staticmethod(stat1) or: def stat1() [staticmethod]: ...body... Some mention was made of classes for a similar extended syntax: klass1 = block: class klass1(): ...body... return I02(I01(klass1)) Obvious problems: All this stinks of trying to come up with a poor substitute of "super-lambda" (a way to make an anonymous function allowing multiple lines and statements). I don't deny it. The main problem is that on the right hand side of the equals sign is something that is not an expression. We could solve that problem with this abomination: block: global d, e a = long_expression01(x0, y0) b = long_expression02(x1, y1) c = long_expression03(x2, y2) d,e = min(a,b,c), max(a,b,c) I associate use of 'global' in my programs with a design flaw, I would not defend this. Finishing up: block: ...body... when you don't care about the return value. g = block: ...body... yield x Would be the same as: def g(): ...body... yield x but probably should raise an error because it doesn't match what happens with 'return', not even matching in spirit.

1 0

RE: [Python-Dev] Proto-PEP regarding writing bytecode files
by Moore, Paul 24 Jan '03

24 Jan '03

From: Skip Montanaro [mailto:skip@pobox.com] > At any rate, I believe I incorporated everybody's suggestions > (thanks for the feedback) and checked it in as pop-0304 (the > number David Goodger assigned me). You can now grab it from > CVS or view it online at > > http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/nondist/peps/p… > > More feedback is welcome. Once the uproar^H^H^H^H^H^Hfeedback > dies down a bit, I will look at implementing the idea. Reading this, it seems to me that this could probably be implemented by an import hook. Just's plan is to expose the (currently internal) filesystem module loading code as a hook for 2.3a2. If you discuss this with him, I'd imagine that it would be possible to a) Ensure that however it gets exposed, it has enough customisability to allow you to implement this feature b) Actually implement this feature as an import hook, which could then be loaded by people who need it, and ignored by people (like me) who don't care. The PEP still needs an entry in the Issues section for import hooks. For now, just putting "Not clear how this interacts with import hooks" should be enough... Paul.

1 0