Mailman 3 October 2004 - Python-Dev

RE: [Python-Dev] scropion
by Michael Chermside Oct. 12, 2004

Oct. 12, 2004

> [Paul introduces himself] > My question is for performance: isn't binary code faster than p code? > So If I can vote, I would vote for c or c++ in the numerics functions and > data transfer areas. > even assembler. The quicker the better. Hello, and welcome. Thanks for your input on what kinds of things you value in Python. (The Python developers, who use the python-dev list) DO pay attention to such opinion (although don't expect sudden changes... you are one in a sea of users). You are not alone in caring about performance -- it's been a popular topic lately. I'll comment briefly on your specific suggestions. Binary code is faster than "p code" usually, but in some cases the "p code" is more ammenable to automated optimization. That is why certain kinds of code can run FASTER in java than in C. Python code too will often run faster than the equivalent C code... not because of a powerful just-in-time optimizer, but because the Python code is written using better algorithms and data structures than one would bother with in C. But you specifically mention numerics functions and data transfer... two areas where well-written C code is likely to be much better performing. We certainly share the goal of better performance (so long as it doesn't come at the cost of other things like usability and maintainability). If you know of some specific areas where we could improve, please submit a patch (through SourceForge's patch manager) -- we'd love to have the help! Finally, you mention the possibility of dropping into assembler. Unfortunately, this is sure to be rejected because it conflicts with an important design principle -- Python is designed to be VERY portable, and should compile on just about any system with a standards-compliant C compiler. -- Michael Chermside

1 0

Re: [Python-Dev] Re: Re: subprocess - updated popen5 module
by Neil Hodgson Oct. 11, 2004

Oct. 11, 2004

Fredrik Lundh: > Thomas Heller wrote: > > > How many _xxxx.pyd windows specific modules do we need? > > as many as it takes to make Python an excellent tool. it's not like one > more entry in the makefile, and 20k in the binary distribution will cause > problem for anyone. Requiring windows interface modules to be written in C rather than Python reduces the number of Python users that can add, modify and fix these modules. It also increases the amount of effort involved in working on these modules. Neil

6 7

compiler bug #1042238 is blocking the 2.4b1 release
by Anthony Baxter Oct. 11, 2004

Oct. 11, 2004

Anyone with knowledge of Lib/compiler is urged to examine www.python.org/sf/1042238. It's breaking on some genexprs, in particular in Lib/_strptime.py The failing code, distilled down, is: import compiler a="""def foo(*x): return x foo((a for b in [[1,2,3],[4,5,6]] for a in b),'Z')""" compiler.compile(a,'<string>', 'exec') It's the combination of the genexps and the function call with another argument. This bug is bad enough (IMHO) to block 2.4b1. Please, if you know about this, have a look at the failure. Anthony -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

2 1

Patch to make GIL API calls noops in single threaded applications
by Gustavo J. A. M. Carneiro Oct. 10, 2004

Oct. 10, 2004

Can someone review my patch[1]? It makes GIL API calls noops in single threaded applications, i.e. if PyEval_InitThreads hasn't been called. It's a very simple patch, but it makes a lot of difference if you want to make a Python extension module thread-aware without incurring an extra (unnecessary) performance penalty in single-threaded programs. Without this in Python, each extension library has to keep track of thread state (enabled or not), and one extension library doesn't work well with another without some explicit synchronization between the two. Please, we should fix this before Python 2.4 final. Regards. [1] https://sourceforge.net/tracker/? func=detail&aid=1011380&group_id=5470&atid=105470 -- Gustavo J. A. M. Carneiro <gjc(a)inescporto.pt> <gustavo(a)users.sourceforge.net> The universe is always one step beyond logic

1 0

assert failure on obmalloc
by Jeremy Hylton Oct. 10, 2004

Oct. 10, 2004

Failure running the test suite today with -u compiler enabled on Windows XP. test_logging Assertion failed: bp != NULL, file \code\python\dist\src\Objects\obmalloc.c, line 604 The debugger says the error is here: msvcr71d.dll!_assert(const char * expr=0x1e22bcc0, const char * filename=0x1e22bc94, unsigned int lineno=604) Line 306 C python24_d.dll!PyObject_Malloc(unsigned int nbytes=100) Line 604 + 0x1b C python24_d.dll!_PyObject_DebugMalloc(unsigned int nbytes=84) Line 1014 + 0x9 C python24_d.dll!PyThreadState_New(_is * interp=0x00951028) Line 136 + 0x7 C python24_d.dll!PyGILState_Ensure() Line 430 + 0xc C python24_d.dll!t_bootstrap(void * boot_raw=0x02801d48) Line 431 + 0x5 C python24_d.dll!bootstrap(void * call=0x04f0d264) Line 166 + 0x7 C msvcr71d.dll!_threadstart(void * ptd=0x026a2320) Line 196 + 0xd C I've been seeing this sort of error on-and-off for at least a year with my Python 2.3 install. It's the usual reason my spambayes popproxy dies. I can't recell seeing it before on Windows or while running the test suite. Jeremy

3 3

Guidelines for Logging Usage
by Michael P. Dubner Oct. 10, 2004

Oct. 10, 2004

Hello, I beg your pardon in advance for my English, which isn't my native language. I'm posting PEProposal for discussion. Last part is reasoning for posting it in this hot time. I'm ready to implement it all by myself... :-) -- Best regards, Michael Dubner (dubnerm(a)mindless.com) Brainbench MVP/HTML+JavaScript (http://www.brainbench.com) PS: Sorry for my English PEP: XXX Title: Guidelines for Logging Usage Version: $Revision: $ Last-Modified: $Date: $ Author: Michael P. Dubner <dubnerm(a)mindless.com> Status: Draft Type: Standards Track Content-Type: text/plain Created: 02-Oct-2004 Python-Version: 2.5 Post-History: Abstract This PEP defines guidelines for using logging system (PEP 282 [1]) in standard library. Implementing this PEP will simplify development of daemon applications. As a downside this PEP requires to slightly modify (however in backportable way) large number of standard modules. After implementing this PEP one can use following filtering scheme:: logging.getLogger('stdlib.BaseHTTPServer').setLevel(logging.FATAL) Rationale There are couple of situations when output to stdout or stderr is very incomfortable. - Daemon application where framework doesn't allows to redirect standard output to some file, but assumes use of some other way of logging. Examples are syslog under *nix'es and EventLog under WinNT+. - GUI application which want to output every new log entry in separate popup window (i.e. fading OSD). Also sometimes application want to filter output enties based on it's source or severity. This requirement can't be implemented using simple redirection. At last sometimes output need to be marked with time of event, which can be acqured with logging system with ease. Proposal Every module usable for daemon and GUI applications should be rewritten to use logging system instead of 'print' or 'sys.stdout.write'. There should be code like this included in the beginning of every modified module:: import logging _log = logging.getLogger('stdlib.<module-name>') Prefix of 'stdlib.' must be used by all modules included in standard library distributed along with Python, and only by such modules (unverifiable). Using of "_log" is intentional as we don't want to auto-export it. For modules that using log only in one class logger can be created inside class definition as following:: class XXX: __log = logging.getLogger('stdlib.<module-name>') Then this class can create access methods to log to this private logger. So print's and "sys.std{out|err}.write" should be replaced with "_log.{debug|info}" and "traceback.print_exception" with "_log.exception" or sometimes "_log.debug('...',exc_info=1)". Module List Here is (possibly incomplete) list of modules to be reworked: - asyncore (dispatcher.log, dispatcher.log_info) - BaseHTTPServer (BaseHTTPRequestHandler.log_request, BaseHTTPRequestHandler.log_error, BaseHTTPRequestHandler.log_message) - cgi (possibly - is cgi.log used by somebody?) - ftplib (if FTP.debugging) - gopherlib (get_directory) - httplib (HTTPResponse, HTTPConnection) - ihooks (_Verbose) - imaplib (IMAP4._mesg) - mhlib (MH.error) - nntplib (NNTP) - pipes (Template.makepipeline) - pkgutil (extend_path) - platform (_syscmd_ver) - poplib (if POP3._debugging) - profile (if Profile.verbose) - robotparser (_debug) - smtplib (if SGMLParser.verbose) - shlex (if shlex.debug) - smtpd (SMTPChannel/PureProxy where print >> DEBUGSTREAM) - smtplib (if SMTP.debuglevel) - SocketServer (BaseServer.handle_error) - telnetlib (if Telnet.debuglevel) - threading? (_Verbose._note, Thread.__bootstrap) - timeit (Timer.print_exc) - trace - uu (decode) Additionaly there are couple of modules with commented debug output or modules where debug output should be added. For example: - urllib At last possibly some modules should be extended to provide more debug information. Doubtful Modules Here should be placed modules that community will propose for addition to module list and modules that community say should be removed from module list. - tabnanny (check) Guidelines for Logging Usage Also we can provide some recommendation to authors of library modules so they all follow same format of naming loggers. I propose that non-standard library modules should use logger named after their full names, so module "spam" in sub-package "junk" of package "dummy" will be named "dummy.junk.spam" and, of cause, "__init__" module of same package will have logger "dummy.junk". Implementation Schedule Proposal As one can see whole bunch of changes required to fullfil this proposal is rather large. I propose to delay these changes until after 2.4 release, and change only modules critical for server applications: - BaseHTTPServer - ftplib - httplib - poplib - smtpd - smtplib - SocketServer This can be done rather fast and reliable. References [1] PEP 282, A Logging System, Vinay Sajip, Trent Mick http://www.python.org/peps/pep-0282.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:

7 8

Toward Python's future article
by daniel narf Oct. 8, 2004

Oct. 8, 2004

Hi i am sure most of you have read the article of Andrew Kuchling about focusing more in the standart library than language newFeatures/tweaking and probably i as many others would like to know what the python hackers/developers think about this proposal.Maybe this post is out of place but oh well.. i am personaly very interested in improving the stdlib which is very messy in my opinion right now. the article(several comments): http://www.amk.ca/diary/archives/cat_python.html#003382

6 7

HAVE_USABLE_WCHAR_T
by Thomas Heller Oct. 8, 2004

Oct. 8, 2004

The Include/unicodeobject.h file says (line 103): /* If the compiler provides a wchar_t type we try to support it through the interface functions PyUnicode_FromWideChar() and PyUnicode_AsWideChar(). */ This isn't true - grepping the CVS sources for this symbol shows that it is used in these ways: - When defined together with the WANT_WCTYPE_FUNCTIONS symbol, the compiler's wctype.h functions are used instead of the ones supplied with Python. Include/unicodeobject.h, line 294. - When defined together with MS_WINDOWS, it makes available mbcs_enocde and mbcs_decode functions (in Modules/_codecsmodule.c), plus the PyUnicode_DecodeMBCS and PyUnicode_AsMBCSString functions in Objects/unicodeobject.c. - Contrary to the comment at the top of this message, the PyUnicode_FromWideChar and PyUnicode_AsWideChar functions are compiled when HAVE_WCHAR_H is defined. The HAVE_USABLE_WCHAR_T symbol is only used to determine whether memcpy is used, or the unicode characters are copied one by one. - Finally, again when defined together with MS_WINDOWS, it sets the filesystem encoding to mbcs. So, it seems that the HAVE_USABLE_WCHAR_T symbol doesn't play any role for the extension programmer *at all*. The preprocessor flag that plays a role for extensions seem to be HAVE_WCHAR_H since they mark whether the PyUnicode_FromWideChar and PyUnicode_AsWideChar are available or not. This has caused me quite some confusion, and so I suggest the comment above in the Include/unicodeobject.h file should be fixed. Finally, the docs also seem to get it wrong (although I haven't followed that in detail). Can't reach python.org at the moment, but Python C/api manual, section 7.3.2, unicode objects says: Py_UNICODE This type represents a 16-bit unsigned storage type which is used by Python internally as basis for holding Unicode ordinals. On platforms where wchar_t is available and also has 16-bits, Py_UNICODE is a typedef alias for wchar_t to enhance native platform compatibility. On all other platforms, Py_UNICODE is a typedef alias for unsigned short. Isn't the size 32 bits for wide unicode builds? Please, please fix this - unicode is already complicated enough even without this confusion! Thomas

2 1

Python Bug Day this month?
by Jeff Epler Oct. 8, 2004

Oct. 8, 2004

I thought the Python Bug Day was going to return this month, and I was hoping to spare some of my time to help out (on one previous bug day I reviewed some doc patches), but I haven't seen anything about it. Jeff

2 1

Version # for PSF license?
by A.M. Kuchling Oct. 7, 2004

Oct. 7, 2004

DOAP (http://usefulinc.com/doap) is yet another schema for recording data about software projects. I'd like to add support for it to PyPI; however, before doing that a URL needs to be allocated for the PSF license in the listing at http://usefulinc.com/doap/licenses . Question: does the PSF license have a version number of its own or do we have to say 'PSF license for Python 2.2' (2.3, 2.4, etc)? --amk

1 0