Mailman 3 January 2001 - Python-Dev

RE: [Python-checkins] CVS: python/dist/src/Lib fileinput.py,1.5,1.6
by Tim Peters Jan. 5, 2001

Jan. 5, 2001

[Guido] > Modified Files: > fileinput.py > Log Message: > Speed it up by using readlines(sizehint). It's still slower than > other ways of reading input. :-( On my box, it's now head-to-head with (maybe even a little quicker than) the while 1: line-at-a-time way: total 117615824 chars and 3237568 lines readlines_sizehint 9.450 9.459 using_fileinput 29.880 29.884 while_readline 30.480 30.506 (stock CVS Python under Win98SE) So that's a huge improvement! the-… [View More]

5 6

new "theme" on SourceForge!
by Fred L. Drake, Jr. Jan. 5, 2001

Jan. 5, 2001

While "theme-ability" is becoming very popular for desktop software (think about the latest Gnome and KDE systems for Unix, and some of the multimedia applications for Windows, and the newest MacOS desktops), it can be a huge drain on Web sites; too many graphics is a pain, and too many tables just makes it worse. SourceForge had definately fallen prey to the overly-fancy themes, and all of us developers paid the price with slow rendering. But they've fixed that! The SF crew has announced … [View More]

2 1

Re: [Python-Dev] xreadlines : readlines :: xrange : range
by Thomas Wouters Jan. 5, 2001

Jan. 5, 2001

On Wed, Jan 03, 2001 at 07:37:22PM -0500, Guido van Rossum wrote: > > In other words: use it! :) > > Mind doing a few platform tests on the (new version of the) patch? Well, only a bit :) It's annoying that BSDI doesn't come with autoconf, but I managed to use all my early-morning wit (it's 6:30AM <wink>) to work around it. I've tested it on BSDI 4.1 and FreeBSD 4.2-RELEASE. > I already know that it works on Red Hat Linux 6.2 (my box) and Solaris > 2.6 (Andrew's box). … [View More] I would be delighted to know that it works on at > least one other platform that has getc_unlocked() and one platform > that doesn't have it! Sorry, I have to disappoint you. FreeBSD does have getc_unlocked, they just didn't document it. Hurrah for autoconf ;P Anyway, it worked like a charm on BSDI: (Python 2.0) total 1794310 chars and 37660 lines count_chars_lines 0.310 0.300 readlines_sizehint 0.150 0.150 using_fileinput 2.013 2.017 while_readline 1.006 1.000 (CVS Python + getc_unlocked) daemon2:~/python/python/dist/src > ./python test.py termcapx10 total 1794310 chars and 37660 lines count_chars_lines 0.354 0.350 readlines_sizehint 0.182 0.183 using_fileinput 1.594 1.583 while_readline 0.363 0.367 But something weird is going on on FreeBSD: (Standard CVS Python) > ./python ~thomas/test.py ~thomas/termcapx10 total 1794310 chars and 37660 lines count_chars_lines 0.265 0.266 readlines_sizehint 0.148 0.148 using_fileinput 0.943 0.938 while_readline 0.214 0.219 (CVS+getc_unlocked) > ./python-getc-unlocked ~thomas/test.py ~thomas/termcapx10 total 1794310 chars and 37660 lines count_chars_lines 0.266 0.266 readlines_sizehint 0.151 0.141 using_fileinput 1.066 1.078 while_readline 0.283 0.281 This was sufficiently unexpected that I looked a bit further. The FreeBSD Python was compiled without editing Modules/Setup, so it was statically linked, no readline etc, but *with* threads (which are on by default, and functional on both FreeBSD and BSDI 4.1.) Here's the timings after I enabled just '*shared*': (CVS + *shared*) > ./python ~thomas/test.py ~thomas/termcapx10 total 1794310 chars and 37660 lines count_chars_lines 0.276 0.273 readlines_sizehint 0.150 0.156 using_fileinput 0.902 0.898 while_readline 0.206 0.203 (This was not a fluke, I repeated it several times, getting hardly any variation.) Enabling readline and cursesmodule had no additional effect. Adding *shared* to the getc_unlocked tree saw roughly the same improvement, but was still slower than without getc_unlocked. (CVS + *shared* + getc_unlocked) > ./python ~thomas/test.py ~thomas/termcapx10 total 1794310 chars and 37660 lines count_chars_lines 0.272 0.273 readlines_sizehint 0.149 0.148 using_fileinput 1.031 1.031 while_readline 0.267 0.266 Increasing the size of the testfile didn't change anything, other than the absolute numbers. I browsed stdio.h, where both getc() and getc_unlocked() are defined as macros. getc_unlocked is defined as: #define __sgetc(p) (--(p)->_r < 0 ? __srget(p) : (int)(*(p)->_p++)) #define getc_unlocked(fp) __sgetc(fp) and getc either as #define getc(fp) getc_unlocked(fp) (without threads) or static __inline int \ __getc_locked(FILE *_fp) \ { \ extern int __isthreaded; \ int _ret; \ if (__isthreaded) \ _FLOCKFILE(_fp); \ _ret = getc_unlocked(_fp); \ if (__isthreaded) \ funlockfile(_fp); \ return (_ret); \ } #define getc(fp) __getc_locked(fp) _FLOCKFILE(x) is defined as flockfile(x), so that isn't the difference. The speed difference has to be in the quick-and-easy test for whether the locking is even necessary. Starting a thread on 'time.sleep(900)' in test.py shows these numbers: (standard CVS python) > ./python-shared-std ~/test.py ~/termcapx10 total 1794310 chars and 37660 lines count_chars_lines 0.433 0.445 readlines_sizehint 0.204 0.188 using_fileinput 1.595 1.594 while_readline 0.456 0.453 (getc_unlocked) > ./python-getc-unlocked-shared ~/test.py ~/termcapx10 total 1794310 chars and 37660 lines count_chars_lines 0.441 0.453 readlines_sizehint 0.206 0.195 using_fileinput 1.677 1.688 while_readline 0.509 0.508 So... using getc_unlocked manually for performance reasons isn't a cardinal sin on FreeBSD only if you are really using threads :-) Lets-outsmart-the-OS-scheduler-next!-ly y'rs -- Thomas Wouters <thomas(a)xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread! [View Less]

4 9

Re: [Python-checkins] CVS: python/dist/src/Include classobject.h,2.33,2.34
by M.-A. Lemburg Jan. 4, 2001

Jan. 4, 2001

Neil Schemenauer wrote: > > Update of /cvsroot/python/python/dist/src/Include > In directory usw-pr-cvs1:/tmp/cvs-serv21006/Include > > Modified Files: > classobject.h > Log Message: > Remove PyInstance_*BinOp functions. > > Index: classobject.h > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Include/classobject.h,v > retrieving revision 2.33 > retrieving revision 2.34 > … [View More]

2 2

SGI's Digital Media SDK
by Andrew Kuchling Jan. 4, 2001

Jan. 4, 2001

SGI just made a source release of their digital media SDK for IRIX and Linux at http://oss.sgi.com/projects/dmsdk/ . According to the FAQ, this is derived from previous SGI libraries, "including the Video Library (VL), the Audio Library (AL), Digital Media Image Convertor (DMIC), Digital Media Audio Convertor (DMAC), and the Compression Library (CL)." Interested parties may want to look into this, because Python still has the al, cd, cl, and sv modules; maybe they'd work with the new software … [View More]

1 0

Re: [Python-checkins] CVS: python/dist/src/Lib/test/output test_coercion,1.2,1.3
by Thomas Wouters Jan. 4, 2001

Jan. 4, 2001

On Wed, Jan 03, 2001 at 05:36:27PM -0800, Neil Schemenauer wrote: > Update of /cvsroot/python/python/dist/src/Lib/test/output > In directory usw-pr-cvs1:/tmp/cvs-serv21710/Lib/test/output > > Modified Files: > test_coercion > Log Message: > Sequence repeat works now for in-place multiply with an integer type > as the left operand. I don't know if this is a feature or a bug. > ! 2 *= [1] => [1, 1] It's a feature. x = 2 * [1] works, so x = 2 x *= [1] does, … [View More]

1 0

xreadlines : readlines :: xrange : range
by Martin von Loewis Jan. 3, 2001

Jan. 3, 2001

> Apparently getc_unlocked() is in the Single Unix spec. Not sure how > widespread that is -- do Linux developers pay attention to this > standard at all? Ulrich Drepper, who is in charge of glibc, is always interested in following Single Unix to the letter; getc_unlocked is supported atleast since glibc 2.0. http://www.sun.com/smcc/solaris-migration/docs/courses/threadsHTML/adv.html claims that getc_unlocked is already in POSIX.1c; Solaris apparently supports it atleast since … [View More]

2 1

Re: [Python-checkins] CVS: python/dist/src/Lib codecs.py,1.13,1.14
by M.-A. Lemburg Jan. 3, 2001

Jan. 3, 2001

"M.-A. Lemburg" wrote: > > Update of /cvsroot/python/python/dist/src/Lib > In directory usw-pr-cvs1:/tmp/cvs-serv26608/Lib > > Modified Files: > codecs.py > Log Message: > ... > > This patch closes the bugs #116285 and #119960. I was too fast... the subject line of #119960 was misleading. It is still open. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://… [View More]

1 0

Re: [Patches] [Patch #103082] speed up readline() using getc_unlocked()
by Andrew Kuchling Jan. 3, 2001

Jan. 3, 2001

On Wed, Jan 03, 2001 at 08:47:30AM -0800, GvR wrote: >Summary: speed up readline() using getc_unlocked() So what does the performance of this version look like? --amk

2 4

Re: [Python-Dev] Help wanted with setup.py script
by Andrew Kuchling Jan. 3, 2001

Jan. 3, 2001

[Cc'ing to python-dev]. On Wed, Jan 03, 2001 at 04:35:10PM +0100, Thomas Heller wrote: >You didn't expect this script run under windows? >(It does not run) It shouldn't matter, I think, since the makesetup stuff doesn't run on Windows either; presumably the compiled-in modules are specified by an MSVC project file, or something similar. Can anyone confirm that I don't care if setup.py works on Windows? (Well, I *know* for a fact I don't care; but should I? :) ) --amk

2 1