Mailman 3 June 2000 - Python-Dev

more on httplib
by Greg Stein 11 Jun '00

11 Jun '00

I would like to propose a few changes to the (new) httplib: *) drop HTTPS() -- this class isn't in 1.5.2, so there isn't a reason to provide backwards-compat for it *) revamp the pipeline support: - record the "last response object" ... when a new getreply() is done, then we store the "last" into response.prior - reading of the "HTTP/1.1 <code> <msg>" line is deferred, and performed by the response object - the read of that line is lazy - if the response line is read *before* the "prior" response (if any) is "closed", then an exception is raised: ResponseNotReady *) address some of Moshe's concerns: - use class-based exceptions - clarify that HTTPConnection is designed for *blocking* sockets - conn.close() followed by conn.send() will reopen the socket. This could lead to programming errors. I'll add a class-based default flag to disable this behavior. - in request(), we check for errno==32 ... what to do on Windows? I will implement these changes in small chunks so that each can be reviewed in my CVS repository. The history is available at: http://www.lyra.org/cgi-bin/viewcvs.cgi/gjspy/httplib.py/ Cheers, -g -- Greg Stein, http://www.lyra.org/

2 3

Greg Stein's correction broke my traceback wrapper
by Eric S. Raymond 11 Jun '00

11 Jun '00

Greg Stein's correction broke my traceback wrapper. I now do *not* see a traceback on an exit due to Python runtime errors. I don't know why this is. -- <a href="http://www.tuxedo.org/~esr">Eric S. Raymond</a> No one who's seen it in action can say the phrase "government help" without either laughing or crying.

3 2

Suggestion to Windows- and Mac-library people
by Eric S. Raymond 11 Jun '00

11 Jun '00

Last night I sent the following patch for urllib to python-patches: # Support for launching a browser if os.environ.has_key("BROWSER"): _browsers = string.split(os.environ["BROWSER"], ":") else: _browsers = ["mozilla %s &", "netscape -remote 'openURL(%s)'", "netscape %s &", "lynx %s &", "w3m %s &"] def urlbrowse(url): """Launch a browser, in background, pointed at the given URL. Accept either a string or a parsed URL tuple. Interpret the BROWSER environment variable, if it exists, as a colon-separated list of browser commands to try. """ from urlparse import urlunparse if type(url) == (): url = urlunparse(url) for browser in _browsers: if not os.system('which 1>/dev/null 2>&1 '+string.split(browser)[0]): if os.system((browser % url)) == 0: return 1 return 0 Obviously this is not portable off Unix. Equally obviously it should be possible to do an equivalent thing under Windows (at least). For anybody writing interactive programs in this Web-mad era, this function is just too useful not to have in the toolbox. So this is a nudge to any Windows and Mac wizards on this list -- let's make this function available cross-platform. I've also copied this note to Chip Salzenberg. Perl ought to have this capability too -- and I'm sufficiently serious about that to (gasp) code in Perl myself to make it happen if I have to. -- <a href="http://www.tuxedo.org/~esr">Eric S. Raymond</a> (Those) who are trying to read the Second Amendment out of the Constitution by claiming it's not an individual right (are) courting disaster by encouraging others to use the same means to eliminate portions of the Constitution they don't like. -- Alan Dershowitz, Harvard Law School

2 4

Re: [Python-checkins] CVS: python/dist/src/Lib/curses __init__.py,NONE,1.1 wrapper.py,NONE,1.1
by Greg Stein 10 Jun '00

10 Jun '00

On Sat, Jun 10, 2000 at 04:06:55PM -0700, A.M. Kuchling wrote: >... > def wrapper(func, *rest): > """Wrapper function that initializes curses and calls another function, > restoring normal keyboard/screen behavior on error. > The callable object 'func' is then passed the main window 'stdscr' > as its first argument, followed by any other arguments passed to > wrapper(). > """ > > res = None > try: > # Initialize curses > stdscr=curses.initscr() > # Turn off echoing of keys, and enter cbreak mode, > # where no buffering is performed on keyboard input > curses.noecho() ; curses.cbreak() > > # In keypad mode, escape sequences for special keys > # (like the cursor keys) will be interpreted and > # a special value like curses.KEY_LEFT will be returned > stdscr.keypad(1) > > res = apply(func, (stdscr,) + rest) > except: > # In the event of an error, restore the terminal > # to a sane state. > stdscr.keypad(0) > curses.echo() ; curses.nocbreak() > curses.endwin() > # Pass the exception upwards > (exc_type, exc_value, exc_traceback) = sys.exc_info() > raise exc_type, exc_value, exc_traceback > else: > # Set everything back to normal > stdscr.keypad(0) > curses.echo() ; curses.nocbreak() > curses.endwin() # Terminate curses > > return res This is too complicated. It should read: ... return apply(func, (stdscr,) + rest) finally: # Set everything back to normal stdscr.keypad(0) curses.echo() curses.nocbreak() curses.endwin() Andrew: can you make this change? Separate from the above simplification, it is important to note: exc_type, exc_value, exc_traceback = sys.exc_info() raise exc_type, exc_value, exc_traceback and raise are equivalent in (most) function. The difference is that the former places the traceback into local variable storage (exc_traceback) which then introduces a reference loop. Whenever sys.exc_info() is used, it is very important to watch out for loops caused by the traceback object. "del exc_traceback" is very handy :-) Cheers, -g -- Greg Stein, http://www.lyra.org/

3 4

Creating curses package: OK?
by Andrew Kuchling 10 Jun '00

10 Jun '00

[Redirecting a private conversation to python-dev] On Sat, Jun 10, 2000 at 11:31:24AM -0400, Eric S. Raymond wrote: >I found a workaround. The two curses enhancements (traceback wrapper >and text pad) are now working and ready to be integrated into the >1.6 library. How should we proceed? OK; python-dev has discussed creating a curses package, and reaction seemed favorable. So, my plan of action is: 1) Rename cursesmodule.c to _cursesmodule.c. (Barry, can you do a 'mv' inside the SourceForge CVS repository so the complete history of the module doesn't get lost with the renaming?) 2) Create a Lib/curses/ directory; __init__.py does 'from _curses import *'. 3) The traceback wrapper can then be put into __init__.py, since I assume it's fairly small and everyone should use it. The text pad stuff could then go in another module in the curses package. Seem reasonable to everyone? --amk

5 13

Japanese Codecs
by M.-A. Lemburg 10 Jun '00

10 Jun '00

Python currently supports lots of different code pages and other more specific codecs. I was wondering what your thoughts on this are: should we proceed to include an even wider range of codecs or limit the available number to an absolute minimum ? I'm asking this, because Tamito KAJIYAMA wrote a set of codecs for EUC-JP and Shift_JIS which would probably be very useful for Japanese Python users -- I just don't know how many there are and whether inclusion of an extra 650k of source code would hurt or not. Thoughts ? -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

6 7

What's New: 2nd draft
by Andrew Kuchling 10 Jun '00

10 Jun '00

We've made a second pass at the "What's New in 1.6" document; the only unwritten section is IDLE, and I don't think either Moshe or I use IDLE. Can anyone tell us what the new features are? Again, suggestions on style, forgotten changes, *anything*, are welcomed: http://starship.python.net/crew/amk/python/writing/new-python/ -- A.M. Kuchling http://starship.python.net/crew/amk/ "How about you -- what're you up to?" "Issue #3." -- Vic and Zot, in ZOT! #3

1 0

revamped Python web site
by Jeremy Hylton 09 Jun '00

09 Jun '00

Say a startup company decided to put some resources behind the Python Web site -- a full-time Web master, some content developers, etc. What would you have them do? What could be done to open up the site to the community and make it more responsive to developers and users? What's your wish list for content, services, etc.? If you've got any ideas, please get back to me soon. Jeremy

11 15

impact of cycle gc...
by Skip Montanaro 09 Jun '00

09 Jun '00

I spent a little time poking around with a very simple test script (appended) that makes use of Neil's cycle gc patch. It measures the time it takes to instantiate and delete a cycle-producing instance 100,000 times (based upon a report in python-bugs). For vanilla 1.6a2 ("./configure" ; make") I got the following (last of six runs): memory usage @ start: 5560 clock: 1.33 elapsed: 1.34805500507 memory usage @ end: 18816 For 1.6a2 configured --with-cycle-gc ("./configure --with-cycle-gc ; make") I got the following (also the last of six runs): memory usage @ start: 5576 clock: 2.34 elapsed: 2.33785700798 memory usage @ end: 5576 I was very impressed with the memory usage, but dismayed at the increased execution time (about 75% more). Of course, this is a test script that is bound to show the potential performance penalty in the worst possible light. Maybe pystone will be more indicative of the performance hit we can expect from it. (There's a new one - pystone.py being a better indicator of true system performance than some other test script!) Plain: Pystone(1.1) time for 10000 passes = 1.76 This machine benchmarks at 5681.82 pystones/second --with-cycle-gc: Pystone(1.1) time for 10000 passes = 1.83 This machine benchmarks at 5464.48 pystones/second That suggests something more like a 4% performance penalty. I suspect that may be the lower limit. While I haven't looked at pystone recently, I doubt it creates any cyclic garbage. My next step will be to try it out on my development database server and see what happens. just-another-data-point-ly y'rs, -- Skip Montanaro, skip(a)mojam.com, http://www.mojam.com/, http://www.musi-cal.com/ "We have become ... the stewards of life's continuity on earth. We did not ask for this role... We may not be suited to it, but here we are." - Stephen Jay Gould class fred: def __init__(self): self.indirectFunc = self.theFunc def theFunc(self): return "blah" def test(): f = fred() del f if __name__ == "__main__": import os, time, sys sys.stdout.write("memory usage @ start: ") sys.stdout.flush() os.system("ps auxww | egrep method | egrep -v egrep | awk '{print $5}'") t = time.clock(), time.time() for x in xrange(100000): test() print "clock:", time.clock()-t[0], "elapsed:", time.time()-t[1] sys.stdout.write("memory usage @ end: ") sys.stdout.flush() os.system("ps auxww | egrep method | egrep -v egrep | awk '{print $5}'")

2 2

impact of cycle gc...
by Neil Schemenauer 09 Jun '00

09 Jun '00

Just a few points and then I will get off python-dev. :) First of all, I don't think it is very meaningful to use leaking applications to do timing comparisons. The collector has be quite careful when freeing structures containing reference cycles. However, using something other than pystone is definitely a good idea. Here the pybench results for the latest patch: PYBENCH 0.6 Benchmark: b_nogc (rounds=4, warp=30) Tests: per run per oper. diff * ------------------------------------------------------------------------ BuiltinFunctionCalls: 286.25 ms 3.37 us -4.22% BuiltinMethodLookup: 367.50 ms 1.05 us -4.55% ConcatStrings: 368.37 ms 3.68 us +42.50% CreateInstances: 477.87 ms 17.07 us -4.45% CreateStringsWithConcat: 322.12 ms 2.42 us +10.27% DictCreation: 382.75 ms 3.83 us +1.29% ForLoops: 536.88 ms 80.73 us +0.99% IfThenElse: 432.75 ms 0.96 us -3.21% ListSlicing: 236.87 ms 104.12 us +11.73% NestedForLoops: 320.00 ms 1.28 us +0.16% NormalClassAttribute: 386.50 ms 0.97 us +0.00% PythonFunctionCalls: 477.87 ms 4.34 us -3.51% PythonMethodCalls: 379.12 ms 7.59 us -12.67% Recursion: 280.75 ms 33.70 us -0.88% SecondImport: 206.62 ms 12.41 us -5.76% SecondPackageImport: 216.75 ms 13.02 us -4.41% SecondSubmoduleImport: 279.25 ms 16.77 us -2.57% SimpleComplexArithmetic: 351.00 ms 2.39 us +0.72% SimpleDictManipulation: 320.75 ms 1.60 us -2.99% SimpleFloatArithmetic: 361.50 ms 0.99 us -20.37% SimpleIntFloatArithmetic: 336.00 ms 0.76 us +0.04% SimpleIntegerArithmetic: 328.38 ms 0.75 us -0.76% SimpleListManipulation: 312.88 ms 1.74 us -1.61% SimpleLongArithmetic: 308.75 ms 2.81 us +9.88% SmallLists: 470.13 ms 2.77 us -5.05% SmallTuples: 374.62 ms 2.34 us -15.74% SpecialClassAttribute: 384.00 ms 0.96 us -1.88% SpecialInstanceAttribute: 446.38 ms 1.12 us -2.75% StringSlicing: 315.50 ms 2.70 us +16.58% TryExcept: 585.37 ms 0.59 us -1.70% TryRaiseExcept: 312.75 ms 31.28 us -5.30% TupleSlicing: 299.38 ms 4.39 us +12.18% ------------------------------------------------------------------------ Average round time: 13615.00 ms -1.13% My AMD-K6-II processor is a pretty quirky beast so I don't think you can conclude too much for those results. Here are the median timings from running Jeremy's compiler on its own source: $ time python compile.py `find . -name '*.py'` Python 1.6 without GC: real 0m16.926s user 0m16.810s sys 0m0.110s Python 1.6 with GC: real 0m21.593s user 0m21.470s sys 0m0.080s Python 1.6 with GC, collection disabled (ie. gc.set_threshold(0)): real 0m18.441s user 0m18.220s sys 0m0.220s We can tune the collection frequency all we want but we won't do any better than the last numbers. Those numbers reflect the cost of keeping track of the objects and the increase in object size. On a related note, I would like to cleanup the PyGC_{NEW, VAR_NEW} macros but I can't figure out a way to transform this code into a macro: op = PyObject_MALLOC(sizeof(PyGCInfo) + _PyObject_SIZE(tp)); if (op) op = PyGC_OBJ((PyGCInfo *)op); If C's || operator was like the Python or I could do something like: #define PyGC_OBJ_SAFE(g) ((PyGCInfo *)(((g) || -1) + 1)) Any ideas? Using an inline function in the header file would be nice but of course it is not portable. GCC has statement expressions but again they are not portable. Neil

2 1