Mailman 3 March 2006 - Python-Dev

PythonCore\CurrentVersion
by "Martin v. Löwis" 12 Mar '06

12 Mar '06

What happened to the CurrentVersion registry entry documented at http://www.python.org/windows/python/registry.html AFAICT, even the python15.wse file did not fill a value in this entry (perhaps I'm misinterpreting the wse file, though). So was this ever used? Why is it documented, and who documented it (unfortunately, registry.html is not in cvs/subversion, either)? Regards, Martin

5 11

Re: [Python-Dev] str.count is slow
by Ben Cartwright 12 Mar '06

12 Mar '06

>From comp.lang.python: chrisperkins99(a)gmail.com wrote: > It seems to me that str.count is awfully slow. Is there some reason > for this? > Evidence: > > ######## str.count time test ######## > import string > import time > import array > > s = string.printable * int(1e5) # 10**7 character string > a = array.array('c', s) > u = unicode(s) > RIGHT_ANSWER = s.count('a') > > def main(): > print 'str: ', time_call(s.count, 'a') > print 'array: ', time_call(a.count, 'a') > print 'unicode:', time_call(u.count, 'a') > > def time_call(f, *a): > start = time.clock() > assert RIGHT_ANSWER == f(*a) > return time.clock()-start > > if __name__ == '__main__': > main() > > ###### end ######## > > On my machine, the output is: > > str: 0.29365715475 > array: 0.448095498171 > unicode: 0.0243757237303 > > If a unicode object can count characters so fast, why should an str > object be ten times slower? Just curious, really - it's still fast > enough for me (so far). > > This is with Python 2.4.1 on WinXP. > > > Chris Perkins Your evidence points to some unoptimized code in the underlying C implementation of Python. As such, this should probably go to the python-dev list (http://mail.python.org/mailman/listinfo/python-dev). The problem is that the C library function memcmp is slow, and str.count calls it frequently. See lines 2165+ in stringobject.c (inside function string_count): r = 0; while (i < m) { if (!memcmp(s+i, sub, n)) { r++; i += n; } else { i++; } } This could be optimized as: r = 0; while (i < m) { if (s[i] == *sub && !memcmp(s+i, sub, n)) { r++; i += n; } else { i++; } } This tactic typically avoids most (sometimes all) of the calls to memcmp. Other string search functions, including unicode.count, unicode.index, and str.index, use this tactic, which is why you see unicode.count performing better than str.count. The above might be optimized further for cases such as yours, where a single character appears many times in the string: r = 0; if (n == 1) { /* optimize for a single character */ while (i < m) { if (s[i] == *sub) r++; i++; } } else { while (i < m) { if (s[i] == *sub && !memcmp(s+i, sub, n)) { r++; i += n; } else { i++; } } } Note that there might be some subtle reason why neither of these optimizations are done that I'm unaware of... in which case a comment in the C source would help. :-) --Ben

16 30

Google ads on python.org?
by Georg Brandl 12 Mar '06

12 Mar '06

Okay, if they were sensible, but: http://www.ph.tum.de/~gbrandl/python-vb.png Not that we want them to use Python... <wink> Georg

2 1

Coverity Open Source Defect Scan of Python
by Ben Chelf 11 Mar '06

11 Mar '06

Hello Python Developers, I'm the CTO of Coverity, Inc., a company that does static source code analysis to look for defects in code. You may have heard of us or of our technology from its days at Stanford (the "Stanford Checker"). The reason I'm writing is because we have set up a framework internally to continually scan open source projects and provide the results of our analysis back to the developers of those projects. Python is one of the 32 projects currently scanned at: http://scan.coverity.com My belief is that we (Coverity) must reach out to the developers of these packages (you) in order to make progress in actually fixing the defects that we happen to find, so this is my first step in that mission. Of course, I think Coverity technology is great, but I want to hear what you think and that's why I worked with folks at Coverity to put this infrastructure in place. The process is simple -- it checks out your code each night from your repository and scans it so you can always see the latest results. Right now, we're guarding access to the actual defects that we report for a couple of reasons: (1) We think that you, as developers of Python, should have the chance to look at the defects we find to patch them before random other folks get to see what we found and (2) From a support perspective, we want to make sure that we have the appropriate time to engage with those who want to use the results to fix the code. Because of this second point, I'd ask that if you are interested in really digging into the results a bit further for your project, please have a couple of core maintainers (or group nominated individuals) reach out to me to request access. As this is a new process for us and still involves a small number of packages, I want to make sure that I personally can be involved with the activity that is generated from this effort. So I'm basically asking for people who want to play around with some cool new technology to help make source code better. If this interests you, please feel free to reach out to me directly. And of course, if there are other packages you care about that aren't currently on the list, I want to know about those too. If this is the wrong list, my sincerest apologies and please let me know where would be a more appropriate forum for this type of message. Many thanks for reading this far... -ben Ben Chelf Chief Technology Officer Coverity, Inc.

7 9

multidict API
by Ian Bicking 10 Mar '06

10 Mar '06

I'm not really making any actionable proposal here, so maybe this is off-topic; if so, sorry. Back during the defaultdict discussion I proposed a multidict object (http://mail.python.org/pipermail/python-dev/2006-February/061264.html) -- right now I need to implement one to represent web form submissions. It would also be ordered in that case. The question then is what the API should look like for such an object -- an ordered, multi-value dictionary. I would really like if this object was in the collections module, but I'm too lazy to try to pursue that now. But if it did show up, I'd like the class I write to look the same. There's some open questions I see: * Does __getitem__ return a list of all matching keys (never a KeyError, though possibly returning []), or does it return the first matching key? * Either way, I assume there will be another method, like getfirst or getall, that will present the other choice. What would it be named? Should it have a default? * Should there be a method to get a single value, that implicitly asserts that there is only one matching key? * Should the default for .get() be None, or something else? * Does __setitem__ overwrite any or all values with matching keys? * If so, there should be another method like .add(key, value) which does not overwrite. Or, if __setitem__ does not overwrite, then there should be a method that does. * Does __delitem__ raise a KeyError if the key is not found? * Does .keys() return all unique keys, or all keys in order (meaning a key may show up more than once in the list)? I really could go either way on all of these questions, though I think there's constraints -- answer one of the questions and another becomes obvious. But you can answer them in whatever order you want. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

3 4

unicodedata.c no longer compiles on Windows
by Tim Peters 10 Mar '06

10 Mar '06

Compiling... unicodedata.c \Code\python\Modules\unicodedata.c(74) : error C2133: 'unicodedata_functions' : unknown size It's griping about this: /* Forward declaration */ static PyMethodDef unicodedata_functions[];

2 1

ctypes is in SVN now.
by Thomas Heller 10 Mar '06

10 Mar '06

ctypes is in SVN now, and the buildbot is green, after I disabled a test that dumped core on sparc solaris. The crash was apparently caused Missing are .vcproj files for Windows, both for the _ctypes.pyd extension and the _ctypes_test.pyd extension needed for testing. IIRC, Martin promised to create them - is this offer still valid? I could do that myself, but only for x86, while other .pyd files also have build settings for Itanium and x86_64. (I find it always very painful to click through the settings dialog in MSVC - isn't there any other way to create these files?) Thomas

5 15

Re: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py
by Tim Peters 09 Mar '06

09 Mar '06

[Tim Peters] >> Added: >> python/trunk/Tools/scripts/svneol.py (contents, props changed) >> Log: >> Simple utility to add svn:eol-style to text files under >> SVN control. Like reindent.py, I expect to run this >> mindlessly from time to time, checking in whatever it >> happens to do ;-) [Thomas Heller] > Should 'sln' and 'vcproj' be added to the extensions list? I think these are > text-files too. Although PCBuild\pcbuild.sln has a binary mime-type property, > so the script would not change that. I don't know whether they're text files in the SVN eol-style "native" sense. The corresponding file types under VC 6 were not: in SVN terms, they should have svn:eol-style set to CRLF (Visual Studio required \r\n line ends for some inscrutable reason -- it would not tolerate \n line ends, and this mattered when, e.g., some Linux-head tried to run Visual Studio from a Linux-native checkout of Python; CRLF was still required, and we "faked that" under CVS by calling those files binary; SVN offers finer control; I don't know how picky the .NET 2003 MSDev is about this).

3 5

Generated code in test_ast.py
by Grant Olson 09 Mar '06

09 Mar '06

I’m finishing up a patch for bug 1441408. I had to change the asdl definitions which in turn caused a failure in test_ast.py. A comment in the file indicates that #### EVERYTHING BELOW IS GENERATED ##### and has a definition for Slice() that is no longer valid. Can anyone tell me how this is generated? Should it be automatically generated when I run asdl_c.py? I'm developing on Windows. I know in the past some of the autogenerated code from the ast-branch worked better via make. Would this have fixed itself on a linux box, or is this a manual process? -Grant P.S. Originally sent this on Monday, still waiting moderator approval. Thought I'd give gmane a try.

2 1

quit() on the prompt
by Ian Bicking 09 Mar '06

09 Mar '06

Frederick suggested a change to quit/exit a while ago, so it wasn't just a string with slight instructional purpose, but actually useful. The discussion was surprisingly involved, despite the change really trully not being that big. And everyone drifted off, too tired from the discussion to make a change. I suppose it didn't help that the original proposal struck some people as too magic, while there were some more substantive problems brought up as well, and when you mix aesthetic with technical concerns everyone gets all distracted and worked up. Anyway, I would like to re-propose one of the ideas that came up (originally from Ping?): class Quitter(object): def __init__(self, name): self.name = name def __repr__(self): return 'Use %s() to exit' % self.name def __call__(self): raise SystemExit() quit = Quitter('quit') exit = Quitter('exit') This is not very magical, but I think is more helpful than the current behavior. It does not satisfy the "just do what I said" argument for not requiring the call (quit() not quit), but eh -- I guess it seemed like everything that didn't require a call had some scary corner case where the interpreter would abruptly exit. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

13 17