Mailman 3 November 2008 - Python-Dev

AST-level type inference optimizations
by David Turner 05 Nov '08

05 Nov '08

I wrote a patch to Tom Lee's AST optimization branch, which I have submitted at http://bugs.python.org/issue4264. Here's a brief explanation of what the patch does, followed by a little general discussion of the future. Python bytecode includes at least one operation which is not directly accessible from Python code: LIST_APPEND, which pops a value and a list off the stack and appends the value to the list. This is used in generating code for list comprehensions: [x for x in somelist] generates the following code for the body of the loop: ... LOAD_FAST 1#a local is generated to hold the new list LOAD_FAST 2 #the iteration variable, x LIST_APPEND ... Whereas if you were to try to write the comprehension directly, you would get: LOAD_FAST 1 LOAD_ATTR 0 #an index into the constant table: “append” LOAD_FAST 2 CALL_FUNCTION 1 #remove x and the append function from the top of the stack and push the result of the call POP_TOP This is much slower. In part, it’s the cost of doing the attribute lookup each time, which is why it’s common to see code like a = [] ap = a.append for x in ...: ap(x + ...) return a But the function call is naturally slower than the simpler LIST_APPEND operation. My patch tries to determine statically if a local is all circumstances holds a list, and if so, generates LIST_APPEND whenever user code would call local.append. It also works with the manual optimization mentioned above -- it tracks whether variables hold a method call to append on a list. It’s not perfect — in particular, I could track local types on a more fine-grained level than per-function. Anyway, the patch is a win on microbenchmarks. There's a lot more we could do with function-local type inference. We could eliminate temp variables, and unnecessary stores to variables that are never used again. We could potentially track the types of what goes into lists, and generate special-case code for numbers (using array), or strings (using stringio or something). And there's probably more than I'm not thinking of right now. What do you think?

1 0

Why don't range and xrange threat floats as floats?
by L V 05 Nov '08

05 Nov '08

Why don't range and xrange threat floats as floats? Is there any good reason range and xrange don't threat floats as floats but as integers? When I enter float arguments in a range, the floats are treated as integers. (+ some warning) This is how I think it should work: >>>range(0, 1, 0.1) [0.0, 0.10000000000000001, 0.20000000000000001, 0.29999999999999999, 0.40000000000000002, 0.5, 0.59999999999999998, 0.69999999999999996, 0.80000000000000004, 0.90000000000000002] I got these results by: >>>[x/10. for x in xrange(10)] I'm not looking for alternatives, just for the reason that it works this way.

3 3

Looking for VCS usage scenarios
by Jim Jewett 05 Nov '08

05 Nov '08

David Ripton wrote: > Time for average user to check out Python sources with bzr: 10 minutes > Time for average user to check out Python sources with git or hg: 1 minute > Time for average user's trivial patch to be reviewed and committed: 1 year > I love DVCS as much as the next guy, but checkout time is so not the > bottleneck for this use case. I think Paul's point is that he wants to support people who have not previously planned to contribute to python. Writing the patch may be a matter of minutes, once they implement the fix for themselves. Downloading a new VCS is a major commitment of time and disk space. (And is there setup, and dealing with proxies?) It doesn't take as long (calendar) as waiting for the review, but it takes long enough (clock) that people may not bother to do it. And if they don't, what was the point of switching to a DCVS? -jJ

1 0

test - please ignore
by skip＠pobox.com 05 Nov '08

05 Nov '08

(working on spam system - just hit 'd')

1 0

file open in python interpreter
by adrian golding 04 Nov '08

04 Nov '08

hi all, i am trying to find out where is the part of the code in the python interpreter that opens up the .py file and parses it. in particular, i am trying to find the file open command in that file. I greped and i just want to make sure this is it: /Python-2.6/Parser/pgenmain.c i am intending to take a hash measurement of the .py file just before i open it to run the script. is the above file the right place to call for the measurement before the file open function? thank you - adrian

4 6

hg branch gone?
by Brett Cannon 04 Nov '08

04 Nov '08

I just tried to update my 3.0 branch in hg from http://code.python.org/hg/branches/py3k/ and hg is telling me it's a 404. Anyone else having trouble? -Brett

8 20

XXX do we need a new policy?
by Benjamin Peterson 04 Nov '08

04 Nov '08

Grepping through Python's sources tells me that we have over 2,000 "XXX" comments. The thing that irks me about them is that the have a very slow rate of being resolved, since they usually act more as "notes to self" rather than easily attainable tasks. So, I propose that we adopt a policy similar to Twisted's: All "XXX" comments must have an issue in the tracker and an accompanying link with the source code. That way we'll have a forum for discussing the changes and deciding whether they are reasonable enough to implemented. -- Cheers, Benjamin Peterson "There's nothing quite as beautiful as an oboe... except a chicken stuck in a vacuum cleaner."

8 11

Python2.5 _sre deepcopy regression?
by Andrew McNamara 04 Nov '08

04 Nov '08

In version of Python prior to 2.5, it would appear that deepcopying compiled regular expressions worked by accident: 2.4: >>> copy.deepcopy(re.compile('')) <_sre.SRE_Pattern object at 0xb7d53ef0> 2.5: >>> copy.deepcopy(re.compile('')) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.5/copy.py", line 173, in deepcopy y = copier(memo) TypeError: cannot deepcopy this pattern object I say "by accident", since the SRE_Pattern object in 2.4 has a __deepcopy__ method which raises the "cannot deepcopy this pattern object" TypeError, however this method isn't found by copy.deepcopy() in 2.4, and copy.deepcopy() falls back to using the pickle logic. The _sre source has #ifdef-out support for __deepcopy__, issue 416670 has the gory details: http://bugs.python.org/issue416670 Changeset 38430 on the release24-maint branch introduced the changes that stopped SRE_Pattern.__deepcopy__ being found. r38430 was a patch forward ported from 2.3, but never ported to the trunk (probably a good thing, too). Thoughts? -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

1 1

New "stage" field in the tracker
by Brett Cannon 03 Nov '08

03 Nov '08

Martin added a new "stage" field in the tracker so that issues can now be more clearly identified in terms of what is needed to move them forward. For now it is probably best to continue to use both keywords and stage values (i.e. the "patch" keyword is implied when the stage of an issue goes beyond "needs patch" and the need for a "patch review" is covered by the "commit review" stage). Once the use of the stage gets far enough along we can discussed ditching the keywords. Anyway, cheers to our first step in improving our workflow! -Brett

1 0

Packaging the PyPI version of the SSL module for Debian
by Bill Janssen 03 Nov '08

03 Nov '08

As some of you know, I've provided a PyPI version of the 2.6/3.x "ssl" module, for use with older versions of Python. I've received this request to tweak it for Debian, and I thought I'd ask those of you who may have already done it for advice on the various issues Cyril raises here. I'm not Debian-knowledgable, myself. Bill ---------- Forwarded message ---------- From: Cyril Brulebois <kibi(a)debian.org> Date: Mon, Jul 14, 2008 at 11:37 AM Subject: Packaging (python-)ssl for Debian To: python.ssl.maintainer(a)gmail.com Hi, I've noticed the following Request For Packaging [1], and it turned out that it was exactly what I was looking for (a real ssl module, not just read/write on a socket). So I started packaging it for Debian, and here are a few comments, and some points that I'd like you to address, so that I can upload it into the archive. 1. http://bugs.debian.org/453101 The main concern currently is the license, since "Python (MIT-like)" is quite vague. One has no clue which Python version is concerned, since as far as I can tell, there are at least 2.4.2 and 2.5.2 [2,3] out with (slightly but still) different licenses. So, pointing to the precise version would really help people. At least the URL to the license would be needed, but it would be even better to include it into the source package, so that there's no possible doubt, and so that one isn't dependent from being online to actually read it. 2. http://www.python.org/download/releases/2.4.2/license/ 3. http://www.python.org/download/releases/2.5.2/license/ I'd suggest including it in a LICENSE file, and then point to it using something like "LICENSE: Python x.y.z (MIT-like), see LICENSE file". It's then up to you to choose whether to state clearly in each file "This file is licensed under the terms of the Python x.y.z license, see LICENSE file." (which I encourage you to do), or to state at the very beginning of this LICENSE file that "the whole (python-)ssl package is licensed under the following terms (Python x.y.z license):" and then quote your favourite license. Also, you're listing the contributors in various places, PKG-INFO and setup.py. That's nice, but it might be a better idea to point to a single AUTHORS file, where you would keep a single list of the contributors. It's not a problem per se, but you may want to consider this. Related to authorship is copyright assignment, and that one is a blocker for the inclusion into Debian. Every file should include copyright information so that one can know who is holding the copyrights on this or that file. Having a very quick look, the following files would need it: *.py, *.h, *.c. PKG-INFO could eventually list all copyright lines, but I'm not sure it's a widely-used practice in the Python world, just tell me ;-); the Makefile might deserve (c) info as well; MANIFEST and *.pem files don't need it. Just for the records, one specifies copyrights in the following manner: Copyright 2008 Some Author Copyright 2004, 2007 Another One Copyright 2004-2008 Yet Another One "Copr." or "(c)" can be used instead of "Copyright". "(C)" can be added (it's harmless) but has no legal meaning, so can't be used alone. No need to list everyone having ever contributed a single patch, it's sufficient to list people having done the real work and worthy modifications to it. Adding people to copyright holders is usally a matter of project policy. If you don't feel like listing the copyright holders in PKG-INFO, you could decide for using something like: ,---[ AUTHORS ]--- | | Main developers: | Copyright 2004 John Doe | Copyright 2004-2008 Alice Wonderland | | Contributors: | Peter Pan | Foo Bar | `--- This way, you just have to do the following to keep your software uptodate WRT legalese things: - Add/update a Copyright line at the top of .c/.h/… file when a significant modification is included. - Add/update the Copyright line in the AUTHORS file accordingly. - Or add/update the Contributors list if that doesn't warrant a Copyright line. So that there's no need to update every file (like PKG-INFO and setup.py), you only have to make them point to AUTHORS for authorship information. Once that is done, I should be able to upload it to Debian, where the first upload will be double-check by ftpmasters, whose role is to make sure that no obvious packaging error is made, but mainly to make sure that the archive is kept OK from a legal point of view. That's why I'm starting with this quite boring mail, and I'm sorry for that. :) Mraw, KiBi. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkh7quoACgkQeGfVPHR5Nd1LQwCaAＤ艊�볁ᜆ뻈働� 2Y0AnitpaRf3zybMhNxstnDrKaJc8K3/ =eriZ -----END PGP SIGNATURE-----

1 0