Mailman 3 July 2003 - Python-Dev

A syntax for function attributes?
by Michael Chermside 01 Aug '03

01 Aug '03

Pat Miller writes: > I can imagine: > > def sqr [inline=true](x): > return x*x [...] > I think it essential to use some dense syntax. It would be messy if it > were > > def sqr(x): > return x*x > sqr.inline = true I don't object to a syntax for function attributes... in fact, I've seen no specific proposal to object to. But I find your point above HIGHLY unconvincing: Python 2.3 (#46, Jul 29 2003, 18:54:32) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> version_1 = """def sqr [inline=true] (x): ... return x*x ... """ >>> version_2 = """def sqr(x): ... return x*x ... sqr.inline = true ... """ >>> len(version_2) - len(version_1) 4 Here's what WOULD convince me. Mark Nottingham writes: > I would very much like to use function > attributes for associating metadata with functions and methods, but the > lack of such syntactic support precludes their use, so I end up > (ab)using __doc__. Rather than introducing the syntax first, let's start by making it POSSIBLE somehow, even if the syntax is awkward. That's already been done -- functions can have attributes. THEN, let's write some cool code that uses feature, and we'll learn how it works out in practice. Only once we see how very useful this really is should we consider introducing special syntax for it, because the less special syntax that Python has, the better it is (if we keep functionality constant). Notice that this is the approach that Guido used for classmethod and its ilk, and we're still working out what the right "special syntax" is for those, but in the meantime we're seeing that they are getting lots of use. I think this approach (first prove it's useful for Python, THEN add syntax to the language) is preferable for ALL new "special syntax" except in three cases. One exception is features like generators, which are instantly recognized by all as being enormously useful, so we skipped the "whether to do it" stage and went right to the discussion of _what_ the special syntax should be. Another exception is features which simply CAN'T be done without introducing new syntax... like PEP 310 where we prevent the race condition. And the third exception is cases where the proponants claim that the syntax change itself is what makes the difference -- for instance those who point to ruby's blocks as very useful while Python's lambda's have just a little bit too much typing. (Not that I agree, necessarily, but they're entitled to make their point.) Other than that, let's always try things out before adding special syntax. Now-my-fourth--no-AMONGST-my-exceptions-lly yours, -- Michael Chermside

3 2

buglet in interaction between pygame, Numeric & copy in Python 2.3
by Alex Martelli 01 Aug '03

01 Aug '03

I'm sending this buglet-report to the discussion lists for each of pygame, Numeric, and Python-Dev, because it ends with a recommendation for changes (at least to docs) in each of the two extension packages and Python 2.3.1. I'm not a regular user of pygame [I am one of Numeric]. I helped some pygame users find out why their program was breaking (in just one point) when migrating Python 2.2 -> 2.3 (Numeric 23.0, pygame 1.5.6 from cvs snapshot). The problem can be narrowed down the following little buglet.py script: import copy, pygame.display, pygame.surfarray screen = pygame.display.set_mode((640,480),0,24) surf = pygame.surfarray.pixels3d(screen) xx = copy.copy(surf[10:20,30]) With python 2.2, this runs without errors. With 2.3: Traceback (most recent call last): File "buglet.py", line 6, in ? xx = copy.copy(surf[10:20,30]) File "/usr/local/lib/python2.3/copy.py", line 101, in copy raise Error("un(shallow)copyable object of type %s" % cls) copy.Error: un(shallow)copyable object of type <type 'array'> Workaround for the problem: add an "import Numeric" at the top of buglet.py. Reason for the problem: copy.copy uses different mechanisms in 2.3 vs 2.2 for shallow copies of Numeric.array instances. In 2.2, it looked for a method __copy__ on the instance, found it, and happily used it. In 2.3, it relies on getting a 'reductor' function from the copy_reg.dispatch_table. Module Numeric installs such a reductor *WHEN IMPORTED* -- but pygame never imports it even though functions such as the above-used pixels3d do return instances of Numeric.array. So, Numeric.py doesn't run, and therefore cannot install its pickle_array function with the copy_reg module. Suggested fix in pygame: I would suggest making sure Numeric has been imported when Numeric.array instances are to be returned. That's up to the pygame maintainers, of course; at the very least, I think this issue should be mentioned in pygame's docs (asking the user to make sure "import Numeric" happens in any pygame-using program that manipulates such instances). Suggested fix in Numeric: Numeric cannot DO anything much about this issue, I think, but it should probably *mention* (in the docs regarding interfacing to Numeric from other C-coded extensions) that other extensions should make sure Numeric is _imported_ (from a Python POV), if said extensions return instances of Numeric.array, to ensure such instances have full functionality (and in particular shallow copyability). Suggested fix in Python: a mention in the "what's new" document, section "porting to Python 2.3", is probably warranted -- not so much about Numeric and/or pygame extensions per se, though these are both popular 3rd party extensions, but rather the excellent changes in the shallow copy mechanics, which may indeed require code changes on the part of extension authors. Thanks, Alex

3 3

urllib2 extensibility patch -- comments?
by John J Lee 01 Aug '03

01 Aug '03

It would be great if anyone could look at this RFE patch I posted a while back: http://python.org/sf/759792 One of urllib2's main reasons-for-being is to enable extensions that are reasonably independent of each other. This patch enables that for a fairly big class of extensions. I won't say any more here -- it's all in the RFE. The reason I'd appreciate comments on it now is that I'm working on cookie code that I hope to get into 2.4. I don't want to release a version of my code with this change if the change isn't going to get into the Python library. John

1 0

Re: zip() in Py2.4
by Guido van Rossum 31 Jul '03

31 Jul '03

[Argh! My followup got swallowed by a disconnect.] The CA air is definitely cool, but I'm not quite settled -- I'm still living in a hotel room with my family, at least until Friday. My rationale for zip() refusing to be called without arguments is that this is more likely a mistake than intentional. Admittedly I hadn't foreseen using zip(*xxx). There are two sides to this. 1) Someone mistakenly types zip(). If they get an empty list, how confused will they be? Will their program finish with bogus output, or will the empty list cause a speedy failure? 2) How realistic is the use case for zip(*[])? --Guido

2 1

Python version number in file names
by Jack Jansen 31 Jul '03

31 Jul '03

Something that came up in the 2.3 alfa/beta/release cycle is that there is a buglet in our version numbering scheme. We use "major.minor" as the indicator of binary compatibility, and hence use "lib/python2.3" to put things, distutils uses "2.3" in its build directory and distribution filenames, etc. There are exceptions to this binary compatibility rule, however, and that are the alfa and beta releases: these are explicitly not guaranteed to be compatible. This problem has always existed, but as far as I know it was theoretical until Apple shipped a Python 2.3 beta with their Panther beta. All betas, but still this means that there is now a much bigger chance than their ever was of people trying to use things built for 2.3final with 2.3b2, or people building modules with 2.3b2 for quite some time, which others will then try to install into 2.3final. I think it's a good idea to fix this. I thought I had an algorithm to do this using '.'.join(version.split('.')[:2]) in stead of version[:3], but unfortunately this will make the release candidates also not use "python2.3", and I think for these we definitely want to use the final names. Still, an algorithm shouldn't be rocket science:-) Opinions? -- - Jack Jansen <Jack.Jansen(a)oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

3 2

How to get the .spec file version number updated.
by Sean Reifschneider 31 Jul '03

31 Jul '03

What has to be done to make sure that the version number in the .spec file gets updated along with other files before pushing a release out? Is there a script I need to patch (or write) for this, or does the PEP template need to be changed or something? Thanks, Sean -- "All I'm saying is that when I'm around you I find myself showing off, which is the idiots version of being interesting." -- _LA_Story_ Sean Reifschneider, Member of Technical Staff <jafo(a)tummy.com> tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin

2 1

A syntax for function attributes?
by Mark Nottingham 31 Jul '03

31 Jul '03

PEP 232 lists syntactic support for function attributions to be a possible future direction. I would very much like to use function attributes for associating metadata with functions and methods, but the lack of such syntactic support precludes their use, so I end up (ab)using __doc__. Has there been much further consideration of this issue? I'm not too particular about the chosen syntax, I just need something that doesn't require qualification with the function name (which tends to reduce readability/typeability, in some cases drastically so). I'm happy to write a PEP if that will help, but wanted to get a sense of what people's thinking was. Cheers, [ pls cc: me in responses ]

5 4

Re: [Python-Dev] zip() in Py2.4
by Raymond Hettinger 30 Jul '03

30 Jul '03

> > After Guido gets settled in, I'll see if the California air has > > changed his thinking. If it hasn't, I'll at least get the > > rationale documented. > > I don't know about the CA air (it's nice and col here though) > and I'm not quite settled yet (still living in a hotel room > with my family at least until Friday) but the rationale > is that I expect zip() to be called by mistake more likely > than with an intention. Now, the question is if that mistake > will be hidden by returning [] or not. IOW whether returning > [] causes a broken program to limp along with bogus data, > or whether it will cause a clear-enough error to happen next. > > What are typical use situations for zip(...) these days? The main use is still lockstep iteration in for-loops. If someone mistakenly omits a zip argument here, the mistake will still surface immediately with a [] return value: for timestamp, user, event in zip(): pass # never gets here because the tuple won't unpack Another use is for joining together keys/indices and values and then undoing the join (like a Schwarzian transform): >>> k = ('pear', 'apple', 'banana', 'coconut') >>> i = range(len(k)) >>> temp = zip(k, i) >>> temp [('pear', 0), ('apple', 1), ('banana', 2), ('coconut', 3)] >>> temp.sort() >>> fruits, indices = zip(*temp) >>> fruits ('apple', 'banana', 'coconut', 'pear') >>> indices (1, 2, 3, 0) Another use is a fast row/column switch in rectangular tables organized as lists of tuples. > Can you elaborate the case for zip() without arguments? Yes, it comes up whenever zip() is used with the * operator (as in the last two examples above). When there are variable length argument lists, it is helpful to have a smooth transition to the null case -- for instance a zero matrix used with: def transpose(mat): return zip(*mat) Transposing is a common first step in when reading CSV files so that the tuples (records) of non-homogenenous data can be re-grouped into homogenous data from each column position (field): dates, rainfall, hightemp, lowtemp = zip(*csv.reader(file("weather.csv"))) print 'Monthly total rainfall', sum(rainfall) Subsequent to the time the PEP was decided, I think the trend has been for python to handle null cases the same as regular cases and not raise exceptions. For instance, sum([]) returns 0 instead of demanding at least one addend. The star operator is also used in the unzip operation, described above as zip(*temp) in the Schwarzian transform example. Unzipping a reasonably common way of breaking out data aggregates. IOW, zip(*something) is handy for re-organizing data and the empty dataset should need not special case handling. This is especially true when data is built-up from the null case: mytable = [] gen_report(mytable) # don't have this raise an exception mytable.append(newrecord()) gen_report(mytable) mytable.append(newrecord()) gen_report(mytable) mytable.append(newrecord()) That about covers the subject. Raymond Hettinger

2 3

RE: [Python-Dev] zip() in Py2.4
by Moore, Paul 30 Jul '03

30 Jul '03

From: Barry Warsaw [mailto:barry@python.org] > Raymond, I remember that we made an explicit decision for zero arg zip() > to raise an exception, but I don't remember what the rationale was. Can > you go back into the archives and see if you can find that discussion (I > believe zip() predates the peps). It's PEP 201, and that contains a statement that Guido pronounced explicitly on 0-arg zip producing an exception. Sadly, there's no rationale given (but Guido doesn't need one :-)) Paul.

3 2

zip() in Py2.4
by Raymond Hettinger 30 Jul '03

30 Jul '03

Currently, zip() with no arguments raises an exception. Does anyone have objections to my having it return an empty list instead? A poster on comp.lang.python raised some valid objections to the current behavior and I see no use case for wanting to have an exception instead of a useful return value. zip() --> [] would make the zip function more useful and predictable when used with variable length argument lists: def transpose(matrix): return zip(*matrix) On python-dev, when other features have been implemented, there have been discussions about whether zero cases should raise an exception or have a return value that is a more natural extension of the non-zero cases. I believe the resolution was typically to choose the latter (i.e. '' in 'abc'). This allows the patterns to extend more naturally: >>> zip('abc', range(3)) [('a', 0), ('b', 1), ('c', 2)] >>> zip('abc') [('a',), ('b',), ('c',)] >>> zip() [] FWIW, I've tried it out and found that it only touches a couple of lines of C, it breaks no test cases other than the two which serve to verify the current behavior; my existing apps and downloaded python apps all run fine. IOW, while it is a backwards incompatible change, I do not expect to find more than a modicum of existing code relying on an exception from zip() with no arguments. Another nice side benefit is that it simplifies the documentation and makes zip() easier to learn in the first place. Code surrounding a zip(*args) becomes more clear without a surrounding try/except. Avoiding unnecessary exceptions is part of the reason for methods like dict.get(k) which make coding a little more straight-forward. Raymond Hettinger

2 1