Mailman 3 November 2007 - Python-Dev

Re: [Python-Dev] Suggestions for Improvements to namedtuple
by Raymond Hettinger Nov. 14, 2007

Nov. 14, 2007

> for efficiency I would prefer to avoid using * to break > up the sequences generated directly by the database interface. There are some use cases that would be better served by an alternate design and others that are better served by the current design. For example, it would really suck to have to type-out an an enclosing set of parenthesis to build a named tuple explicitly: opt = StockOption((daysleft/360., strike/100.0, bool(putcall))). The alternate signature was discussed and … [View More]rejected previously. See some of the details on previous python-dev posting and in the ASPN recipe discussion: at some length in the ASPN recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/500261 FWIW, the current design does work efficiently with your use case if you use itertools to apply the named tuple cast to large record sets. See the example in the docs: http://docs.python.org/dev/library/collections.html#named-tuple-factory-fun… Also, fwiw, I've had good success using this starmap() approach in some performance critcal applications. Almost every class in Python could be subjected to the same discussion. Why doesn't every class and function accept a tuple of args instead of separate, serial arguments. The * operator and starmap() function were provided specifically to address the inter-conversion between the two approaches. > It would be nice to be able to have default values This came-up before, but I'll take another look at it this weekend. If it significantly complicates the API or impacts performance, then it is a non-starter; otherwise, I'll put together an implementation and float it on ASPN for comments. > In my opinion __replace__ should be able to replace multiple fields. This may be do-able. Will take a look at it. The current design is easily extended for this without impacting performance. The question is whether is makes sense in normal applications to do multiple replacements at once. FWIW, the str.replace function only has one target/replacement pair. > To me, exec is a last resort It was a last resort here also. Earlier versions of the recipe used type() but they were less flexible, slower, had crummy error reporting, and were much less clear. The exec version has the advantage of being able to print-out the template so you can see *exactly* what it is doing. I ended-up putting in a lot f field validation to prevent injection attacks (they are also they to provide more actionable error messages). > crucial stuff like __fields__ and __signature__ fully > read-write. It feels like those should be read-only properties. > ... On the other hand, I'm a recovering Java programmer, Welcome to the Python world of consenting adults. These fields could be made read-only but that is just a matter of taste. Looking through the standard library, you'll see making attributes read-only is not the typical practice. If users demand it, I don't see a problem with it, but until then, I don't see a reason to unnecessarily complicate and slow-down the code. FWIW, you're welcome to email me directly. I've been getting tons of user feedback and have already incorporated the better suggestions. Also, the recipe itself is a combination of the best of prior attempts. Unfortunately, each of several commenters revisit everything from scratch and propose their own complete rewrites that ignore the requests and commments of all those who preceded them. Fortunately, we're homing in on a darned good tool. Raymond [View Less]

1 0

CPython crash with '%o'
by Mike Stall Nov. 14, 2007

Nov. 14, 2007

This is my first post, so here's a quick intro: I've recently joined the IronPython team at Microsoft, after about 6 years as a developer on the the .NET runtime (CLR). I'm currently trying to make IronPython match CPython's behavior regarding some % format string to a level of detail not documented by the spec. (http://docs.python.org/lib/typesseq-strings.html) While exploring to determine what CPython's behavior is, I'm hitting a SystemError in CPython. Given the following snippet: class c(… [View More]

3 3

Suggestions for Improvements to namedtuple
by Isaac Morland Nov. 14, 2007

Nov. 14, 2007

I was working on something very similar to namedtuple for a project of my own, when it occurred to me that it's generally useful idea and maybe somebody else was working on it too. So I searched, and found Raymond Hettinger's addition to collections.py, and it does pretty much what I want. I have a few suggestions which I hope are improvements. I will discuss each one, and then at the bottom I will put my version of namedtuple. It is not based on the one in Python SVN because it was … [View More]mostly written before I found out about that one. If my suggestions meet with approval, I could check out a copy of collections.py and make a patch for further comment and eventual submission to somebody with checkin privileges. 1. I think it is important that there be a way to create individual namedtuple instances from an existing sequence that doesn't involve splitting the sequence out into individual parameters and then re-assembling a tuple to pass to the base tuple constructor. In my application, I'm taking database rows and creating named tuples from them, with the named tuple type being automatically created as appropriate. So there will be *lots* of named tuple instances created, so for efficiency I would prefer to avoid using * to break up the sequences generated directly by the database interface. I would like to pass those sequences directly to the base tuple constructor. To restore to my code the feature of being able to use individual parameters as in collections.py, I added a classmethod to the generated classes called __fromvalues__. This uses Signature, my other idea (next message) to convert a call matching a procedure signature of (fieldname1, ...) into a dictionary, and passes that dictionary into another classmethod __fromdict__ which creates a named tuple instance from the dictionary contents. The problem I see with this is that having to say Point.__fromvalues__ (11, y=22) instead of Point (11, y=22) is a little verbose. Perhaps there could be an __fromsequence__ instead for the no-unpacking method of instance creation, as the most common use of direct-from-sequence creation I think is in a more general circumstance. 2. It would be nice to be able to have default values for named tuple fields. Using Signature it's easy to do this - I just specify a dictionary of defaults at named tuple class creation time. 3. In my opinion __replace__ should be able to replace multiple fields. My version takes either two parameters, same as collections.py, or a single dictionary containing replacements. 4. I put as much of the implementation as possible of the named tuple classes into a base class which I've called BaseNamedTuple. This defines the classmethods __fromvalues__ and __fromdict__, as well as the regular methods __repr__, __asdict__, and __replace__. 5. It didn't occur to me to use exec ... in so I just create the new type using the type() function. To me, exec is a last resort, but I'm a Python newbie so I'd be interested to hear what people have to say about this. 6. Not an improvement but a concern about my code: the generated classes and instances have all the crucial stuff like __fields__ and __signature__ fully read-write. It feels like those should be read-only properties. I think that would require namedtuple to be a metaclass instead of just a function (in order for the properties of the generated classes to be read-only). On the other hand, I'm a recovering Java programmer, so maybe it's un-Pythonic to want stuff to be read-only. Here I would especially appreciate any guidance more experienced hands can offer. And now, here is the code, together with a rudimentary example of how this could be used to improve the "addr" functions in email.utils: #!/usr/bin/env python from operator import itemgetter class BaseNamedTuple (tuple): @classmethod def __fromvalues__ (cls, *args, **keys): return cls.__fromdict__ (cls.__signature__.expand_args (*args, **keys)) @classmethod def __fromdict__ (cls, d): return cls ([d[name] for name in cls.__fields__]) def __repr__ (self): return self.__reprtemplate__ % self def __asdict__ (self): return dict (zip (self.__fields__, self)) def __replace__ (self, *args): slist = list (self) if len (args) == 1: sdict = args[0] elif len (args) == 2: sdict = {args[0]: args[1]} else: raise TypeError for key in sdict: slist[self.__indices__[key]] = sdict[key] return self.__class__ (slist) def namedtuple (name, fields, defaults=None): fields = tuple (fields) result = type (name, (BaseNamedTuple,), {}) for i in range (len (fields)): setattr (result, fields[i], property (itemgetter (i), None, result)) result.__fields__ = fields result.__signature__ = Signature (fields, defaults=defaults) result.__reprtemplate__ = "%s(%s)" % (name, ", ".join ("%s=%%r" % name for name in fields)) result.__indices__ = dict ((field, i) for i, field in enumerate (fields)) return result from email.utils import formataddr class test (namedtuple ("test", ("realname", "email"), {'realname': None})): @property def valid (self): return self.email.find ("@") >= 0 __str__ = formataddr if __name__ == "__main__": e1 = test (("Smith, John", "jsmith(a)example.com")) print "e1.realname =", e1.realname print "e1.email =", e1.email print "e1 =", repr (e1) print "str(e1) =", str (e1) e2 = test.__fromvalues__ (email="test(a)example.com") print "e2 =", repr (e2) print "str(e2) =", str (e2) Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist [View Less]

1 0

Summary of Tracker Issues
by Tracker Nov. 14, 2007

Nov. 14, 2007

ACTIVITY SUMMARY (11/07/07 - 11/14/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1325 open (+18) / 11600 closed (+25) / 12925 total (+43) Open issues with patches: 421 Average duration of open issues: 689 days. Median duration of open issues: 797 days. Open Issues Breakdown open 1320 (+18) pending 5 ( +0) Issues Created Or Reopened (44) _______________________________ … [View More]

1 0

Re: [Python-Dev] New python developer
by Raymond Hettinger Nov. 14, 2007

Nov. 14, 2007

> As you may have seen, I have recently been granted developer > privileges on python svn. Hello Amaury. Welcome aboard. Raymond

1 0

Summary of Tracker Issues
by Tracker Nov. 13, 2007

Nov. 13, 2007

ACTIVITY SUMMARY (11/06/07 - 11/13/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1323 open (+17) / 11597 closed (+23) / 12920 total (+40) Open issues with patches: 418 Average duration of open issues: 689 days. Median duration of open issues: 795 days. Open Issues Breakdown open 1318 (+17) pending 5 ( +0) Issues Created Or Reopened (41) _______________________________ … [View More]

1 0

XML codec?
by Walter Dörwald Nov. 13, 2007

Nov. 13, 2007

I have a patch ready (http://bugs.python.org/issue1399) that adds an XML codec. This codec implements encoding detection as specified in http://www.w3.org/TR/2004/REC-xml-20040204/#sec-guessing and could be used for the decoding phase of an XML parser. Other use cases are: The codec could be used for transcoding an XML input before passing it to the real parser, if the parser itself doesn't support the encoding in question. A text editor could use the codec to decode an XML file. When the … [View More]

8 49

Fwd: volunteers for a test
by Guido van Rossum Nov. 12, 2007

Nov. 12, 2007

Please indulge this off-topic post. I know there are a lot of Python developers here whose mother tongue is not English; would those folks mind participating in the experiment below? --Guido ---------- Forwarded message ---------- From: Jose Oriol Lopez Berengueres <jose.oriol.lopez.berengueres(a)cern.ch> Date: Nov 12, 2007 12:22 PM Subject: volunteers for a test To: guido(a)python.org Dear Mr. Guido, A coleague and me at CERN are trying to measure the productivity differential … [View More]

1 0

Proposal for new 2to23 tool
by Graham Horler Nov. 12, 2007

Nov. 12, 2007

I have been developing in Python since 1.5, and now have to support 2.1 as a minimum version. I do like to keep my code runnable on newer versions however, and am considering the feasability of forward compatibility with Python 3.0. I also notice the Leo[1] project could use some assistance with forward compatibility. So I was wondering if anyone else had a need for a 2to23.py tool to help make code compatible with 3.0 but not break it for 2.x. Such a tool could also include implementations … [View More]

6 7

Summary of Tracker Issues
by Tracker Nov. 12, 2007

Nov. 12, 2007

ACTIVITY SUMMARY (11/05/07 - 11/12/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1321 open (+17) / 11592 closed (+21) / 12913 total (+38) Open issues with patches: 418 Average duration of open issues: 689 days. Median duration of open issues: 791 days. Open Issues Breakdown open 1316 (+17) pending 5 ( +0) Issues Created Or Reopened (39) _______________________________ … [View More]

1 0