Mailman 3 September 2004 - Python-Dev

PEP 265 - Sorting dicts by value
by David Harrison 13 Sep '04

13 Sep '04

Hi all, Quick pep265 summary : People frequently want to count the occurrences of values in a dict, or sort the results of a d.items() call by value. This could be done by extending the current items() definition, or by creating a new function for the dict object (both requiring a C implementation). I've had a read through pep265 a few times now, and every time I've had two immediate reactions. First, that I've been there too. I've found myself innumerable times needing to count the occurrences of values in a dict. However second, that dicts shouldn't be naturally sortable. A dict does not guarantee the order that it returns calls such as items() keys() or values(). It's my feeling that we should not encourage people to rely on a dict returning a set ordering, since as a hash based data structure they are designed for key lookup not sequential traversal - if you want to sort something, massage the data into a list and then sort the list (I've seen a proposal before that the sort function be able to handle objects which would allow sorting of 2 dimensional lists). With regards to the two arguments put forward by Grant, the first - that it is an idiom known only to experienced campaigners - does not seem to be a supportable argument to me. I think the problem has quite a simple elegant solution which is rather easily discovered - there are lots of differences in Python that require an inexperienced programmer to learn a new idiom (such as the looping construct). The second, that the solution is full of 'grunge', seems a matter of taste and use to me. As mentioned in the pep there are different kinds of comparison that may be wanted, but could not be supported. Further, it is a natural use case of a dict that items held within it need not be of the same type (and therefore makes the idea of a comparison between them meaningless). With respect to implementation suggestions, numbers 1 2 and 3 definitely don't work for me. To extend the usage of items() without similarly extending the usage of keys() and values() would mean that we are special casing the items() function in a way that makes it inconsistant with the other dict functions. Number 5 seems too specific to me. I could live with 4 ;-) I think in the end it's my feeling that these kind of idioms belong in the cookbook - which, incidentally, it already is to a certain extent under 'Sorting a Dictionary', another recipe could always be added for this ;-) cheers Dave Harrison

3 3

PEP 265 - Sorting dicts by value
by David Harrison 13 Sep '04

13 Sep '04

> > In the sense that items() can still be used as before, it remains > > consistent. However since the same argument could be used to equally > > promote the modification of other dict functions to accept such > > arguments - such as keys(values_first=0) - to make the change to > > items() alone is (in my humble opinion) inconsistent. > > But that's just my opinion, others may feel differently. > > To me, neither mydict.keys(values_first=whatever) nor > mydict.values(values_first=whatever) make any sense: these methods > return a list of only keys or only values, so saying "values first" > when you're getting a list of keys is meaningless. Oops my mistake, I was thinking along the lines of requesting a sort order based on value (i.e. keys() returns in the order of its values, increasing or decreasing). So my misunderstanding aside ;-) ... We would have the following situation (just to clarify) : >>> d = { 'a' : 1 , 'b' :2, 'c':0 } >>> itemList = d.items(values_first=1) >>> itemList [(1, 'a'), (2, 'b'), (0, 'c')] >>> itemList.sort() >>> itemList [(0, 'c'), (1, 'a'), (2, 'b')] That is probably my preferred option for this usage. But to play devil's advocate for a minute. Considering that a dict is a key based data structure and not a sequential structure, does it really make sense to be able to request its inversion ? For example : >>> d = { 'a' : [1,2,3] , 'b' : [4,5,6], 'c':[7,8,9] } >>> d.items(values_first=1) [ ([1,2,3], 'a'), ([4,5,6], 'b'), ([7,8,9], 'c') ] This just doesnt make sense, nor would it make sense if the values were objects. The only use case raised was for counting instances of an item in a dict, and then inverting _that_ dict (ie. the one that stored the count values) . e.g. for key in d.keys(): d[key] = d.get(key, 0) + 1 items = [(v,k) for k,v in d.items()] items.sort() items.reverse() items = [(k,v) for v,k in items] So I'll also raise again my question of whether it is reasonable to implement a functionality that is not going to be used as a part of the primary purpose of a dict. This functionality extension is just an implementation of one way of using a dict - not necessarily an example of 'missing functionality' to me.

1 0

Re: [Python-Dev] PEP 265 - Sorting dicts by value
by Andrew Durdin 13 Sep '04

13 Sep '04

On Mon, 13 Sep 2004 12:29:11 +1000, David Harrison <dave.l.harrison(a)gmail.com> wrote: > > In the sense that items() can still be used as before, it remains > consistent. However since the same argument could be used to equally > promote the modification of other dict functions to accept such > arguments - such as keys(values_first=0) - to make the change to > items() alone is (in my humble opinion) inconsistent. > But that's just my opinion, others may feel differently. To me, neither mydict.keys(values_first=whatever) nor mydict.values(values_first=whatever) make any sense: these methods return a list of only keys or only values, so saying "values first" when you're getting a list of keys is meaningless.

1 0

PEP 265 - Sorting dicts by value
by David Harrison 12 Sep '04

12 Sep '04

> > With respect to implementation suggestions, numbers 1 2 and 3 definitely > > don't work for me. To extend the usage of items() without similarly > > extending the usage of keys() and values() would mean that we are > > special casing the items() function in a way that makes it inconsistant > > with the other dict functions. Number 5 seems too specific to me. I > > could live with 4 ;-) > > To quote the PEP: > """ > Alternatively, items() could simply let us control the (key, value) > order: > > (3) items(values_first=0) > """ > This suggestion No. 3 from the PEP does not special case the items() > function in a way that makes it "inconsistent with the other dict > functions" (i.e. keys(), values()); however it would suggest that > dict() then also ought take such an inverted, values-first list of > tuples if given an optional values_first parameter. But this IMHO > makes the dict() constructor too complicated, as well as having a > potential conflict with named keywords. In the sense that items() can still be used as before, it remains consistent. However since the same argument could be used to equally promote the modification of other dict functions to accept such arguments - such as keys(values_first=0) - to make the change to items() alone is (in my humble opinion) inconsistent. But that's just my opinion, others may feel differently.

1 0

PEP 328 - Relative Imports
by M.-A. Lemburg 12 Sep '04

12 Sep '04

Hi there, I know that this has been discussed a few times in the past, but the more I have to deal with building applications using third-party libs or packages, the more I get the feeling that the choice of making "import module" absolute is the wrong path to follow. The typical scenario goes like this: * you build an application that uses various third-party packages and has to maintain them inside another package, e.g. ThirdPartyCode * you don't have access to the (third-party) package source code or it's not feasable to make changes to it for maintenance reasons Another common case is that you have to deal with third-party code that is not properly packaged as Python package, but comes as a set of top-level modules. In this scenario you typically put all those files into a newly created Python package directory and access the modules in that directory using the package name. In Python 2.3 and 2.4 (as well as all previous versions), both scenarios can easily be implemented without having to change the third-party code. The PEP however suggests that starting with 2.5, the interpreter will issue a warning and 2.6 should default to absolute paths. I'd like to request that the latter change be postponed to Python 3k, or that some other way of supporting the above scenarios is provided that can be enabled in the application. Please remember that changes to application code are well possible. What's not possible is making changes to the packaged third-party code. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 08 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

7 11

Re: [Python-Dev] SHA-256 module
by Tim Peters 12 Sep '04

12 Sep '04

[Michael Hudson, on 30 June 2004] >> Nevertheless, am I right to still believe that there are no known >> distinct strings which even MD5 to the same hash? [Andrew Kuchling] > Correct. And two months later, the world is all different again: """ import md5 S = ('\xd11\xdd\x02\xc5\xe6\xee\xc4i=\x9a\x06\x98\xaf\xf9\\' '/\xca\xb5\x87\x12F~\xab@\x04X>\xb8\xfb\x7f\x89U\xad4' '\x06\t\xf4\xb3\x02\x83\xe4\x88\x83%qAZ\x08Q%\xe8\xf7' '\xcd\xc9\x9f\xd9\x1d\xbd\xf2\x807<[\x96\x0b\x1d\xd1' '\xdcA{\x9c\xe4\xd8\x97\xf4ZeU\xd55s\x9a\xc7\xf0\xeb' '\xfd\x0c0)\xf1f\xd1\t\xb1\x8fu\'\x7fy0\xd5\\\xeb"' '\xe8\xad\xbay\xcc\x15\\\xedt\xcb\xdd_\xc5\xd3m\xb1' '\x9b\n\xd85\xcc\xa7\xe3') T = ('\xd11\xdd\x02\xc5\xe6\xee\xc4i=\x9a\x06\x98\xaf\xf9\\' '/\xca\xb5\x07\x12F~\xab@\x04X>\xb8\xfb\x7f\x89U\xad4' '\x06\t\xf4\xb3\x02\x83\xe4\x88\x83%\xf1AZ\x08Q%\xe8\xf7' '\xcd\xc9\x9f\xd9\x1d\xbdr\x807<[\x96\x0b\x1d\xd1\xdcA{' '\x9c\xe4\xd8\x97\xf4ZeU\xd55s\x9aG\xf0\xeb\xfd\x0c0)' '\xf1f\xd1\t\xb1\x8fu\'\x7fy0\xd5\\\xeb"\xe8\xad\xbayL' '\x15\\\xedt\xcb\xdd_\xc5\xd3m\xb1\x9b\nX5\xcc\xa7\xe3') assert S != T print md5.new(S).hexdigest() print md5.new(T).hexdigest() print "oops" """ A number of hash functions got cracked since this thread started, by some researchers in China: http://eprint.iacr.org/2004/199.pdf MD5 is truly dead now for "secure" applications. Maybe someone who gives a rip <wink> could update the docs. Best I understand it, SHA-1 still stands, although a variant with half the rounds has been cracked. It does increase the desirability (IMO) of adding SHA-256, lest SHA-1 get cracked too while Python 2.4.j is still current.

3 3

doctest and inspect.getmodule
by Johannes Gijsbers 11 Sep '04

11 Sep '04

I just checked in a change to inspect.getmodule (without running the tests beforehand, not a smart move) which broke a whole bunc of tests for doctest. The tests mostly seem to fail because doctest can find modules for objects it previously couldn't. I think the change is basically correct, but I'm not sure how to fix doctest. Should doctest omit the module, or should the doctest tests be changed to expect the module being printed? Oh, I promise I'll run the tests before checking in next time. Johannes P.S.: here's the checkin message for the change: Modified Files: inspect.py Log Message: Use __module__ attribute when available instead of using isclass() predicate (functions and methods have grown the __module__ attribute too). See bug #570300. Index: inspect.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/inspect.py,v retrieving revision 1.54 retrieving revision 1.55 diff -u -d -r1.54 -r1.55 --- inspect.py 18 Aug 2004 12:40:30 -0000 1.54 +++ inspect.py 11 Sep 2004 15:53:22 -0000 1.55 @@ -370,7 +370,7 @@ """Return the module an object was defined in, or None if not found.""" if ismodule(object): return object - if isclass(object): + if hasattr(object, '__module__'): return sys.modules.get(object.__module__) try: file = getabsfile(object)

2 1

RE: [Python-Dev] Re: Missing arguments in RE functions
by Edward Loper 11 Sep '04

11 Sep '04

> Right now, there are some things you can do with the RE functions > and a DIFFERENT set of things you can do with the compiled REs. > That's TWO sets of functionality to learn. If Noam's patch can > make the feature set of the RE functions the SAME as the feature > set of the compiled REs, then there's only ONE set of features to > memorize. On the whole, there are MORE indiviual "pieces" to the > API but because of orthogonality the API as a whole is simpler. > Therefore in this case I favor using Noam's patch. +1. Consistency makes the API conceptually simpler, even if the absolute number of parameters is larger. -Edward

5 9

Adding status code constants to httplib
by Andrew Eland 11 Sep '04

11 Sep '04

Hi, Over in web-sig, we're discussing PEP 333, the Web Server Gateway Interface. Rather than defining our own set of constants for the HTTP status code integers, we thought it would be a good idea to add them to httplib, allowing other applications to benefit. I've uploaded a patch[1] to httplib.py and the corresponding documentation. Do people think this is a good idea? -- Andrew Eland (http://www.andreweland.org) [1] http://sourceforge.net/tracker/index.php?func=detail&aid=1025790&group_id=5…

6 8

Re: [Python-checkins] python/dist/src/Lib string.py, 1.73, 1.74
by Fred L. Drake, Jr. 10 Sep '04

10 Sep '04

On Thursday 09 September 2004 11:07 pm, bwarsaw(a)users.sourceforge.net wrote: > - Adopt Tim Peter's idea for giving Template a metaclass, which makes the > delimiter, the identifier pattern, or the entire pattern easy to > override and document, while retaining efficiency of class-time > compilation of the regexp. Good documentation would really help for this as well. One simple and one... interesting example would be nice. ;-) -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

2 1