Mailman 3 April 2009 - Python-ideas

string.swap()
by spir April 6, 2009

April 6, 2009

Hello, Some time ago on this list was mentionned swapping sub-strings inside a string. A short while later I had a use case case for that feature; this led me to explore this topic in a general case defined as: * Sub-strings to be swapped must not be single characters. * The string can hold any char or sequence of chars: there is then no safe choice usable as temporary place holder. So that the common scheme s1->temp / s2->s1 / temp->s2 using string.replace() cannot be used. * But the issue of overlapping substrings is not taken into account. I asked for algorithms on the tutor mailing list (see below). In addition to the 'naive', but tricky, step-by-step process, there was: ~ A proposal with regex, 'cheating' with the sub() method (I had some similar method, but Kent's is much smarter than mine). ~ A very clever proposal that splits the string into a list and joins back. My conclusions about this feature are as follows: -0- It seems rarely needed. -1- It is of general use, as opposed to domain-specific. -2- It is much less trivial/obvious than it seems. -3- It maps conceptually to a single, whole, operation. What do you think? Is it worth having it Python? Below exemples of implementation. Note that behind the scene the regex method must perform something analog to the the naive one. Timing produced the following results: ~ For any reason, the regex version's time is very unstable on smaller strings / number of replacements / number of trials (up to ~10000). Time varies from 60% to 120% of the naive version time. ~ On bigger trials (~ 100000) regex and naive versions are about as (in)efficient. ~ The list version is about 5 times faster in all cases. def swapNaive(text, s1, s2): new_text = "" l1, l2 = len(s1), len(s2) while (text.count(s1) > 0) or (text.count(s2) > 0): i1, i2 = text.find(s1), text.find(s2) if i1>=0 and (i2<0 or i1 < i2): new_text += text[:i1] + s2 text = text[i1+l1:] else: new_text += text[:i2] + s1 text = text[i2+l2:] new_text += text return new_text ### proposed by Albert T. Hofkamp ### on the tutor mailing list def swapList(text, s1, s2): pieces = text.split(s1) pieces = [p.replace(s2, s1) for p in pieces] return s2.join(pieces) ### proposed by Kent Johnson ### on the tutor mailing list import re def swapRegex(text, s1, s2): def replace(m): return s2 if m.group()==s1 else s1 matcher = re.compile('%s|%s' % (re.escape(s1), re.escape(s2))) return matcher.sub(replace, text) Denis ------ la vita e estrany

4 3

Historical revisionism and source file formatting
by Talin April 6, 2009

April 6, 2009

As you know, the current Python source tree contains a mixture of various different tab/space conventions and source formatting styles. The Python community has opted not to perform any sort of mass conversion of source file to a uniform style, because the community values time-wise consistency (of versions and branches) over space-wise consistency (between files in the same revision). However, I have often wondered if it would be reasonable to reformat *all prior* versions of the source code. Imagine some kind of tool that would iterate over all previous versions, copying each version from one repository to another, and doing an automatic reformat on each version. Of course, in order for this to work, the automatic reformatter would have to be very, very trustworthy, since any errors would likely go undetected for a long time. One objection to this plan is that the history thus created would be "revisionist" - that it would be, in fact, an unauthentic historical record of what had happened. I don't have a good answer to that other than to say that this is an argument of theory over practicality. In any case, I don't mean this as a serious suggestion that we reformat all Python revisions in the current SVN repository. However - since we're going to be setting up a new DVCS anyway, I wonder if something along these lines would make sense as part of the migration. It might be a one-time opportunity to deal with this matter once and for all. -- Talin

2 1

about repr
by spir April 6, 2009

April 6, 2009

Hello, -1- repr writing function Wonder if you would like a statement (py2) or function (py3) similar to print, except that it would output the repr form instead of str. I would enjoy having such a nicety to avoid "print repr(x)" or "print(repr(x))". As a side note, there is imo something missing in the // below: __str__ str() %s print __repr__ repr() %r (show?) If only for consistency... it wouldn't hurt. This seems easier for python 3: repr output would then be implemented as a function, like print, so that we wouldn't not need a dedicated keyword. -2- use for test purpose I just read the following by Ka-Ping Yee on python-dev (http://mail.python.org/pipermail/python-dev/2000-April/003238.html): """ repr() is for the human, not for the machine. [Serialization] is for the machine. repr() is: "Please show me as much information as you reasonably can about this object in an accurate and unambiguous way, but if you can't readably show me everything, make it obvious that you're not." """ I see str() as intended to produce a view of an object that has a kind of "natural" (read: cultural) textual form, like a date, and rather for the user. repr() instead seems to me more "rough", informative, and programmer-oriented. For this reason, I heavily use repr for test, usually have a __repr__ method on custom classes only for that purpose. I would like to know how much debug work is done without help of a debugger. I suspect it's a rather high percentage for several reasons, especially with a clear and friendly language like python. At test/debug/setup time it's often enough to have some variable output at the proper time & place. But we often need several values at several points. The one thing missing is then the names of the variables showed. So that we have to insert eg print "x:%r" % x You will probably find it weird, but I would really like a way to have a variable name automatically written together with its value. I can see 2 ways for that: ~ Either repr is really considered as programmer test output. Then when its argument is a simple name instead of another kind of expression, this name would be printed to stdout together with the value. (*) ~ Or a brand new pair of functions/methods: one to produce the string, and one to print it out. (*) This special casing of identifier alone, as opposed to general expression, may seem strange; but it already exists in python assignments in order to alias instead of yielding a new value. Denis PS: I cannot simply write a tool func, for ordinary objects do not know their own name: only functions, methods and classes have a __name__. I can nevertheless simulate it using eval(): def show(name): sys.stdout.write( "%s:%r" % (name, eval(name)) ) but then show() must be called with "show('x')", using additional ''. Also, this cannot work as a method, so cannot be specialized. What I wish is instead: def show(obj): sys.stdout.write( "%s:%r" % (obj.__name__, obj) ) that can be called with "show(x)". Why do only funcs & types know their names? ------ la vita e estrany

3 2

Are we hammering thumbtacks? [was: Re: x=(yield from) confusion]
by Jim Jewett April 5, 2009

April 5, 2009

On Fri, Apr 3, 2009 at 8:32 PM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > That is, yield, send() and throw() involve communication between the > currently active subcoroutine and the client of the whole coroutine. > They bypass the current stack in the coroutine itself. The return > values, on the other hand, *do* involve unwinding the coroutine stack, > just like they do with normal function calls. ... > # Note how similar this is to the normal version above > def average_diff_cr(start): > avg1 = yield from average_cr(start, seq1) > avg2 = yield from average_cr(start, seq2) > return avg2 - avg1 Yes, this explanation and example finally made that clear to me. But it also makes me wonder if we aren't using the wrong tool. It seems that you want the yields to pass opaquely through average_diff -- possibly without any actual value; just a go-ahead-and-suspend-me flag. Meanwhile, you do want some values to be shared between average_diff and average_cr. average_diff can send in some initial values when it creates/initializes average_cr, but there isn't any way to get information back without hijacking the stream of yields. The proposals below are more ambitious than the current PEP, but they still somehow feel less contorted by special cases: Option 1: Recognize that yield is serving two purposes, and find another way to spell go-ahead-and-suspend-me. It may be too late to do this cleanly. Option 2: Find another way for average_diff and average_cr to share information. Option 2a: Add anoher set of methods, similar to send, but specific to co-routines. This starts to look like the Actor that Bruce Eckel asked about a a while ago, and Kamaelia may have a good pattern. Option 2b: Let the caller and callee share a scope, similar to exec in locals. (Previous discussions talked about thunks or macros.) -jJ

1 0

list.index() extension
by Benjamin Peterson April 5, 2009

April 5, 2009

I would like to propose a extra parameter `predicate` to list.index() like this: def index(self, obj, predicate=operator.eq): for idx, item in enumerate(self): if predicate(item, obj): return idx raise IndexError My main use-case is 2to3 where a node has to locate its self in the parents node list by identity. Instead of writing out the search manually as is done now, it would be nice to just write `self.parent.nodes.index(self, operator.is_)`. I can also imagine this might be useful: print "The first number less than 5 in this list is %s" % (my_list.index(5, operator.lt),)

12 20

Adding a functional list type to the standard library.
by Alexandre Vassalotti April 5, 2009

April 5, 2009

Hello, I would like to have your opinion whether a functional list type would be welcomed in the standard library. A functional list type would provide an efficient data-structure for keeping many lists with partial modifications. In particular, a such data-structure would be obsolete many, if not all, use-cases of copy.deepcopy on lists. Here is a simple example how a such list would be used: # The constructor does a deep conversion of the list to its internal representation. state = FunctionalList([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, None]]) print(goal[0]) # This would print a FunctionalList object, i.e., FunctionalList([1, 2, 3, 4]) # Do a lazy copy of the list. new_state = state.copy() # Modify the list—many (possibly all) unchanged items are shared between # the original list and its copy. The original list remains unchanged. new_state[3][3] = new_state[3][2] new_state[2][3] = None # Show the unchanged list. This prints: # FunctionalList([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, None]]) print(state) # Show the modified list. This prints: # FunctionalList([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, None], [13, 14, 15, 12]]) Attentive readers will remark the above example comes from a attempt of an 15-puzzle solver. There are many use-cases for a such data-structure. In particular, any problem that requires backtracking search would be benefit from having a functional list. Peter Norvig [1], for example, had to resort to use strings to implement his soduku solver efficiently. Fortunately for him, the state of a soduko puzzle fits well in a string as each soduko's cell can be stored in a single single character. For more complex problems, using strings is not always a suitable option. Also, it would also be interesting to consider the addition of functional dictionaries and sets as well. These twos would probably more useful to have since the most common usage of copy.deepcopy seems to be on dictionaries. [2] Finally, I would like to say that I am willing to write a Python and an optimized C implementation of my proposal. Regards, -- Alexandre [1]: http://norvig.com/sudoku.html [2]: http://google.com/codesearch?q=copy\.deepcopy+lang:python

4 3

clear() method for lists
by Andre Roberge April 5, 2009

April 5, 2009

Hi everyone, On the general Python list, a suggestion was made to add a clear() method to list, as the "obvious" way to do del some_list[:] or some_list[:] = [] since the clear() method is currently the obvious way to remove all elements from dict and set objects. I believe that this would be a lot more intuitive to beginners learning the language, making Python more uniform. André

10 18

name export
by spir April 5, 2009

April 5, 2009

Hello, When I write tool modules that export useful names to client code, I usually use __all__ to select proper names. Sure, it's a potential source of identifier conflict. I have another custom __*names__ module attribute that allows the client at least to control which names are defined in the imported module: # module M __Mnames__ = [...] __all__ = ["__Mnames__"] + __M_names__ Then from M import * ; print __Mnames__ outputs needed naming information: from M import * ; print __Mnames__ ; print dir() ==> ['a', 'b', 'c'] ['__Mnames__', '__builtins__', '__doc__', '__file__', '__name__', 'a', 'b', 'c'] [Indeed, you'd have the same info with M.__all__, but I find it strange to have both "from M import *" and "import M" only to access its __all__ attribute. Also, it happens that a module name and it's main defined name are identical, like time.time.] A complication arises when the toolset is structured as a hierarchy of modules. Then I have a top module that (oft) only serves as name interface. Each module exports a "name summary" attribute. For instance, in the case below (where M22 is an internal tool module, not to be exported): M0 M1 M2 M21 M22 the import/export schedule would be: # M21 __M21names__ = [...] __all__ = ["__M21names__"] + __M21names__ # M22 __all__ = [...] # M2 from M21 import * from M22 import * __M2names__ = [...] + __M21names__ __all__ = ["__M2names__"] +__M2names__ # M1 __M1names__ = [...] __all__ = ["__M1names__"] + __M1names__ # M0 from M1 import * from M2 import * __M0names__ = [...] + __M1names__ + __M2names__ __all__ = ["__M0names__"] + __M0names__ Now, when I modify a module in a way that leads to change/delete/add exported names, I only need to care with this update _locally_. The update will automatically climb up the export chain. [Only a module name change remains problematic -- I'm thinking at an automatic module name lookup/update tool.] Well, all of this looks a bit forced and rather unpythonic to me. It works fine, but I'm not satisfied. I wonder whether I'm overlooking something obvious. And if yes, what? Or conversely, do you think there could/should be a nicer (and builtin) way of doing such things? Denis ------ la vita e estrany

5 5

Re: [Python-ideas] list.index() extension
by Leif Walsh April 4, 2009

April 4, 2009

On Sat, Apr 4, 2009 at 5:58 PM, Benjamin Peterson <benjamin(a)python.org> wrote: > 2009/4/4 Leif Walsh <leif.walsh(a)gmail.com>: >> On Sat, Apr 4, 2009 at 5:38 PM, Benjamin Peterson <benjamin(a)python.org> wrote: >>> print "The first number less than 5 in this list is %s" % (my_list.index(5, >>> operator.lt),) >> >> print "The first number less than 5 in this list is my_list[%d]=%s" % >> ((idx, elt) for idx, elt in enumerate(my_list) if elt < 5).next() > > That does something different. My would tell you the first index of a > number less than 5 and your what tell you what that was. Mine does both, actually, and you can get whichever part you need out of it. > Did you see the 2to3 one? Yes. I don't know the details of the implementations of those classes, but I think you could easily cook up something quite similar to what I did above. I'm not really -1 or +1 on this, I just think it's probably easier for you to use a generator expression than to try to convince python-ideas that this needs to happen. My finger could be way off the list's pulse though. -- Cheers, Leif

1 0

Revised**7 PEP on Yield-From
by Greg Ewing April 4, 2009

April 4, 2009

I've made another couple of tweaks to the formal semantics (so as not to over-specify when the iterator methods are looked up). Latest version of the PEP, together with the prototype implementation and other related material, is available here: http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/ -- Greg

10 137