Mailman 3 July 2011 - Python-ideas

A function that gives you the number of items in an arbitrary iterable (not len)
by Mike Graham 29 Jul '11

29 Jul '11

There are a couple patterns, some uglier than others, which I have seen occasionally pop up which could be solved by a new builtin or stdlib that works like def exhaust(it): count = 0 for item in it: count += 1 return count Here are some situations where such a function could come in handy: 1. I want to know how many conditions satisfy a condition. I can currently determine this with code like sum(itertools.imap(cond, it)) or one of several other options. Oftentimes these take longer to read than they should. 2. I simply want to evaluate all the items in a lazy iterable. The nicest way to do this is to the effect of "for item in it: pass", though I have seen creative solutions like collections.deque(iterator, 0) (to try to speed up the loop). 3. You simply want to know the length of a non-sequence iterable. One current way to do this is sum(1 for item in it) (although I think I've seen "for count, _ in enumerate(it): pass;; count += 1"). I think the introduction of a function that works like this might make Python easier to read occasionally. I've seen these all pop up, but only occasionally. None are hard to perform as is, but each can feel a little messy. Does anyone have any input? Mike

3 2

Re: [Python-ideas] anonymous object support
by Herman Sheremetyev 26 Jul '11

26 Jul '11

[adding python-ideas back to CC] On Mon, Jul 25, 2011 at 5:40 PM, Paul Moore <p.f.moore(a)gmail.com> wrote: > On 25 July 2011 08:41, Herman Sheremetyev <herman(a)swebpage.com> wrote: >>> >>> class Dummy: pass >>> >>> It's not really worth including something this short in the stdlib: >>> >>> from somewhere import Dummy >>> >>> is actually longer! >> >> This doesn't actually help with the "create object as an expression" >> requirement. Steven D'Arpano's message describes some implementation >> details, and it would be nice if you didn't have to import it though I >> could live with that as well. > > I'm not quite sure I follow. Are you insisting that you need to create > an instance and then assign an instance attribute/method <b>in a > single expression</b>, and have the resulting instance be the value of > that expression? > > If so, then I don't really see what you gain apart from saving a line > or two in your source code, and confusing your reader. But if you must > do that, then Nick's suggestion of using the type builtin would > probably do what you want. (I make no claims that it's particularly > readable, but I'd say you gave up on that when you insisted on a > one-line solution :-)) > > To compare, we're talking about: > > d = Dummy() > d.foo = lambda x: x > expression_involving(d) > > vs > > expression_involving(Dummy(foo = lambda x:x)) You could do that if it seems reasonable or useful in that particular case, but more often than not I think you'd do something like: variable = Dummy(foo=lambda x: x) expression_involving(variable) Having one line of code that accomplishes a simple task (creating an object and saving it in a variable) instead of having to write N lines for each attribute makes for more readability not less IMO. You can still break lines inside the parens and add each attribute on its own line, but the indentation would at least clue the reader to the fact that it's really a single operation. Having N lines, especially if N is more than 2 or 3, is begging for the whole thing to be moved off into a helper function. And in tests this kind of operation happens all the time. > Even using trivial examples like this, I find the first more readable > (and I'd curse the writer if I had to maintain code like the second). > If expression_involving() was anything more complex than a function > call, or foo was more than the identity function, this would be > utterly unreadable. > > Can you point at any real code that implements a class-making function > such as you describe, and then uses it? I'd be curious to see any > real-world examples to check if my intuition that such code is > difficult to understand holds up. https://github.com/has207/flexmock/blob/master/tests/flexmock_test.py You can take a look at the tests starting at line 56 -- the flexmock() function does exactly what I'm proposing when given keyword args. > PS Feel free to quote any of this mail back on-list. I would have > added the list back into my reply, on the assumption that you simply > forgot to include the list, but didn't want to do so without your > agreement... It wasn't my message that took the list off the CC :) Cheers, -Herman

6 13

use "as" for block scope support
by 海韵 26 Jul '11

26 Jul '11

New submission from HaiYun Yan <lyricconch(a)gmail.com>: just like mozilla javascript "let" extension https://developer.mozilla.org/en/new_in_javascript_1.7#Block_scope_with_let… according function args can have the default value let_stmt is unnecessary for python. ============example 1=========== f = None with open(...) as f: assert hasattr(f, "read") f = 1 assert f == 1 assert f is None ==========example 2============== try: e = None 1/0 except ZeroDivisionError as e: assert isinstance(e, ZeroDivisionError) except ... as e: # the same as except: but has the exception instance store to e assert isinstance(e, BaseException) else: assert e is None finally: assert e is None assert e is None ===========example 3============== while SEQ.next_availd() as tid: assert bool(tid) is True assert "tid" not in dir() ===========example 4============== assert "zero" not in dir() A = (id(None) % 1 as zero) zero + 5 assert A == 5 assert "zero" not in dir() ======example 5================== K = 1 A = (K+1 as K, K+2 as Q) K*Q assert A == 6 # (1+1) * (1+2) assert K==1 ============example 6=============== K = 1 A = (K+1 as K) (K+2 as K) K * K assert A == 16 # ((1+1) + 2) * ((1+1) + 2) assert K == 1

6 10

Fwd: use "as" for block scope support
by 海韵 25 Jul '11

25 Jul '11

1. but what about ( t * s + t / s )? temp var always keep in memory. (should not reduce ( t + s ) to ( 2 * x ) since x, y may not a number. any operator here should be conside as independent function. ) 2. try convert this to the lambda form: (x+y as t, x-y as s)(t * s as u, t/s as v)(u << v as p, u>>v as q) p ** q In runtime: lambda = MAKE_FUNCTION/MAKE_CLOSURE = PyFunction_New lambda call = CALL_FUNCTION = PyFrame_New + PyEval_EvalFrameEx all of them are heavy, (you can check the source code) but: as = PyCell_New as enter = SETUP_FINALLY = PyFrame_BlockSetup as leave = END_FINALLY = PyFrame_BlockPop which is much much much more more more cheaper than above. 2011/7/26 Bruce Leban <bruce(a)leapyear.org>: > > > On Mon, Jul 25, 2011 at 9:50 AM, 海韵 <lyricconch(a)gmail.com> wrote: >> >> Dupilcate K is used only for describe the scope rule. >> Advantage: >> >> 1. the same grammer, the same behave >> "t" of """(x+y as t, x-y as s) t * s + s + t""" is only valid in the >> expression which require >> just as >> "e" of """except ... as e""" only live in the suite which it belongs >> >> 2. readability includes laconic >> a = (x+y as t, x-y as s) t * s + s + t >> it doesnt break thinking. from left to right, people finish thinking >> inplace. >> temp var will not keep in memory, people can forget them since they >> are not valid outside the expression. everything is clean. >> t, s = x+y, x-y >> a = t * s + s + t >> in this case, thinking is break and temp var keep in memory, >> people should take some care about it (for example, in nested loop) > > (x + y) * (x - y) + 2 * x > is IMHO more readable - especially note the absence of y in the third term. >> >> 3. this enchant the "lambda" syntax >> list.sort(key = lambda p: (sqrt(p.x**2+p.y**2) as r) r**2+ A*r + B) > > I don't know what "enchant" means here, but you can do this if you want: > list.sort(key = lambda p: (lambda r: r**2 + A*r + B) (sqrt(p.x**2+p.y**2))) > I'm not recommending that as I think it would be more readable to name that > function so that it would be more clear why you're using that function as > the sort key. But, to me, this example does not justify new syntax. > --- Bruce

1 1

Re: [Python-ideas] Deprecate str.find
by Larry Bugbee 20 Jul '11

20 Jul '11

> Steven D'Aprano wrote: > In my wishlist for Python 4000 I have: and sooner would be nicer. > * list.index renamed to list.find > * str.find and list.find return None if the argument is not found +1 ...and forget throwing an exception if not found. None is much preferred.

2 1

Left division operator
by Sturla Molden 18 Jul '11

18 Jul '11

I was wondering if anyone else has felt the need for a leftward division operator in Python. That is, y\x in addition to x/y. Thus the operators: y \ x y \\ x Why the statement "y\x" is different from "x/y": 1. Opposite order of evaluation: This is important if evaluating one of the operands has side-effects. 2. In linear algebra, (Y**-1)*X is in general not equal to X*(Y**-1). Matrix and vector algebra in fundamental to computer graphics and scientific computing. 3. NumPy solves the lack of "\" operator by having a function np.linalg.solve. But using it in matrix expressions has the effect of mixing Pythonic infix operators with a Lisp-like prefix operator. Two other issues: 1. The expressions y /= x y //= x are evaluated in the same order as "y \ x", not "x / y". 2. The expressions y \= x y \\= x should perhaps be illegal due to implied side-effects on RHS. Sturla

10 16

Deprecate str.find
by Mike Graham 18 Jul '11

18 Jul '11

str.find (and bytes.find) is worse than the alternatives in every way. It should be explicitly deprecated in favour of str.__contains__ and str.index. str.find when used to check for substring is inferior to the in operator. "if sub in s:" is shorter, easier-to-read, and more efficient than "if s.find(sub) != -1:" and is not prone to the error "if s.find(sub):" I have occasionally seen. str.index is better for finding indices in that it supports an idiomatic exception-based API rather than a return-code API. Every usage of str.find should look like "index = s.find(sub); if index == -1: (exception code)", which is an antipattern in Python. This problem is compounded by the fact that the returned value is actually a valid value; consider s = 'bar'--s[s.find('x')] is somewhat surprisingly 'r'. Additionally, the existence of str.find violates the there's-one-way-to-do-it principle. Mike

17 61

Re: [Python-ideas] None-as-sentinel alternatives? (was: deferred default arguments)
by Don Spaulding 16 Jul '11

16 Jul '11

On Wed, Jul 13, 2011 at 8:23 PM, Steven D'Aprano <steve(a)pearwood.info>wrote: > Eric Snow wrote: > > A New Solution >> --------------- >> >> Provide a builtin version of a Deferred singleton, like None or >> NotImplemented. When resolving arguments for a call, if an argument >> is this Deferred object, replace it with the default argument for that >> parameter on the called function, if there is one. If there isn't, >> act as though that argument was not passed at all. >> > > I don't believe that should be a public object. If it's public, people will > say "Yes, but what do you do if you want the default to actually be the > Deferred singleton?" It's the None-as-sentinel problem all over again... > sometimes you want None to stand in for no value, and sometimes you want it > to be a first class value. When I come across a situation where I want a sentinel that doesn't preclude None from being a valid value, I've always spelled it like this: SENTINEL = object() # Guaranteed to be unique and not None. def f(arg=SENTINEL): if arg is SENTINEL: arg = 'foo' x(arg) It's always worked well, but that little voice in my head tells me there's probably something I'm not thinking of that will come back to bite me in the future. I don't think this helps the OP in any meaningful way, but I was wondering if you bright people might shed some light on the drawbacks of this approach for me. What are the better alternatives?

3 3

deferred default arguments
by Eric Snow 15 Jul '11

15 Jul '11

I'll get this out of the way now. This is probably a dumb idea. However, I have found myself working around a use case lately and have a thought of a cleaner solution. The Situation --------------- As you know, you can use default arguments when defining a function. Sometimes you want to pass an argument through from a function's arguments to another function that the original ("outer") calls. I run into this during inheritance-related calls and when interfacing between two compatible APIs. Here's an example: class X: def f(self, name="N/A"): ... class Y(X): def f(self, name="N/A"): super().f(name) Y().f() So, the name argument is passed through from Y's f() to X's. In this case, we want to keep the default argument the same. Current Solutions --------------- Keeping the default arguments the same can be accomplished in a few different ways right now, but they are less than ideal: * Manually keep the two sync'ed (like above), a DRY violation. * Use introspection at definition time (using X.f.__defaults__ and X.f.__kwdefaults__), which is fragile (if the X.f attributes are changed later or the function definition's signature changes). * Use a dummy value to indicate that the default should be supplied by X's f(). This requires some clutter in the function and possibly a less obvious default argument. Here's an example of that last one: class DeferredType: pass DEFERRED = DeferredType() class X1: def f(self, name="N/A"): if name is DEFERRED: name = "N/A" # DRY #name = X1.f.__defaults__[0] # fragile ... class Y1(X1): def f(self, name=DEFERRED): super().f(name) Of course, the DEFERRED object could be any other specific object, particularly None. Here's another similar example that reminds me of Jack's example[1] from the recent thread on mutable default arguments. It is the solution I am using currently. class X2: def f(self, name=None): if name is None: name = "N/A" ... class Y2(X2): def f(self, name=None): super().f(name) Of course, using DEFERRED instead of None would allow you to use None as the actual default argument, but you get the idea. Notice that the actual default argument is no longer in the argument list, so it is less obvious and no longer introspect-able. I have also run into this use-case outside of inheritance situations. A New Solution --------------- Provide a builtin version of a Deferred singleton, like None or NotImplemented. When resolving arguments for a call, if an argument is this Deferred object, replace it with the default argument for that parameter on the called function, if there is one. If there isn't, act as though that argument was not passed at all. Here's how it would look: class X: def f(self, name="N/A"): ... class Y(X): def f(self, name=Deferred): super().f(name) [2] This way, Y's f has a sensible default argument for "name"[3] and implies where to find the actual default argument (on super().f)[4]. X's f is not cluttered up and has a clear, introspect-able default argument. Thoughts? -eric p.s. while I usually pursue random flights of fancy (answers looking for questions), this idea was born of an honest-to-goodness practical use-case I use semi-frequently. One thing I've learned about the Python community is that ideas that come from real [and common] use-cases have a much better chance of not dying a quick and painful death (not holding my breath here though <wink>). [1] http://mail.python.org/pipermail/python-ideas/2011-May/010263.html [2] As a bonus, an "outer" function could take advantage of different default arguments when calling more than one function: def f(name="N/A"): ... def g(name="Undefined"): ... def h(name=Deferred): f(name) g(name) [3] "Deferred" indicates what kind of argument it is in a straightforward way. However, I don't want the name to be confused with anything like Twisted Deferred objects, so whatever name is clear would, of course, be appropriate. [4] That is true from a people perspective, but not for introspection; e.g. if you want to introspect the actual default argument for Y.f, you would have to be able to programmatically determine how that parameter is used in the function body...

6 9

os.path.isparent and os.path.ischild
by Giampaolo Rodolà 15 Jul '11

15 Jul '11

>>> isparent('/a', '/a/b') True >>> isparent('/a/b', '/a') False >>> ischild('/a/b', '/a') True >>> ischild('/a', '/a/b') False Thoughts? Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/

16 30