Mailman 3 January 2012 - Python-ideas

Retrying EAFP without DRY
by Mike Meyer Jan. 25, 2012

Jan. 25, 2012

Just running another half-baked "easy" idea up to see what others think of it. One of the cases where EAFP is clumsy is if the "forgiveness" portion is actually retrying what failed after fixing things. I just ran into this in real code, and the resulting discussion suggested this change. Eiffel's exception-handling mechanism is built assuming that's the *standard* way to do things. Seems to work ok, but if I had to choose one of the two, I'll take try/except. So here's the proposal: A single new keyword/clause, "retry". It has the same syntax as an "except" clause, can be used anywhere "except" can be used, and can be intermingled with them in the same try statement. There's probably a better syntax, but this is easy to describe. The behavior change from except is that instead of exiting the "try" statement when the "retry" clause ends, it restarts the try clause. In python, this code: try: # try block except: # except block retry: # retry block else: # else block finally: # finally block Would provide the same behavior as this code: while True: try: # try block except: # except block retry: # retry block else: # else block finally: # finally block break Where except & retry can be repeated, include exceptions to check against, and intermixed with each other. The break would propagate up to the else block and any except blocks if there was no finally. If there's no else, the break from it winds up at the end of the try. The use case, as mentioned, is avoiding doing EAFP without repeating the code in the exception handler. I.e., the choices without retry are things like (LBYL): if not isinstance(x, my_class): x = fixup(x) mangle(x) other_stuff(x) or EAFP: try: mangle(x) except: x = fixup(x) mangle(x) other_stuff(x) or: while True: try: mangle(x) break except: x = fixup(x) other_stuff(x) with: try: mangle(x) retry: e = fixup(x) other_stuff(x) My gut reaction is cool, but not clear it's worth a new keyword. So maybe there's a tweak to the except clause that would have the same effect, but I don't see anything obvious. <mike

17 59

Allow Profile() to be used as a context manager
by David Townshend Jan. 25, 2012

Jan. 25, 2012

I often find it useful to profile small sections to code in a running application to pinpoint a known bottleneck. The cProfile and profile modules don't make this easy as they stand, requiring the section of code to be wrapped into a single statement with can be called as a string using exec(). This always feels a bit clumsy to me, so my idea is simply to provide __enter__ and __exit__ methods to cProfile.Profile() and profile.Profile(). From a quick look at the code it seems that this should be very easy for cProfile, and slightly less easy for profile (I can't quite figure out how to deal with the set_cmd() line, or how important it is - maybe someone can help?). Any major objections to this? David

5 10

-m pstats should combine all the profiles given as arguments
by Matt Joiner Jan. 25, 2012

Jan. 25, 2012

Frequently when profiling multiple threads, I need to combine several dump stat files. Currently -m pstats reads the profiling data at only the first path given. It should merge all the profiling data from all the paths given. $ python3.3 -m pstats prof/5506-7f00f* http://bugs.python.org/issue13839

1 0

Combining test and assignment
by Eike Hein Jan. 24, 2012

Jan. 24, 2012

Hello, this is very spur of the moment, and so I apologize if I am missing the obvious reasons for why this would be horribly dumb. I sometimes find myself annoyed that I cannot access the result of an expression in a conditional statememt. Consider: if spam(): ... frobulate the return value of spam() ... else: ... value too low to frobulate ... In order to frobulate spam(), either need to call it again in the indented block, or preassign: x = spam() if x: ... frobulate x ... The first option is usually inefficient, and the second feels ugly to me because something relevant to a block is happening outside it and its header. Instead, I'd love to be able to: if spam() as x: ... frobulate x ... Further, there are two reasons this seems a particularly good fit in Python: - There is a _syntactic_ precedent, or at least great similarity to the 'with' statement. - Because calls in Python return None if they don't ex- plicitly return anything else, there is no "can't assign void to x" to worry about. I haven't given much thought to scoping, particularly for else clauses, yet, at least not enough to make up my mind. The other question is if this reads good "as English" or needs to be 'if spam() then as x:', or if that's even an issue. Thoughts? Best regards, Eike

14 19

Re: [Python-ideas] Combining test and assignment
by Chris Rebert Jan. 22, 2012

Jan. 22, 2012

> On Sun, Jan 22, 2012 at 1:36 PM, Chris Rebert <pyideas(a)rebertia.com> wrote: >> On Sun, Jan 22, 2012 at 4:04 AM, Paul Moore <p.f.moore(a)gmail.com> wrote: >> > On 22 January 2012 04:51, Steven D'Aprano <steve(a)pearwood.info> wrote: >> >> Assignment as an expression feels unnatural to me: >> >> >> >> if spam() as x != 'ham': >> >> frobulate(x) >> >> >> >> doesn't really correspond to any natural English order. >> > >> > While in general I agree with you, there *is* a natural reading of this: >> > >> > "if spam() isn't 'ham', frobulate it" >> > >> > The fact that you have to choose a name is because computer languages >> > have to be more explicit than natural languages, and can't deal with >> > the implicit referent involved in the English usage of "it". The need >> > for assignment comes from the need for a name. You could actually deal >> > with the if-or-while-expression-assignment case by defining a special >> > variable __it__ to mean the value of the condition in the containing >> > if or while expression, but it has all the same lack of generality >> > issues as any proposal that limits itself to if/while conditions. And >> > short of explicitly marking the expression somehow, you can't >> > generalise any further. >> >> This is reminiscent of Perl's $_ variable, which represents the "current >> topic". >> http://www.perl.com/pub/2002/10/30/topic.html On Sun, Jan 22, 2012 at 1:24 PM, Edward Lesmes <ehlesmes(a)gmail.com> wrote: > why not make python do something like > _ = test() > if _: > ... > Automatically, so you can do something like > > if test(): > frobulate(_) > > > It seems natural to me, because in prompt, > python assigns the last result to the > _ variable I think "Explicit is better than implicit"[0] pretty much covers why that sort of thing is generally disfavored. Cheers, Chris -- [0]: http://www.python.org/dev/peps/pep-0020/

1 0

Symbolic expressions (or: partials and closures from the inside out)
by Nathan Rice Jan. 18, 2012

Jan. 18, 2012

Greetings, I have been writing a lot of code lately that involves creating symbolic expressions of one form or another, which are then fully evaluated at a later time. Examples of this include Elementwise, where I create expressions that act on every member of an iterable (there is a much improved new version coming soon, by the way), and a design by contract/validation lib I'm working on (which shall remain nameless :D) that uses symbolic expressions in the *args of the metaclass __new__ method to generate a constraint class which validates input using __instancecheck__. I do most of this with lambdas, a little hacking with closures and FunctionType(), and chainable objects. I am very impressed that python is this flexible, but there are some issues with the approach that I would like to rectify, namely: 1. Because of the early binding behavior of most things in Python, if I want to include isinstance(X, someclass) in a symbolic expression, I have to wrap it in a lambda (or use .apply(), in the case of Elementwise). This is not a huge deal for me, but it forces me to create wrappers for lots of functions (e.g. isinstance_(X, someclass)) and/or have users wrap every such function they want to use in a symbolic expression. Having to do this also bloats the code a lot; the github version of Elementwise is over 3,000 LoC at this point (including prodigious documentation, but still...). 2. Python expects that certain functions, such as int(), str(), etc, will have a specific return type. While in general I agree with this, it makes Elementwise somewhat inconsistent (and it will do the same to anything else that wants to work with symbolic expressions). I'm interested in fixing both issues. I believe both issues I've had could be solved by having a robust "symbolic object". These objects would basically usable like ordinary objects, however upon any attribute access or other form of interaction, the object would basically short circuit the calling function, and return a symbolic object directly to the outer scope. The symbolic object would behave like a generator function frozen at the point of attribute access, and upon send()-ing (or whatever method), it would behave exactly as if the values sent had been the ones passed in originally (ideally without consuming the generator). I have thought about ways to approximate this behavior python currently, and while I could hack something together using inspect to pull relevant info from the stack, then break out using a special exception (potentially passing a generator with state as an exception arg), this approach strikes me as VERY brittle, implementation dependent, ugly and difficult to work with. Additionally, you would need to catch the special exception somewhere in the stack, so this trick wouldn't work on the first thing in an expression to be evaluated. As an aside, I'd like to solicit some feedback on the validation syntax I've been working on. Currently, I have code that support things like: X = SymbolicObject() const = Constraints(X * 2 + 1 >= 5, X % 2 != 0) const2 = Constraints(X[-1] == "h") const3 = Constraints(X[-1].upper() == "H") >>> print isinstance(3, const) True >>> print isinstance(2, const) False >>> print isinstance(1, const) False >>> print isinstance("bleh", const2) True >> print isinstance("bleh", const3) True Callables are supported as well, so if you wanted to do something like: Constraints(isinstance(X.attr, someclass), somefunc(X[-2].attr, args)) You could approximate that with: Constraints(lambda x: isinstance(x.attr, someclass), lambda x: somefunc(x[-2].attr, args)) As I mentioned in the first paragraph, Constraints is a metaclass, so your validations are checked using __instancecheck__. I'm also considering having __init__ generate mock objects (for certain straight-forward cases, anyhow). Thanks for your time, Nathan

8 15

Fwd: quantifications, and tuple patterns
by Paul Moore Jan. 17, 2012

Jan. 17, 2012

Sorry, should have replied on list. On 14 January 2012 05:25, Annie Liu <liu(a)cs.stonybrook.edu> wrote: > For example, even the simplest "some x in s has pred", when a witness > for x is not needed, could end up like "{x for x in s if pred}", > "any(pred for x in s)", "len(...)!=0", "len...>0", len...>=1", ... not > to mention the many more kinds of loops and tests written by the > majority of students not used to writing queries/comprehensions. Even > more kinds of loops and bigger hassles are involved when a witness for > x is needed. I'm confused as to why any(pred for x in s) is unacceptable here? > For another example, a simple "each x in s has x > y" tends to end up > like "min(s)>y", which is incorrect, since s could be empty, so one > has to either add this condition or invent some boundary value for > min, which is error-prone since it is sensitive to the possible values > of elements of s and the uses of these values. all(x > y for x in s) seems to me both idiomatic and correct. I'm not sure quite what you're proposing here - it seems like you're asking for new syntax to do the same job as any and all, which already do exactly what's needed. New syntax has to provide something much, much better if it's likely to be accepted. Please explain more clearly. Paul.

10 23

Re: [Python-ideas] Fwd: quantifications, and tuple patterns
by Tom Rothamel Jan. 17, 2012

Jan. 17, 2012

(Sorry about the formatting - I'm a former student of Annie's, who just joined the list.) Jim Jewett wrote: > Even in this thread, there have been multiple posts that would shown > bugs when the witness itself had a boolean evaluation of False. > What if any and all started to return a Witness object that evaluated > to True/False, but made the value available? I could see an object > like this being useful in other code as well. > class Witness(object): > def __bool__(self): return self.truth > def __init__(self, truth, value): > self.truth=truth > self.value=value > Of course, it might be nicer to inherit from type(True), and it still > doesn't quite solve the laziness problem with for, or the > assign-in-the-right-place problem with while. I'm wondering if would make sense to add an "as" clause to the while and if statements, similar to the as clause in the with statement. It would essentially retrieve the value field from the witness object described above. This would let one write the main loop in a topological sort (a classic workset algorithm) as: while any(v for v in vertices if v.incoming_count == 0) as v1: result.append(v) for v2 in v1.outgoing: v2.incoming_count -= 1 Another use of this could be for database queries that return 0 or 1 results: if db.query(page_name=page_name) as page: render_template(page) else: error404() This is a common pattern that often involves either exception handling or a sentinel value like None.

2 1

Special method to delegate PEP 3118 buffer access to another object?
by Nick Coghlan Jan. 15, 2012

Jan. 15, 2012

I'm still working on fixing CPython's operand precedence bug for pure sequence types implemented in C [1], and, while trying to test bytearray, came across the problem that there doesn't appear to be a way to write a class in pure Python that redirects the PEP 3118 buffer access slots to another object (e.g. a memoryview instance or a bytes object). Did I miss something? And, if not, would it make sense to have the buffer retrieval operations look for a __buffer__ method in the type dictionary as a last resort if the relevant C level slots aren't provided? Cheers, Nick. [1] http://bugs.python.org/issue11477 -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 1

Re: [Python-ideas] Pythonic buffering in Py3 print()
by Cameron Simpson Jan. 15, 2012

Jan. 15, 2012

[ Taken back on list - I can't see anyway personal here, and the objection is a good one. - Cameron ] On 11Jan2012 08:41, Paul Moore <p.f.moore(a)gmail.com> wrote: | On 11 January 2012 06:15, Cameron Simpson <cs(a)zip.com.au> wrote: | > | flush=None? | > | > +1 from me, if None means "honour the file's default flushing | > behaviour". | | The trouble with this is that we then have: | - no parameter (default None) => default behaviour | - flush=True => Always flush | | and that begs the question of what an explicit flush=False means, and | people will assume it means "never flush" even thought you can't force | that as the underlying stream might auto-flush (e.g., at a newline). | | +1 for flush=True/False, with a default of False (meaning do the | stream's default behaviour). Hmm. Good point about False suggesting "never flush", but I think it should just be considered invalid. What about flush=3, or flush=tuple? Better if we only specify behaviour for True (flush now) and None/default/missing (flush default) and make it slightly clear that other values do not have defined behaviour. That leaves scope later for defining a sane (or at least clear) meaning for False if a _useful_ one is ever agreed upon. Call it "reserved". Cheers, -- Cameron Simpson <cs(a)zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ Copyright and Patents: To prevent the Progress of Science and useful Arts, by securing for unlimited Times to Authors and Inventors and Trolls the exclusive Right to all Writings and Discoveries. - http://science.slashdot.org/story/11/04/26/211258/Copyright-Law-Is-Killing-…

4 5