Python-ideas December 2011

python-ideas@python.org

57 participants
30 discussions

Should stdlib provide something to properly handle SIGINT/SIGTERM signals?

by Giampaolo Rodolà

Since this is a problem which occurs very often, I'd like to hear your opinion as to whether something like this might find some place in the python stdlib (signal module maybe?). Please read the discussion included in this recipe: http://code.activestate.com/recipes/577997/ It should provide a description of the problem and the general use case. Thanks in advance for your comments, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/

8 years, 7 months

defaultattrgetter

by John O'Connor

Pythons: What are your thoughts on the concept of a `defaultattrgetter`? It would be to operator.attrgetter what getattr(foo, x, default) is to getattr(foo, x). I dont like that attrgetter requires the attribute to exist or else the getter be wrapped in a try catch with respect to how defaultdict solves roughly the same problem for dictionary keys. The semantics could be something such as: from operator import defaultattrgetter _x = defaultattrgetter({'x': 0}) _y = defaultattrgetter({'y': 1}) - or - _xy = defaultattrgetter({'x': 0, 'y': 1}) One use case I am thinking of is functions that may be decorated with attributes x and/or y. Obviously a python implementation of defaultattrgetter would be trivial to implement but one of the benefits of these functions is the speed. It also seems like it would fit in well with the rest of the operator module. Generally speaking, would anyone else have a use for this? - John

8 years, 7 months

Adding "Typed" collections/iterators to Python

by Nathan Rice

I believe it would be a good idea in instances where it is known that a collection of a single type is going to be returned, to return a subclass with type information and type specific methods "mixed in". You could provide member methods as collection methods that operate in a vectorized manner, returning a new collection or iterator with the results much like the mathematical functions in NumPy. This would also give people a reliable method to make functions operate on both scalar and vector values. I believe this could be implemented without needing subclasses for everything under the sun with a generic collection "type contract" mix-in. If a developer wanted to provide additional type specific collection/iterator methods they would of course need to subclass that. To avoid handcuffing people with types (which is definitely un-pythonic) and maintain backwards compatibility, the standard collection modification methods could be hooked so that if an object of an incorrect type is added, a warning is raised and the collection gracefully degrades by removing mixed-in type information and methods. Additionally, a method could be provided that lets the user "terminate the contract" causing the collection to degrade without a warning. I have several motivations for this: -- Performing a series of operations using comprehensions or map tends to be highly verbose in an uninformative way. Compare the current method with what would be possible using "typed" collections: L2 = [X(e) for e in L1] L3 = [Y(e) for e in L2] vs L2 = X(L1) # assuming X has been updated to work in both vector/scalar L3 = Y(L2) # context... L2 = [Z(Y(X(e))) for e in L1] vs L2 = Z(Y(X(L1))) L2 = [e.X().Y().Z() for e in L1] vs L2 = L1.X().Y().Z() # assuming vectorized versions of member methods #are folded into the collection via the mixin. -- Because collections are type agnostic, it is not possible to place methods on them that are type specific. This leads to a lot of cases where python forces you to read inside out or a the syntax gets very disjoint in general. A good example of this is: "\n".join(l.capitalize() for l in my_string.split("\n")) which could reduce to something far more readable, such as: my_string.split("\n").capitalize().join_items("\n") Besides the benefits to basic language usability (in my opinion) there are tool and development benefits: -- The additional type information would simplify static analysis and provide cues for optimization (I'm looking at pypy here; their list strategies play to this perfectly) -- The warning on "violating the contract" and without first terminating it would be a helpful tool in catching and debugging errors. I have some thoughts on syntax and specifics that I think would work well, however I wanted to solicit more feedback before I go too far down that path. Nathan

8 years, 7 months

Columnize in module "cmd"

by Tom Zhou

Hi~ alls recently, i focus on the module "cmd", and find some confused things-- the function named "columnize". Why we need a multiloop as "for nrows .. for col .. for row.." ?? i think we can make a easier method, for example, first, find out the maxlen str in list, and use its length as the standard size to format the list. Ok, maybe i ignore something, so please give me some hints. --- thanks tom

8 years, 7 months

Re: [Python-ideas] Adding "Typed" collections/iterators to Python

by Nathan Rice

>> L2 = [X(e) for e in L1]>> L3 = [Y(e) for e in L2]>> vs>> L2 = X(L1) # assuming X has been updated to work in both vector/scalar>> L3 = Y(L2) # context...>>> L = ['a', 'bc', ['ada', 'a']]>> What is len(L)? 3 or [1, 2, 2] or [1, 2, [3, 1]]?>>>> L2 = [Z(Y(X(e))) for e in L1]>> vs>> L2 = Z(Y(X(L1)))>>>> L2 = [e.X().Y().Z() for e in L1]>> vs>> L2 = L1.X().Y().Z() # assuming vectorized versions of member methods>> #are folded into the collection via the mixin.>>> What is L.count('a')? 1 or [1, 0, 1] or [1, 0, [2, 1]]? A fair concern; if the vectorized version of the child method were given the same name as the child method, I agree that this could result in ambiguity. There are multiple ways that member methods could be made available on the collection, including a proxy attribute, renaming, etc. ... >> I find >> using operator& functools _far_ clearer in intent than using lambda, >> >> _and it works right now_, which was the point I was trying to make >> here. > > > I find using list comprehensions and generator expressions even more > clearer. In general I think comprehensions are superior to map because they are more intuitive. Based on my experience reading code in the community I think that is well supported. Sadly, even though I have issues with the way they are used in many cases, lambdas are superior to a lot of other options because they are simple and cover many use cases. When used inline with simple expressions they provide a lot of mileage.

8 years, 7 months

context managers used to allocate resources can be abused - a solution

by Alon Horev

Hello, while I was implementing a connection pool i've noticed a pitfall of our beloved with statements: with pool.get_connection() as conn: ....conn.execute(...) conn.execute(...) # the connection has been returned to the pool and does not belong to the user! a proposed solution: with protected_context(pool.get_connection()) as conn: ....conn.execute(...) conn.execute(...) # raises OutOfContextError() with protected_context(file("/tmp/bla.txt", "w")) as f: ....file.write("blo") file.write("blu") # raises OutOfContextError() the solution is to basically proxy all methods to the real object until the context ends and then the proxy expires. what do you think about adding it to contextlib? Thanks, Alon Horev

8 years, 7 months

Python Isn't Perfect: adding a 'gotchas' section to the tutorial

by Richard Prosser

Although I love Python there are some aspects of the language design which are disappointing and which can even lead to problems in some cases. A classic example is a mutable default argument having the potential to produce unexpected side-effects, as a consequence of the non-intuitive scoping rules. Another awkward 'feature' is the requirement for a trailing comma in singleton tuples, due I believe to the use of expression parentheses rather than (say) the use of special brackets like chevrons. Something that I personally wish for is the ability to declare variable types 'up front' but that facility is missing from Python. This is an important issue, so I propose that the Python tutorial be updated to highlight such problems. I would be willing to write a draft section myself but obviously it would need to be reviewed. I am not sure if this is the appropriate place to make such a comment but it seems to be a good starting point. Any advice on making a more formal proposal would be welcome. Cheers, Richard Prosser PS Is it too late to fix such warts in version 3?

8 years, 7 months

contextlib.maybe

by Julian Berman

Twice recently I've found myself wanting to write the following code: def fn(a_file=None): responsible_for_closing = False if a_file is None: a_file = open(a_default_location) responsible_for_closing = True do_stuff(a_file) if responsible_for_closing: a_file.close() which can be written slightly shorter I know, but it's still a tiny bit messy and repetitive. What I'd prefer to write is something more like: def fn(a_file=None): with contextlib.maybe(a_file, open, default) as a_file: do_stuff(a_file) where `maybe` takes an object and conditionally runs a context manager if a check fails. Implementation would be: @contextlib.contextmanager def maybe(got, contextfactory, *args, checkif=bool, **kwargs): if checkif(got): yield got else: with contextfactory(*args, **kwargs) as got: yield got It's hard to gauge utility for such simple functions (though contextlib already has closing(), so I figured it'd be worth asking at least). Would this be useful to others? Or perhaps I'm completely missing something and you've got suggestions on how to better have an API where an argument can be fetched if not provided but a context manager would preferably need to be run to do so.

8 years, 7 months

list / array comprehensions extension

by Alexander Heger

Hi, in IDL/GDL, if at have an array x = [1,2,3] ?I can simply construct a new array by y = [0,x] which would give [0,1,2,3] I know you can do this case with x[0:0] = [0] but obviously more complicated cases are conceivable, common for me, in fact x=[1,2,3] y=[5,6,7] z=[0,x,4,y,8] result [0,1,2,3,4,5,6,7,8] or maybe even z=[0,x[0:2],5] and so forth. On the other hand, if you have a function you can pass the list of arguments to another function def f(*args): ? g(*args) which expands the list args. ?So, would it be possible - not even reasonable to have something similar for list and arrays, maybe dictionaries, in comprehensions as well? x = [1,2,3] y = [0,*x] to obtain [0,1,2,3] or similar for lists x = (1,2,3) y = (0,*x) to obtain (0,1,2,3) (for dictionaries, which are not sorted, the extend function seems fine) Or is there a way of doing this that in a similarly compact and obvious way I did not yet discover? -Alexander

8 years, 7 months

An Alternate Suite Delineation Syntax For Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)

by Nick Coghlan

A more appropriate way to phrase this suggestion would be one that clearly states the problems with the status quo and explores them, rather than assuming an answer, and indulging in a long rambling diatribe against those that will shout you down for adding nothing new to the discussion. Step 1: post to the right mailing list (i.e. python-ideas, not python-dev) Step 2: don't assume the conclusion in the subject line. Ask a question, don't state an imperative. For example: "An Alternate Suite Delineation Syntax For Python?" Step 3: respect the time of others, by trimming your message to its core essence For example: An Alternate Suite Delineation Syntax For Python? ================================================= Python's whitespace based delineation of suites is one of its greatest strengths. It aligns what the human reader perceives with what the computer is actually executing, reducing the frequency of semantic errors due to mismatches between the use of separate block delimiters and the human readable indentation. However, this benefit comes at quite a high price: it is effectively impossible to embed arbitrary Python statements into any environment where leading whitespace is *not* significant, including Python's own expression syntax. It can be argued that this restriction has led directly to the introduction of "expression friendly" variants of several Python top level constructs (for example, lambda expressions, conditional expressions and as a contributing factor in creating the various forms of comprehension). It is also one of the reasons Python-based templating languages almost always create their own custom syntax - embedding Python's own whitespace sensitive statement syntax into environments where leading whitespace is either ignored or forms a significant part of the template output is a formidable challenge. In other languages, this kind of issue is handled by using explicit suite and statement delimiters (often braces and semi-colons, respectively) to allow full suites to be used as expressions. While Python uses braces for another purpose (dictionary and set definitions), it is already the case that semi-colons (';') can be used as statement terminators, both optionally at the end of any simple statement, and also to combine multiple simple statements into a single larger statement (e.g. "x += y; print(x)"). It seems that this existing feature could be combined with a brace-based notation to create an unambiguous "suite expression" syntax that would enjoy the same semantics as ordinary Python suites (i.e. doesn't create a new scope, doesn't directly affect control flow), but allows *all* Python statements to be embedded inside expressions. Currently, the character sequence "{:" is a Syntax Error: you are attempting to end a compound statement header line while an opening brace remains unmatched, or else trying to build a dictionary without specifying the key value. This creates an opportunity to re-use braces for a suite expression syntax without conflicting with their use for set and dictionary construction. Specifically, it should be possible to create a variant of the top-level Python syntax that: 1. Explicitly delimits suites using the notation "{:" to open the suite and "}" to end it 2. Requires the use of ";" to separate simple statements (i.e. newline characters would not end a statement, since we would be inside an expression) 3. Requires that all subordinate suites also be suite expressions (i.e. leading whitespace would not be significant, since we would be inside an expression) 4. Returns the value of the last statement in the suite as the result of the suite expression (since return statements would affect the containing scope) 5. Anonymous class and function definitions would be permitted in a suite expression (but accepting only a suite expression instead of an ordinary suite) (Ruby block notation and C's comma expressions are pretty much direct inspiration for the above list) Some examples: Raise expressions: x = y if y is not None else {: raise ValueError("y must not be None!")} Try expressions: x = {: try {: y.hello} except AttributeError {: "world!"}} With expressions: data = {: with open(fname) as f {: f.read()}} Embedded assignments: if {: m = pat.search(data); m is not None}: # do something with m else: # No match! In-order conditional expressions: {: if a {:b} else {:c}} One-liner accumulator function: def acc(n) {: s=n; return {: def (i) {: nonlocal s; s += i; return s}}} Cheers, Nick. P.S. I actually think the above idea is kinda cryptic and ugly. It is, however, extraordinarily powerful, especially when it comes to embedding Python code inside other environments that may not be friendly to significant leading whitespace. Applying the excellent set of criteria suggested by Mike Meyer: 1. What's the use case? Cleanly embedding arbitrary Python statements inside environments that are not generally sensitive to leading whitespace, such as templating languages, Python expressions and Python strings. 2. Does it make such code more readable? Yes, I believe it does, by providing "one obvious way to do it". Currently there are a mish-mash of alternatives out there (or else people give up and use something else entirely, like Javascript). 3. Does it encourage writing unreadable code? Perhaps. You certainly end up with a lot of braces, colons and semi-colons floating around. However, that's also something that can be addressed by style guides - if people are using the embedded syntax when the indented syntax would work fine, the problem isn't really due to the embedded syntax, it's due to people not caring about maintainability. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

8 years, 7 months

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas December 2011