Mailman 3 July 2012 - Python-ideas

Where has packaging gone?
by Michele Lacchia 16 Jul '12

16 Jul '12

Hi, I read this: http://www.gossamer-threads.com/lists/python/dev/992979 but at the end nobody posted to it anymore. I would like to know what will now happen, since as far as I understood, the development of packaging has stopped and Bento (David Cournapeau library) has appeared. So what will happen now? Packaging will continue the development? Or will be Bento replace it? Personally I don't care which tool will become standard, but I do feel the need for a standard to be created. So I would like to point out that to me these three PEPs seem really important, and hope that will be taken into account: - PEP 345 (http://www.python.org/dev/peps/pep-0345/) - PEP 376 (http://www.python.org/dev/peps/pep-0376/) - PEP 386 (http://www.python.org/dev/peps/pep-0386/) What do you think about all this? Regards, Michele Lacchia

1 0

Syntax for defining parametric decorators
by Mike Graham 12 Jul '12

12 Jul '12

A common stumbling block for new users is writing decorators that take arguments. To create a decorator like @timesn(n) def f(y): ... We write code like def timesn(n) def decorator(f): def inner(y): return n * f(y) return inner return decorator which confuses many users and can be a handful to type. I wonder if it would be clearer for people to write def timesn(n)(f): def inner(y): return n * f(y) return inner which is more concise and looks a lot more like a non-parametric decorator someone might have written already. The syntax is mostly self-explaining and could potentially be useful in other contexts. There exist tools like the decorator library to try to simplify this already, but in my experience they mostly serve to confuse people using decorators for the first time more. One thing I didn't specify was whether `n` was nonlocal or not and the behavior of something that keeps and reuses timesn(some_specific_n) multiple times. Does anyone think a feature like this may be useful? Regards, Mike

16 30

Explanation on how to search the archives
by Christopher Reay 12 Jul '12

12 Jul '12

I have heard two suggestions on how to search the achives one is: If you have time to post and discuss an idea, you have time to search gmane.comp.python.ideas at search.gmane.org and perhaps find out why not to bother posting. And the Other: It's more a matter of working out how to point Google (or the search engine of your choice) at the archives in a useful way. In this case: https://www.google.com/search?q=inurl%3Apython-ideas%20site%3Amail.python.o… This is great information. Is there a page suggesting how to search archives effectively? Whilst much of that skill is generic, and clear concise explanation of what is expected from the polite user of the mailing list (and other Python mailing lists) could help. A link to that page could be put in the footer of the python mailing list emails.. and voila, a very easy thing to reference and get righteous about people not having read. What do you think? Christopher -- Be prepared to have your predictions come true

4 5

itertools recipes: why not add them to the stdlib *somewhere*?
by Jürgen A. Erhard 10 Jul '12

10 Jul '12

I just now ran into (since learning about the recipes) having use for a recipe (then I found out that I can't use it anyway, but that's a different thing) So... I consider it very strange and annoying that there's this code. Ready to use. But you have to copy-and-paste it into your code *somewhere* instead of easily importing it. As to the "it makes using Python harder to learn", I beg to differ: an addition to the *language* (like, say, metaclasses) *can* make it more complicated. But additions to the stdlib? What about "batteries included"? Not a motto anymore (I heard rumors, so maybe that's the case) Putting these in some official(!) iterutils (or name it what you want) package would be a solution for so many people. Yes, not everyone needs grouper's current functionality. But for the many who do, it'd be there already. And someone thought those recipes are useful, didn't you? So why not make them more easily available? And then there's Alex Martelli at http://stackoverflow.com/a/1641242/238406, and I agree with him wholeheartedly. Bye, J

12 14

Re: [Python-ideas] Syntax for defining parametric decorators
by Haoyi Li 09 Jul '12

09 Jul '12

I admit calling it all at once would be very strange, but i still think the "call me three times to get result" comes across waaaay more clearly than with nested defs. Multiple arguments in one list is about the same, it's true, until you really need the curried version, like in decorators, then you end up with the nasty tower-of-defs which sparked this discussion. Regarding "not knowing how many times it should be called", the idea is that you break it up when you have a meaningful, reusable unit. That's to say you wouldn't do a def f(a)(b)(c) Unless f f(1) f(1)(2) f(1)(2)(3) Are all meaningful, useful functions. Doesn't happen very often, granted, but when it does (e.g. for decorator factories and decorators) it'd be nice to have. I view curried defs like default parameters: Doesnt add anything that cant already be done (you can emulate default args with telescoping methods) but cuts boilerplate, makes things clearer and overall nice to have. Whether it's nice enough to justify extra syntax is of course open for debate. From: Stephen J. Turnbull Sent: 7/9/2012 1:48 AM To: Haoyi Li Cc: Nick Coghlan; Python-Ideas Subject: Re: [Python-ideas] Syntax for defining parametric decorators Haoyi Li writes: > However, I think that in the general case, being able to define > curried functions in one line, a.l.a. [...] > def f(a)(b)(c) > > looks much more similar to how you would use it: > > f(1)(2)(3) I don't understand. I wouldn't use it that way! I would think if that were common usage, and the curried form only occasional, def f(a,b,c): pass f(1,2,3) would be more natural in Python (and use functools.partial for the occasional currying). Presumably if you were using this syntax, you'd be doing something more like def f(a)(b)(c): pass def g(h, x): h(x) g(f(1)(2), 3) which doesn't look very nice to me, nor do I find the def to be a particularly intuitive way of reminding me that the proper usage is a call of a function in curried form (specifically, I would not be reminded of whether the expected curried form is a 1st-, 2nd-, or 3rd-order function). I also suppose that you wouldn't be able to do def f(a)(b)(c): pass def g(x,y): pass h(f(1)) h(g) (ie, without LBYL or EAFP constructs in h, which would simply be throwing the complexity into the caller's backyard). Even though def bar(a)(b): pass def baz(x): pass quux(bar(1)) quux(baz) would work fine. So to me, there may be something in this syntax, but my initial impression is that it's trying to turn Python into something it's not.

1 0

the optional "as" statement inside "if" statements
by fiatjaf＠yahoo.com.br 07 Jul '12

07 Jul '12

the idea is to make an variable assignment at the same time that the existence of that variable -- which is being returned by a function -- is made. suppose we are returning a variable from the method 'get' from the 'request' object and them making some stuff with it, but that stuff we will only do if it exists, if not, we'll just pass, instead of writing: variable = self.request.get('variable') if variable: print variable we could write if self.request.get('variable') as variable: print variable seems stupid (or not?), but with lots of variables to process, this pre-assignment could be very unpleasant -- especially if, as the in the example case, very little use will be made of the tested variable. also, the "as" expression already exists and is very pythonic.

11 16

Bloody FAQ (Was: [Python-Dev] itertools.chunks(iterable, size, fill=None))
by anatoly techtonik 06 Jul '12

06 Jul '12

On Thu, Jul 5, 2012 at 7:50 PM, Stefan Behnel <stefan_ml(a)behnel.de> wrote: > anatoly techtonik, 05.07.2012 15:36: >> On Sun, Jul 1, 2012 at 12:09 AM, Terry Reedy wrote: >>> From Raymond's first message on http://bugs.python.org/issue6021 , add >>> grouper: >>> >>> "This has been rejected before. >> >> I quite often see such arguments and I can't stand to repeat that >> these are not arguments. It is good to know, but when people use that >> as a reason to close tickets - that's just disgusting. > > The *real* problem is that people keep bringing up topics (and even spell > them out in the bug tracker) without searching for existing discussions > and/or tickets first. That's why those who do such a search (or who know > what they are talking about anyway) close these tickets with the remark > "this has been rejected before", instead of repeating an entire heap of > arguments all over again to feed a discussion that would only lead to the > same result as it did before, often several times before. Make the bloody FAQ and summarize this stuff? Why waste each others time? If people don't enjoy repeating themselves over and over - there is a bloody wiki. What should happen to people to start extracting gems of knowledge from piles of dusty sheets called list "archives" for others to admire. No, it is easier to say "it was already discussed many times", "why don't you Google yourself", "so far you're only complaining", etc. If people can't find anything - why everybody thinks they are ignorant and lazy. Even if it so, why nobody thinks that maybe that bloody Xapian index is dead again for a bloody amount of moons nobody knows why and how many exactly? Why nobody thinks that lazy coders can also help with development? Maybe that laziness is the primary reason some major groups actually prefer Python to Java, C++ and other more interesting languages (such as PHP) when it comes to typing? Make it easy and the patches will follow. Answers like "this was discussed before" don't make it easy to understand, and leaving users rereading old 19xx archives that people don't reread themselves will likely make users bounce and never (NEVER!) come up with some proposal again. An "organic" way to keep traffic low. Miscommunication is a bad experience for users, bad experience for developers, everybody is annoyed and as a result such nice language as Python loses points on TIOBE (and convenient chunk() functions to munch-munch on the sequence data). Wheew. :-F

3 3

Re: [Python-ideas] [Python-Dev] itertools.chunks(iterable, size, fill=None)
by anatoly techtonik 05 Jul '12

05 Jul '12

On Wed, Jul 4, 2012 at 9:31 PM, Terry Reedy <tjreedy(a)udel.edu> wrote: > On 7/4/2012 5:57 AM, anatoly techtonik wrote: >> >> On Fri, Jun 29, 2012 at 11:32 PM, Georg Brandl <g.brandl(a)gmx.net> wrote: > > >>> Anatoly, so far there were no negative votes -- would you care to go >>> another step and propose a patch? >> >> >> Was about to say "no problem", > > > Did you read that there *are* strong negative votes? And that this idea has > been rejected before? I summarized the objections in my two responses and > pointed to the tracker issues. One of the objections is that there are 4 > different things one might want if the sequence length is not an even > multiple of the chunk size. Your original 'idea' did not specify. I actually meant that there is a problem to propose a patch in the sense of getting checkout, working on a diff, sending it by attaching to bug tracker as developer guide says. >> For now the best thing I can do (I don't risk even to mention anything >> with 3.3) is to copy/paste code from the docs here: >> >> from itertools import izip_longest >> def chunks(iterable, size, fill=None): >> """Split an iterable into blocks of fixed-length""" >> # chunks('ABCDEFG', 3, 'x') --> ABC DEF Gxx >> args = [iter(iterable)] * size >> return izip_longest(fillvalue=fill, *args) > > > Python ideas is about Python 3 ideas. Please post Python 3 code. > > This is actually a one liner > > return zip_longest(*[iter(iterable)]*size, fillvalue=file) > > We don't generally add such to the stdlib. Can you figure out from the code what this stuff does? It doesn't give chunks of strings. >> BTW, this doesn't work as expected (at least for strings). Expected is: >> chunks('ABCDEFG', 3, 'x') --> 'ABC' 'DEF' 'Gxx' >> got: >> chunks('ABCDEFG', 3, 'x') --> ('A' 'B' 'C') ('D' 'E' 'F') ('G' 'x' 'x') > > > One of the problems with idea of 'add a chunker' is that there are at least > a dozen variants that different people want. That's not the problem. People always want something extra. The problem that we don't have a real wish distribution. If 1000 people want chunks and 1 wants groups with exception - we still account these as equal variants. Therefore my idea is deliberately limited to "string to chunks" user story, and SO implementation proposal. > I discussed the problem of > return types issue in my responses. I showed how to get the 'expected' > response above using grouper, but also suggested that it is the wrong basis > for splitting strings. Repeated slicing make more sense for concrete > sequence types. > > def seqchunk_odd(s, size): > # include odd size left over > for i in range(0, len(s), size): > yield s[i:i+size] > > print(list(seqchunk_odd('ABCDEFG', 3))) > # > ['ABC', 'DEF', 'G'] Right. That's the top answer on SO that people think should be in stdlib. Great we are talking about the same thing actually. > def seqchunk_even(s, size): > # only include even chunks > for i in range(0, size*(len(s)//size), size): > yield s[i:i+size] > > print(list(seqchunk_even('ABCDEFG', 3))) > # > ['ABC', 'DEF'] This is deducible from seqchunk_odd(s, size) > def strchunk_fill(s, size, fill): > # fill odd chunks > q, r = divmod(len(s), size) > even = size * q > for i in range(0, even, size): > yield s[i:i+size] > if size != even: > yield s[even:] + fill * (size - r) > > print(list(strchunk_fill('ABCDEFG', 3, 'x'))) > # > ['ABC', 'DEF', 'Gxx'] Also deducible from seqchunk_odd(s, size) > Because the 'fill' value is necessarily a sequence for strings, > strchunk_fill would only work for lists and tuples if the fill value were > either required to be given as a tuple or list of length 1 or if it were > internally converted inside the function. Skipping that for now. > > Having written the fill version based on the even version, it is easy to > select among the three behaviors by modifying the fill version. > > def strchunk(s, size, fill=NotImplemented): > # fill odd chunks > q, r = divmod(len(s), size) > even = size * q > for i in range(0, even, size): > yield s[i:i+size] > if size != even and fill is not NotImplemented: > yield s[even:] + fill * (size - r) > > print(*strchunk('ABCDEFG', 3)) > print(*strchunk('ABCDEFG', 3, '')) > print(*strchunk('ABCDEFG', 3, 'x')) > # > ABC DEF > ABC DEF G > ABC DEF Gxx I now don't even think that fill value is needed as argument. if len(chunk) < size: chunk.extend( [fill] * ( size - len(chunk)) ) > I already described how something similar could be done by checking each > grouper output tuple for a fill value, but that requires that the fill value > be a sentinal that could not otherwise appear in the tuple. One could modify > grouper to fill with a private object() and check the last item of each > group for that sentinal and act accordingly (delete, truncate, or replace). > A generic api needs some thought, though. I just need to chunk strings and sequences. Generic API is too complex without counting all usecases and iterating over them. > An issue I did not previously mention is that people sometimes want > overlapping chunks rather than contiguous disjoint chunks. The slice > approach trivially adapts to that. > > def seqlap(s, size): > for i in range(len(s)-size+1): > yield s[i:i+size] > > print(*seqlap('ABCDEFG', 3)) > # > ABC BCD CDE DEF EFG > > A sliding window for a generic iterable requires a deque or ring buffer > approach that is quite different from the zip-longest -- grouper approach. That's why I'd like to drastically reduce the scope of proposal. itertools doesn't seem to be the best place anymore. How about sequence method? string.chunks(size) -> ABC DEF G list.chunks(size) -> [A,B,C], [C,D,E],[G] If somebody needs a keyword argument - this can come later without breaking compatibility.

2 1

Re: [Python-ideas] [Python-Dev] itertools.chunks(iterable, size, fill=None)
by anatoly techtonik 05 Jul '12

05 Jul '12

On Sun, Jul 1, 2012 at 3:01 PM, Stefan Behnel <stefan_ml(a)behnel.de> wrote: > > To address the main problem of users not finding what they need, what about > simply extending the docstring of the grouper() function with a sentence > like this: > > "This functionality is also called 'chunking' or 'blocking' and can be used > for load distribution and sharding." > > That would make it easy for users to find what they are looking for when > they search the page for "chunk". I find that a much more common and less > ambiguous name than "grouping", which reminds me more of "group by". In appeared that "chunking" and "grouping" are different kind of tasks. You can chunk a sequence (sting) by slicing it into smaller sequences, but you can not chunk in iterable - you can only group it. There is an data loss about the structure that occurs when a sequence (string) becomes an iterator: chunks ABCDE -> AB CD E group ABCDE -> A B C D E -> (A B) (C D) (D E)

1 0

Re: [Python-ideas] [Python-Dev] itertools.chunks(iterable, size, fill=None)
by Raymond Hettinger 03 Jul '12

03 Jul '12

On Jul 1, 2012, at 5:01 AM, Stefan Behnel wrote: > To address the main problem of users not finding what they need, what about > simply extending the docstring of the grouper() Here's a small change to the docstring: http://hg.python.org/cpython/rev/d32f21d87363 FWIW, if you're interested in load balancing applications, George Sakkis's itertools recipe for roundrobin() may be of interest. Another interesting iterator technique that is not well known is the two-argument form of iter() which is a marvel for transforming callables into iterators: for block in iter(partial(f.read, 1024), ''): ... for diceroll in iter(partial(randrange(1, 7), 4): ... Raymond

1 0

isascii()/islatin1()/isbmp()
by Serhiy Storchaka 02 Jul '12

02 Jul '12

As shown in issue #15016 [1], there is a use cases when it is useful to determine that string can be encoded in ASCII or Latin1. In working with Tk or Windows console applications can be useful to determine that string can be encoded in UCS2. C API provides interface for this, but at Python level it is not available. I propose to add to strings class new methods: isascii(), islatin1() and isbmp() (in addition to such methods as isalpha() or isdigit()). The implementation will be trivial. Pro: The current trick with trying to encode has O(n) complexity and has overhead of exception raising/catching. Contra: In most cases after determining characters range we still need to encode a string with the appropriate encoding. New methods will complicate already overloaded strings class. Objections? [1] http://bugs.python.org/issue15016

10 19

Happy leap second
by Alexander Belopolsky 01 Jul '12

01 Jul '12

Even though many have hoped that the authorities would stop fiddling with our clocks, today a leap second will be inserted in UTC. Systems using Olson/IANA timezone database have a way to deal with this without adjusting their clocks, but few systems are configured that way: $ TZ=right/UTC date -d @1341100824 Sat Jun 30 23:59:60 UTC 2012 (1341100824 is the number of seconds since epoch including the leap seconds.) Python's time module works fine with the "right" timezones: >>> import time >>> print(time.strftime('%T', time.localtime(1341100824))) 23:59:60 but the datetime module clips the leap second down to the previous second: >>> from datetime import datetime >>> from datetime import datetime >>> print(datetime.fromtimestamp(1341100824).strftime('%T')) 23:59:59 >>> print datetime.fromtimestamp(1341100823).strftime('%T') 23:59:59 BDFL has been resisting adding support for leap seconds to the datetime module [1], but as the clocks become more accurate and synchronization requirements become stricter, we may want to revisit this issue. [1] http://mail.python.org/pipermail/python-ideas/2010-June/007307.html

4 9