Mailman 3 July 2012 - Python-ideas

the optional "as" statement inside "if" statements
by fiatjaf＠yahoo.com.br 07 Jul '12

07 Jul '12

the idea is to make an variable assignment at the same time that the existence of that variable -- which is being returned by a function -- is made. suppose we are returning a variable from the method 'get' from the 'request' object and them making some stuff with it, but that stuff we will only do if it exists, if not, we'll just pass, instead of writing: variable = self.request.get('variable') if variable: print variable we could write if self.request.get('variable') as variable: print variable seems stupid (or not?), but with lots of variables to process, this pre-assignment could be very unpleasant -- especially if, as the in the example case, very little use will be made of the tested variable. also, the "as" expression already exists and is very pythonic.

11 16

Bloody FAQ (Was: [Python-Dev] itertools.chunks(iterable, size, fill=None))
by anatoly techtonik 06 Jul '12

06 Jul '12

On Thu, Jul 5, 2012 at 7:50 PM, Stefan Behnel <stefan_ml(a)behnel.de> wrote: > anatoly techtonik, 05.07.2012 15:36: >> On Sun, Jul 1, 2012 at 12:09 AM, Terry Reedy wrote: >>> From Raymond's first message on http://bugs.python.org/issue6021 , add >>> grouper: >>> >>> "This has been rejected before. >> >> I quite often see such arguments and I can't stand to repeat that >> these are not arguments. It is good to know, but when people use that >> as a reason to close tickets - that's just disgusting. > > The *real* problem is that people keep bringing up topics (and even spell > them out in the bug tracker) without searching for existing discussions > and/or tickets first. That's why those who do such a search (or who know > what they are talking about anyway) close these tickets with the remark > "this has been rejected before", instead of repeating an entire heap of > arguments all over again to feed a discussion that would only lead to the > same result as it did before, often several times before. Make the bloody FAQ and summarize this stuff? Why waste each others time? If people don't enjoy repeating themselves over and over - there is a bloody wiki. What should happen to people to start extracting gems of knowledge from piles of dusty sheets called list "archives" for others to admire. No, it is easier to say "it was already discussed many times", "why don't you Google yourself", "so far you're only complaining", etc. If people can't find anything - why everybody thinks they are ignorant and lazy. Even if it so, why nobody thinks that maybe that bloody Xapian index is dead again for a bloody amount of moons nobody knows why and how many exactly? Why nobody thinks that lazy coders can also help with development? Maybe that laziness is the primary reason some major groups actually prefer Python to Java, C++ and other more interesting languages (such as PHP) when it comes to typing? Make it easy and the patches will follow. Answers like "this was discussed before" don't make it easy to understand, and leaving users rereading old 19xx archives that people don't reread themselves will likely make users bounce and never (NEVER!) come up with some proposal again. An "organic" way to keep traffic low. Miscommunication is a bad experience for users, bad experience for developers, everybody is annoyed and as a result such nice language as Python loses points on TIOBE (and convenient chunk() functions to munch-munch on the sequence data). Wheew. :-F

3 3

Re: [Python-ideas] [Python-Dev] itertools.chunks(iterable, size, fill=None)
by anatoly techtonik 05 Jul '12

05 Jul '12

On Wed, Jul 4, 2012 at 9:31 PM, Terry Reedy <tjreedy(a)udel.edu> wrote: > On 7/4/2012 5:57 AM, anatoly techtonik wrote: >> >> On Fri, Jun 29, 2012 at 11:32 PM, Georg Brandl <g.brandl(a)gmx.net> wrote: > > >>> Anatoly, so far there were no negative votes -- would you care to go >>> another step and propose a patch? >> >> >> Was about to say "no problem", > > > Did you read that there *are* strong negative votes? And that this idea has > been rejected before? I summarized the objections in my two responses and > pointed to the tracker issues. One of the objections is that there are 4 > different things one might want if the sequence length is not an even > multiple of the chunk size. Your original 'idea' did not specify. I actually meant that there is a problem to propose a patch in the sense of getting checkout, working on a diff, sending it by attaching to bug tracker as developer guide says. >> For now the best thing I can do (I don't risk even to mention anything >> with 3.3) is to copy/paste code from the docs here: >> >> from itertools import izip_longest >> def chunks(iterable, size, fill=None): >> """Split an iterable into blocks of fixed-length""" >> # chunks('ABCDEFG', 3, 'x') --> ABC DEF Gxx >> args = [iter(iterable)] * size >> return izip_longest(fillvalue=fill, *args) > > > Python ideas is about Python 3 ideas. Please post Python 3 code. > > This is actually a one liner > > return zip_longest(*[iter(iterable)]*size, fillvalue=file) > > We don't generally add such to the stdlib. Can you figure out from the code what this stuff does? It doesn't give chunks of strings. >> BTW, this doesn't work as expected (at least for strings). Expected is: >> chunks('ABCDEFG', 3, 'x') --> 'ABC' 'DEF' 'Gxx' >> got: >> chunks('ABCDEFG', 3, 'x') --> ('A' 'B' 'C') ('D' 'E' 'F') ('G' 'x' 'x') > > > One of the problems with idea of 'add a chunker' is that there are at least > a dozen variants that different people want. That's not the problem. People always want something extra. The problem that we don't have a real wish distribution. If 1000 people want chunks and 1 wants groups with exception - we still account these as equal variants. Therefore my idea is deliberately limited to "string to chunks" user story, and SO implementation proposal. > I discussed the problem of > return types issue in my responses. I showed how to get the 'expected' > response above using grouper, but also suggested that it is the wrong basis > for splitting strings. Repeated slicing make more sense for concrete > sequence types. > > def seqchunk_odd(s, size): > # include odd size left over > for i in range(0, len(s), size): > yield s[i:i+size] > > print(list(seqchunk_odd('ABCDEFG', 3))) > # > ['ABC', 'DEF', 'G'] Right. That's the top answer on SO that people think should be in stdlib. Great we are talking about the same thing actually. > def seqchunk_even(s, size): > # only include even chunks > for i in range(0, size*(len(s)//size), size): > yield s[i:i+size] > > print(list(seqchunk_even('ABCDEFG', 3))) > # > ['ABC', 'DEF'] This is deducible from seqchunk_odd(s, size) > def strchunk_fill(s, size, fill): > # fill odd chunks > q, r = divmod(len(s), size) > even = size * q > for i in range(0, even, size): > yield s[i:i+size] > if size != even: > yield s[even:] + fill * (size - r) > > print(list(strchunk_fill('ABCDEFG', 3, 'x'))) > # > ['ABC', 'DEF', 'Gxx'] Also deducible from seqchunk_odd(s, size) > Because the 'fill' value is necessarily a sequence for strings, > strchunk_fill would only work for lists and tuples if the fill value were > either required to be given as a tuple or list of length 1 or if it were > internally converted inside the function. Skipping that for now. > > Having written the fill version based on the even version, it is easy to > select among the three behaviors by modifying the fill version. > > def strchunk(s, size, fill=NotImplemented): > # fill odd chunks > q, r = divmod(len(s), size) > even = size * q > for i in range(0, even, size): > yield s[i:i+size] > if size != even and fill is not NotImplemented: > yield s[even:] + fill * (size - r) > > print(*strchunk('ABCDEFG', 3)) > print(*strchunk('ABCDEFG', 3, '')) > print(*strchunk('ABCDEFG', 3, 'x')) > # > ABC DEF > ABC DEF G > ABC DEF Gxx I now don't even think that fill value is needed as argument. if len(chunk) < size: chunk.extend( [fill] * ( size - len(chunk)) ) > I already described how something similar could be done by checking each > grouper output tuple for a fill value, but that requires that the fill value > be a sentinal that could not otherwise appear in the tuple. One could modify > grouper to fill with a private object() and check the last item of each > group for that sentinal and act accordingly (delete, truncate, or replace). > A generic api needs some thought, though. I just need to chunk strings and sequences. Generic API is too complex without counting all usecases and iterating over them. > An issue I did not previously mention is that people sometimes want > overlapping chunks rather than contiguous disjoint chunks. The slice > approach trivially adapts to that. > > def seqlap(s, size): > for i in range(len(s)-size+1): > yield s[i:i+size] > > print(*seqlap('ABCDEFG', 3)) > # > ABC BCD CDE DEF EFG > > A sliding window for a generic iterable requires a deque or ring buffer > approach that is quite different from the zip-longest -- grouper approach. That's why I'd like to drastically reduce the scope of proposal. itertools doesn't seem to be the best place anymore. How about sequence method? string.chunks(size) -> ABC DEF G list.chunks(size) -> [A,B,C], [C,D,E],[G] If somebody needs a keyword argument - this can come later without breaking compatibility.

2 1

Re: [Python-ideas] [Python-Dev] itertools.chunks(iterable, size, fill=None)
by anatoly techtonik 05 Jul '12

05 Jul '12

On Sun, Jul 1, 2012 at 3:01 PM, Stefan Behnel <stefan_ml(a)behnel.de> wrote: > > To address the main problem of users not finding what they need, what about > simply extending the docstring of the grouper() function with a sentence > like this: > > "This functionality is also called 'chunking' or 'blocking' and can be used > for load distribution and sharding." > > That would make it easy for users to find what they are looking for when > they search the page for "chunk". I find that a much more common and less > ambiguous name than "grouping", which reminds me more of "group by". In appeared that "chunking" and "grouping" are different kind of tasks. You can chunk a sequence (sting) by slicing it into smaller sequences, but you can not chunk in iterable - you can only group it. There is an data loss about the structure that occurs when a sequence (string) becomes an iterator: chunks ABCDE -> AB CD E group ABCDE -> A B C D E -> (A B) (C D) (D E)

1 0

Re: [Python-ideas] [Python-Dev] itertools.chunks(iterable, size, fill=None)
by Raymond Hettinger 03 Jul '12

03 Jul '12

On Jul 1, 2012, at 5:01 AM, Stefan Behnel wrote: > To address the main problem of users not finding what they need, what about > simply extending the docstring of the grouper() Here's a small change to the docstring: http://hg.python.org/cpython/rev/d32f21d87363 FWIW, if you're interested in load balancing applications, George Sakkis's itertools recipe for roundrobin() may be of interest. Another interesting iterator technique that is not well known is the two-argument form of iter() which is a marvel for transforming callables into iterators: for block in iter(partial(f.read, 1024), ''): ... for diceroll in iter(partial(randrange(1, 7), 4): ... Raymond

1 0

isascii()/islatin1()/isbmp()
by Serhiy Storchaka 02 Jul '12

02 Jul '12

As shown in issue #15016 [1], there is a use cases when it is useful to determine that string can be encoded in ASCII or Latin1. In working with Tk or Windows console applications can be useful to determine that string can be encoded in UCS2. C API provides interface for this, but at Python level it is not available. I propose to add to strings class new methods: isascii(), islatin1() and isbmp() (in addition to such methods as isalpha() or isdigit()). The implementation will be trivial. Pro: The current trick with trying to encode has O(n) complexity and has overhead of exception raising/catching. Contra: In most cases after determining characters range we still need to encode a string with the appropriate encoding. New methods will complicate already overloaded strings class. Objections? [1] http://bugs.python.org/issue15016

10 19

Happy leap second
by Alexander Belopolsky 01 Jul '12

01 Jul '12

Even though many have hoped that the authorities would stop fiddling with our clocks, today a leap second will be inserted in UTC. Systems using Olson/IANA timezone database have a way to deal with this without adjusting their clocks, but few systems are configured that way: $ TZ=right/UTC date -d @1341100824 Sat Jun 30 23:59:60 UTC 2012 (1341100824 is the number of seconds since epoch including the leap seconds.) Python's time module works fine with the "right" timezones: >>> import time >>> print(time.strftime('%T', time.localtime(1341100824))) 23:59:60 but the datetime module clips the leap second down to the previous second: >>> from datetime import datetime >>> from datetime import datetime >>> print(datetime.fromtimestamp(1341100824).strftime('%T')) 23:59:59 >>> print datetime.fromtimestamp(1341100823).strftime('%T') 23:59:59 BDFL has been resisting adding support for leap seconds to the datetime module [1], but as the clocks become more accurate and synchronization requirements become stricter, we may want to revisit this issue. [1] http://mail.python.org/pipermail/python-ideas/2010-June/007307.html

4 9